Using a Proxy with wget: Commands and Examples

wget is a command-line tool, usually pre-installed on Linux and macOS systems, used to download files and fetch web pages. It's commonly used to download a file from within a script on a server, to mirror a site, or to run a simple reachability test. This article covers three ways to run wget through a proxy, authentication, HTTPS details, what to watch for with bulk downloads, and common errors, with example commands.

All of the options covered are documented in the GNU Wget manual.

Before you start: is wget installed?#

Check the version in the terminal:

wget --version

If the command isn't found, you can use sudo apt install wget on Debian and Ubuntu, or brew install wget with Homebrew on macOS. wget isn't built into Windows; curl.exe, which comes built in on Windows 10 and later, can be used for the same jobs instead. The wget spelling in PowerShell isn't real wget — it's an alias for the Invoke-WebRequest command and doesn't accept the options in this article.

Using a test address that returns your IP address makes testing easier. If the proxy is working, this address shows the proxy's IP, not yours:

wget -qO- https://httpbin.org/ip

Here -q silences the output, and -O- prints the downloaded content to the screen instead of writing it to a file. Run it once before defining a proxy and note your own IP; you'll immediately see the difference on later attempts.

The format of a proxy address#

All three methods write the proxy the same way:

http://kullanici:parola@sunucu:port

• The http:// scheme is the protocol of the connection established with the proxy. This scheme is used even when going to HTTPS sites; we explain why below.
• The kullanici:parola@ part is optional. If you've authorized your server's IP from your proxy panel, you can leave this part out entirely.
• The sunucu:port information varies by package; you can find the correct values in your customer panel.

Method 1: with environment variables#

wget reads the system's standard proxy environment variables: http_proxy, https_proxy, ftp_proxy, and no_proxy. This is the fastest method, and many other tools running in the same terminal use these variables too.

export http_proxy="http://kullanici:parola@pr.proxynet.io:8000"
export https_proxy="http://kullanici:parola@pr.proxynet.io:8000"

wget -qO- https://httpbin.org/ip

Three things to note:

• The https_proxy value also starts with http://. This variable specifies the proxy used when going to HTTPS sites; the connection with the proxy itself is HTTP, and HTTPS traffic is tunneled inside that connection.
• The variables are only valid for that terminal session. If you want them to persist, add them to ~/.bashrc or ~/.zshrc.
• Prefer lowercase. wget reads lowercase variables; some systems may also have the uppercase HTTP_PROXY defined, and if the two definitions conflict, it causes confusion. Setting both to the same value is the safest approach.

If you want certain addresses to skip the proxy, use no_proxy:

export no_proxy="localhost,127.0.0.1,.sirketiniz.local"

This variable prevents internal network services or local development servers from being routed through the proxy. A domain name starting with a dot covers all of that domain's subdomains.

Method 2: on the command line, one-off#

To route just a single command through the proxy without changing environment variables, you can pass wget's settings directly to the command with the -e (--execute) option:

wget -e use_proxy=yes \
     -e https_proxy=http://pr.proxynet.io:8000 \
     --proxy-user=kullanici \
     --proxy-password=parola \
     https://httpbin.org/ip -O -

This method is especially useful in scripts: the proxy setting doesn't affect other commands, and you can give a different proxy on every call. For example, if you want to make each download in a loop from a different exit point, you'd take the -e https_proxy= value from the loop variable.

Every line given with -e is a setting that can also be written into a .wgetrc file. So every key you'll see in the next method is also valid here.

Method 3: with a .wgetrc file, permanently#

If you're using wget with a proxy continuously on the same machine, you can write the settings into a ~/.wgetrc file in your user directory:

use_proxy = on
http_proxy = http://pr.proxynet.io:8000
https_proxy = http://pr.proxynet.io:8000
proxy_user = kullanici
proxy_password = parola

Since the file contains a password, restrict its permissions so only you can read it:

chmod 600 ~/.wgetrc

If a system-wide setting is needed, the same lines can be written to /etc/wgetrc; the file in the user directory overrides the values in the system file.

If you want to temporarily disable the proxy while this file is defined, add the --no-proxy option for a single command:

wget --no-proxy https://httpbin.org/ip -O -

Which method should you choose?#

If all three methods are defined at once, the priority order is: command-line options override environment variables, and environment variables override the .wgetrc file.

How does a proxy work on HTTPS sites?#

When going to an HTTPS address through a proxy, wget first sends a CONNECT hedef:443 request to the proxy. The proxy opens a TCP connection to the target, and from then on only relays encrypted bytes. So the proxy can't see the site's content; it only knows which domain name is being connected to.

The practical result of this is: you can safely reach HTTPS sites even if the proxy address starts with http://. Certificate validation happens between your machine and the target site; the proxy has no part in that validation. We compared this behavior of the proxy against SOCKS in our SOCKS vs. HTTP Proxy article.

Combined with commonly used options#

A proxy setting alone is rarely enough on its own; real work usually needs a few more options. The example below downloads a file through a proxy with retry and timeout settings:

wget --timeout=30 --tries=3 --waitretry=5 \
     --user-agent="Mozilla/5.0 (X11; Linux x86_64)" \
     -O rapor.pdf https://example.com/rapor.pdf

• --timeout limits the wait time for each network operation.
• --tries sets how many times a failed download is retried; --waitretry adds a wait between attempts.
• --user-agent changes the browser identity. wget's default identity is directly blocked on some sites; using a value that identifies you is both more transparent and less likely to cause problems.
• -O names the output file; adding -c resumes a partial download from where it left off.

If cookies are needed while fetching a page, the --load-cookies and --save-cookies options keep the session in a file; this is how you can maintain a logged-in session together with a proxy.

Does wget support a SOCKS proxy?#

GNU wget has no built-in SOCKS support; all the methods above are for HTTP and HTTPS proxies. If you need SOCKS5, you have two options:

• Use cURL. cURL directly supports the socks5:// and socks5h:// schemes and does almost everything wget does. See the details in our How to Use a Proxy with cURL article.
• Use a routing tool. Tools like proxychains on Linux can route the connections of programs without SOCKS support to a SOCKS proxy. On Windows, Proxifier does the same job.

For packages that work directly with wget, take a look at the HTTPS Proxy page.

Common errors and their fixes#

"407 Proxy Authentication Required"#

Proxy credentials are missing or wrong. Check the username and password. If the password contains special characters like @, :, or #, percent-encode them within the address (%40 for @), or use the --proxy-user and --proxy-password options, which don't require encoding.

The command isn't using the proxy at all#

• Make sure you defined the variable with export; writing just http_proxy=... doesn't pass it to child processes.
• If you're going to an HTTPS address, the https_proxy variable needs to be defined, not http_proxy.
• Check whether there's a use_proxy = off line in .wgetrc.
• If you're running sudo wget, sudo may not carry your environment variables by default; carry them with sudo -E, or write the setting into /etc/wgetrc.
• The target address may be matching a domain in the no_proxy list.

"Unable to establish SSL connection"#

First check whether the target address opens without the proxy. If the problem is the certificate, the --no-check-certificate option skips validation; but this option weakens the connection's security and should only be used for testing. Remember that the proxy has no part in certificate validation: this error usually comes from the system's root certificate store being outdated or from the target site's own configuration.

The connection is timing out#

Double-check the proxy address and port. Make sure the firewall allows traffic to the proxy port. On long downloads, explicitly setting the timeout and retry count prevents scripts from getting stuck:

wget --timeout=30 --tries=3 https://httpbin.org/ip -O -

"ERROR 403: Forbidden"#

The target site is rejecting the request. The cause may not be the proxy; wget's default User-Agent value is directly blocked on some sites. Give an identifying value with --user-agent. If the problem continues, the target site may be behaving based on the proxy's IP type; we explained why data center addresses are rejected more often in our Residential vs. Datacenter Proxy article.

What to watch for with bulk downloads#

wget's -r (recursive download) option can fetch an entire site with a single command. This power can put a serious load on the target server. In bulk downloads, the following options benefit both you and the target:

wget -r -l 2 --wait=2 --random-wait --limit-rate=500k \
     --no-parent -A pdf https://example.com/belgeler/

• -l 2 limits the depth to two levels; unlimited depth downloads the entire site.
• --wait=2 puts two seconds between requests, --random-wait varies that duration randomly.
• --limit-rate=500k limits the download speed.
• --no-parent prevents going up to parent directories, -A pdf fetches only the specified extension.

wget respects robots.txt rules by default; don't turn that behavior off. We covered which data can be collected under which conditions in our Is Web Scraping Legal? article.

In projects that need a large number of requests, distributing addresses with Rotating Proxy instead of piling traffic onto a single IP both reduces the risk of getting blocked and puts a more balanced load on the target site. For more extensive needs, take a look at our data-scraping solutions.

Frequently asked questions#

wget or cURL?#

For downloading files, mirroring, and recursive fetching, wget is more practical; for API requests, custom HTTP methods, and SOCKS proxy, cURL is more capable. Both read an HTTP proxy from the same environment variables, so a setting defined once works in both tools.

Does the proxy setting affect other programs?#

A proxy defined with an environment variable affects every program started from the same shell that reads those variables (cURL, pip, git, for example). The .wgetrc and -e methods only affect wget.

Can I use a proxy without saving my password?#

Yes. If you authorize your server's IP address from your proxy provider's panel, a username and password aren't needed; the address is written simply as http://sunucu:port. This is the cleanest method for servers with a fixed IP.

Can I get a different IP on every request with wget?#

wget itself doesn't do rotation. There are two paths: use a rotating proxy (a single address, a different exit IP on every connection), or give a different proxy to each call in a script with -e https_proxy=.

Can I download over FTP through a proxy?#

If the ftp_proxy variable is defined, wget fetches FTP addresses through the HTTP proxy; the proxy needs to support this usage. Since FTP sources have become less common today, most proxy packages focus on HTTP and HTTPS.

In short#

There are three ways to use wget with a proxy: environment variables for quick tests, the -e option for one-off use in scripts, and a .wgetrc file for continuous use. Don't forget to define https_proxy for HTTPS addresses, store credentials securely, limit the rate on bulk downloads, and switch to cURL when SOCKS is needed.