How to Use a Proxy with Python Requests

A python requests proxy setup is the fix when your script receives a 403 error, captcha, or geo-restriction. A page that loads for a user in Germany can fail when your code sends traffic from a datacenter IP. The mechanism is a proxies dictionary passed to Python requests, with both http and https keys pointing to the same proxy server. The one-line change is adding proxies= to the request call. This routes the connection through the chosen endpoint and changes the exit IP.

This guide explains how to use proxy with python requests through HTTP or SOCKS5 URLs without leaking your real address or breaking authentication. Verification and failure handling determine reliability.

Quick Answer: Set a Proxy in Python Requests

The fastest way to set proxy in Python requests is a python requests proxy dictionary passed to requests.get. It maps each URL scheme to one endpoint. These python requests proxies should include both http and https keys. The https key usually still contains an http:// address. It describes how to reach the intermediary, not the destination scheme.

import requests

proxies = {
    "http": "http://203.0.113.10:8080",
    "https": "http://203.0.113.10:8080",
}
response = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=10)
print(response.json())

{
  "origin": "203.0.113.10"
}

Replace 203.0.113.10:8080 with a working HTTP proxy before running. The timeout argument prevents a dead route from hanging the script. Call httpbin.org/ip or api.ipify.org first without the dictionary. Then call again with it. If the origin value changes, the request left through the new address. An unchanged result means the direct connection won.

Always include the scheme in each URL. The sample output shows only the exit address. A direct run prints your own public IP. A proxied call prints the route’s address. This comparison proves whether the mapping worked. For HTTPS targets, the same HTTP proxy usually opens a CONNECT tunnel. TLS then happens inside it.

Prerequisites and Installing the Requests Library

Before you add a requests library proxy, confirm Python 3 and pip are available in your terminal. Use any editor that can run scripts. Requests is not bundled with Python, so install it inside a venv. This keeps test packages away from the system interpreter.

python -m venv .venv
# Linux/macOS: source .venv/bin/activate
# Windows: .venv\Scripts\activate

python -m pip install requests
python -m pip show requests

Verify the package metadata command output to confirm the installation. Then make one baseline request to api.ipify.org without proxy settings. This endpoint returns only the caller address. Print your public IP and save the value. For example, a developer keeps this address in a test note or comment. Use the saved value as the control for every later check. Later, compare it with the exit address produced through a proxy. If both match, traffic did not route through it.

import requests

resp = requests.get("https://api.ipify.org", timeout=10)
print(resp.text)

Python Requests Proxies Dictionary: Step-by-Step Setup

The python requests proxies dictionary maps scheme keys to proxy URLs, and the library matches them literally. Define http and https because omitting the second one lets encrypted targets go direct. Follow Step 1, Step 2, and Step 3. Pass this argument to requests.get with timeout.

Step 1: Create the proxies dictionary

Create the proxies dict with an http key and an https key. For python requests proxies, both values normally point to an HTTP endpoint at host:port, even for HTTPS pages.

proxies = {
    "http": "http://203.0.113.10:8080",  # placeholder
    "https": "http://203.0.113.10:8080",
}

The https value uses http:// because it describes how to reach the proxy, not the target scheme. Keys can also use scheme://hostname to route one site separately. For SOCKS, use socks5h://. The two entries can be identical, but they may also point to different addresses if your network requires separate routes. Replace the placeholder value with your proxy.

Step 2: Send the request with proxies and a timeout

In a python requests proxy setup, pass the dictionary to the request method and always add timeout. Without it, a dead endpoint can hang the script indefinitely. The tuple form gives separate limits: the first is the connect timeout, the second is the read timeout.

import requests

proxies = {
    "http": "http://203.0.113.10:8080",
    "https": "http://203.0.113.10:8080",
}

response = requests.get(
    "https://api.ipify.org",
    proxies=proxies,
    timeout=(5, 15),
)

response = requests.post(
    "https://httpbin.org/post",
    json={"page": 1},
    proxies=proxies,
    timeout=(5, 15),
)

Use the same pattern for requests.post. For example, post JSON to an API through the proxy with timeout=(5, 15). The same arguments also apply to put, delete, head, and patch.

Step 3: Verify that the proxy is really used

Call an IP-echo endpoint and compare the returned address with your baseline public IP. If they match, the proxy was skipped. Test both https://api.ipify.org and http://httpbin.org/ip, because a missing http or https key only fails for one scheme. Also confirm the response status is 200-299 before trusting the result.

def check_ip(proxies, url="https://api.ipify.org"):
    response = requests.get(url, proxies=proxies, timeout=(5, 15))
    response.raise_for_status()
    return response.text.strip()

print(check_ip(proxies))
print(check_ip(proxies, "http://httpbin.org/ip"))

203.0.113.10
{
  "origin": "203.0.113.10"
}

The origin value is the exit IP seen by the remote service. Use this check at the start of every script run. Repeat after any credential change.

Python Requests Proxy Authentication (Username and Password)

For python requests proxy authentication, put the username and password into the proxy URL: http://username:password@host:port. If a developer receives credentials by email, add that string to the proxies dict without extra setup. Wrong values usually return 407 Proxy Authentication Required.

Put credentials in the proxy URL

This format tells the library which route to contact and which login to send. A working python requests proxies dictionary uses that same pattern.

# URL template
http://username:password@proxy-host:port

# Working dictionary
proxies = {
    "http": "http://user:pass@203.0.113.10:8080",
    "https": "http://user:pass@203.0.113.10:8080",
}

Use the same authenticated URL for both keys when one endpoint handles HTTP and HTTPS traffic.

Do not hardcode the login. In a CI job, read PROXY_USER and PROXY_PASS from stored secrets with os.environ, then build the URL with an f-string. The script injects the values only at runtime.

import os

user = os.environ["PROXY_USER"]
password = os.environ["PROXY_PASS"]
proxy_url = f"http://{user}:{password}@203.0.113.10:8080"
proxies = {"http": proxy_url, "https": proxy_url}

The same URL pattern also works for SOCKS5 after installing requests[socks].

URL-encode special characters in the password

Special characters such as @, :, /, #, ?, and % can break python requests proxy authentication. The parser treats them as URL syntax. The request can then fail with a 407 or a connection error. It can also raise InvalidURL before any request leaves your machine.

Encode the password with urllib.parse.quote(password, safe=""). Pass safe="" so no character is left unencoded. This percent-encoding makes the password safe inside the URL. A literal percent sign becomes %25. Insert only the encoded string into the proxy URL. The example below shows a broken URL and the encoded form. Do not encode the separator between user and host. Use URL encoding only for the credential fields.

from urllib.parse import quote

password = "p@ss:word%1"

# Broken: raw @, :, and % corrupt the proxy URL
# "http://user:p@ss:word%1@203.0.113.10:8080"

encoded_password = quote(password, safe="")
# "p%40ss%3Aword%251"

proxy_url = f"http://user:{encoded_password}@203.0.113.10:8080"

If authentication still fails, check four common causes:

• Wrong scheme for the proxy type.
• Unencoded special characters in the username or password.
• Wrong username or password.
• Wrong host or port.

Set a Proxy with Environment Variables (HTTP_PROXY and HTTPS_PROXY)

To set proxy in python requests without touching code, rely on the environment. The requests library proxy configuration reads HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, and NO_PROXY automatically. Each value must be a URL. Uppercase and lowercase names both work. If both exist and disagree, lowercase wins.

# Linux/macOS
export HTTPS_PROXY="http://203.0.113.10:8080"
export NO_PROXY="localhost,127.0.0.1"

# Windows cmd
set HTTPS_PROXY=http://203.0.113.10:8080
set NO_PROXY=localhost,127.0.0.1

# Windows PowerShell
$env:HTTPS_PROXY="http://203.0.113.10:8080"
$env:NO_PROXY="localhost,127.0.0.1"

Keep credentials out of committed shell history by loading them from secrets at runtime. This suits containers and CI, where one image can route different jobs through different exit points. Use NO_PROXY for localhost, internal APIs, health checks, or package indexes. Excluded hosts bypass the configured route entirely.

If a Docker container already defines HTTPS_PROXY, bypass one call with proxies={'http': None, 'https': None}. To ignore all environment settings, set session.trust_env = False before sending.

Confirm the active route with an IP-echo endpoint. If the returned address is unchanged, a variable, session default, or missing key is still interfering.

Use a Proxy with requests.Session

Use requests.Session when one script makes many calls through the same proxy. The object stores the routing config once. It also keeps cookies, default headers, and open TCP connections. That avoids a fresh TLS handshake on repeated calls. This matters for dashboards, APIs, and multi page forms.

import requests

proxies = {
    "http": "http://203.0.113.10:8080",
    "https": "http://203.0.113.10:8080",
}

session = requests.Session()
session.proxies.update(proxies)

r1 = session.get("https://api.ipify.org", timeout=(5, 15))
r2 = session.get("https://httpbin.org/ip", timeout=(5, 15))
print(r1.text)
print(r2.json())

For a login flow, the same object keeps authentication state. This example posts credentials once. Later pages are fetched without logging in again. If a dashboard sets a session cookie, three report pages can reuse the same exit IP. That is how a single login survives subsequent GETs without sending credentials again.

with requests.Session() as session:
    session.proxies.update(proxies)
    session.post(
        "https://example.com/login",
        data={"user": "bot", "password": "example"},
        timeout=(5, 15),
    )

    dashboard = session.get("https://example.com/dashboard", timeout=(5, 15))
    report = session.get("https://example.com/report", timeout=(5, 15))
    settings = session.get("https://example.com/settings", timeout=(5, 15))

A Session remembers:

• cookies set by earlier responses
• default headers and authentication
• connection pooling by host
• one shared proxy setting

Environment variables can override session.proxies. This surprises scripts that set a python requests proxy explicitly. HTTP_PROXY and HTTPS_PROXY are read by default. Set session.trust_env = False to ignore them. For a requests library proxy setup, that also ignores .netrc credentials. An explicit proxies argument still wins over the stored mapping. It also makes failures easier to reproduce because all calls share the same routing. Close unused sessions or use a context manager to release sockets. A with block makes cleanup happen even after an exception. Skipping this cleanup can leave sockets open longer than necessary.

Use SOCKS5 Proxies with Python Requests

Python requests supports HTTP and HTTPS proxies out of the box. SOCKS needs an extra package. Install requests[socks], which brings in PySocks:

python -m pip install "requests[socks]"

If the package is missing, requests raises an error when it sees a SOCKS URL.

For SOCKS, your python requests proxies dictionary uses socks5h:// URLs. Include credentials in the URL when the endpoint uses SOCKS5 authentication:

import requests

proxies = {
    "http": "socks5h://user:pass@host:1080",
    "https": "socks5h://user:pass@host:1080",
}

response = requests.get("https://httpbin.org/ip", proxies=proxies, timeout=(5, 15))
print(response.json())

The same endpoint can serve both keys because the scheme describes the SOCKS transport, not the target page. Keep the timeout argument, because a dead SOCKS endpoint can hang.

Use socks5h:// when the script must not resolve the target hostname on the local network. The h form sends the hostname to the SOCKS endpoint. DNS resolution happens on the proxy side. Plain socks5:// resolves the name locally first. That can leak the lookup to your local resolver. It can also expose the destination before the tunnel is used.

Example: a crawler checks a private API and must not leave a local DNS trace. It uses socks5h://. It then calls httpbin.org/ip and confirms the returned address is not the machine address. That test proves the python requests proxy carries both the lookup and the connection.

Rotate Proxies in Python Requests

A single source is easy to profile. If a price-monitoring script makes 500 calls per hour from one address, the target may return 429 rate limit responses or impose an IP ban. Use proxy rotation to spread python requests proxies across multiple exits.

Pick a random proxy with retries

A proxy pool stores several proxy dictionaries. This trades predictable routing for resilience: one bad endpoint no longer stops the job. For a python requests proxy rotation loop, keep the list in config or environment variables, and let random.choice pick one for each call.

import random, time, requests

def fetch_with_rotation(url, proxy_list, attempts=3, timeout=None):
    for attempt in range(attempts):
        proxy = random.choice(proxy_list)
        try:
            return requests.get(url, proxies=proxy, timeout=timeout)
        except requests.exceptions.RequestException:
            print("Proxy failed; retrying")
            time.sleep(attempt)
    return None

In a pool of three endpoints, the first item may be dead. The retry loop catches RequestException, waits briefly, and succeeds on the second attempt. A single timeout does not prove a permanent failure, so a small backoff helps.

The flip side is complexity. Random retries can hit the same broken endpoint again. To reduce that, use a cooldown for failed proxies, or pick two random candidates and use the less-used one. This power of two choices idea avoids scanning the whole pool. Free proxies die within hours, so use them only to test the code.

• Track use counts and prefer the less-used candidate.
• Put failed entries into cooldown instead of retrying them immediately.
• Create a per-proxy Session when each endpoint needs its own cookies.

Sticky sessions vs rotating IPs: choose by task

The core trade-off is session consistency versus IP spread.

Sticky session

Rotating IP

Example: a scraper logs in once, keeps one IP through checkout, then changes proxy before the next job.

Rotation is simpler when the endpoint supports IP rotation via API and can keep or change the IP on demand. OnlineProxy mobile proxies use real 4G/5G carrier IPs through modems and SIM routers, with username and password authentication in the proxy URL. With a dedicated device, the IP can stay sticky, change by link, or change by timer.

Python Requests Proxy Not Working: Troubleshooting

A Python requests proxy failure usually has three causes. The call bypasses the proxy, the endpoint is unreachable, or a status such as 407 is returned. Server environment variables can override local settings and trigger ProxyError or Max retries exceeded.

Requests still shows your real IP

A common mistake is setting only the HTTP route because the first test page works. The script then leaks your IP on every HTTPS call. That produces false geo checks, repeated login challenges, failed paid jobs, and avoidable blocks.

1. Missing HTTPS key: requests matches keys literally. Secure URLs bypass the proxy. Fix: the same address under both keys.
2. Environment variable override: exported variables can beat saved settings. Fix: per-request proxies argument or session.trust_env False.
3. Wrong scheme: socks5 resolves names locally and can leak DNS. Fix: socks5h://.

proxies = {'http': 'http://proxy-host:port'}
print(requests.get('http://httpbin.org/ip', proxies=proxies).text)
print(requests.get('https://api.ipify.org', proxies=proxies).text)
print(requests.get('https://api.ipify.org').text)

If both match, the IP leak means python requests proxies are not applied.

ProxyError and 'Max retries exceeded'

A port typo often produces Connection refused or Errno 61. Nothing is listening at that endpoint. A wrong password often produces Tunnel connection failed: 407. The endpoint is reachable, but it rejects authentication. ProxyError wraps these tunnel and connection failures. ConnectTimeout means the connection phase exceeded its time budget. ReadTimeout means the response stalled after a connection existed. For an HTTPS target, the intermediary can reject the CONNECT tunnel before any HTTP response exists. If the host or port is uncertain, test it with a free online proxy checker. All network errors subclass RequestException. Catch it last.

requests.exceptions.ProxyError: HTTPSConnectionPool(host='api.example.com', port=443):
Max retries exceeded with url: /ip
Caused by: Connection refused (Errno 61)

try:
    response = requests.get(url, proxies=proxies, timeout=(5, 15))
except requests.exceptions.ConnectTimeout:
    print("Connect timeout")
except requests.exceptions.ReadTimeout:
    print("Read timeout")
except requests.exceptions.ProxyError:
    print("Proxy or tunnel error")
except requests.exceptions.RequestException:
    print("Network error")

Proxy status codes and what they mean

A status code tells you which layer rejected the call. A developer who sees 401 may change proxy credentials, but the target site wants its own login. For 407, fix python requests proxy authentication.

SSLError and verify=False

SSLError usually means requests cannot validate the certificate chain. A TLS-intercepting proxy replaces the site certificate with its own, moving the trust boundary to the proxy operator.

For corporate TLS interception, install the corporate root certificate and point verify at the CA bundle instead:

requests.get(url, proxies=proxies, verify="/path/to/corp-ca.pem")

For noisy tests, suppress urllib3's InsecureRequestWarning only temporarily.

Which Proxy to Use with Python Requests: Free vs Paid

A python requests proxy setting passes traffic through an endpoint you choose. Free options suit quick checks, not production. If a developer tests a rotation loop on a public list, most entries may time out. The code may be correct while the addresses are dead. Unknown routes may log or alter traffic. Never send real credentials or sensitive data except through a trusted proxy.

Choose by task. The Balancing Security and Privacy study (2025) notes that datacenter addresses are easier to detect. Residential sources can be harder to detect, but consent matters. Mobile carrier IPs usually look like ordinary mobile traffic.

FAQ: Python Requests Proxy Questions

How do I set proxy in Python requests for one call?

Pass the proxies argument to that call. To bypass routing for one internal API, map both schemes to null. The rest of the script can keep its normal proxy. Set NO_PROXY for excluded host suffixes. A null proxies=None value does not bypass environment variables.

requests.get(url, proxies={'http': None, 'https': None})

Why is my proxy still showing my real IP?

Usually the target URL scheme has no matching key. If only the HTTP key is set, HTTPS requests may go direct. Environment variables can also override a session. Confirm the exit IP with an IP-echo URL, then test both HTTP and HTTPS endpoints.

How does Python requests proxy authentication work?

Place the username and password in the proxy URL. That creates an authenticated proxy. Load credentials from environment variables instead of hardcoding them. If the password contains reserved characters, URL-encode it first. Wrong encoding often produces a 407 response.

Can I use SOCKS5 with requests?

Yes, install the SOCKS extra. Then use the socks5h scheme in the same dictionary. The h form resolves DNS at the proxy, which avoids local DNS leaks. Plain SOCKS5 resolves names on your machine before connecting.

What happens if I set only the http key?

Only HTTP URLs are routed through the proxy. HTTPS URLs can bypass it, depending on environment settings. That causes a real-IP leak. Set both keys to the same proxy endpoint unless you intentionally need different routing rules.

Does the requests library proxy from environment variables by default?

Yes. It reads HTTP_PROXY, HTTPS_PROXY, ALL_PROXY, and their lowercase forms when trust_env is true. It does not automatically read every OS system setting. To ignore them for a session, disable environment trust. Explicit per-request settings still override those variables.

Conclusion

Before you call a python requests proxy setup done, check these four practical points.

• Set both keys. The proxies dict needs both http and https entries.
• Encode credentials. Percent-encode special characters in usernames and passwords inside the proxy URL.
• Apply precedence. Per-request proxies beat environment variables, which beat session.proxies.
• Confirm IP verification. Call an IP-echo endpoint and compare the returned IP with your baseline.

Add a timeout and retry logic so one dead route does not hang the whole script. Test your setup.

import requests
proxy = "http://user:pass@host:port"
proxies = {"http": proxy, "https": proxy}
response = requests.get("https://api.ipify.org", proxies=proxies, timeout=10)
print(response.text)