Selenium Proxy Setup and Rotation

Selenium drives a real browser, so it inherits both the benefits and the awkward parts of browser automation. Proxy configuration is mostly straightforward, with one stubborn exception: authenticated proxies, which browsers do not support natively in the way HTTP clients do.

This covers the working approaches and the rotation pattern that keeps sessions coherent.

Chrome

Chrome takes proxy settings as launch arguments.

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument("--proxy-server=http://proxy.lightningbytes.com:1080")
options.add_argument("--headless=new")

driver = webdriver.Chrome(options=options)
driver.get("https://api.ipify.org")

The proxy applies to the whole browser, so every navigation goes through it. Note that the scheme in --proxy-server describes how you reach the proxy, not the destinations you can visit.

Firefox

Firefox uses a preferences dictionary, which is slightly more verbose.

from selenium import webdriver
from selenium.webdriver.firefox.options import Options

options = Options()
options.set_preference("network.proxy.type", 1)
options.set_preference("network.proxy.http", "proxy.lightningbytes.com")
options.set_preference("network.proxy.http_port", 1080)
options.set_preference("network.proxy.ssl", "proxy.lightningbytes.com")
options.set_preference("network.proxy.ssl_port", 1080)
options.set_preference("network.proxy.share_proxy_settings", True)

driver = webdriver.Firefox(options=options)

Setting both HTTP and SSL is the equivalent of configuring both keys in requests. Omitting the SSL settings is a common cause of HTTPS traffic bypassing the proxy, which we warn about in Using Python requests with Proxies.

The authentication problem

Neither browser shows a proxy login dialog you can automate reliably, and launch arguments do not carry credentials. Three approaches work.

IP allowlisting. Register your egress IPs with the provider so no credentials are needed. This is the cleanest option when your workers have stable addresses. The comparison is in Proxy Authentication.

A local relay. Run a small local proxy that adds the credentials and forwards to the real gateway, then point the browser at 127.0.0.1. This works everywhere and keeps credentials out of the browser configuration.

A browser extension. Some setups inject credentials via an extension loaded at launch. It works but adds a moving part to your automation.

For most teams, allowlisting where possible and a local relay otherwise is the pragmatic combination.

Rotating across driver instances

Rotation happens at the browser level, so the pattern is one driver per endpoint rather than changing the proxy mid-session.

def build_driver(proxy_url):
    options = Options()
    options.add_argument(f"--proxy-server={proxy_url}")
    options.add_argument("--headless=new")
    return webdriver.Chrome(options=options)

for endpoint in endpoints:
    driver = build_driver(endpoint)
    try:
        scrape(driver)
    finally:
        driver.quit()

Three notes on that shape:

Create a driver per endpoint, not one shared driver with a changing proxy. Chrome does not support changing the proxy at runtime.

Use a session value per endpoint so each driver gets a distinct exit IP and holds it. The mechanism is in Understanding Proxy Session IDs.

Close drivers properly. Leaked browser processes consume memory and can exhaust the machine long before your proxy limits do.

Why you probably should rotate per instance

Rotating mid-session contradicts what browser automation is for. A browser builds state: cookies, local storage, a session on the server. Changing the network identity partway through is exactly the inconsistency that triggers suspicion, as we explain in Rotating vs Sticky Proxies.

The clean mapping is one browser instance, one identity, one endpoint, for the duration of a unit of work.

Reducing the cost of browser automation

Browsers are heavy, so a few optimisations pay off quickly:

• Block unnecessary resources. Images, fonts and media usually dominate bandwidth and load time, and none of it is data you want. Request interception makes this trivial.
• Run headless unless you specifically need a visible browser.
• Reuse the driver across pages within a session rather than launching per page.
• Set page load strategies that do not wait for every subresource when you only need the DOM.

The bandwidth argument is in What Is Proxy Bandwidth.

Detecting a proxy that is not being used

A silent failure mode is the browser ignoring the configuration and connecting directly. The symptom is a request that succeeds but exits from your real address.

Always verify. Load a page that echoes the exit IP at the start of a session and compare. The proxy checker reports the exit IP and added latency for an endpoint, and the WebRTC leak test catches browser-level leaks that bypass the proxy entirely, which is a distinct failure from a misconfigured proxy.

Selenium versus the alternatives

Selenium is mature and has the widest language support. Playwright has better ergonomics for parallel contexts and auto-waiting, and Puppeteer is a natural fit in Node projects. The comparison is in Playwright vs Selenium, with the Playwright-specific proxy setup in Playwright Scraping with Proxies.

If you only need to fetch HTML and the page is not JavaScript-rendered, skip the browser entirely. Web Scraping with Python covers the lighter path.