A Proxy Strategy for E-Commerce Teams

Most e-commerce teams that struggle with collection are not short of proxies. They are using one type for everything: usually whichever pool was set up first, applied to tasks with incompatible requirements.

The strategy is a mapping, task to address type, and it falls out of two questions: does the data differ by location, and does the request need a session?

The two questions

Does the data differ by location? A public product page on a global site returns the same HTML everywhere. A localised retailer returns different prices, currencies and availability per region. If location changes the data, your address is part of the measurement.

Does the request need a session? A plain page fetch does not. A cart, a logged-in view, a multi-step flow, or anything where state carried across requests does. If it does, the address must be stable for the life of that state.

Everything below follows from those two.

The allocation

Four properties of that table are worth stating explicitly.

Datacenter addresses are correct for bulk reads. They are cheap, fast and abundant, and there is no reason to spend residential bandwidth on data that does not depend on location. The comparison is in Datacenter vs Residential vs Mobile Proxies.

Residential is for correctness, not stealth. In localised-pricing collection, the residential address is what makes the measurement match the market. That framing matters, because it also tells you the address must be in the region being measured, not merely residential.

ISP addresses solve the session problem. They combine a stable address with a fast connection and a business-plausible range, which is what account and dashboard work needs. The case is in ISP Proxies for Account Management.

Verification traffic never uses an account's address. Ad and creative checks are read-only, they rotate, and keeping them off the account path prevents a research operation from being linked to a monetised account. This is the mistake described in Ad Verification with Proxies.

Cost-aware scheduling

The allocation decides which pool a task uses. The schedule decides how much of it you consume, and that is where the budget is won or lost.

Three scheduling rules:

Match cadence to volatility, not to habit. A category whose prices move monthly does not need daily collection. A fast-moving electronics category does. Cadence should come from observed change frequency, which you can measure from your own stored history.

Front-load cheap capacity. Use the datacenter pool for anything that does not need locality or a session, and reserve residential bandwidth for the requests where the address changes the answer.

Bound the coverage. Tracking a defined product set is cheaper than sweeping a catalogue, and usually more useful. Decide what the decision requires before increasing volume.

For most teams, the surprise is how much of the current spend is on residential bandwidth used for requests that were neither location-sensitive nor session-bound.

Bandwidth versus requests

Two providers bill differently, and the difference matters for this workload.

Per gigabyte. Common for residential. Favours HTML-light work and penalises browser rendering, because a rendered page is orders of magnitude heavier than an API response. The mechanics are in What Is Proxy Bandwidth.

Per IP or per request. Common for ISP and datacenter. Favours high-volume, light requests.

The practical consequence: if you render pages through a residential pool, most of the cost is image and asset payload you did not need. Finding the JSON endpoint instead of rendering is the single largest cost reduction available to most e-commerce collectors, and the method is in Scraping Browsers Explained.

Instrumenting the allocation

You cannot tune an allocation you do not measure. Log three things per request: the task, the pool used, and whether the response was validated.

That gives you, per task, success rate and cost. Two findings usually follow within a week.

A task on the wrong pool. Common examples are localised pricing running on datacenter addresses and returning the wrong region's data, or a session-dependent view on a rotating pool that fails intermittently for reasons that look random.

A task on the wrong cadence. A pool with a low success rate may simply be running too fast for what the target tolerates, which is the distinction in Rate Limiting vs Blocking.

Both diagnoses come from the per-task numbers rather than from aggregate usage, which is the same principle as Monitoring Scraper Health.

Starting point

If you have one pool in use, the migration order that produces results fastest:

1. Split the account and dashboard traffic onto static ISP addresses, one per account. This protects the highest-value access first.
2. Split the location-sensitive collection onto geo-matched residential, and verify the exit country per market.
3. Leave everything else on datacenter and confirm the success rate holds.
4. Measure per task, then adjust cadence before increasing pool size.

For the products, see Residential and Mobile, and for the solution context, E-Commerce.