If you know how many requests you need to make in order to collect the data you're after, you can easily estimate the cost. We'll use 20,000 requests in this example and assume you want to run them all in a single workflow execution.

Shared Proxy

If using a shared proxy, pricing is straightforward and you would pay $0.0010 per request, or 20,000 x $0.0010 = $20.00 in usage credits for the workflow execution. This is a flat fee independent of how long it takes (e.g. if you're rate limited, the price will remain constant).

Dedicated Proxy

You'll typically find dedicated proxies to be more cost effective when working with large workflows, as they are charged by the hour and meant for performing a large number of requests in a small time window.

Dedicated proxies are priced at $0.0750 usage credits per hour, allowing requests to be made at no charge while the dedicated proxy is on. You only pay for each hour the proxy is left on, and not by the request made.

Assuming that you'll be able to make more than 75 requests per hour (e.g. you are not severely rate limited), then dedicated proxies will be more cost effective.

Going back to our example of making 20,000 requests, let's assume the throughput works out to 1 request every 2 seconds (or 0.5 requests per second), then it would take 40,000 seconds (or 11.11 hours) to complete.

This rounds up to 12 hours or 12 x $0.0750 = $0.90 in usage credits. We can see this makes a huge difference and is about 20x cheaper than using the shared proxy!

Estimating Price from Throughput

Once we know the total number of requests and the throughput, we can estimate the price as follows:

PRICE = ( REQUEST_COUNT / THROUGHPUT ) x 0.0000208333333333334

Where throughput is the number of requests made per second, or 0.5 in this example. Depending on your specific workflow though, throughput may be much lower if you are waiting between each request a few seconds and if a lot of data is returned (there's added network time).

We typically see throughput range from 0.1 to 0.5, so you may want to use 0.25 to be conservative in your estimate.

Custom Proxy

Lastly, you can run your workflow with a third-party proxy at a 50% discount of the shared proxy price, meaning you only pay $0.0005 usage credits per request. This means it would only cost $10 in usage credits to process the 20,000 requests using a custom proxy.

Did this answer your question?