Let's say you want to do a deep crawl on some Twitter hashtag data using the Tweet Search Pagination Workflow Formula. Specifically, you want to perform a deep crawl for 20 very popular hashtags and want to process them all in parallel (run 20 workflow executions in parallel). We'll walk you through how to do this via the API using Python code.

1 - Get your Workflow Ready

You only need one workflow set up to perform multiple runs, so either import it from the formula you're interested in or set it up on your account. We'll use the Tweet Search Workflow in this example, so you can import it and follow along if you'd like.

2 - Find & Copy the Workflow ID

Navigate to the workflow you want to perform multiple executions for and take note of the ID in the URL, e.g. the ID will be in this form: https://stevesie.com/cloud/workflows/WORKFLOW_ID 

3 - Get the Endpoint's Variable Names

Go to the endpoint responsible for your workflow (in this case it's the Tweet Search Endpoint) and scroll down to the Inputs section. Take note of the variable names in red, courier text:

In our case, we want to perform executions of the workflow and override the query  and access_token  input variables on each run.

4 - Write Some Python

We now have everything we need to trigger our workflow executions. You'll also need your API Key which you can find from the account page.

You'll want to substitute the first 4 variables with your own definitions and change the execution_parameters  below to match your specific workflow's API endpoint.

import requests

API_TOKEN = 'XXX'
WORKFLOW_ID = 'XXX'

TWITTER_ACCESS_TOKEN = 'XXX'
TWITTER_HASHTAG_SEARCHES = [
    '#beer',
    '#ipa',
    ...
]

URL_BASE = 'https://stevesie.com/cloud/api/v1'

auth_headers = {
  'Token': API_TOKEN,
}

for query in TWITTER_HASHTAG_SEARCHES:
    requests.post(
        '{}/workflows/{}/executions'.format(URL_BASE, WORKFLOW_ID),
        json={
            'proxy': {
                'type': 'dedicated',  # change to shared if you want
                'location': 'nyc',
            },
            'execution_parameters': {
                'access_token': TWITTER_ACCESS_TOKEN,
                'query': query,
            },
            'send_completion_email': False,
            'execution_name': query,
        },
        headers=auth_headers,
    )

5 - Check Your Workflows

You'll now see your workflows running in your account, and each will do a deep crawl for that hashtag (paginating through all results), and collect the results in a different CSV file at the end.

Did this answer your question?