Oxylabs Amazon Scraper

Use the amazon_sellers data source to retrieve information from Amazon Seller pages. You must provide a 13-character seller ID in the query parameter. To receive structured data, set the parse parameter to true (note: structured parsing is currently only supported for the desktop user_agent_type).

import requests
from pprint import pprint

# Structure payload.
payload = {
    'source': 'amazon_sellers',
    'domain': 'de',
    'query': 'ABNP0A7Y0QWBN',
    'parse': True
}

# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())

You can scrape product data from a specific Amazon department page using the scrape command. You must provide the department URL as a parameter and ensure the URL is enclosed in quotation marks to prevent parsing errors.

Command Syntax:

make scrape URL="<amazon_department_page_url>"

Example:

make scrape URL="https://www.amazon.com/s?i=specialty-aps&bbn=16225009011&rh=n%3A%2116225009011%2Cn%3A541966&ref=nav_em__nav_desktop_sa_intl_computers_and_accessories_0_2_5_6"

To use the free Amazon Scraper tool, ensure you have Python 3.11 installed on your system. Navigate to the repository directory in your terminal and run the following command to install dependencies:

make install

Use the requests library to send a POST request to the Oxylabs API. You must provide your credentials via HTTP Basic Auth and include a JSON payload containing the source, url, and parse parameters.

import requests
from pprint import pprint

# Structure payload.
payload = {
    'source': 'amazon',
    'url': 'https://www.amazon.co.uk/dp/B0BDJ279KF',
    'parse': True
}

# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('YOUR_USERNAME', 'YOUR_PASSWORD'), #Your credentials go here
    json=payload,
)

Use the amazon_pricing data source to retrieve Amazon product offer listings. You must provide a 10-symbol ASIN code in the query parameter. Setting parse to true will return structured data.

import requests
from pprint import pprint

# Structure payload.
payload = {
    'source': 'amazon_pricing',
    'domain': 'nl',
    'query': 'B09RX4KS1G',
    'parse': True,
}

# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())

Use the amazon_questions data source to retrieve a specific product's Questions & Answers pages. You must provide a 10-symbol ASIN code in the query parameter. Setting parse to true will return structured data.

import requests
from pprint import pprint

# Structure payload.
payload = {
    'source': 'amazon_questions',
    'domain': 'nl',
    'query': 'B09RX4KS1G',
    'parse': True,
}

# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

# Print prettified response to stdout.
pprint(response.json())

Use the amazon_bestsellers data source to retrieve Amazon Best Sellers pages. The query parameter expects a Department name (e.g., Clothing, Shoes & Jewelry). You can also use the context parameter with category_id to search for items in a specific browse node.

import requests
from pprint import pprint

# Structure payload.
payload = {
    'source': 'amazon_bestsellers',
    'domain': 'de',
    'query': 'automotive',
    'start_page': 2,
    'parse': True,
    'context': [
        {'key': 'category_id', 'value': 82400031},
    ],
}

# Get response.
response = requests.request(
    'POST',
    'https://realtime.oxylabs.io/v1/queries',
    auth=('user', 'pass1'),
    json=payload,
)

After running the scraper, the tool generates a file named amazon_products.csv in your current directory. The CSV contains the following columns:

• title: The title of the product.
• url: The URL pointing to the product's Amazon page.
• asin_code: The product's unique ASIN code.
• image_url: The URL of the product image.
• price: The price of the product (this field may be empty if the product is out of stock).

The following parameters are available for the amazon_pricing data source:

The following parameters are available for the amazon_bestsellers data source: