Oxylabs AI Studio Python SDK

Use AiCrawler.crawl to navigate a website based on a natural language prompt. This is useful for finding specific types of pages or content across a domain.

Parameters:

• url (str): Starting URL (required)
• user_prompt (str): Natural language prompt to guide extraction (required)
• output_format (Literal["json", "markdown", "csv", "toon"]): Output format (default: "markdown")
• schema (dict | None): JSON schema for structured extraction (required if output_format is "json", "csv" or "toon")
• render_javascript (bool): Render JavaScript (default: False)
• return_sources_limit (int): Max number of sources to return (default: 25)
• geo_location (str): Proxy location in ISO2 format or country canonical name.
• max_credits (int | None): Maximum credits to use.

from oxylabs_ai_studio.apps.ai_crawler import AiCrawler

crawler = AiCrawler(api_key="<API_KEY>")

url = "https://oxylabs.io"
result = crawler.crawl(
    url=url,
    user_prompt="Find all pages with proxy products pricing",
    output_format="markdown",
    render_javascript=False,
    return_sources_limit=3,
    geo_location="United States",
)
print("Results:")
for item in result.data:
    print(item, "\n")

Use AiMap.map to discover and map the structure of a website or domain using keywords and natural language prompts.

Parameters:

• url (str): Starting URL or domain to map (required)
• search_keywords (list[str]): Keywords for URL path filtering.
• user_prompt (str | None): Natural language prompt for keyword search.
• max_crawl_depth (int): Max crawl depth (1..5, default: 1)
• limit (int): Max number of URLs to return (default: 25)
• geo_location (str): Proxy location in ISO2 format or country canonical name.
• render_javascript (bool): JavaScript rendering (default: False)
• include_sitemap (bool): Whether to include sitemap as seed (default: True)
• max_credits (int | None): Maximum credits to use.
• allow_subdomains (bool): Include subdomains (default: False)
• allow_external_domains (bool): Include external domains (default: False)

from oxylabs_ai_studio.apps.ai_map import AiMap


ai_map = AiMap(api_key="<API_KEY>")
payload = {
    "url": "https://career.oxylabs.io",
    "search_keywords": ["career", "jobs", "vacancy"],
    "user_prompt": "job ad pages",
    "max_crawl_depth": 2,
    "limit": 10,
    "geo_location": "Germany",
    "render_javascript": False,
    "include_sitemap": True,
    "max_credits": None,
    "allow_subdomains": False,
    "allow_external_domains": False,
}
result = ai_map.map(**payload)
print(result.data)

Use AiScraper.scrape to extract specific data from a single target URL. You can use generate_schema to automatically create a JSON schema based on a prompt.

Parameters:

• url (str): Target URL to scrape (required)
• output_format (Literal["json", "markdown", "csv", "screenshot", "toon"]): Output format (default: "markdown")
• schema (dict | None): JSON schema for structured extraction (required if output_format is "json", "csv" or "toon")
• render_javascript (bool | string): Render JavaScript. Can be set to "auto" to detect if rendering is needed (default: False)
• geo_location (str): Proxy location in ISO2 format or country canonical name.
• user_agent (str): User-Agent request header.
• optimize_content (bool): Return cleaner markdown by focusing on main content. Reduces output size (default: True)
• browser_instructions (list[BrowserInstruction] | None): Browser actions (click, input, wait, etc.) to run before capture. Requires render_javascript=True.

from oxylabs_ai_studio.apps.ai_scraper import AiScraper

scraper = AiScraper(api_key="<API_KEY>")

schema = scraper.generate_schema(prompt="want to parse developer, platform, type, price game title, genre (array) and description")

url = "https://sandbox.oxylabs.io/products/3"
result = scraper.scrape(
    url=url,
    output_format="json",
    schema=schema,
    render_javascript=False,
    optimize_content=True,
)
print(result)

The AiScraper is designed to scrape website content and return it as Markdown or structured JSON. If you request JSON, you must provide a valid JSON schema.

Parameters

• url (str, required): Target URL to scrape.
• output_format (Literal["json", "markdown", "csv", "screenshot"], default: "markdown"): The desired output format.
• schema (dict | None, required if output_format is "json"): OpenAPI schema for structured extraction.
• render_javascript (bool, default: False): Whether to render JavaScript on the page.
• geo_location (str): Proxy location in ISO2 format.

Output Structure

The result is an AiScraperJob containing:

• run_id: Unique identifier for the job.
• message: Status message (optional).
• data: The extracted content. The type depends on output_format:
  • "json": returns a dict.
  • "markdown": returns a str.
  • "csv": returns a str formatted as CSV.
  • "screenshot": returns a str.

from oxylabs_ai_studio.apps.ai_scraper import AiScraper

scraper = AiScraper(api_key="<API_KEY>")

url = "https://sandbox.oxylabs.io/products/3"
result = scraper.scrape(
    url=url,
    output_format="json",
    schema={"type": "object", "properties": {"price": {"type": "string"}}, "required": []},
    render_javascript=False,
)
print(result)

Use AiSearch.search to perform web searches.

Note: If limit <= 10 and return_content=False, the SDK automatically uses the instant_search endpoint for faster response times without polling.

Parameters:

• query (str): What to search for (required)
• limit (int): Max results to return (default: 10, max: 50)
• render_javascript (bool): Render JavaScript (default: False)
• return_content (bool): Whether to return markdown contents in results (default: True)
• geo_location (str): ISO 2-letter format, country name, or coordinate formats.

from oxylabs_ai_studio.apps.ai_search import AiSearch

search = AiSearch(api_key="<API_KEY>")

query = "lasagna recipe"
result = search.search(
    query=query,
    limit=5,
    render_javascript=False,
    return_content=True,
)
print(result.data)

# For fast search (instant endpoint)
result = search.instant_search(
    query=query,
    limit=10,
)
print(result.data)

The BrowserAgent is a browser automation tool that controls a browser to perform actions like clicking, scrolling, and navigation based on a textual prompt. It is ideal for complex interactions where a simple scrape is insufficient.

Parameters

• url (str, required): Target URL to scrape.
• user_prompt (str, required): Textual instructions for the agent (e.g., "Click the login button"). Focus on actions rather than extraction goals.
• output_format (Literal["json", "markdown"], default: "markdown"): The format of the result.
• schema (dict | None, required if output_format is "json"): An OpenAPI schema for structured extraction.

Output Structure

The result is a BrowserAgentJob containing a DataModel. The DataModel includes:

• type: One of "json", "markdown", "html", "screenshot", or "csv".
• content: The extracted data (dict, str, or None).

from oxylabs_ai_studio.apps.browser_agent import BrowserAgent

browser_agent = BrowserAgent(api_key="<API_KEY>")

prompt = "Find if there is game 'super mario odyssey' in the store."
url = "https://sandbox.oxylabs.io/"
result = browser_agent.run(
    url=url,
    user_prompt=prompt,
    output_format="json",
    schema={"type": "object", "properties": {"page_url": {"type": "string"}}, "required": []},
)
print(result.data)

Use BrowserAgent.run to perform complex browser-based tasks. The agent can interact with the page (e.g., using search bars) to find information based on a natural language prompt.

Parameters:

• url (str): Starting URL to browse (required)
• user_prompt (str): Natural language prompt for extraction (required)
• output_format (Literal["json", "markdown", "html", "screenshot", "csv", "toon"]): Output format (default: "markdown")
• schema (dict | None): JSON schema for structured extraction (required if output_format is "json", "csv" or "toon")
• geo_location (str): Proxy location in ISO2 format or country canonical name (e.g., 'Germany').

from oxylabs_ai_studio.apps.browser_agent import BrowserAgent

browser_agent = BrowserAgent(api_key="<API_KEY>")

schema = browser_agent.generate_schema(
    prompt="game name, platform, review stars and price"
)

prompt = "Find if there is game 'super mario odyssey' in the store. If there is, find the price. Use search bar to find the game."
url = "https://sandbox.oxylabs.io/"
result = browser_agent.run(
    url=url,
    user_prompt=prompt,
    output_format="json",
    schema=schema,
)
print(result.data)