AgentQL

website·Indexed 34 days ago

https://www.agentql.com/blog/

An AI-powered web data extraction and automation platform that enables natural language query generation to treat every web page as an endpoint. It features a JavaScript SDK, a REST API, and a Playwright Smart Locator for simplified web automation. AgentQL provides capabilities for structured data extraction from PDFs and images, bot detection evasion via Stealth Mode, and integrations with MCP (for Claude, Cursor, and Windsurf), Zapier, Dify, and AgentStack.

Tokens
362K
Snippets
3.6K
Records
5.3K
Agent score
66%

What's inside AgentQL

  1. Overview of AgentQL JavaScript SDK

    AgentQL is an AI-powered query language and parser designed to extract data from web pages and automate browser interactions. Unlike traditional CSS selectors or XPath, AgentQL uses natural language and AI to locate elements based on descriptions and relationships, making queries 'self-healing' and resilient to changes in page structure or layout. It is designed for Node.js environments and integrates with Playwright to handle dynamic content, client-side JavaScript, and user interactions (such as clicks and logins).
  2. Overview of AgentQL capabilities

    AgentQL is a suite of tools for connecting AI agents and LLMs to the web. It provides a semantic query language and parser that allows developers to extract structured data from any web page (public or private, including those behind authentication) without relying on fragile XPath or CSS selectors. Key features include self-healing queries that persist despite dynamic content changes and the ability to use the same query across multiple similar pages.
  3. Overview of AgentQL MCP Server

    The AgentQL MCP Server implements the Model Context Protocol (MCP), allowing AI assistants (such as Claude, Cursor, and Windsurf) to extract real-time, structured data from unstructured HTML web pages. It eliminates the need for custom scraping scripts, external APIs, or vector storage by transforming live web content into usable data formats.
  4. Overview of AgentQL Semantic Selectors

    AgentQL provides Python and Node.js SDKs that use semantic selectors and natural language queries. This approach ensures deterministic results for web automation and E2E testing without relying on fragile XPath or DOM selectors.
  5. Extract company information from About Amazon using AgentQL

    AgentQL allows you to query website data using natural language descriptors instead of fragile XPath or DOM selectors. For aboutamazon.com, you can define a query object to extract specific company details such as name, address, and contact information.
    // Query
    {
      company_information {
        name
        address
        contact_info {
          phone_number
          email
        }
      }
    }
    
    // Expected Return
    {
      "company_information": {
        "name": "Amazon",
        "address": "410 Terry Avenue North, Seattle, WA 98109-5210",
        "contact_info": {
          "phone_number": "(206) 266-1000",
          "email": "NA"
        }
      }
    }
  6. Extract data from Citizens' Stamp Advisory Committee using AgentQL

    AgentQL allows you to query websites using natural language descriptions instead of fragile XPath or DOM selectors. For the Citizens' Stamp Advisory Committee page (https://about.usps.com/who/csac/), you can define the data you want to extract by describing it in a JSON-like query structure.
    {
      mission_statement(A brief description of the USPS's core mission.)
    }
  7. Evade bot detection with AgentQL Stealth Mode

    AgentQL includes a Stealth Mode specifically designed to reduce the risk of detection by bot detection systems, allowing for more reliable data capture and collection from web pages.
  8. Query Facebook and other websites with AgentQL

    AgentQL allows developers to query data from websites like Facebook using natural language descriptions instead of fragile XPath or DOM selectors. This ensures that data extraction remains consistent even when the website's UI changes.
  9. Query QuickBooks data using AgentQL

    AgentQL allows you to extract data from QuickBooks (and other websites) using natural language queries instead of fragile XPath or DOM selectors. This ensures that data extraction remains consistent even when the website's UI changes.
    // Example Query
    {
      products[] {
        name
        price
      }
    }
    
    // Example Return
    {
      "products": [
        {
          "name": "QuickBooks Online",
          "price": "$30/month"
        }
      ]
    }
  10. Query Matomo and other websites using AgentQL

    AgentQL allows developers to extract data from websites like Matomo using natural language descriptions (smart selectors) instead of fragile XPath or DOM selectors. This ensures that data extraction remains consistent even when the website's UI changes.
    // Example Query
    {
      website_name
      tagline
      features[]
    }
    
    // Example Result
    {
      "website_name": "Matomo",
      "tagline": "The Open Source Web Analytics Platform",
      "features": [
        "Real-time analytics",
        "Custom dashboards",
        "Data privacy"
      ]
    }
  11. Extract data from Congressional Research Service using AgentQL

    AgentQL allows you to query websites like the Congressional Research Service (https://loc.gov/crsinfo/about/) using natural language descriptions instead of fragile XPath or DOM selectors. You define the data you want in a JSON-like structure with descriptions, and AgentQL returns the matching data from the page.
    // Query
    {
      mission_statement(A brief description of CRS's mission.)
      contact_information {
        email
        phone_number
        mailing_address
      }
    }
    
    // Expected Return
    {
      "mission_statement": "To provide timely, nonpartisan, and objective information and analysis to support the deliberations of the U.S. Congress.",
      "contact_information": {
        "email": "crsinfo@crs.loc.gov",
        "phone_number": "(202) 707-7000",
        "mailing_address": "Library of Congress, Washington, DC 20540"
      }
    }
  12. Query EOIR and other websites using AgentQL natural language selectors

    AgentQL allows developers to extract data from websites like the Executive Office for Immigration Review (EOIR) using natural language descriptions instead of XPath or DOM selectors. This ensures that data extraction remains consistent even if the website's UI changes.
    // Example AgentQL query to extract page title and sections
    {
      title(Main title of the page)
      sections[] {
        heading
        content
      }
    }