website·Indexed 34 days ago
https://docs.agentql.comAn AI-powered query language and parser that uses natural language selectors for web scraping and automation. It provides Python and JavaScript SDKs, a Chrome Debugger Extension, and a REST API to extract structured data and interact with web elements. Features include integration with Playwright, remote browser allocation, bot detection avoidance, and compatibility with Google's Agent Development Kit (ADK).
The AgentQL JavaScript SDK provides a set of modules and classes to automate web interactions and extract data at scale. Key components include:
agentql module: Used for global configurations (such as setting the API key) and converting standard Playwright Page objects into AgentQL Page objects.agentql/tools module: Provides utilities for data extraction, web automation, and remote browser session management.AgentQL Page: A wrapper around the Playwright Page class that enables the use of AgentQL's querying API.AQLResponseProxy: A metadata structure returned by the queryElements() method on an AgentQL Page, allowing developers to access web elements using intuitive dot notation rather than raw data.The AgentQL Python SDK provides a suite of tools to automate web interactions and extract data. Key components include:
agentql module: Contains utility methods to convert a standard Playwright Page into an AgentQL Page.AgentQL Page: A wrapper around Playwright's Page that enables the use of AgentQL's querying API.agentql.tools module: Provides utility methods specifically for web automation and data extraction.PaginationInfo: A class used to manage pagination and navigate to subsequent pages.AgentQL is an AI-powered toolset for retrieving web elements and data using natural language queries instead of traditional DOM, XPath, or CSS selectors. It consists of three main components:
To deploy an AgentQL script, build the Docker image, push it to a registry (e.g., Docker Hub or AWS ECR), and run it on a cloud service (e.g., AWS EC2).
Key Considerations:
arm64) matches the cloud service instance architecture.AGENTQL_API_KEY in the cloud service environment.The AQLResponseProxy class is returned by the page.queryElements() method. It acts as a metadata structure that allows developers to access web elements using dot notation based on the names defined in their AQL queries.
When accessing a leaf node (an innermost element) in the query, the proxy returns a Playwright Locator, allowing direct interaction via the Playwright Locator API (e.g., .click(), .type()). For container nodes, it returns another AQLResponseProxy object to allow further chaining.
const { configure, wrap } = require('agentql');
const { chromium } = require('playwright');
const QUERY = `
{
search_box
header {
search_btn
}
}
`;
(async () => {
configure({ apiKey: process.env.AGENTQL_API_KEY });
const browser = await chromium.launch({ headless: false });
const page = await wrap(await browser.newPage());
await page.goto('https://duckduckgo.com');
// Get AQLResponseProxy
const aqlResponse = await page.queryElements(QUERY);
// Access elements with dot notation as Playwright Locators
await aqlResponse.search_box.type('AgentQL');
await aqlResponse.header.search_btn.click();
await browser.close();
})();AgentQL provides several tools to help developers test, debug, and optimize queries for web data extraction:
() immediately after a term. This is used to refine results, exclude specific items, or disambiguate between similar elements on a page using plain English.{
products(exclude sponsored results)[]
}AgentQL is an AI-powered parser that replaces traditional CSS/XPath selectors with semantic, natural language queries.
Key capabilities include:
Playwright automatically performs actionability checks before actions like clicking, hovering, or typing to prevent timeout errors and ensure reliability. An element is considered actionable only if it meets the following criteria:
visibility:hidden in its computed style.disabled property.