Clairvoyance Documentation

repository·main·Indexed 23 days ago

https://github.com/nikitastupin/clairvoyance

Clairvoyance is a tool designed to obtain GraphQL API schemas even when introspection is disabled. It performs blind introspection by brute-forcing field names using wordlists to reconstruct a JSON schema compatible with tools like GraphQL Voyager or InQL. The tool is available via pip and Docker, supporting concurrent requests, custom headers, and proxy routing.

Tokens
1.9K
Snippets
4
Records
12
Agent score
81%

What's inside Clairvoyance

  1. Optimize schema reconstruction with wordlists

    main

    Clairvoyance relies on wordlists to guess GraphQL names. To improve accuracy, consider these three approaches:

    1. Specialized GraphQL wordlists: Use lists specifically collected for GraphQL (e.g., from Escape Technologies).
    2. General English wordlists: Use common English wordlists (e.g., google-10000-english).
    3. Target-specific wordlists: Create a custom list by extracting valid GraphQL names from the target application's HTTP traffic or mobile application static files.

    Note: The regex for a valid GraphQL name is [_A-Za-z][_0-9A-Za-z]*.

  2. Install and use Clairvoyance via pip

    main

    Install Clairvoyance using pip. Once installed, you can run it from the command line by providing the target GraphQL endpoint URL and an output file path using the -o flag. The tool will attempt to reconstruct the schema and save it in JSON format.

    pip install clairvoyance
    clairvoyance https://rickandmortyapi.com/graphql -o schema.json
  3. Perform blind GraphQL introspection

    main

    Use the clairvoyance CLI to perform blind introspection on a GraphQL endpoint. This process attempts to reconstruct the GraphQL schema by brute-forcing field names using a wordlist, which is useful when standard introspection queries are disabled.

    Key capabilities:

    • Uses a wordlist to discover schema elements.
    • Supports custom input documents (queries) and existing schema files.
    • Supports concurrent requests for performance.
    • Can output the discovered schema to a file.
    • Can validate wordlist items against the GraphQL name regex ([_A-Za-z][_0-9A-Za-z]*) to ensure compatibility.
  4. Troubleshoot issues with a specific website

    main

    If Clairvoyance fails only when targeting a specific website, investigate these areas:

    1. Scope Verification: Rule out a general tool failure by testing against a known working target. For example, try:
      clairvoyance https://rickandmortyapi.com/graphql -o schema.json
    2. URI Validity: Check the endpoint URI for typos or errors introduced during copying.
    3. Authorization: Ensure you are providing the correct headers, cookies, or other required authorization credentials.
    4. WAF (Web Application Firewall): The target's WAF might be blocking the request. Ensure the program imitates a modern browser or a specific web client by checking that the User-Agent header matches a legitimate browser/app.
    clairvoyance https://rickandmortyapi.com/graphql -o schema.json
  5. Troubleshoot Clairvoyance installation and connectivity issues

    main

    If Clairvoyance fails to run entirely, check the following:

    1. Installation: Verify that the installation process completed without errors and that all dependency versions are correct.
    2. Internet Connection: Ensure you have a stable internet connection and that the environment running Clairvoyance has the necessary permissions/rights to access your target URL.
    3. Usage Syntax: Verify your command syntax against the instructions in the README.md.
  6. Troubleshoot Clairvoyance stopping during execution

    main

    If Clairvoyance was working but suddenly stops, determine if a restart fixes the issue:

    If a restart fixes the issue (Rate Limiting)

    The server may be rate-limiting requests from your IP or user identity. To mitigate this:

    • Lower the number of threads used.
    • Run the program in a "slow mode" (if supported).

    If a restart does NOT fix the issue (Server/IP issues)

    • Server Availability: The target server might be temporarily unavailable. Manually verify the endpoint using a browser or other tools.
    • IP/User Ban: Your IP address or user identity may have been banned by the target server.
  7. Configure Clairvoyance logging via environment variables

    main

    You can control the logging behavior of Clairvoyance using the following environment variables:

    • LOG_FMT: A string format for logging.
    • LOG_DATEFMT: A string format for logging the date.
    • LOG_LEVEL: The logging level (e.g., INFO).
    LOG_FMT=`%(asctime)s \t%(levelname)s\t| %(message)s` # A string format for logging.
    LOG_DATEFMT=`%Y-%m-%d %H:%M:%S` # A string format for logging date.
    LOG_LEVEL=`INFO` # A string level for logging.
  8. Initialize the Client for GraphQL introspection

    main

    The Client class is used to perform blind introspection against a GraphQL endpoint. It manages an aiohttp.ClientSession and handles concurrency, retries, and backoff logic.

    Parameters

    • url (str): The target GraphQL endpoint URL.
    • max_retries (Optional[int]): Maximum number of retry attempts for failed requests. Defaults to 3.
    • headers (Optional[Dict[str, str]]): HTTP headers to include in requests.
    • concurrent_requests (Optional[int]): Maximum number of concurrent requests allowed via an internal semaphore. Defaults to 50.
    • proxy (Optional[str]): Proxy URL to route requests through.
    • backoff (Optional[int]): An integer used to calculate exponential backoff delay. If provided, the delay is calculated as 0.5 * backoff**retries.
    • disable_ssl_verify (Optional[bool]): If True, SSL verification is disabled in the aiohttp.TCPConnector. Defaults to False.
  9. Reference: clairvoyance CLI arguments and flags

    main

    The clairvoyance command-line tool accepts the following arguments and flags to control the blind introspection process:

    Argument/FlagDescription
    urlThe target GraphQL endpoint URL.
    --wordlistPath to a file containing the wordlist. Items are de-duplicated.
    --validateIf present, removes wordlist items that do not conform to the GraphQL name regex ([_A-Za-z][_0-9A-Za-z]*).
    --headersA list of headers in the format Key: Value.
    --concurrent-requestsNumber of concurrent requests to perform.
    --documentThe input GraphQL document template (defaults to query { FUZZ }).
    --input-schemaPath to an existing JSON schema file to use as a starting point.
    --outputPath to save the discovered schema.
    --proxyProxy URL to route requests through.
    --max-retriesMaximum number of retries for failed requests.
    --backoffBackoff delay between retries.
    --no-sslDisables SSL verification.
    --verboseEnables verbose logging.
  10. Post a GraphQL document using Client.post()

    main

    The post method sends a GraphQL document (query or mutation) to the server and returns the response as a JSON dictionary.

    Signature

    async def post(self, document: Optional[str], retries: int = 0) -> Dict

    Behavior

    • Document: If document is provided, it is wrapped in a {"query": document} JSON object. If None, it sends a null payload.
    • Retries: If the server returns a status code $\ge 500$, the client automatically retries up to max_retries.
    • Error Handling:
      • If a JSONDecodeError occurs, it logs a warning and a hint suggesting authentication issues or rate-limiting (e.g., Cloudflare).
      • Connection errors (aiohttp.ClientConnectionError, aiohttp.ClientPayloadError) and asyncio.TimeoutError are caught and logged.
    • Backoff: If backoff was configured during initialization, the client waits for an exponentially increasing delay between retries.
    • Concurrency: Requests are governed by the concurrent_requests semaphore set during initialization.