OpenOSINT Documentation

repository·main·Indexed 21 days ago

https://github.com/openosint/openosint

An AI-powered OSINT agent, MCP server, and CLI for authorized security research. OpenOSINT provides 19 investigation tools for identity, infrastructure, web search, and security analysis through a natural-language interface. It supports an interactive REPL, Web UI, and direct tool execution, with AI backend compatibility for Anthropic Claude, local Ollama, and OpenAI-compatible endpoints.

Tokens
37K
Snippets
116
Records
168
Agent score
77%

What's inside OpenOSINT

  1. Overview of the AI OSINT Prompt Pack

    main

    The AI OSINT Prompt Pack is a collection of 30+ structured prompts designed to facilitate repeatable, disciplined, and auditable AI-assisted investigations. Instead of ad-hoc prompting, the pack provides a methodology that forces AI assistants (like Claude or ChatGPT) to follow a specific investigation loop: scope → collect → pivot → verify → document.

    While the pack is designed to work with any AI assistant, it is optimized to pair with OpenOSINT.

  2. Overview of OpenOSINT features

    main

    OpenOSINT provides a suite of OSINT capabilities designed for security researchers:

    • AI Tool Chaining: Uses natural language to drive tool selection and multi-step investigations.
    • 19 Modular Tools: Includes capabilities for Email, username, breach, WHOIS, IP, subdomain, dorks, paste, phone, Shodan, VirusTotal, Censys, IP2Location, AbuseIPDB, GitHub, DNS, live dork search, URL scraping, and SERP footprint.
    • AI Backend Support: Supports Anthropic Claude (default), local Ollama, or any OpenAI-compatible endpoint (via LiteLLM, vLLM, LM Studio, etc.).
    • MCP Support: Acts as a native Model Context Protocol (MCP) server for use in Claude Desktop, Claude Code, Cursor, and Windsurf.
    • Parallel Execution: Use the --parallel flag to run complementary tools concurrently.
    • Session Management: REPL sessions are saved to ~/.openosint/history/. Use openosint history to browse them.
  3. List and use OpenOSINT tools

    main

    OpenOSINT provides a suite of 19 specialized OSINT tools for investigating various targets. Tools are categorized by their purpose, such as email enumeration, username searching, IP intelligence, and domain analysis.

    Available Tools Overview:

    • Identity: search_email (via holehe), search_username (via sherlock), search_phone (via phoneinfoga), search_github (via GitHub API).
    • Infrastructure: search_ip (via ipinfo.io), search_whois, search_domain (via sublist3r), search_dns (via dnspython), search_shodan, search_censys, search_abuseipdb, search_ip2location.
    • Web & Search: generate_dorks (local), search_dorks_live (via Bright Data), scrape_url (via Bright Data), search_footprint (via Bright Data), search_paste (via psbdmp.ws).
    • Security: search_breach (via HIBP), search_virustotal.

    For full per-tool documentation, CLI flags, and output formats, visit openosint.tech.

  4. How Playbook reports and errors are handled

    main

    When running a playbook via the runner.py, the system handles missing configurations or tool errors gracefully to ensure the final report is still produced:

    • Missing Configuration: If a tool requires an environment variable that is not set (e.g., BRIGHTDATA_API_KEY), the report will include an ℹ️ Skipped block for that section instead of failing. It may also include a note regarding the missing requirement.
    • Tool/Binary Errors: If a tool fails (e.g., a RuntimeError due to a missing binary), the report will render a ⚠ Step error block for that specific section. The run_playbook method will not crash and will continue to process other steps.
    • Empty Results: If a tool returns no data, the report will display No results found for that section.
    • Report Output: Reports are written to a reports/ directory with a filename following the pattern {YYYY-MM-DD}_{target}_{recipe}_report.md.
  5. Understand Playbook Step States

    main

    Playbook steps do not use a simple pass/fail binary. Instead, each step results in one of the following StepState values, which determines how the output is rendered in the final report:

    • NOT_CONFIGURED: A required environment variable or binary is missing. The report renders an informational block explaining how to enable the tool (e.g., which API key to set).
    • EMPTY: The tool ran successfully but returned no output.
    • ERROR: An unexpected exception occurred during execution. The report renders a warning with the exception message.
    • SUCCESS (Implicit): The tool ran and returned non-empty output. The output is rendered verbatim in a fenced Markdown block.
  6. Understand the snapshot data schema

    main

    Snapshots are stored as JSON files in data/metrics/YYYY-MM-DD.json. The schema includes GitHub repository metadata, traffic statistics (views and clones), and PyPI download data.

    Note on Uniques: The trailing_30_days metric for unique visits is a daily-summed value. Because the GitHub API does not provide cross-day deduplication, summing daily uniques is not a true 30-day unique visitor count. The report labels this as unique visits (daily-summed) to avoid misinterpretation.

    {
      "date": "2026-07-25",
      "repo": "OpenOSINT/OpenOSINT",
      "github": {
        "stars": 0,
        "forks": 0,
        "watchers": 0,
        "traffic": {
          "views": {"total": 0, "unique_total": 0, "daily": [{"date": "2026-07-20", "count": 0, "uniques": 0}]},
          "clones": {"total": 0, "unique_total": 0, "daily": [{"date": "2026-07-20", "count": 0, "uniques": 0}]},
          "referrers": [{"referrer": "google.com", "count": 0, "uniques": 0}],
          "paths": [{"path": "/", "count": 0, "uniques": 0}]
        }
      },
      "pypi": {
        "downloads_last_day": 0,
        "downloads_last_week": 0,
        "downloads_last_month": 0
      }
    }
  7. Understand the Playbooks Feature structure

    main

    The Playbooks feature in OpenOSINT allows for executing automated OSINT workflows defined as 'recipes' in YAML format. A playbook consists of multiple steps, where each step targets a specific tool and a target entity (e.g., a domain).

    Key components include:

    • Recipe-as-Data: Workflows are defined in YAML files located in openosint.playbooks/recipes/*.yaml.
    • Loader: Responsible for loading YAML recipes into frozen dataclasses (Recipe and PlaybookStep).
    • Runner: Executes the steps in a recipe, collects tool outputs, and generates a Markdown report (and optionally a PDF).
    • Executive Summary: An automated summary generated by the runner that counts findings (like subdomains, IP addresses, and emails) using the EXTRACTOR_REGISTRY.
  8. Compare Free (MIT) vs. Commercial Plans

    main

    OpenOSINT is free and open source under the MIT License for any use, including commercial, SaaS, and closed-source applications. You do not need a commercial plan to use the software commercially.

    Commercial plans are intended for organizations that require a formal vendor relationship, such as procurement compliance, written warranties, indemnification, or priority support.

    Comparison Summary

    FeatureFree (MIT)Commercial Plan
    Personal, Academic, & Commercial use
    Closed-source / SaaS use
    Modification & redistribution
    Community support (GitHub Issues)
    Written commercial use agreement
    License assurance letter
    Priority email support
    Defined response-time target
    Named maintainer contact
    Basic indemnification clause
    SLA, Security, & Compliance docsEnterprise
    Vendor onboarding / InvoicingEnterprise
  9. Understand the Demo Recording Workflow

    main

    The recording pipeline automates a browser session to capture a high-quality demonstration of the OpenOSINT interface. The workflow follows these steps:

    1. Initialization: Opens http://localhost:8080 in a headed Chromium browser (1440×860 @2x). It seeds the OPENOSINT_DEMO_KEY into sessionStorage.openosint_byok via addInitScript to ensure the key is never visible in logs or on screen.
    2. Interaction: Waits for the ES module bridge (window._agentLoopReady === true), then sends the command "Investigate openosint.tech" to the #chat-input.
    3. Graph Expansion: Waits for the graph to reach at least 6 nodes, then waits 1700ms for the Cytoscape layout to settle. It performs a real mouse click on the first non-root node to ensure the cursor is visible.
    4. Completion: Waits for 9+ nodes (post-pivot expansion) or a 20s timeout, holds the final state for 2.5s, takes a poster screenshot, and saves the raw video to out/raw.webm.
  10. Implement the Prompt Pack conversion funnel

    main

    To maximize lead generation, follow this funnel strategy:

    1. Traffic Source: Direct all blog Prompt Pack CTAs (Call to Actions) to the /free-prompts capture page rather than directly to the checkout.
    2. Conversion: Use the free 5-prompt set as a lead magnet to capture email addresses from cold readers.
    3. Upsell: Include a direct link to the full paid pack (https://tommasodev.gumroad.com/l/ai-osint-prompt-pack) on the capture page for users who are ready to purchase immediately.

    This approach ensures that even during launch spikes, you build an email list instead of losing potential customers who are not yet ready to purchase.

  11. Use RapidProxy as an upstream proxy in OpenOSINT

    main

    RapidProxy does not have a dedicated search_* tool. Instead, it is used via OpenOSINT's generic upstream proxy layer. By configuring a proxy URL, you can route outbound HTTP/SOCKS5 traffic for most OpenOSINT tools through RapidProxy residential IPs.

    Supported Tools

    Setting a proxy applies to any tool that uses the standard proxy retrieval methods, including:

    • search_abuseipdb
    • search_breach
    • search_censys
    • search_domain
    • search_email
    • search_github
    • search_ip
    • search_ip2location
    • search_paste
    • search_phone
    • search_shodan
    • search_username
    • search_virustotal

    Excluded Tools

    The following are not proxyable via this method:

    • search_dns (requires raw DNS resolution)
    • generate_dorks (no network calls)
    • Anthropic client in agent.py (LLM traffic)
    • Bright Data-backed tools (scrape_url, search_dorks_live, search_footprint) as they already use a residential network layer.
    # Using the environment variable
    export OPENOSINT_PROXY_URL=http://user:pass@<rapidproxy-gateway-host>:<port>
    openosint ip2location 8.8.8.8
    
    # Using the CLI flag (overrides environment variable)
    openosint --proxy socks5://user:pass@host:1080 email target@example.com
  12. Configure environment variables for OSINT tools

    main

    To enable advanced intelligence gathering in playbooks, ensure the following environment variables are set in your shell or deployment environment:

    • SHODAN_API_KEY: Required for search_shodan tool.
    • VIRUSTOTAL_API_KEY: Required for search_virustotal tool.