ScopeSentry Documentation

repository·main·Indexed 23 days ago

https://github.com/autumn-27/scopesentry

A distributed security scanning platform for asset mapping, subdomain enumeration, vulnerability scanning, and information leakage detection. It features a multi-node architecture, a plugin system for extensibility, and a CTWatcherSDK for Certificate Transparency log monitoring. The platform supports asset management, directory scanning, web crawling, and integrates with MongoDB and Redis for data persistence.

Tokens
5.8K
Snippets
16
Records
31
Agent score
81%

What's inside ScopeSentry

  1. Overview of ScopeSentry features

    main

    ScopeSentry is a distributed security tool designed for asset mapping and vulnerability management. Key capabilities include:

    • Asset Management: Subdomain enumeration, subdomain takeover detection, port scanning, asset identification, and asset grouping.
    • Scanning & Detection: Directory scanning, vulnerability scanning, sensitive information leakage detection, and POC import.
    • Monitoring & Crawling: URL extraction, web crawling, and page monitoring.
    • Extensibility: A plugin system that allows adding new tools via extensions and custom web fingerprints.
  2. ScopeSentry Core Features Overview

    main

    ScopeSentry is a distributed asset mapping and security scanning tool. Key capabilities include:

    • Asset Mapping: Subdomain enumeration, subdomain takeover detection, port scanning, and asset identification.
    • Scanning & Detection: Directory scanning, vulnerability scanning, sensitive information leak detection (including PDF support), and POC import.
    • Automation: ICP/APP/Mini-program automated collection, crawler, and page monitoring.
    • Extensibility: A plugin system to integrate any external tool.
    • Management: Asset grouping, multi-node scanning, webhook support, and data export.
  3. Best Practice: Two-stage scanning for Root Domains

    main

    When performing full information collection on root domains, do not run a single massive pipeline. Because tasks are distributed by target, a single node handling a root domain will also handle all its discovered subdomains, leading to uneven load and potential errors.

    Recommended Workflow:

    1. Stage 1: Subdomain Collection

      • targetSource: general
      • target: Your list of root domains.
      • Template: Use a template containing only SubdomainScan and SubdomainSecurity.
      • Wait for completion using get_task.
    2. Stage 2: Subsequent Modules

      • targetSource: subdomain
      • search: task=="<Stage 1 Task Name>" (exact match).
      • Template: Use templates for port scanning, asset mapping, or vulnerability scanning (exclude SubdomainScan).

    This approach allows subdomains to be distributed as independent targets across all available nodes, significantly increasing parallel efficiency.

  4. Draco decoder file variations

    main

    The Draco utilities consist of three main files, provided in two different build variations depending on your requirements:

    Files

    • draco_decoder.js: An Emscripten-compiled decoder compatible with any modern browser.
    • draco_decoder.wasm: A WebAssembly decoder for newer browsers and devices.
    • draco_wasm_wrapper.js: A JavaScript wrapper used to interface with the WASM decoder.

    Build Variations

    • Default: Latest stable builds tracking the Draco master branch.
    • glTF: Builds specifically targeted by the glTF mesh compression extension, tracking the gltf_2.0_draco_extension branch.
  5. Use the search DSL for custom queries

    main

    The search parameter uses a custom Domain Specific Language (DSL), not SQL.

    Operators:

    • ==: Exact match (Uses Index)
    • ^: Prefix match (e.g., domain=^example.com) (Uses Index)
    • =: Fuzzy/Regex match (Does NOT use index)
    • !=: Exclusion
    • &&: Logical AND
    • ||: Logical OR

    Common Search Fields by Type:

    • asset: domain, ip, port, service, app, title, statuscode, icon, banner, type, body, header
    • vulnerability: url, vulname, matched, request, response, level
    • DirScanResult: url, statuscode, redirect, length
    • subdomain: domain, ip, type, value
    • Universal Fields: tag, task (task name), rootDomain
    domain==www.example.com && port==443
    domain=^example.com
    level==high
    statuscode==200
  6. Add new scanning nodes

    main

    ScopeSentry supports a distributed architecture where you can add multiple scanning nodes. To add a new node, clone the ScopeSentry-Scan repository and use its specific docker-compose file.

    Configuration Steps:

    1. Clone ScopeSentry-Scan.
    2. Navigate to the build directory.
    3. Edit the .env file to provide connection information for your existing MongoDB and Redis instances.
    4. Set a unique NodeName in the .env file. If left empty, a name will be randomly generated (though you can change it later in the web interface).
    5. Run the node using scan-docker-compose.yml.
    git clone https://github.com/Autumn-27/ScopeSentry-Scan.git
    cd ScopeSentry-Scan/build
    
    # Edit the connection information for MongoDB and Redis in the .env file. 
    # NodeName is the node name, and each node name should be unique.
    docker-compose -f scan-docker-compose.yml up -d
  7. Install ScopeSentry via Docker Compose

    main

    To deploy a single-host instance of ScopeSentry, clone the repository and use the provided single-host-deployment.yml file. This deployment includes four containers: mongodb, redis, scope-sentry (the server), and scopesentry-scan (the scanning node).

    Important: Before running, you must edit the .env file to change the default MongoDB and Redis account passwords.

    git clone https://github.com/Autumn-27/ScopeSentry.git
    cd ScopeSentry
    # Change the MongoDB and Redis account passwords in the .env file.
    docker-compose -f single-host-deployment.yml up -d
  8. Retrieve initial login credentials

    main

    After the containers are running, you can retrieve the initial user password and the secondary verification password for the plugin by checking the logs of the scope-sentry container.

    docker logs scope-sentry
  9. Create a scanning task

    main

    To create a scan task using create_scan_task, you must provide a name, a node (from list_nodes), and a template (the template ObjectID, not the name).

    The targetSource determines where the scan targets are pulled from. Common sources include:

    • general: Direct input via the target field.
    • project: Targets pulled from a project (requires project ObjectID array).
    • asset: Targets searched from the asset library (requires search).
    • subdomain: Targets searched from the subdomain library (requires search).
    • RootDomain: Targets searched from the root domain library (requires search).
    • UrlScan: Targets searched from URL scan results (requires search).
    • *Source (e.g., subdomainSource): Uses targetTp=search with search or targetTp=select with targetIds to pull from selected/searched assets.
  10. Create a scanning template

    main

    To build a custom scanning template via create_scan_template:

    1. Call list_plugin_modules to see available module names.
    2. Call list_plugins (optionally filtered by module) to find the specific hash and default parameter for each plugin.
    3. Use create_scan_template by defining the modules array, where each entry maps a module to its corresponding plugin hash array.