DOCSEARCHbyAlgolia
Submit Docs

Nuclei Vulnerability Scanner

repository·dev·Indexed 12 days ago

https://github.com/projectdiscovery/nuclei

A high-performance, template-based vulnerability scanner used to detect security flaws across various protocols using YAML definitions. It provides a Go library (v3) featuring a thread-safe engine for concurrent scanning and supports multiple input formats for fuzzing, including Burp Suite XML, OpenAPI 3.0, Postman Collections, and Swagger specifications.

Tokens
43.4K
Snippets
127
Records
199
Agent score
95%

What's inside Nuclei

  1. Overview of Nuclei

    dev

    Nuclei is a modern, high-performance vulnerability scanner that utilizes simple YAML-based templates. It allows users to design custom vulnerability detection scenarios that simulate real-world conditions to minimize false positives.

    Key Features:

    • YAML-based Templates: Simple format for creating and customizing vulnerability templates.
    • Community Driven: Thousands of security experts contribute to keep up with trending vulnerabilities.
    • High Performance: Supports ultra-fast parallel scanning and request clustering.
    • Protocol Support: Handles multiple protocols including TCP, DNS, HTTP, SSL, WHOIS, JavaScript, and Code.
    • CI/CD Integration: Can be integrated into pipelines for vulnerability detection and regression testing.
    • Integrations: Connects with Jira, Splunk, GitHub, Elastic, and GitLab.

  2. What is Nuclei

    dev

    Nuclei is a high-performance vulnerability scanner that uses simple YAML-based templates. It allows users to design custom vulnerability detection scenarios that mimic real-world conditions to minimize false positives.

    Key Features:

    • YAML Templates: Simple format for creating and customizing vulnerability templates.
    • Community Driven: Contributions from thousands of security professionals.
    • High Performance: Super-fast parallel scanning and request clustering.
    • Protocol Support: Supports various protocols including TCP, DNS, HTTP, SSL, WHOIS, JavaScript, Code, and more.
    • CI/CD Integration: Can be integrated into pipelines for vulnerability detection and regression testing.
    • Integrations: Connects with Jira, Splunk, GitHub, Elastic, and GitLab.

  3. What is tsgen?

    dev

    tsgen is a developer tool designed to generate dummy TypeScript code for Goja Node modules that are written in Go.

    This generated TypeScript code serves two primary purposes:

    1. Intellisense: It can be compiled into .d.ts files to provide type definitions and autocompletion in editors like VS Code.
    2. Documentation: It provides a way to document the available interfaces of the Node modules.

  4. Nuclei Project Structure Overview

    dev

    The Nuclei repository is organized into several functional packages. Key modules include:

    • Reporting (pkg/reporting): Handles result exporting (SARIF, Markdown, Elasticsearch) and issue tracking (GitLab, Jira, GitHub).
    • Templates & Workflows:
      • pkg/parsers: Template and workflow loading/validation.
      • pkg/catalog: Disk-based template loading and filtering.
      • pkg/workflows: Workflow execution logic.
    • Protocols (pkg/protocols): Core protocol implementations including http, dns, file, network, and headless.
    • Operators (pkg/operators): Implementation of DSL functions, matchers, and extractors used within templates.
    • Core Logic:
      • pkg/types: CLI options and helpers.
      • pkg/model: Template information models.
      • pkg/output: Output management.

  5. Supported Input Formats for Fuzzing

    dev

    Nuclei supports several request source formats that can be used as input providers for fuzzing-related testing. When using these formats, Nuclei parses the files to generate and send HTTP requests for testing.

    Supported formats include:

    • Burp Suite XML: Request/Response files.
    • Proxify JSONL: Output files.
    • OpenAPI Specification: OpenAPI 3.0 schemas.
    • Postman Collection: JSON files.
    • Swagger Specification: Converted to OpenAPI 3.0 for processing.

  6. What is Nuclei Flow and how does it work?

    dev

    Nuclei flow is a request execution orchestration engine that provides two-way interaction between JavaScript (ECMAScript 5.1) and Nuclei templates. It allows you to move beyond simple conditional execution to complex logic, such as:

    • Iterating over slices of values (arrays, maps, etc.) to execute requests for each value.
    • Extracting values from one request and using them to drive subsequent requests.
    • Getting and setting values in the template context (global variables).
    • Implementing custom runtime logic (e.g., re-running a request if a status code is 403).
    • Updating variables at runtime (e.g., refreshing a JWT).
    • Using standard JavaScript objects and arrays to transform inputs.

    Essentially, flow allows you to write JavaScript code within a template to orchestrate how different protocol requests (http(), dns(), ssl(), etc.) interact and share data.

  7. Understand Nuclei Flow for conditional execution and orchestration

    dev

    Introduced in Nuclei v3, flow is a template engine/backend that allows for advanced request orchestration. It enables two primary capabilities:

    1. Conditional Execution: Execute specific requests only if certain conditions are met (e.g., flow: dns() && http()).
    2. Request Orchestration: Control the order of execution, iterate over slices, and use control structures like if and for statements.

    flow logic is implemented using JavaScript (ECMAScript 5.1) via the goja backend, allowing you to write complex logic directly within your YAML templates.

  8. Protocol-specific exceptions in multi-protocol execution

    dev

    The file protocol intentionally skips the first two integration steps required for multi-protocol execution to prevent performance issues and data redundancy:

    1. Statement 1 (Context Merging): Skipped because file/directory input paths do not currently contain variables or use them in paths.
    2. Statement 2 (Response Exposure): Skipped because files are processed by scanning each line; adding this would unintentionally load the entire content of all files into the context.

  9. Input data transformation and normalization

    dev

    Before input data is sent to a protocol executor, it passes through a transformation step. This process normalizes the data and may include:

    • Adding default ports if they are missing from the input.
    • Validating whether the input is a file, directory, or URL.
    • Adjusting the input structure accordingly to ensure compatibility with the protocol executor.

  10. Understand the Nuclei JavaScript Protocol Architecture

    dev

    The Nuclei JavaScript protocol is implemented using goja (a pure Go JavaScript VM). The architecture is designed to bridge Go-native functionality with a JavaScript runtime, allowing users to write scripts that leverage powerful Go libraries.

    Key architectural components include:

    • Compiler: Handles the compilation and execution of JavaScript code, including loading Node modules and injecting global types/functions.
    • Global/Builtin Scope: A set of types and functions available in the runtime without requiring an explicit require() statement. These are composed of both JavaScript-based logic (global/js) and Go-based implementations (global/scripts.go).
    • Bindings: Go-native packages are exposed to the JavaScript runtime via bindings located in generated/go/. These bindings use gojs interfaces to register Go packages as if they were Node modules.
    • Implementation Layer: The actual logic for all exposed JavaScript functions and types resides in the libs/ directory, which contains the native Go packages.

  11. How the Operators package works for matching and extraction

    dev

    The pkg/operators package provides the core logic for evaluating template conditions. It uses Matchers to determine if a request is successful/vulnerable and Extractors to pull data from responses.

    To use this functionality in a protocol, embed the operators.Operators type. The core logic is driven by the Execute function, which takes an input dictionary and matching/extraction functions to produce a Result object.

    Result Structure:

    • Matched: Boolean indicating if matchers passed.
    • Extracted: Boolean indicating if extractors found data.
    • Matches: Map of matcher names to matched strings.
    • Extracts: Map of extractor names to extracted values.
    • DynamicValues / PayloadValues: Maps containing values used during execution.
    // Operators contain the operators that can be applied on protocols
type Operators struct {
	Matchers []*matchers.Matcher
	Extractors []*extractors.Extractor
	MatchersCondition string
}

// Execute executes the operators on data and returns a result structure
func (operators *Operators) Execute(data map[string]interface{}, match MatchFunc, extract ExtractFunc, isDebug bool) (*Result, bool)

  12. How multi-protocol execution works in Nuclei

    dev

    Multi-protocol execution allows a single Nuclei template to utilize multiple protocols (e.g., combining DNS, HTTP, and SSL) in a specific sequence.

    When a template is unmarshalled, the order of protocols defined in the template is preserved and passed to the Executor. The multiproto engine acts as the backend for TemplateExecutor, managing shared logic and ensuring protocols are executed in the correct order.

    Execution Flow:

    1. All protocol requests in the Queue are executed sequentially.
    2. Dynamic values extracted from one protocol are added to the template context for subsequent protocols.
    3. Protocol Responses: Beyond extracted internal:true values, the response fields/values of a protocol are added to the template context at ExecutorOptions.TemplateCtx. To prevent collisions, all response fields are prefixed with the template type prefix (e.g., ssl_subject_dn).