Sealighter Documentation

repository·main·Indexed 18 days ago

https://github.com/pathtofile/sealighter

A research-oriented tool for capturing, filtering, and automatically parsing Event Tracing for Windows (ETW) and Windows PreProcessor Tracing (WPP) events into JSON. Built with native C++ using the Krabs ETW library, it features multi-provider subscription, robust filter chaining (any_of, all_of, none_of), event buffering to reduce noise, and the ability to retrieve event stack traces for user-mode providers.

Tokens
9.7K
Snippets
25
Records
34
Agent score
63%

What's inside Sealighter

  1. Overview of Sealighter features

    main

    Sealighter is a tool designed for security researchers to perform Event Tracing for Windows (ETW) and Windows PreProcessor Tracing (WPP) research without writing custom parsing code. It leverages the Krabs ETW library to provide several key capabilities:

    • Multi-Provider Subscription: Subscribe to multiple ETW and WPP Providers simultaneously.
    • Automatic JSON Parsing: Automatically captures and parses events into JSON format without requiring prior knowledge of the event structure or format.
    • Robust Filtering: Supports filter chaining, filter negation, and filtering by Event ID, Opcode, property values, or arbitrary string searches across the entire event.
    • Flexible Output: Direct events to Standard out (stdout), a File, or the Windows Event Log (ideal for high-volume traces like FileIO).
    • Stack Traces: Ability to retrieve event stack traces.
    • Event Buffering: Configurable buffering to aggregate multiple similar events into a single event based on a count, reducing noise.

    Captured JSON data can be processed using Python, PowerShell, or ingested into SIEM tools like Splunk or ELK.

  2. What is Buffering in Sealighter

    main

    Buffering allows Sealighter to collapse multiple similar events occurring within a specific time window into a single event containing a buffered_count field. This reduces noise and volume in traces by reporting the frequency of events rather than every individual occurrence.

    Events are considered 'the same' and thus buffered together if they share identical values for all properties specified in the properties_to_match configuration.

  3. Understand the design philosophy of Sealighter vs SilkETW

    main

    Sealighter is specifically designed as a research platform for investigating ETW, TraceLogging, and WPP providers/events. While tools like SilkETW are geared toward production environments, Sealighter prioritizes deep visibility and research capabilities.

    Key differences include:

    • Implementation: Sealighter is written in native C++ (using the KrabsETW library) for higher performance and lower resource usage, whereas SilkETW is .NET-based.
    • Filtering Strategy: Sealighter filters events as early as possible using a static list of filters before JSON serialization to maximize performance. SilkETW relies on Yara rule matching against JSON-serialized events, which incurs higher CPU/memory overhead.
    • Data Depth: Sealighter supports features like event stack traces for user-mode providers and event buffering (grouping similar events with a count) to manage high-volume data.
  4. How Sealighter filters work

    main

    Filters are used to reduce the volume of ETW events by only reporting those that match specific criteria. Filters are applied per-provider and are defined within the user_traces section of your configuration file.

    Filters are processed in a specific order:

    1. The keyword_any and keyword_all fields are checked first.
    2. If those pass, the filters object is evaluated.

    Filters are organized into three logical lists: any_of, all_of, and none_of.

    "user_traces": [
        {
            "trace_name": "proc_trace",
            "provider_name": "Microsoft-Windows-Kernel-Process",
            "keywords_any": 16
        },
        {
            "trace_name": "guid_trace",
            "provider_name": "{382b5e24-181e-417f-a8d6-2155f749e724}",
            "filters": {
                "any_of": {
                    "opcode_is": [1, 2]
                }
            }
        }
    ]
  5. Understand the Sealighter JSON Event Format

    main

    Sealighter outputs events as JSON objects. Each event consists of three primary sections:

    1. header: Contains metadata common to every event (e.g., process_id, timestamp, provider_name).
    2. properties: Contains the actual event data. These fields are unique to the event type and are parsed based on their TDH_INTYPE.
    3. property_types: Maps each key in properties to its corresponding TDH_INTYPE (e.g., UINT32, STRINGW), which is useful for writing filters or custom parsers.

    If the report_stacktrace option is enabled in the provider configuration, an additional stack_trace array containing memory addresses of the generating functions will be present.

    {
        "header": {
            "activity_id": "{00000000-0000-0000-0000-000000000000}",
            "event_flags": 576,
            "event_id": 1,
            "event_name": "",
            "event_opcode": 1,
            "event_version": 3,
            "process_id": 17964,
            "provider_name": "Microsoft-Windows-Kernel-Process",
            "task_name": "ProcessStart",
            "thread_id": 25932,
            "timestamp": "2020-05-17 11:54:24Z",
            "trace_name": "proc_trace",
            "buffered_count": 2
        },
        "properties": {
            "CreateTime": "2020-05-17 11:54:24Z",
            "ImageName": "\\Device\\HarddiskVolume4\\Windows\\System32\\notepad.exe"
        },
        "property_types": {
            "CreateTime": "FILETIME",
            "ImageName": "STRINGW"
        },
        "stack_trace": [
            "0x7FFA18BAB944",
            "0x7FFA1868902A"
        ]
    }
  6. Use buffering to aggregate similar events

    main

    Buffering allows Sealighter to report multiple similar events as a single event with a buffered_count field, reducing noise.

    To use buffering:

    1. Set buffering_timout_seconds in session_properties to define the reporting interval.
    2. Define a buffers array within a user_trace.
    3. Specify the event_id to buffer and the properties_to_match (the field values that must be identical for events to be grouped together).

    Example: Buffering ProcessStart events (Event ID 1) from the Microsoft-Windows-Kernel-Process provider, grouping them by ImageName every 10 seconds.

    {
        "session_properties": {
            "session_name": "Sealighter-Trace",
            "output_format": "stdout",
            "buffering_timout_seconds":  10
        },
        "user_traces": [
            {
                "trace_name": "ProcTrace01",
                "provider_name": "Microsoft-Windows-Kernel-Process",
                "keywords_any": 16,
                "filters": {
                    "any_of": {
                        "event_id_is": 1
                    }
                },
                "buffers": [
                    {
                        "event_id": 1,
                        "max_before_buffering": 0,
                        "properties_to_match": [
                            "ImageName"
                        ]
                    }
                ]
            }
        ]
    }
  7. Configure filter lists: any_of, all_of, and none_of

    main

    You can group filters using one of three logical lists inside the filters key:

    • any_of: An event is reported if it matches at least one filter in the list. If a filter value is an array, the event only needs to match one item in that array.
    • all_of: An event is reported only if it matches every filter in the list. If a filter value is an array, the event must match one item in that array, but it must still satisfy all other top-level keys in the all_of list.
    • none_of: An event is reported only if it does not match any of the filters in the list. If a filter value is an array, the event is excluded if it matches any one of the items in that array.
    // any_of example: matches if opcode is 1 OR event_id is 1
    "filters": {
        "any_of": {
            "opcode_is": 1,
            "event_id_is": 1
        }
    }
    
    // all_of example: matches only if opcode is 1 AND event_id is 1
    "filters": {
        "all_of": {
            "opcode_is": 1,
            "event_id_is": 1
        }
    }
    
    // none_of example: matches if it is NOT (opcode 1 AND event_id 1)
    "filters": {
        "none_of": {
            "opcode_is": 1,
            "event_id_is": 1
        }
    }
  8. Use event buffering to manage high-volume ETW data

    main
    Sealighter includes a buffering mechanism to handle high-volume ETW providers. It can buffer many similar events occurring within a specific time period and consolidate them into a single event that includes a count of the occurrences. This prevents the output from being overwhelmed by repetitive data.
  9. How to handle WPP traces in Sealighter

    main

    Sealighter cannot automatically parse WPP (Windows Software Trace Preprocessor) traces at runtime. You cannot extract event formats from a session automatically.

    To work around this, you must manually define the event format or extract it from a PDB file. Alternatively, you can capture the raw data for post-processing by setting the dump_raw_event option to true on the provider. This will provide hex-encoded bytes that you can parse after the trace has been captured.

    // Example configuration concept
    {
      "provider": {
        "dump_raw_event": true
      }
    }
  10. Run Sealighter with a JSON configuration

    main

    To run Sealighter, execute sealighter.exe as an Administrator and provide the path to a JSON configuration file.

    Example configuration to log all process creations and terminations using stdout:

    {
        "session_properties": {
            "session_name": "My-Process-Trace",
            "output_format": "stdout"
        },
        "user_traces": [
            {
                "trace_name": "proc_trace",
                "provider_name": "Microsoft-Windows-Kernel-Process",
                "keywords_any": 16
            }
        ]
    }
    sealighter.exe path\to\config.json
  11. How to get started with Sealighter

    main

    To begin using Sealighter, you should follow the specialized guides for each stage of your research workflow:

    1. Installation: Follow the Installation guide to set up the tool and configure Windows Event logging if needed.
    2. Configuration: Refer to Configuration to define which Providers to log and specify your output destination.
    3. Filtering: Use the Filtering guide to learn how to implement complex triage logic using chains and negations.
    4. Buffering: If dealing with high-volume noise, consult Buffering to learn how to aggregate similar events.
    5. Data Analysis: Once traces are captured, use Parsing Data to learn how to extract information from the JSON output using Python or PowerShell.
    6. Research Scenarios: For practical applications, see Scenarios for walkthroughs of real-world research use cases.
  12. Trace and parse WPP events

    main

    Sealighter does not currently auto-parse WPP (Windows Software Trace Preprocessor) events because their format is stored in PDB or TMF files rather than the trace itself. To work with WPP, you must capture the raw event data and parse it manually (e.g., using Python).

    Workflow:

    1. Enable the provider: For some providers like OLE32, you may need to modify the registry first.
    2. Configure Sealighter:
      • Log both the main WPP provider GUID and the specific message provider GUID.
      • Set keywords_any and level to 255 (0xff) to capture all events.
      • Use dump_raw_event: true to capture the hex-encoded bytes.
      • Use max_events_total in a filter to prevent flooding.
    3. Parse the output: Use a script to convert the raw hex field into a readable format (like a UTF-16 string).
    # Enable OLE32 tracing
    reg add HKEY_LOCAL_MACHINE\Software\Microsoft\OLE\Tracing /v ExecutablesToTrace /t REG_MULTI_SZ /d * /f
    {
        "session_properties": {
            "session_name": "Sealighter-Trace",
            "output_format": "file",
            "output_filename": "ole.json"
        },
        "user_traces": [
            {
                "trace_name": "ole32",
                "provider_name": "{BDA92AE8-9F11-4D49-BA1D-A4C2ABCA692E}",
                "keywords_any": 255,
                "level": 255
            },
            {
                "trace_name": "ole32-message",
                "provider_name": "{0F480EA8-F109-39A3-8A27-36DC7E84A294}",
                "dump_raw_event": true,
                "filters": {
                    "any_of": {
                        "max_events_total": 10
                    }
                }
            }
        ]
    }
    import json
    import binascii
    
    with open("ole.json", "r") as f:
        for line in f:
            event = json.loads(line)
            process_id = event["header"]["process_id"]
            raw_hex = event["raw"]
            raw_bytes = binascii.unhexlify(raw_hex)
            raw_string = raw_bytes.decode("utf16")
            print(f"[{process_id}] - {raw_string}")
    # Cleanup: Delete the registry key
    reg delete HKEY_LOCAL_MACHINE\Software\Microsoft\OLE\Tracing /v ExecutablesToTrace /f