Timesketch Documentation

repository·master·Indexed 23 days ago

https://github.com/google/timesketch

An open-source platform for collaborative forensic timeline analysis. This documentation covers deployment via PowerShell and Bash, importing forensic data from Google Cloud Storage and local directories, Nginx reverse proxy configuration, and the SecGemini BYOT client. It also includes guides for setting up development and e2e testing environments using Docker, running application services (Celery and Gunicorn), and importing investigative questions using DFIQ.

Tokens
58.1K
Snippets
171
Records
384
Agent score
85%

What's inside Timesketch

  1. Overview of Timesketch

    master
    Timesketch is an open-source tool designed for collaborative forensic timeline analysis. It allows users and collaborators to organize timelines using 'sketches' and add context to raw data through annotations, comments, tags, and stars.
  2. Frontend-v3 technology stack overview

    master

    The frontend-v3 package uses the following modern web stack:

    • Framework: Vue 3 and Vuetify 3
    • State Management: Pinia
    • Routing: Vue Router
    • Layouts: vite-plugin-vue-layouts
    • Build Tool: Vite (provides fast cold starts and HMR)
    • Component Management: unplugin-vue-components for automated component importing
  3. Understand Timesketch Index Mappings

    master

    Timesketch uses OpenSearch index mappings to define how data from different sources is stored and indexed. This ensures that data from Plaso, CSV, or JSONL imports is searchable and correctly typed.

    Timesketch uses two primary mapping files located in the data/ directory of the repository:

    • plaso.mappings: Used for .plaso file imports.
    • generic.mappings: Used for CSV and JSONL file imports.
  4. Getting started with Timesketch

    master

    To begin using Timesketch, follow these primary workflows:

    • Installation: Follow the official Install Timesketch guide for administrative setup.
    • Data Ingestion: Learn how to Upload data to create timelines.
    • Core Usage: Refer to the Users guide to understand basic concepts and how to navigate the tool.
    • Notebook Integration: If you need to add a Notebook container for advanced analysis, follow the Notebook Installation guide.
  5. Start the legacy VueJS development server

    master

    To run the legacy frontend development environment, you need two active shells:

    1. Main Webserver: Start the API server using gunicorn in the first shell.
    2. Development Webserver: Start the Vue.js development server in the second shell. This spawns a listener on port 5001.

    Access the application at http://localhost:5001/login.

    # 1. Start the main webserver (API, etc.)
    $ CONTAINER_ID="$(docker container list -f name=timesketch-dev -q)"
    $ docker exec -it $CONTAINER_ID gunicorn --reload -b 0.0.0.0:5000 --log-file - --timeout 600 -c /usr/local/src/timesketch/timesketch/gunicorn.conf.py timesketch.wsgi:application
    
    # 2. Start the development webserver
    $ CONTAINER_ID="$(docker container list -f name=timesketch-dev -q)"
    $ docker compose exec timesketch yarn run --cwd=/usr/local/src/timesketch/timesketch/frontend serve
  6. Generate a Timesketch analyzer template using l2t_scaffolder.py

    master

    To create a new analyzer, use the l2t_scaffolder.py tool. This tool generates the necessary Python files and automatically registers the new analyzer in timesketch/lib/analyzers/__init__.py.

    Steps to scaffold a sketch_analyzer:

    1. Install l2tscaffolder from source in a virtualenv.
    2. Run the scaffolder from your Timesketch source directory:
      $ cd ~/timesketch/
      $ l2t_scaffolder.py
    3. Follow the interactive prompts:
      • Definition choice: Select timesketch (option [1]).
      • Path to the project root: Enter the path to your local Timesketch repository (e.g., .).
      • Module Name: Enter the name for your analyzer (e.g., best_analyzer).
      • Scaffolder choice: Select sketch_analyzer (option [1]).

    The tool will create a new git branch for your analyzer and generate three files: timesketch/lib/analyzers/<module_name>.py, timesketch/lib/analyzers/<MODULE_NAME>_test.py, and update timesketch/lib/analyzers/__init__.py.

    $ cd ~/timesketch/
    $ l2t_scaffolder.py
  7. Create a timeline from JSONL or CSV files

    master

    You can ingest timeline data into Timesketch using .csv or .jsonl files. While you can include any number of custom attributes, the following three fields are mandatory for the UI to render events correctly:

    • message: A string containing an informative message about the event.
    • datetime: The event timestamp in ISO8601 format (e.g., 2015-07-24T19:01:01+00:00).
    • timestamp_desc: A string explaining the timestamp type (e.g., file created).

    Requirements:

    • File Extensions: Files must end in .csv or .jsonl or the import will fail.
    • Missing Fields: If mandatory fields are missing, you must use the Headers Mapping functionality in the UI to map existing columns to these mandatory names.
  8. Troubleshoot Import issues

    master

    If imports are failing, check the following:

    • Ensure the Celery worker is running and check its logs.
    • Check the WebUI status of the import.
    • Attempt to upload the same file using import_client.py instead of the WebUI to isolate the issue.
    • Test with a known sample file (e.g., sigma_events.csv).

    CSV Import specific checks:

    • Verify CSV headers meet minimum requirements.
    • Check for encoding issues.
    • Provide headers and a few lines of sample content (scrambled for PII) when reporting issues.

    Plaso Import specific checks:

    • Verify the Plaso version installed on the server using psort.py --version inside the Docker container.
    • Identify the Plaso version used to create the file.
    • Use pinfo.py $FILENAME inside the Docker container to check file compatibility.
    pinfo.py $FILENAME
  9. Add Intelligence (IOCs) to a sketch

    master

    Intelligence items in Timesketch are typically Indicators of Compromise (IOCs) like IP addresses or file hashes. You can add them in two ways:

    1. Using automatic suggestions: Timesketch parses event attributes for potential IOCs. When a match is found (e.g., a SHA-256 hash or IP address), the string is displayed with a grey highlight. Click the highlighted string to open the add local intelligence dialog.
    2. Manual selection: Select any string with your cursor. The string will become highlighted, allowing you to add it as intelligence.

    You can optionally change the IOC type before confirming the suggestion.