ccl_chromium_reader

repository·master·Indexed 42 days ago

https://github.com/cclgroupltd/ccl_chromium_reader

A Python library for programmatic access to data stores used by Chromium-based applications such as Chrome, Edge, and Electron. Designed for digital forensics, it provides tools to extract and verify data from Bookmarks, History, Cookies, Local Storage, Session Storage, IndexedDB, and Cache. It includes the ccl_chrome_audit tool for scanning profile folders and supports both high-level APIs via ChromiumProfileFolder and low-level forensic APIs.

Tokens
20.5K
Snippets
63
Records
87
Agent score
54%

What's inside ccl_chromium_reader

  1. Filter ChromiumProfileFolder data with KeySearch

    master

    Methods in ChromiumProfileFolder that retrieve data support a KeySearch interface for filtering. You can pass the following types to search/filter results:

    • str: Performs an exact match on the value.
    • Collection of str (e.g., list or tuple): Matches if the value is exactly one of the items in the collection.
    • re.pattern: Performs a regex search (equivalent to re.search).
    • Function: A callable that takes a str and returns a bool indicating a match.
    import re
    import pathlib
    from ccl_chromium_reader import ChromiumProfileFolder
    
    profile_path = pathlib.Path("profile path goes here")
    
    with ChromiumProfileFolder(profile_path) as profile:
        # Match one of two possible hosts exactly, then a regular expression for the key
        for ls_rec in profile.iter_local_storage(
                storage_key=["http://not-a-real-url1.com", "http://not-a-real-url2.com"], 
                script_key=re.compile(r"message\d{1,3}?-text")
            ):
            print(ls_rec.value)
            
        # Match all urls which end with "&read=1" using a lambda function
        for hist_rec in profile.iterate_history_records(url=lambda x: x.endswith("&read=1")):
            print(hist_rec.title, hist_rec.url)
  2. Read Local Storage data with ccl_chromium_localstorage

    master

    Use the ccl_chromium_localstorage module to access Local Storage data from a Chromium/Chrome profile folder. You can use the LocalStoreDb class as a context manager to iterate through storage keys and their associated records. For each record, you can optionally attempt to find an associated batch using find_batch(leveldb_seq_number) to retrieve an approximate timestamp (accurate within 5-60 seconds).

    import sys
    import pathlib
    from ccl_chromium_reader import ccl_chromium_localstorage
    
    level_db_in_dir = pathlib.Path(sys.argv[1])
    
    # Create the LocalStoreDb object which is used to access the data
    with ccl_chromium_localstorage.LocalStoreDb(level_db_in_dir) as local_storage:
        for storage_key in local_storage.iter_storage_keys():
            print(f"Getting records for {storage_key}")
          
            for record in local_storage.iter_records_for_storage_key(storage_key):
                # we can attempt to associate this record with a batch, which may
                # provide an approximate timestamp (withing 5-60 seconds) for this
                # record.
                batch = local_storage.find_batch(record.leveldb_seq_number)
                timestamp = batch.timestamp if batch else None
                print(record.leveldb_seq_number, record.script_key, record.value, sep="\t")
  3. Read Session Storage data with ccl_chromium_sessionstorage

    master

    Use the ccl_chromium_sessionstorage module to access Session Storage data from a Chromium/Chrome profile folder. Use the SessionStoreDb class as a context manager. You can iterate through hosts using iter_hosts() and then iterate through the records for each host using iter_records_for_host(host). Each record provides leveldb_sequence_number, key, and value.

    import sys
    import pathlib
    from ccl_chromium_reader import ccl_chromium_sessionstorage
    
    level_db_in_dir = pathlib.Path(sys.argv[1])
    
    # Create the SessionStoreDb object which is used to access the data
    with ccl_chromium_sessionstorage.SessionStoreDb(level_db_in_dir) as session_storage: 
        for host in session_storage.iter_hosts():
            print(f"Getting records for {host}")
            for record in session_storage.iter_records_for_host(host):
              print(record.leveldb_sequence_number, record.key, record.value)
  4. Understand SnssFileType and command types

    master

    Chromium uses two main types of SNSS files, which determine how commands are interpreted:

    • SnssFileType.Session: Used for session files (typically starting with Session_). Commands are identified using SessionRestoreIdType.
    • SnssFileType.Tab: Used for tab files (typically starting with Tabs_). Commands are identified using TabRestoreIdType.

    When iterating through commands, most will be returned as UnprocessedEntry unless they match specific navigation update types. Navigation updates are returned as NavigationEntry objects.

  5. How WrappedIndexDB and WrappedDatabase relate

    master

    The library follows a hierarchical model to represent Chromium's IndexedDB structure:

    1. WrappedIndexDB: The top-level container representing the entire IndexedDB directory. It manages multiple databases and handles origin-based disambiguation.
    2. WrappedDatabase: Represents a specific database within the WrappedIndexDB. It is identified by a name, an origin, and a unique numerical ID.
    3. WrappedObjectStore: Represents an individual data store within a WrappedDatabase.

    Lifecycle: Always use WrappedIndexDB as a context manager to ensure that the underlying IndexedDb (raw) instance is properly closed.

  6. Understand LocalStorageRecord and StorageMetadata

    master

    The library uses two primary data classes to represent parsed data:

    LocalStorageRecord

    Represents an individual key-value pair in Local Storage.

    • file: Path to the origin file.
    • storage_key: The host/origin string.
    • script_key: The key defined in the web script.
    • value: The string value (is None if the record is a deletion).
    • leveldb_seq_number: The LevelDB sequence ID.
    • is_live: Boolean indicating if the record is currently active (not deleted).
    • record_location: A property returning an ArtifactLocation for forensics.

    StorageMetadata

    Represents the metadata associated with a storage host.

    • storage_key: The host string.
    • timestamp: The timestamp of the metadata.
    • size_in_bytes: Total size of the storage for this host.
    • leveldb_seq_number: The sequence number where this metadata was recorded.
  7. Manage SessionStoreDb lifecycle with context managers

    master

    The SessionStoreDb class supports the context manager protocol. Using with ensures that the underlying LevelDB connection is properly closed when the block exits.

    from ccl_chromium_reader.ccl_chromium_sessionstorage import SessionStoreDb
    import pathlib
    
    with SessionStoreDb(pathlib.Path("/path/to/db")) as ssdb:
        for host in ssdb:
            print(host)
    # Connection is automatically closed here
  8. Access Chromium cache data programmatically with ccl_chromium_cache

    master
    The ccl_chromium_cache module allows for programmatic access to Chromium cache data and metadata, including HTTP headers. The module automatically detects the cache type (block file or simple cache) during processing.
  9. Access IndexedDB data using the Raw API

    master

    The IndexedDb class provides a low-level API that does not mask the underlying Chromium/LevelDB structures. This is useful for deep forensics where metadata is required.

    Workflow:

    1. Initialize IndexedDb(leveldb_path, blob_path).
    2. Iterate through db.global_metadata.db_ids to get DatabaseID objects.
    3. Use db.get_database_metadata(...) with DatabaseMetadataType.MaximumObjectStoreId to find the range of object stores.
    4. Iterate through object store IDs (starting at 1) and use db.iterate_records(db_id, obj_store_id) to retrieve data.
    import sys
    from ccl_chromium_reader import ccl_chromium_indexeddb
    
    leveldb_folder_path = sys.argv[1]
    blob_folder_path = sys.argv[2]
    
    db = ccl_chromium_indexeddb.IndexedDb(leveldb_folder_path, blob_folder_path)
    
    for db_id_meta in db.global_metadata.db_ids:
        max_objstore_id = db.get_database_metadata(
            db_id_meta.dbid_no, 
            ccl_chromium_indexeddb.DatabaseMetadataType.MaximumObjectStoreId
        )
        
        if max_objstore_id is None:
            continue
    
        for obj_store_id in range(1, max_objstore_id + 1):
            for record in db.iterate_records(db_id_meta.dbid_no, obj_store_id):
                print(f"key: {record.user_key}")
                print(f"key: {record.value}")
    
                if "file" in record.value:
                    with record.get_blob_stream(record.value["file"]) as f:
                        file_data = f.read()