mevlog

repository·main·Indexed 41 days ago

https://github.com/pawurb/mevlog-rs

An open-source blockchain indexer that downloads raw EVM transaction data from RPC endpoints into a local SQLite database, allowing developers to query 2k+ chains using standard SQL. It includes a CLI for indexing, purging, and inspecting blocks and transactions, as well as a Rust-based backend service featuring a server and scheduler.

Tokens
42.3K
Snippets
153
Records
202
Agent score
51%

What's inside mevlog

  1. What you can do with mevlog

    main

    mevlog provides several core capabilities for blockchain data analysis:

    • Efficient Data Downloading: Pull block ranges from an RPC node into a local per-chain SQLite store.
    • Chain Synchronization: Follow the chain head live and prune old blocks to manage storage.
    • SQL Querying: Run read-only SQL against transactions, blocks, and logs tables. It includes custom SQLite helpers for 256-bit integers, token amounts, ETH/gwei/USD formatting, and ENS names.
    • Custom Table Definitions: Decode specific logs (e.g., swaps, transfers) into typed columns via configuration.
    • Transaction & Block Inspection: Render transactions, logs, blocks, and perform deep EVM analysis (state diffs, call traces, coinbase payments).
    • EVM Tracing Insights: Extract storage state diffs, decoded call traces, and coinbase payments. Tracing can be performed via a node's debug_traceTransaction RPC or fully locally using [Revm].
    • MCP Integration: Expose mevlog as a Model Context Protocol (MCP) server to allow AI assistants to query chain data.
  2. What are Custom Tables and how do they work?

    main

    Custom tables are derived data tables defined in ~/.mevlog/config.toml that allow you to extract specific event parameters from raw logs into typed, queryable SQL columns.

    Key Concepts:

    • Mechanism: They function as an INSERT INTO ... SELECT operation over the existing logs table. No additional RPC calls are made.
    • Data Source: They slice the topic0..topic3 and data fields from the logs table. This allows you to access non-indexed ABI parameters (like swap amounts or tick values) that are otherwise packed in the raw data blob.
    • Lifecycle: Tables are tracked via a fingerprint (hash of topic0, addresses, and columns). If the config changes, the fingerprint changes, and you must rebuild the table.
    • Identity: Rows are uniquely identified by (block_number, log_index) using ON CONFLICT DO NOTHING, making re-indexing idempotent.
    • Implicit Columns: Every custom table automatically includes block_number, tx_index, log_index, and address (the emitter). Do not use these names for your custom columns.
  3. Index EVM data from RPC

    main

    The indexing process pulls transactions, blocks, and logs for a specific block range from an EVM-compatible chain.

    • Automatic RPC Selection: By default, mevlog auto-selects the fastest RPC endpoint from ChainList.
    • Custom Endpoints: You can pin specific RPC endpoints via config.toml.
    • Backfilling vs. Live: Use the index command to backfill a specific range, or use the --live flag to continuously watch for and index new blocks as they arrive.

    Note: Most high-level commands (like tx or block) will automatically ensure the relevant blocks are indexed before proceeding, so you may not need to call index manually.

  4. Use pre-query macros for dynamic values

    main

    Macros are expanded by the query command before the SQL is executed. They fetch values via RPC only when the specific token is present in the query. Macros must be wrapped in braces {} and are case-sensitive. The substituted literal is echoed back in the query.sql field of the response for auditability.

    -- Example usage of macros
    SELECT * FROM blocks WHERE block_number = {LATEST_BLOCK()};
  5. The mevlog local storage model

    main

    Indexed data is stored in a local, per-chain SQLite database located at ~/.mevlog/mevlog-txs-v1-{chain_id}.db.

    Database Structure

    The store consists of three primary tables:

    • transactions
    • blocks
    • logs

    Caching and Metadata

    • Caching: Once a block is indexed, it is cached locally. Subsequent queries for the same block range are near-instant and do not require RPC calls.
    • Signatures Database: A separate database (mevlog-sqlite-v5.db) contains method/event signatures and chain metadata. This is automatically downloaded from a CDN during the first run.
  6. How mevlog manages data and RPCs

    main

    Data Ownership and Storage

    mevlog downloads raw chain data from RPC endpoints and stores it in a local SQLite database located at ~/.mevlog/. This allows for offline querying and eliminates API keys or rate limits.

    RPC Discovery

    With zero configuration, mevlog automatically detects the fastest available RPC endpoint using data from ChainList.

    Caching

    All downloaded data is cached in the local SQLite database. Subsequent queries targeting the same block ranges will be nearly instantaneous because they read from the local disk rather than the network.

  7. How the mevlog pipeline works

    main

    The mevlog tool operates on a single three-step pipeline: index → store → query.

    1. Index: Fetches blocks, transactions, and logs from an EVM-compatible chain via RPC.
    2. Store: Persists the indexed data into a local SQLite database.
    3. Query: Executes SQL queries against the local database to retrieve data.

    Understanding this flow is essential because most commands automatically trigger the indexing step if the required data is not already present in the local store.

  8. How mevlog works: The index → store → query model

    main

    mevlog operates using a three-step lifecycle:

    1. Index: Fetches a block range from an RPC endpoint (utilizing cryo internally).
    2. Store: Writes the fetched transactions, blocks, and logs into a local, per-chain SQLite database located at ~/.mevlog/.
    3. Query: Executes SQL queries against the local SQLite store. Since the data is stored locally on your disk, queries are fast and do not incur RPC costs.
  9. How implicit indexing works in mevlog

    main

    When you run mevlog query or display commands (like tx, block, or block-txs), mevlog performs implicit indexing. It checks the requested block range against the local SQLite database and only fetches missing blocks via RPC. This ensures that repeat queries over the same range hit no RPC and use the local cache.

    To bypass this behavior and run SQL strictly against the existing local store without any network activity, use the --skip-index flag. Note that --skip-index is mutually exclusive with the --blocks parameter.

    # Query a range (triggers implicit indexing for missing blocks)
    mevlog query --blocks 22030800:22030900
    
    # Query existing data only (skips all indexing/RPC)
    mevlog query --skip-index