mevlog
repository·main·Indexed 41 days ago
https://github.com/pawurb/mevlog-rsAn open-source blockchain indexer that downloads raw EVM transaction data from RPC endpoints into a local SQLite database, allowing developers to query 2k+ chains using standard SQL. It includes a CLI for indexing, purging, and inspecting blocks and transactions, as well as a Rust-based backend service featuring a server and scheduler.
What's inside mevlog
- mevlog is an open-source (MIT) tool for querying data from any EVM-compatible chain locally. It serves as an alternative to commercial blockchain indexers by downloading raw chain data directly from an RPC endpoint and storing it in a local SQLite database. This allows for offline, unlimited SQL querying without API keys, rate limits, or vendor lock-in.
What you can do with mevlog
mainmevlog provides several core capabilities for blockchain data analysis:
- Efficient Data Downloading: Pull block ranges from an RPC node into a local per-chain SQLite store.
- Chain Synchronization: Follow the chain head live and prune old blocks to manage storage.
- SQL Querying: Run read-only SQL against
transactions,blocks, andlogstables. It includes custom SQLite helpers for 256-bit integers, token amounts, ETH/gwei/USD formatting, and ENS names. - Custom Table Definitions: Decode specific logs (e.g., swaps, transfers) into typed columns via configuration.
- Transaction & Block Inspection: Render transactions, logs, blocks, and perform deep EVM analysis (state diffs, call traces, coinbase payments).
- EVM Tracing Insights: Extract storage state diffs, decoded call traces, and coinbase payments. Tracing can be performed via a node's
debug_traceTransactionRPC or fully locally using [Revm]. - MCP Integration: Expose mevlog as a Model Context Protocol (MCP) server to allow AI assistants to query chain data.
What are Custom Tables and how do they work?
mainCustom tables are derived data tables defined in
~/.mevlog/config.tomlthat allow you to extract specific event parameters from raw logs into typed, queryable SQL columns.Key Concepts:
- Mechanism: They function as an
INSERT INTO ... SELECToperation over the existinglogstable. No additional RPC calls are made. - Data Source: They slice the
topic0..topic3anddatafields from thelogstable. This allows you to access non-indexed ABI parameters (like swap amounts or tick values) that are otherwise packed in the rawdatablob. - Lifecycle: Tables are tracked via a
fingerprint(hash oftopic0, addresses, and columns). If the config changes, the fingerprint changes, and you must rebuild the table. - Identity: Rows are uniquely identified by
(block_number, log_index)usingON CONFLICT DO NOTHING, making re-indexing idempotent. - Implicit Columns: Every custom table automatically includes
block_number,tx_index,log_index, andaddress(the emitter). Do not use these names for your custom columns.
- Mechanism: They function as an
Index EVM data from RPC
mainThe indexing process pulls transactions, blocks, and logs for a specific block range from an EVM-compatible chain.
- Automatic RPC Selection: By default,
mevlogauto-selects the fastest RPC endpoint from ChainList. - Custom Endpoints: You can pin specific RPC endpoints via
config.toml. - Backfilling vs. Live: Use the
indexcommand to backfill a specific range, or use the--liveflag to continuously watch for and index new blocks as they arrive.
Note: Most high-level commands (like
txorblock) will automatically ensure the relevant blocks are indexed before proceeding, so you may not need to callindexmanually.- Automatic RPC Selection: By default,
Backfilling coinbase transfer data
mainIf you have already indexed blocks without tracing, you can backfill thecoinbase_transferdata incrementally. Re-runningquerywith the--evm-traceflag over the desired block range will only trace rows where thecoinbase_transfercolumn is currentlyNULL. This prevents the need to re-trace transactions that were already successfully processed.Understand the Signatures DB
mainThemevlog-sqlite-v5.dbis a separate database that holds method/event signatures and chain metadata. Unlike the per-chain transaction stores, this database is downloaded prebuilt from a CDN and is not intended to be queried via thequery --sqlcommand.Use pre-query macros for dynamic values
mainMacros are expanded by the
querycommand before the SQL is executed. They fetch values via RPC only when the specific token is present in the query. Macros must be wrapped in braces{}and are case-sensitive. The substituted literal is echoed back in thequery.sqlfield of the response for auditability.-- Example usage of macros SELECT * FROM blocks WHERE block_number = {LATEST_BLOCK()};The mevlog local storage model
mainIndexed data is stored in a local, per-chain SQLite database located at
~/.mevlog/mevlog-txs-v1-{chain_id}.db.Database Structure
The store consists of three primary tables:
transactionsblockslogs
Caching and Metadata
- Caching: Once a block is indexed, it is cached locally. Subsequent queries for the same block range are near-instant and do not require RPC calls.
- Signatures Database: A separate database (
mevlog-sqlite-v5.db) contains method/event signatures and chain metadata. This is automatically downloaded from a CDN during the first run.
How mevlog manages data and RPCs
mainData Ownership and Storage
mevlogdownloads raw chain data from RPC endpoints and stores it in a local SQLite database located at~/.mevlog/. This allows for offline querying and eliminates API keys or rate limits.RPC Discovery
With zero configuration,
mevlogautomatically detects the fastest available RPC endpoint using data from ChainList.Caching
All downloaded data is cached in the local SQLite database. Subsequent queries targeting the same block ranges will be nearly instantaneous because they read from the local disk rather than the network.
How the mevlog pipeline works
mainThe
mevlogtool operates on a single three-step pipeline: index → store → query.- Index: Fetches blocks, transactions, and logs from an EVM-compatible chain via RPC.
- Store: Persists the indexed data into a local SQLite database.
- Query: Executes SQL queries against the local database to retrieve data.
Understanding this flow is essential because most commands automatically trigger the indexing step if the required data is not already present in the local store.
How mevlog works: The index → store → query model
mainmevlog operates using a three-step lifecycle:
- Index: Fetches a block range from an RPC endpoint (utilizing cryo internally).
- Store: Writes the fetched transactions, blocks, and logs into a local, per-chain SQLite database located at
~/.mevlog/. - Query: Executes SQL queries against the local SQLite store. Since the data is stored locally on your disk, queries are fast and do not incur RPC costs.
How implicit indexing works in mevlog
mainWhen you run
mevlog queryor display commands (liketx,block, orblock-txs),mevlogperforms implicit indexing. It checks the requested block range against the local SQLite database and only fetches missing blocks via RPC. This ensures that repeat queries over the same range hit no RPC and use the local cache.To bypass this behavior and run SQL strictly against the existing local store without any network activity, use the
--skip-indexflag. Note that--skip-indexis mutually exclusive with the--blocksparameter.# Query a range (triggers implicit indexing for missing blocks) mevlog query --blocks 22030800:22030900 # Query existing data only (skips all indexing/RPC) mevlog query --skip-index