EVM Blockchain Indexer

High-performance SQL indexer for EVM-compatible blockchains

A production-ready indexer that fetches and stores blockchain data for later analysis. Optimized for performance with support for parallel block processing, dynamic RPC capability detection, and efficient memory usage.

Features

• ✅ Complete blockchain primitives: blocks, transactions, receipts, logs, traces, withdrawals
• ✅ Token transfers: ERC20, ERC721, ERC1155
• ✅ DEX trades: Uniswap V2/V3, Curve, Balancer, and multiple high-volume AMMs
• ✅ DEX pairs: Track pair/pool creation across all supported DEXes
• ✅ Liquidity tracking: Monitor liquidity additions, removals, and reserve updates
• ✅ Contract tracking: Automatically indexes deployed contracts
• ✅ Parallel processing: Configurable batch size for optimal throughput
• ✅ Smart RPC usage: Auto-detects eth_getBlockReceipts support

Requirements

• Rust 1.70+
• ClickHouse 23.0+

Quick Start

Using Docker Compose (Recommended)

• Clone the repository:

git clone https://github.com/eabz/evm-indexer && cd evm-indexer

• Start the services:

docker-compose up -d

This will start:

• ClickHouse database on ports 8123 (HTTP) and 9000 (native)
• Indexer configured for Ethereum mainnet

• Monitor logs:

docker-compose logs -f indexer

Local Development

• Clone the repository:

git clone https://github.com/eabz/evm-indexer && cd evm-indexer

• Build the program:

cargo build --release

• Run the indexer:

./target/release/indexer \
  --chain 1 \
  --database clickhouse://user:password@localhost:8123/indexer \
  --rpcs https://eth.llamarpc.com \
  --start-block 0 \
  --batch-size 100

Configuration

CLI Flags

| Flag | Default | Description | |------|---------|-------------| | --chain | 1 | Chain ID to index (1=Ethereum, 56=BSC, 137=Polygon, etc.) | | --database | required | ClickHouse connection string: clickhouse://user:pass@host:port/db | | --rpcs | required | Comma-separated list of RPC endpoints | | --start-block | 0 | Block number to start syncing from | | --end-block | 0 | Last block to sync (0 = continuous sync) | | --batch-size | 200 | Number of blocks to fetch in parallel | | --ws | "" | WebSocket endpoint for real-time block updates | | --traces | true | Fetch transaction traces (requires archive node) | | --fetch-uncles | false | Fetch uncle blocks (adds 5-10% RPC calls) | | --new-blocks-only | false | Only index new blocks (skip historical sync) | | --debug | false | Enable debug logging |

Environment Variables

Create a .env file (see .env.example):

# ClickHouse Configuration CLICKHOUSE_DB=indexer CLICKHOUSE_USER=indexer CLICKHOUSE_PASSWORD=indexer

Indexer Configuration
CHAIN_ID=1
START_BLOCK=0
BATCH_SIZE=10
RPC_URL=https://eth.llamarpc.com
DEBUG=true
FETCH_UNCLES=false
TRACES=true

Database Schema

The indexer creates the following tables in ClickHouse:

• blocks - Block headers and metadata
• transactions - Transaction data with gas info
• logs - Event logs emitted by contracts
• traces - Internal transaction traces
• contracts - Deployed contract addresses
• withdrawals - Validator withdrawals (post-merge)
• erc20_transfers - ERC20 token transfers
• erc721_transfers - NFT transfers
• erc1155_transfers - Multi-token transfers
• dex_trades - DEX swap transactions
• dex_pairs - DEX pair/pool creation events
• dexliquidityupdates - Liquidity additions, removals, and sync events

Performance Tuning

Batch Size

• Small batches (10-50): Lower memory, slower throughput
• Medium batches (100-200): Balanced (recommended)
• Large batches (500+): Higher memory, faster throughput

RPC Endpoints

• Use multiple RPCs for better reliability
• Archive nodes required for traces
• eth_getBlockReceipts support = 2x faster

ClickHouse

• Use SSD storage for better performance
• Increase maxinsertblock_size for large batches
• Enable compression for storage savings

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

MIT License - see LICENSE file for details

Support

• GitHub Issues: Report bugs
• Discussions: Ask questions