eth-indexer

eth-indexer is an Ethereum blockchain indexer project to crawl blocks, transactions & state difference per block/address into MySQL database.

Getting Started

There are 3 main components in the project:

• geth: modified geth to get state difference per block/address
• idx-database: MySQL to store all indexed data
• indexer: indexer to crawl from geth then push to database

Prerequisites

• docker
• docker-compose

Before Building

Before building, please make sure environment variables MYSQLDATAPATH and GETHDATAPATH are setup properly, which are used to mount local data folder to MySQL and Geth containers for data persistence. One way to set this up is to have a .env file in the same folder of the docker-compose.yml

Example .env file:

MYSQLDATAPATH=~/indexer-data/mysql
GETHDATAPATH=~/indexer-data/geth

Configs and Flags

ethindexer_ supports two kinds of input:

• static config YAML files
• dynamic flags through command line

Build

$ git clone git@github.com:getamis/eth-indexer.git
$ cd eth-indexer
$ # Set MYSQLDATAPATH and GETHDATAPATH environment variables
$ docker-compose build

Usage

We use docker-compose for testing and developing. MYSQLDATAPATH & GETHDATAPATH environment variables are necessary, create them out of eth-indexer directory to store database and geth data.

first time to run indexer you need to create the database schema

$ mkdir -p ~/indexer-data/mysql ~/indexer-data/geth
Create database sechema
MYSQLDATAPATH="$HOME/indexer-data/mysql" docker-compose up idx-database idx-migration
press Ctrl + C when see eth-indexeridx-migration1 exited with code 0

then use docker-compose up with environment variables to start indexer:

$ MYSQLDATAPATH="$HOME/indexer-data/mysql" GETHDATAPATH="$HOME/indexer-data/geth" docker-compose up

wait few minutes, then you can see indexing messages from indexer:

Inserted TD for block                    number=0       TD=17179869184 hash=0xd4e56740f876aef8c010b86a40d5f56745a118d0906a34e69aec8c0db1cb8fa3

Example

Once there are some data in MySQL, you can query specific data from it, e.g., you can get data from blockheaders and transactions table. Balance is slightly different, and you can take a look at example folder to see how to query them.

package main
import ( 	"context" 	"fmt"
"github.com/ethereum/go-ethereum/common" 	"github.com/getamis/eth-indexer/model" 	"github.com/getamis/eth-indexer/store/account" 	"github.com/getamis/eth-indexer/store/sqldb" 	"github.com/getamis/sirius/database" 	"github.com/getamis/sirius/database/mysql" )
func main() { 	db, _ := sqldb.New("mysql", 		database.DriverOption( 			mysql.Database("ethdb"), 			mysql.Connector(mysql.DefaultProtocol, "127.0.0.1", "3306"), 			mysql.UserInfo("root", "my-secret-pw"), 		), 	) 	addr := common.HexToAddress("0x756f45e3fa69347a9a973a725e3c98bc4db0b5a0") 	store := account.NewWithDB(db)
account, err := store.FindAccount(context.Background(), model.ETHAddress, addr) 	if err != nil { 		fmt.Printf("Failed to find account: %v\n", err) 	} else { 		fmt.Printf("Find account, block_number: %v, balance: %v, \n", account.Balance, account.BlockNumber) 	} }