Fastest RPC for @near based on Redis/LMDB
What is fast-near?
fast-near aims to provide the fastest RPC implementation for @NEARProtocol using high performance storage backends like: - in-memory storage in Redis.
- SSD optimized storage using LMDB.
It currently doesn't sync with network on its own, data needs to be loaded either from NEAR Lake or from https://github.com/vgrichina/near-state-indexer.
Why
nearcore RPC performance isn't good enough for novel use cases like https://web4.near.page.
fast-near achieves better performance by using
- in-memory storage using Redis
- client-side caching to save on Redis I/O
- V8 WebAssembly implementation
- disabled gas metering (timeout works fine for view calls)
- simpler REST API (no JSON wrapper if passing large binaries, etc)
- good compatibility with caching at HTTP layer (using Nginx, etc)
How to
Run directly from npm:
FASTNEARREDISURL=<redisip> FASTNEARNODEURL=<rpcendpoint> npx fast-nearBuild and run via yarn:
yarn
FASTNEARREDISURL=<redisip> FASTNEARNODEURL=<rpcendpoint> yarn startBuild and run with docker:
docker build -t fastrpc .
docker run -d -e FASTNEARREDISURL=<redisip> -e FASTNEARNODEURL=<rpcendpoint> fastrpcPull data into Redis
To load from NEAR Lake (use --help to learn more about options):
node scripts/load-from-near-lake.js near-lake-data-mainnet --batch-size 50 --history-length 1 --dump-changesPull data into LMDB (experimental)
To load from NEAR Lake (use --help to learn more about options):
FASTNEARSTORAGE_TYPE=lmdb node scripts/load-from-near-lake.js near-lake-data-mainnet --batch-size 50 --history-length 1 --dump-changesRun server:
FASTNEARSTORAGE_TYPE=lmdb yarn startLoad data only for your app's smart contracts
To load data for app1.near, app2.near and all subaccounts of superapp.near:
node scripts/load-from-near-lake.js near-lake-data-mainnet --include app1.near --include app2.near --include '*.superapp.near' --dump-changesExclude some undesired accounts
To load data for all accounts except aurora and sweat subaccounts:
node scripts/load-from-near-lake.js near-lake-data-mainnet --exclude aurora. --exclude sweat. --dump-changesDifferent data sink options
Currently there are such options to dump data loaded from NEAR Lake:
--dump-changes- dumps state changes into storage. UseFASTNEARSTORAGE_TYPEto specify storage type. Defaults toredis
Load data from a local node
See https://github.com/vgrichina/near-state-indexer for Rust implementation running full nearcore node.
CLI options
Environment variables
PORT- port to listen on (default:3000)FASTNEARSTORAGE_TYPE- storage type to use (default:redis). Supported values:redis,lmdb.FASTNEARENABLE_CACHE- enable client-side caching (default:true).FASTNEARLMDBPATH- path to LMDB database (default:./lmdb-data). This is only used ifFASTNEARSTORAGETYPEis set tolmdb.FASTNEARREDIS_URL- Redis URL (default:redis://localhost:6379)FASTNEARNODE_URL- NEAR RPC endpoint (default:https://rpc.mainnet.near.org). This is only used as a fallback for JSON-RPC endpoint.FASTNEARARCHIVALNODEURL- NEAR RPC endpoint for archival node (default:https://rpc.mainnet.internal.near.org). This is only used as a fallback for JSON-RPC endpoint for data unavailable in Redis or on non-archival RPC.FASTNEARALWAYSPROXY- Always proxy JSON-RPC requests toFASTNEARNODEURL(default:false).FASTNEARSTARTBLOCKHEIGHT- Minimum block height expected to be present in Redis (default:0).FASTNEARWORKER_COUNT- Number of workers to use for execution of WASM code. (default:4).FASTNEARCONTRACTTIMEOUTMS- Timeout for contract execution in milliseconds (default:1000).
HTTP API
Public endpoints
- Testnet: https://rpc.web4.testnet.page/account/testnet
- Mainnet: https://rpc.web4.near.page/account/near
Call view method
POST
You can post either JSON or binary body, it's passed raw as input to given method.
URL format:
https://rpc.web4.near.page/account/<contractaccountid>/view/<method_name>Examples
http post https://rpc.web4.near.page/account/vlad.tkn.near/view/ftbalanceof account_id=vlad.nearGET
Parameters are passed as part of URL query.
URL format:
https://rpc.web4.near.page/account/<contractaccountid>/view/<methodname>?<argname>=<stringargvalue>&<argname.json>=<jsonarg_value>Examples
String parameters:
curl 'https://rpc.web4.near.page/account/vlad.tkn.near/view/ftbalanceof?account_id=vlad.near'https://rpc.web4.near.page/account/vlad.tkn.near/view/ftbalanceof?account_id=vlad.near
JSON parameters:
curl --globoff 'https://rpc.web4.near.page/account/lands.near/view/web4_get?request.json={"path":"/"}'https://rpc.web4.near.page/account/lands.near/view/web4_get?request.json={"path":"/"}
Number parameters (passed as JSON):
curl 'https://rpc.web4.near.page/account/lands.near/view/getChunk?x.json=0&y.json=0'https://rpc.web4.near.page/account/lands.near/view/getChunk?x.json=0&y.json=0
Get account info
GET
URL format:
https://rpc.web4.near.page/account/<account_id>Example
curl 'https://rpc.web4.near.page/account/vlad.near'https://rpc.web4.near.page/account/vlad.near
Get access key info
GET
URL format:
https://rpc.web4.near.page/account/<accountid>/key/<publickey>Example
curl 'https://rpc.web4.near.page/account/vlad.near/key/ed25519:JBHUrhF61wfScUxqGGRmfdJTQYg8MzRr5H8pqMMjqygr'Download contract WASM code
GET
URL format:
https://rpc.web4.near.page/account/<account_id>/contractExample
curl 'https://rpc.web4.near.page/account/vlad.tkn.near/contract'https://rpc.web4.near.page/account/vlad.tkn.near/contract
Get contract methods list
GET
URL format:
https://rpc.web4.near.page/account/<account_id>/contract/methodsExample
curl 'https://rpc.web4.near.page/account/lands.near/contract/methods'https://rpc.web4.near.page/account/lands.near/contract/methods
Roadmap
Some of the planned and already implemented components. Is not exhaustive list.
- Loading data
- REST API
- JSON-RPC API
- NEAR P2P Protocol
- WASM Runtime
- Storage
- Tests
