broxus/octusbridge-relay — 24 Stars

Topics

#bnb #bridge #defi #eth #everscale #fantom-opera #octusbridge #polygon

Language breakdown

Rust 97.1%

Shell 2.6%

Dockerfile 0.3%

Solidity 0.1%

▸ Files click to expand

README

Octus Bridge relay

Runtime requirements

CPU: 8 cores, 2 GHz
RAM: 16 GB
Storage: 200 GB fast SSD
Network: 100 MBit/s

How to run

To simplify the build and create some semblance of standardization in this repository there is a set of scripts for configuring the relay.

NOTE: scripts are prepared and tested on Ubuntu 20.04. You may need to modify them a little for other distros.

##### Setup relay service

- Native version (a little more complex way, but gives some performance gain and reduces the load):

./scripts/setup.sh -t native

- OR docker version (the simplest way, but adds some overhead):

./scripts/setup.sh -t docker

Not recommended for machines with lower specs than required

> At this stage, a systemd service relay is created. Configs and keys will be in /etc/relay and > Everscale node DB will be in /var/db/relay.

Do not start this service yet!

##### Prepare config

Either add the environment variables to the [Service] section of unit file. It is located at /etc/systemd/system/relay.service.

> This method is recommended because you can just make a backup of /etc/relay and > not be afraid for it's safety, all keys in it are encrypted

file (systemd)
   [Service]
   ...
   Environment=RELAYMASTERKEY=stub-master-key
   Environment=RELAYSTAKERADDRESS=your-staker-address
   Environment=ETHMAINNETURL=eth-http-rpc-endpoint
   Environment=POLYGON_URL=polygon-http-rpc-endpoint
   ...

Or simply replace the ${..} parameters in the config. It is located at /etc/relay/config.yaml.

##### Generate keys

./scripts/generate.sh -t docker # or native, depends on your installation type

> if you already have seed phrases that you want to import, then add the -i flag

This script will print unencrypted data. Be sure to write it down somewhere! And also make a backup of the /etc/relay folder!

##### Link relay keys

Use ETH address and Everscale public key from the previous step to link this relay setup with your staker address at https://octusbridge.io/relayers/create. When you start the linking process go to step 5 and start the relay. It will begin to confirm the public key and address on the air. It may take some time to sync at first (~40 minutes).

> During linking you will need to send at least 0.05 ETH to your relay ETH address so that the relay can confirm his ownership. > > If you think that this is a lot or if the gas price is more than 300 GWEI now, > then you can change the values in the config.

##### Enable and start relay service

systemctl enable relay systemctl start relay

# Optionally check if it is running normally. It will take some time to start. # Relay is fully operational when it prints Initialized relay journalctl -fu relay

Relay will be running on UDP port 30000 by default, so make sure that this port is not blocked by firewall.

NOTE: docker installation uses port 30000 in unit file, so you may need to update the service if you decide to change it. All environment variables must also be passed to container (e.g. -e RELAYMASTERKEY).

> Relay has a built-in Prometheus metrics exporter which is configured in the metrics_settings section of the config. > By default, metrics are available at http://127.0.0.1:10000/ > >

Response example:

>

> >

> ethsubscriberlastprocessedblock{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",chain_id="56"} 13790361    > ethsubscriberpendingconfirmationcount{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",chain_id="56"} 0    > ethsubscriberlastprocessedblock{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",chain_id="137"} 22954791    > ethsubscriberpendingconfirmationcount{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",chain_id="137"} 0    > ethsubscriberlastprocessedblock{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",chain_id="250"} 26020394    > ethsubscriberpendingconfirmationcount{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",chain_id="250"} 0    > ethsubscriberlastprocessedblock{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",chain_id="1"} 13875962    > ethsubscriberpendingconfirmationcount{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",chain_id="1"} 0    > solsubscriberunrecognizedproposalscount{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 0    > tonsubscriberready{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 1    > tonsubscribercurrent_utime{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 1640456699    > tonsubscribertime_diff{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 3    > tonsubscribershardclienttime_diff{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 7    > tonsubscribermcblockseqno{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 13426600    > tonsubscribershardclientmcblockseqno{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 13426600    > tonsubscriberpendingmessagecount{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 0    > bridgependingethtonevent_count{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 0    > bridgependingtonethevent_count{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 0    > bridgependingsoltonevent_count{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 0    > bridgependingtonsolevent_count{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 0    > bridgetotalactiveethtoneventconfigurations{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 86    > bridgetotalactivetonetheventconfigurations{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 11    > bridgetotalactivesoltoneventconfigurations{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 1    > bridgetotalactivetonsoleventconfigurations{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 1    > stakinguserdatatokensbalance{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",round_num="13"} 100000000000000    > stakingcurrentrelay_round{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246"} 13    > stakingelectionsstarttime{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",roundnum="13"} 1640380268    > stakingelectionsendtime{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",roundnum="13"} 1640553068    > stakingelectionsstatus{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",round_num="13"} 1    > stakingignoreelections{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",round_num="13"} 0    > stakingparticipatesinround{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",roundnum="13"} 1    > stakingelected{staker="0:7a9701bede7f86bf039aba200c1bb421a388bbb4b0580bfaeafa66f908d2b246",roundnum="13"} 1    >

> >

>

Example config

NOTE: The syntax ${VAR} can also be used everywhere in config. It will be

replaced by the value of the environment variable VAR.

---
Keystore password
masterpassword: "${RELAYMASTER_KEY}"
Your address from which you specified keys
stakeraddress: "${RELAYSTAKER_ADDRESS}"
bridge_settings:
  # Keystore data path
  keys_path: "/etc/relay/keys.json"
  # Bridge contract address
  bridge_address: "0:1d51fb47566d0d283ebbf83c641c01ebebaad6c3cec55895b0074b802036094e"
  # If set, relay will not participate in elections. Default: false
  ignore_elections: false
  # Shard split depth
  shardsplitdepth: 10
  # Ton token metadata endpoint base url
  tokenmetabase_url: "https://ton-tokens-api.meta"
  # Solana network config
  sol_network:
    # Public endpoint
    endpoints: ["https://api.mainnet-beta.solana.com"]
    # Events polling interval
    pollintervalsec: 30
  # EVM network configs
  evm_networks:
    # Ethereum
    - chain_id: 1
      # RPC node HTTP endpoint
      endpoint: "${ETHMAINNETURL}"
      # Timeout, used for simple getter requests. Default: 10
      gettimeoutsec: 10
      # Timeout, used for processing eth_getLogs response. Default: 120
      blocksprocessingtimeout_sec: 120
      # Max simultaneous connection count. Default: 10
      pool_size: 10
      # Idle polling interval. Default: 60
      pollintervalsec: 60
      # Maximum blocks range for getLogs request
      maxblockrange: 1000
    # Smart Chain
    - chain_id: 56
      # Public endpoint (see https://docs.bscscan.com/misc-tools-and-utilities/public-rpc-nodes)
      endpoint: https://bsc-dataseed1.binance.org
      gettimeoutsec: 10
      pool_size: 10
      pollintervalsec: 60
      maxblockrange: 1000
    # Fantom Opera
    - chain_id: 250
      # Public endpoint
      endpoint: https://rpc.ftm.tools
      gettimeoutsec: 10
      pool_size: 10
      pollintervalsec: 60
      maxblockrange: 1000
    # Polygon
    - chain_id: 137
      endpoint: "${POLYGON_URL}"
      gettimeoutsec: 10
      pool_size: 10
      pollintervalsec: 60
      maxblockrange: 1000
    # Milkomeda
    - chain_id: 2001
      endpoint: https://rpc-mainnet-cardano-evm.c1.milkomeda.com
      gettimeoutsec: 10
      pool_size: 10
      pollintervalsec: 60
      maximumfailedresponsestimesec: 604800
      maxblockrange: 1000
    # Avalanche Network
    - chain_id: 43114
      endpoint: https://api.avax.network/ext/bc/C/rpc
      gettimeoutsec: 10
      pool_size: 10
      pollintervalsec: 60
      maximumfailedresponsestimesec: 604800
      maxblockrange: 1000
    # BlockPI
    - chain_id: 8217
      endpoint: https://klaytn.blockpi.network/v1/rpc/public
      gettimeoutsec: 10
      pool_size: 10
      pollintervalsec: 60
      maximumfailedresponsestimesec: 604800
      maxblockrange: 5000
node_settings:
  # Root directory for relay DB. Default: "./db"
  db_path: "/var/db/relay"
  # UDP port, used for ADNL node. Default: 30303
  adnl_port: 30000
  # Path to temporary ADNL keys.
  # NOTE: Will be generated if it was not there.
  # Default: "./adnl-keys.json"
  tempkeyspath: "/etc/relay/adnl-keys.json"
  db_options:
    rocksdblrucapacity: "2 GB"
    cellscachesize: "4 GB"
  adnl_options:
    forceusepriority_channels: false
storage:
  persistentdbpath: "/var/db/relay-events"
metrics_settings:
  # Listen address of metrics. Used by the client to gather prometheus metrics.
  # Default: "127.0.0.1:10000"
  listen_address: "127.0.0.1:10000"
  # URL path to the metrics. Default: "/"
  # Example: curl http://127.0.0.1:10000/
  metrics_path: "/metrics"
  # Metrics update interval in seconds. Default: 10
  collectionintervalsec: 10

Architecture overview

The relay is simultaneously the Everscale node and can communicate with all EVM networks specified in the config. Its purpose is to check and sign transaction events.

At startup, it synchronizes the Everscale node and downloads blockchain state. It searches Bridge contract state in it, all connectors, active configurations and pending events. Then it subscribes to the Bridge contract in Everscale and listens for connector deployment events. Each new connector can produce activation event which signals that relay should subscribe to the event configuration contract. Each configuration contract produces event deployment events, relay sees and checks them.

For Everscale-to-EVM events, only the correctness of data packing is checked (all other stuff is verified on the contracts

side). You can enforce token metadata check though (currently for TON only, use flag 0x01 on contract level to enable). If the event is correct, the relay converts this data into ETH ABI encoded bytes and signs it with its ETH key. This signature is sent along with a confirmation message. If the data in the event was invalid then the relay sends a rejection message.

##### Everscale-to-EVM ABI mapping rules:

bytes => same
  string => same
  uintX => same
  intX => same
  bool => same
  fixedbytes => same
  fixedarray => same but mapped
  array => same but mapped
  tuple => same but mapped
  _ => unsupported

For EVM-to-Everscale events, all event parameters are checked on the relay side. It waits for or looking for a transaction

on the specified EVM network, converts event data to the TVM cell and sends a confirmation message if everything was correct. Otherwise, it sends a rejection message.

##### ETH-to-Everscale ABI mapping rules:

> You can use https://github.com/broxus/eth-ton-abi-converter to convert data from web page

address => bytes (of length 20)
  bytes => bytes or cell (*)
  string => same
  intX => same
  uintX => same
  bool => same
  array => same but mapped
  fixedbytes1 => depends on context (**)
  fixedbytesX => same
  fixedarray => same but mapped
  tuple => same but mapped or cell (*)
  _ => unsupported

> When converting ABI from EVM format to Everscale, there is a mechanism for controlling this process. > You can add a bytes1 (\\) element which sets context flags to its value. > > Currently, there are only four flags: > > - 0x01 - place tuples to new cell (\\\*) > - 0x02 - interpret bytes as encoded TVM cell (\*) > - 0x04 - insert default cell in case of error with flag 0x02 (\*) > - 0x08 - check token root for token wallet > > NOTE: Flags can't be changed inside an array element! This would lead to inconsistent array items ABI.

The decision to make the relay an Everscale node was not made by chance. In the first version, several relays were connected to the one "light" node which was constantly restarted or to the graphql which also was quite unreliable. As a result, relays instantly see all events and vote for them almost simultaneously in one block. The implementation is more optimized than C++ node, so they don't harm the network.

Changelog

2.3.2 (2024-03-07)

Features:

Close expired Solana events
Improve Solana metrics

Bugfixes:

Fixed clearing Solana events from pending buffer

2.3.1 (2024-02-08)

Features:

Improved shard states GC

Bugfixes:

Fixed Solana events parsing

2.3.0 (2024-01-08)

Features:

Fully reworked bridge with Solana

2.2.0 (2023-04-04)

Bugfixes:

Stability fixes

2.1.2 (2022-12-23)

Features:

Improved logger
Added support for configuration feature flags
Extend rejection info

Bugfixes:

Fixed transport issues

2.1.1 (2022-10-14)

Bugfixes:

Fixed errors with SOL events during scanning all events
Increased default polling interval

2.1.0 (2022-09-06)

Features:

Add Solana

Bugfixes:

Fixed outgoing RLDP transfers

2.0.13 (2022-07-22)

Features:

Replace tiny-adnl with everscale-network
Optimize DB layout

2.0.12 (2022-06-03)

Features:

Backport transport fixes

2.0.11 (2022-04-13)

Features:

Added archives assembly
Added new account model support

Bugfixes

Fixed ADNL channels

2.0.10 (2022-04-03)

Bugfixes

Fixed memory leaks. (New peers queue was read at a fixed rate).

2.0.9 (2022-03-26)

Features

Added packets compression support (enabled by default).
Various optimizations.

2.0.8 (2022-02-12)

Features

Updated ABI version to 2.2
Fixed memory leaks. (Shard states were slowly filling with loaded storage cells).

2.0.7 (2022-02-02)

Features

ADNL security improvements

2.0.6 (2021-12-31)

Features

Optimized DB structure

2.0.5 (2021-11-27)

Features

Improved ETH events verification
Updated events ABI
Added tonsubscribershardclienttimediff, tonsubscribermcblockseqno and tonsubscribershardclientmcblock_seqno