mirror of https://github.com/arnaucube/hermez-node.git synced 2026-02-07 03:16:45 +01:00

Files

Eduard S f0e79f3d55 Update coordinator to work better under real net

- cli / node
    - Update handler of SIGINT so that after 3 SIGINTs, the process terminates
      unconditionally
- coordinator
    - Store stats without pointer
    - In all functions that send a variable via channel, check for context done
      to avoid deadlock (due to no process reading from the channel, which has
      no queue) when the node is stopped.
    - Abstract `canForge` so that it can be used outside of the `Coordinator`
    - In `canForge` check the blockNumber in current and next slot.
    - Update tests due to smart contract changes in slot handling, and minimum
      bid defaults
    - TxManager
        - Add consts, vars and stats to allow evaluating `canForge`
        - Add `canForge` method (not used yet)
        - Store batch and nonces status (last success and last pending)
        - Track nonces internally instead of relying on the ethereum node (this
          is required to work with ganache when there are pending txs)
        - Handle the (common) case of the receipt not being found after the tx
          is sent.
        - Don't start the main loop until we get an initial messae fo the stats
          and vars (so that in the loop the stats and vars are set to
          synchronizer values)
        - When a tx fails, check and discard all the failed transactions before
          sending the message to stop the pipeline.  This will avoid sending
          consecutive messages of stop the pipeline when multiple txs are
          detected to be failed consecutively.  Also, future txs of the same
          pipeline after a discarded txs are discarded, and their nonces reused.
        - Robust handling of nonces:
            - If geth returns nonce is too low, increase it
            - If geth returns nonce too hight, decrease it
            - If geth returns underpriced, increase gas price
            - If geth returns replace underpriced, increase gas price
        - Add support for resending transactions after a timeout
        - Store `BatchInfos` in a queue
    - Pipeline
        - When an error is found, stop forging batches and send a message to the
          coordinator to stop the pipeline with information of the failed batch
          number so that in a restart, non-failed batches are not repated.
        - When doing a reset of the stateDB, if possible reset from the local
          checkpoint instead of resetting from the synchronizer.  This allows
          resetting from a batch that is valid but not yet sent / synced.
    - Every time a pipeline is started, assign it a number from a counter.  This
      allows the TxManager to ignore batches from stopped pipelines, via a
      message sent by the coordinator.
    - Avoid forging when we haven't reached the rollup genesis block number.
    - Add config parameter `StartSlotBlocksDelay`: StartSlotBlocksDelay is the
      number of blocks of delay to wait before starting the pipeline when we
      reach a slot in which we can forge.
    - When detecting a reorg, only reset the pipeline if the batch from which
      the pipeline started changed and wasn't sent by us.
    - Add config parameter `ScheduleBatchBlocksAheadCheck`:
      ScheduleBatchBlocksAheadCheck is the number of blocks ahead in which the
      forger address is checked to be allowed to forge (apart from checking the
      next block), used to decide when to stop scheduling new batches (by
      stopping the pipeline).  For example, if we are at block 10 and
      ScheduleBatchBlocksAheadCheck is 5, eventhough at block 11 we canForge,
      the pipeline will be stopped if we can't forge at block 15.  This value
      should be the expected number of blocks it takes between scheduling a
      batch and having it mined.
    - Add config parameter `SendBatchBlocksMarginCheck`:
      SendBatchBlocksMarginCheck is the number of margin blocks ahead in which
      the coordinator is also checked to be allowed to forge, apart from the
      next block; used to decide when to stop sending batches to the smart
      contract.  For example, if we are at block 10 and
      SendBatchBlocksMarginCheck is 5, eventhough at block 11 we canForge, the
      batch will be discarded if we can't forge at block 15.
    - Add config parameter `TxResendTimeout`: TxResendTimeout is the timeout
      after which a non-mined ethereum transaction will be resent (reusing the
      nonce) with a newly calculated gas price
    - Add config parameter `MaxGasPrice`: MaxGasPrice is the maximum gas price
      allowed for ethereum transactions
    - Add config parameter `NoReuseNonce`: NoReuseNonce disables reusing nonces
      of pending transactions for new replacement transactions.  This is useful
      for testing with Ganache.
    - Extend BatchInfo with more useful information for debugging

- eth / ethereum client
    - Add necessary methods to create the auth object for transactions manually
      so that we can set the nonce, gas price, gas limit, etc manually
    - Update `RollupForgeBatch` to take an auth object as input (so that the
      coordinator can set parameters manually)
- synchronizer
    - In stats, add `NextSlot`
    - In stats, store full last batch instead of just last batch number
    - Instead of calculating a nextSlot from scratch every time, update the
      current struct (only updating the forger info if we are Synced)
    - Afer every processed batch, check that the calculated StateDB MTRoot
      matches the StateRoot found in the forgeBatch event.

2021-02-16 16:10:23 +01:00

.gitignore

Update coordinator to work better under real net

2021-01-20 17:29:02 +01:00

cfg.buidler.toml

Update coordinator to work better under real net

2021-02-16 16:10:23 +01:00

load-sk-example.sh

Update node and apply some fixes

2021-01-07 16:09:28 +01:00

main.go

Update coordinator to work better under real net

2021-01-20 17:29:02 +01:00

README.md

Document how to build node

2021-02-15 15:41:32 +01:00

README.md

node cli

This is the main cli for the node

Go version

The hermez-node has been tested with go version 1.14

Usage

NAME:
   hermez-node - A new cli application

USAGE:
   node [global options] command [command options] [arguments...]

VERSION:
   0.1.0-alpha

COMMANDS:
   importkey  Import ethereum private key
   genbjj     Generate a new BabyJubJub key
   wipesql    Wipe the SQL DB (HistoryDB and L2DB), leaving the DB in a clean state
   run        Run the hermez-node in the indicated mode
   help, h    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --mode MODE    Set node MODE (can be "sync" or "coord")
   --cfg FILE     Node configuration FILE
   --help, -h     show help (default: false)
   --version, -v  print the version (default: false)

The node has two main modes of running:

sync: Synchronizer mode. In this mode the node will only synchronize the state of the hermez smart contracts, mainly processing the transactions in the batches.
coord: Coordinator mode. In this mode, apart from doing all the synchronization work, the node will also act as a coordinator, accepting L2 transactions in the pool, and trying to forge batches when the proper conditions arise.

Configuration

The node requires a single configuration file to run.

You can find a testing working configuration example at cfg.buidler.toml

To read the documentation of each configuration parameter, please check the type Node and type Coordinator at config/config.go. All the sections that are prefixed with Coordinator are only used in coord mode, and don't need to be defined when running the coordinator in sync mode

Notes

The private key corresponding to the parameter Coordinator.ForgerAddress needs to be imported in the ethereum keystore
The private key corresponding to the parameter Coordinator.FeeAccount.Address needs to be imported in the ethereum keystore
The public key corresponding to the parameter Coordinator.FeeAccount.BJJ can be generated with the command genbjj
There are two sets of debug parameters (Debug for all modes, and Coordinator.Debug for coord mode). Some of these parameters may not be suitable for production.
The parameter Coordinator.Debug.BatchPath, when set, causes the coordinator to store dumps of a lot of information related to batches in json files. This files can be around 2MB big. If this parameter is set, be careful to monitor the size of the folder to avoid running out of space.
The node requires a PostgreSQL database. The parameters of the server and database must be set in the PostgreSQL section.

Building

All commands assume you are at the cli/node directory.

Building the node requires using the packr utility to bundle the database migrations inside the resulting binary. Install the packr utility with:

cd /tmp && go get -u github.com/gobuffalo/packr/v2/packr2 && cd -

Make sure your $PATH contains $GOPATH/bin, otherwise the packr utility will not be found.

Now build the node executable:

cd ../../db && packr2 && cd -
go build .
cd ../../db && packr2 clean && cd -

The executable is node.

Usage Examples

The following commands assume you have built the node previously. You can also run the following examples by replacing ./node with go run . and executing them in the cli/node directory to build from source and run at the same time.

Run the node in mode synchronizer:

./node --mode sync --cfg cfg.buidler.toml run

Run the node in mode coordinator:

./node --mode coord --cfg cfg.buidler.toml run

Import an ethereum private key into the keystore:

./node --mode coord --cfg cfg.buidler.toml importkey --privatekey  0x618b35096c477aab18b11a752be619f0023a539bb02dd6c813477a6211916cde

Generate a new BabyJubJub key pair:

./node --mode coord --cfg cfg.buidler.toml genbjj

Wipe the entier SQL database (this will destroy all synchronized and pool data):

./node --mode coord --cfg cfg.buidler.toml wipesql