36 Commits

Author SHA1 Message Date
arnaucube
93b7866d3e Add lll to linter checks
Updated:
	batchbuilder
	common
	coordinator
	db/statedb
	eth
	log
	node
	priceupdater
	prover
	synchronizer
	test/*
	txprocessor
	txselector

Pending (once
https://github.com/hermeznetwork/hermez-node/tree/feature/serveapicli is
merged to master):
	Update golangci-lint version to v1.37.1
	api
	apitypes
	cli
	config
	db/historydb
	db/l2db
2021-03-01 18:02:10 +01:00
Danilo Pantani
72862147f3 fix possible runtime error 2021-02-22 13:44:36 -03:00
Eduard S
f0e79f3d55 Update coordinator to work better under real net
- cli / node
    - Update handler of SIGINT so that after 3 SIGINTs, the process terminates
      unconditionally
- coordinator
    - Store stats without pointer
    - In all functions that send a variable via channel, check for context done
      to avoid deadlock (due to no process reading from the channel, which has
      no queue) when the node is stopped.
    - Abstract `canForge` so that it can be used outside of the `Coordinator`
    - In `canForge` check the blockNumber in current and next slot.
    - Update tests due to smart contract changes in slot handling, and minimum
      bid defaults
    - TxManager
        - Add consts, vars and stats to allow evaluating `canForge`
        - Add `canForge` method (not used yet)
        - Store batch and nonces status (last success and last pending)
        - Track nonces internally instead of relying on the ethereum node (this
          is required to work with ganache when there are pending txs)
        - Handle the (common) case of the receipt not being found after the tx
          is sent.
        - Don't start the main loop until we get an initial messae fo the stats
          and vars (so that in the loop the stats and vars are set to
          synchronizer values)
        - When a tx fails, check and discard all the failed transactions before
          sending the message to stop the pipeline.  This will avoid sending
          consecutive messages of stop the pipeline when multiple txs are
          detected to be failed consecutively.  Also, future txs of the same
          pipeline after a discarded txs are discarded, and their nonces reused.
        - Robust handling of nonces:
            - If geth returns nonce is too low, increase it
            - If geth returns nonce too hight, decrease it
            - If geth returns underpriced, increase gas price
            - If geth returns replace underpriced, increase gas price
        - Add support for resending transactions after a timeout
        - Store `BatchInfos` in a queue
    - Pipeline
        - When an error is found, stop forging batches and send a message to the
          coordinator to stop the pipeline with information of the failed batch
          number so that in a restart, non-failed batches are not repated.
        - When doing a reset of the stateDB, if possible reset from the local
          checkpoint instead of resetting from the synchronizer.  This allows
          resetting from a batch that is valid but not yet sent / synced.
    - Every time a pipeline is started, assign it a number from a counter.  This
      allows the TxManager to ignore batches from stopped pipelines, via a
      message sent by the coordinator.
    - Avoid forging when we haven't reached the rollup genesis block number.
    - Add config parameter `StartSlotBlocksDelay`: StartSlotBlocksDelay is the
      number of blocks of delay to wait before starting the pipeline when we
      reach a slot in which we can forge.
    - When detecting a reorg, only reset the pipeline if the batch from which
      the pipeline started changed and wasn't sent by us.
    - Add config parameter `ScheduleBatchBlocksAheadCheck`:
      ScheduleBatchBlocksAheadCheck is the number of blocks ahead in which the
      forger address is checked to be allowed to forge (apart from checking the
      next block), used to decide when to stop scheduling new batches (by
      stopping the pipeline).  For example, if we are at block 10 and
      ScheduleBatchBlocksAheadCheck is 5, eventhough at block 11 we canForge,
      the pipeline will be stopped if we can't forge at block 15.  This value
      should be the expected number of blocks it takes between scheduling a
      batch and having it mined.
    - Add config parameter `SendBatchBlocksMarginCheck`:
      SendBatchBlocksMarginCheck is the number of margin blocks ahead in which
      the coordinator is also checked to be allowed to forge, apart from the
      next block; used to decide when to stop sending batches to the smart
      contract.  For example, if we are at block 10 and
      SendBatchBlocksMarginCheck is 5, eventhough at block 11 we canForge, the
      batch will be discarded if we can't forge at block 15.
    - Add config parameter `TxResendTimeout`: TxResendTimeout is the timeout
      after which a non-mined ethereum transaction will be resent (reusing the
      nonce) with a newly calculated gas price
    - Add config parameter `MaxGasPrice`: MaxGasPrice is the maximum gas price
      allowed for ethereum transactions
    - Add config parameter `NoReuseNonce`: NoReuseNonce disables reusing nonces
      of pending transactions for new replacement transactions.  This is useful
      for testing with Ganache.
    - Extend BatchInfo with more useful information for debugging

- eth / ethereum client
    - Add necessary methods to create the auth object for transactions manually
      so that we can set the nonce, gas price, gas limit, etc manually
    - Update `RollupForgeBatch` to take an auth object as input (so that the
      coordinator can set parameters manually)
- synchronizer
    - In stats, add `NextSlot`
    - In stats, store full last batch instead of just last batch number
    - Instead of calculating a nextSlot from scratch every time, update the
      current struct (only updating the forger info if we are Synced)
    - Afer every processed batch, check that the calculated StateDB MTRoot
      matches the StateRoot found in the forgeBatch event.
2021-02-16 16:10:23 +01:00
Eduard S
48a538faa3 Pass StateDB constructor parameters as Config type
- KVDB/StateDB
        - Pass config parameters in a Config type instead of using many
          arguments in constructor.
	- Add new parameter `NoLast` which disables having an opened DB with a
	  checkpoint to the last batchNum for thread-safe reads.  Last will be
	  disabled in the StateDB used by the TxSelector and BatchBuilder.
	- Add new parameter `NoGapsCheck` which skips checking gaps in the list
	  of checkpoints and returning errors if there are gaps.  Gaps check
	  will be disabled in the StateDB used by the TxSelector and
	  BatchBuilder, because we expect to have gaps when there are multiple
	  coordinators forging (slots not forged by our coordinator will leave
	  gaps).
2021-02-08 13:46:24 +01:00
arnaucube
7f46c3028e Add tests connecting TxSelector, BatchBuilder, ZKInputs, ProofServer
- Add tests connecting TxSelector, BatchBuilder, ZKInputs, ProofServer
- Added test to check that the signatures of the PoolL2Txs from the L2DB
pool can be verified, to check that the parameters of the PoolL2Tx match
the original parameters signed before inserting them into the L2DB
2021-01-18 17:52:14 +01:00
Eduard S
18c854fbaa Add circuit configuration to node config
- Remove MaxL1CoordTxs parameter from the TxSelector because this parameter
  doesn't exist
- Use ChainID in l1tx byte encoding
- Pass txprocessor configuration to batch builder via an existing parameter
2020-12-31 10:44:19 +01:00
arnaucube
0cf1ed217b Abstract TxProcessor from StateDB
- Abstract TxProcessor from StateDB
- Upgrade to last version of go-merkletree for the key-value DB usage
2020-12-29 11:14:38 +01:00
Eduard S
2205fcadbc Delete old checkpoints in stateDB automatically
Introduce a constructor parameter for the StateDB called `keep`, which tells
how many checkpoints to keep.  When doing a new checkpoint, if the number of
existing checkpoints exeeds `keep`, the oldest ones will be deleted.
2020-12-24 13:35:47 +01:00
arnaucube
150597c282 Parametrize ChainID 2020-12-23 15:23:48 +01:00
Eduard S
6a9b5ce420 Extend Coordinator tests 2020-12-16 13:13:40 +01:00
arnaucube
593a477e6c ZKInputs generation ISFinalAccFee update
- Update IntermediateState FinalAccFee values to fit in the Circom circuit expected inputs
- Add ZKInputs generation test with transactions generated by Til
(TestZKInputs6)
- BatchBuilder: remove redundant MakeCheckpoint call
2020-12-15 11:49:09 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
arnaucube
ce772b1d19 Add HashGlobalInputs for ZKInputs
Add HashGlobalInputs for ZKInputs compatible with js & circom circuits version.

Compatible with hermeznetwork/commonjs at version: c6a1448db5bae4cda839ce36c1f35d8defccc9cd
(c6a1448db5)
2020-11-17 16:33:29 +01:00
arnaucube
e09092732e Group ProcessTxs output 2020-10-27 13:48:07 +01:00
arnaucube
0ff3fb5ae7 ProcessTxs return an array of created accounts 2020-10-27 12:50:16 +01:00
arnaucube
8f24aa93c9 Add ProcessTxs send fees to Coordinator accounts 2020-10-27 10:28:32 +01:00
Eduard S
88ddbbcddc Delete tmp folders from tests, Resolve #105 2020-10-20 13:13:10 +02:00
Eduard S
0277210c39 Extend ethclient test, implement new TxID spec
- Implement new TxID spec that distinguishes L1UserTx and L1CoordinatorTx
- Replace some type []*Foo by []Foo
- Fix HistoryDB & L2DB bug: in case of error, a rollback was applied and the returned error was nil
- Reorder inserts in historydb.NewHistoryDB() to follow foreign key dependencies
- Add initial synchronizer test with test.Client (for now, only tested l1UserTxs, blocks, addToken)
- Update L1UserTx event in test.Client
2020-10-07 16:41:51 +02:00
arnaucube
3374a4754d Add StateDB compute nonces on ProcessTx L2Tx
Add StateDB compute nonces on ProcessTx L2Tx, and update StateDB for
type TypeSynchronizer, TypeTxSelector, TypeBatchBuilder
2020-10-02 11:54:58 +02:00
arnaucube
aa0bde61d2 Add StateDB ZKInputs generation for L1 & L2 Txs 2020-09-09 14:40:55 +02:00
arnaucube
26dbb16618 Add Coordinator goroutines & channels processes
Add Coordinator goroutines & channels processes
- Add Coordinator goroutines & channels
- Add Coordinator test to debug batches
- Add MakeCheckpoint call after BatchBuilder BuildBatch process
- Update ethClient BlockByNumber to return hermez/common.Block instead of go-ethereum/core/types.Block
2020-08-27 10:56:08 +02:00
arnaucube
0d2004721c Add statedb ExitTree implementation 2020-08-25 12:09:49 +02:00
arnaucube
b4044a2ef7 Add abstraction method of processTxs to StateDB
- Update GHA lint.yml increasing timeout time to avoid GHA Lint errors
- Update common.BatchNum & common.Idx & common.Nonce usage in StateDB
- Add abstraction method of processTxs to StateDB
    - Which will be used by Synchronizer & BatchBuilder
2020-08-24 13:06:51 +02:00
arnaucube
cfa441e1ac Migrate TxProcessors from BatchBuilder to StateDB
Migrate TxProcessors from BatchBuilder to StateDB in order to be used by
BatchBuilder & Synchronizer
2020-08-20 12:06:28 +02:00
arnaucube
c702442287 Update Coordinator & BatchBuilder & TxSelector to new SQL schemas changes 2020-08-17 16:16:25 +02:00
a_bennassar
cb1b820256 Update SQL schemas 2020-08-17 16:16:17 +02:00
arnaucube
aa003063f6 Update BatchBuilder & TxSelector to new StateDB 2020-08-17 13:38:48 +02:00
arnaucube
8390e4a784 Add initial structure Coordinator Forge Sequence 2020-08-14 14:06:28 +02:00
arnaucube
a1c339c918 Add BatchQueue implementation & minor updates 2020-08-13 18:25:50 +02:00
arnaucube
60fa76b4d2 Integrate TxSelector with StateDB 2020-08-13 13:07:30 +02:00
arnaucube
350bac631e Integrate BatchBuilder with StateDB 2020-08-11 16:44:51 +02:00
arnaucube
d8cb7298e0 Add StateDB & LocalStateDB 2020-08-07 10:59:30 +02:00
arnaucube
16555b51e3 Update Leaf to new spec (Sign+Ay) 2020-08-05 16:06:31 +02:00
arnaucube
bcd9f96e1b Add batchbuilder applyCreateLeaf, applyDeposit, applyTransfer 2020-08-05 15:31:27 +02:00
arnaucube
37cefb0057 Add common> Idx parsers 2020-08-05 15:26:15 +02:00
arnaucube
f3cfba1bbe Add initial structure of BatchBuilder 2020-08-05 15:22:51 +02:00