Commit Graph

68 Commits

Author SHA1 Message Date
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
Eduard S
8f1cf2f145 Update coordinator, call all api update functions
- Common:
	- Rename Block.EthBlockNum to Block.Num to avoid unneeded repetition
- API:
	- Add UpdateNetworkInfoBlock to update just block information, to be
	  used when the node is not yet synchronized
- Node:
	- Call API.UpdateMetrics and UpdateRecommendedFee in a loop, with
	  configurable time intervals
- Synchronizer:
	- When mapping events by TxHash, use an array to support the possibility
	  of multiple calls of the same function happening in the same
	  transaction (for example, a smart contract in a single transaction
	  could call withdraw with delay twice, which would generate 2 withdraw
	  events, and 2 deposit events).
	- In Stats, keep entire LastBlock instead of just the blockNum
	- In Stats, add lastL1BatchBlock
	- Test Stats and SCVars
- Coordinator:
	- Enable writing the BatchInfo in every step of the pipeline to disk
	  (with JSON text files) for debugging purposes.
	- Move the Pipeline functionality from the Coordinator to its own struct
	  (Pipeline)
	- Implement shouldL1lL2Batch
	- In TxManager, implement logic to perform several attempts when doing
	  ethereum node RPC calls before considering the error. (Both for calls
	  to forgeBatch and transaction receipt)
	- In TxManager, reorganize the flow and note the specific points in
	  which actions are made when err != nil
- HistoryDB:
	- Implement GetLastL1BatchBlockNum: returns the blockNum of the latest
	  forged l1Batch, to help the coordinator decide when to forge an
	  L1Batch.
- EthereumClient and test.Client:
	- Update EthBlockByNumber to return the last block when the passed
	  number is -1.
2020-11-26 18:06:30 +01:00
Eduard S
3b99953007 Redo coordinator structure, connect API to node
- API:
	- Modify the constructor so that hardcoded rollup constants don't need
	  to be passed (introduce a `Config` and use `configAPI` internally)
- Common:
	- Update rollup constants with proper *big.Int when required
	- Add BidCoordinator and Slot structs used by the HistoryDB and
	  Synchronizer.
	- Add helper methods to AuctionConstants
	- AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL
	  table: `default_slot_set_bid_slot_num`), which indicates at which
	  slotNum does the `DefaultSlotSetBid` specified starts applying.
- Config:
	- Move coordinator exclusive configuration from the node config to the
	  coordinator config
- Coordinator:
	- Reorganize the code towards having the goroutines started and stopped
	  from the coordinator itself instead of the node.
	- Remove all stop and stopped channels, and use context.Context and
	  sync.WaitGroup instead.
	- Remove BatchInfo setters and assing variables directly
	- In ServerProof and ServerProofPool use context instead stop channel.
	- Use message passing to notify the coordinator about sync updates and
	  reorgs
	- Introduce the Pipeline, which can be started and stopped by the
	  Coordinator
	- Introduce the TxManager, which manages ethereum transactions (the
	  TxManager is also in charge of making the forge call to the rollup
	  smart contract).  The TxManager keeps ethereum transactions and:
	  	1. Waits for the transaction to be accepted
		2. Waits for the transaction to be confirmed for N blocks
	- In forge logic, first prepare a batch and then wait for an available
	  server proof to have all work ready once the proof server is ready.
	- Remove the `isForgeSequence` method which was querying the smart
	  contract, and instead use notifications sent by the Synchronizer to
	  figure out if it's forging time.
	- Update test (which is a minimal test to manually see if the
	  coordinator starts)
- HistoryDB:
	- Add method to get the number of batches in a slot (used to detect when
	  a slot has passed the bid winner forging deadline)
	- Add method to get the best bid and associated coordinator of a slot
	  (used to detect the forgerAddress that can forge the slot)
- General:
	- Rename some instances of `currentBlock` to `lastBlock` to be more
	  clear.
- Node:
	- Connect the API to the node and call the methods to update cached
	  state when the sync advances blocks.
	- Call methods to update Coordinator state when the sync advances blocks
	  and finds reorgs.
- Synchronizer:
	- Add Auction field in the Stats, which contain the current slot with
	  info about highest bidder and other related info required to know who
	  can forge in the current block.
	- Better organization of cached state:
		- On Sync, update the internal cached state
		- On Init or Reorg, load the state from HistoryDB into the
		  internal cached state.
2020-11-24 18:59:30 +01:00
Eduard S
6c0d48f4a2 Add Sync stats, and report them in DebugAPI 2020-11-16 11:48:36 +01:00
Eduard S
5b6639a947 Add DebugAPI to Node, fix StateDB
- Allow starting the DebugAPI from the node via config
- In StateDB:
    - Make checkpoints when ProcessTxs() succeeds
    - Remove extra hardcoded `statedb` path that was redundant
    - Replace hardcoded `[:4]` by `[:]` when parsing idx, which failed because
      idx is 6 bytes length now.
- Extra: In node, use waitgroup instead of `stoppedXXX` channels to wait for
syncrhonizer goroutines to finish.
2020-11-06 13:00:11 +01:00
Eduard S
6e4b9b4b70 Update Synchronizer (node) implementation
- node:
    - Extend config to add initial variables of the smart contracts used as
      defaults before they are changed via events.
    - In stopped channels, set size 1 so that panics are not witheld until the
      node stops completely.
- common:
    - In Smart Contract variables, comment:
      - `WDelayerVariables.HermezRollupAddress` because it's not needed.
      - `RollupVariables.Buckets` because there are no events for it, and for
        now it's not used.
- historydb:
    - Add functions to get and set smart contract variables.
- db:
    - Add `Rollback` function in `utils.go` to reduce boilerplate in sql
      transaction rollbacks in defers in db functions.
    - Update `rollup_vars` and `auction_vars` (renamed from `consensus_vars`)
      table, and add `wdelayer_vars` table.
- synchronizer:
    - Synchronize WDelayer
    - Handle SC variables properly
- test/ethclient:
    - Add essential implementation of WDelayer
2020-11-04 15:12:51 +01:00
Eduard S
e6fb0a03de Reorganize smart contract types, udate eth tests, etc.
- Move smart contract constants and structs for variables to
  common/{ethrollup.go, ethauction.go, ethwdelayer.go}:
    - This removes repeated code of the structs for variables
    - Allows reusing the constants and variables from all modules without
      import cycles
- Remove unused common/scvars.go
- In common.BlockData, split data from each smart contract into a sepparate
  field (Rollup, Auction, WDelayer).  This affects the structures that til uses
  as output, and HistoryDB in the AddBlockSCData.
- In Synchronizer:
    - Pass starting block of each smart contract as config, instead of
      incorrectly using the genesis block found in the acution constant (which
      has a very different meaning)
    - Use variable structs from common instead of an internal copy
    - Synchronize more stuff (resolve some TODOs)
    - Fix some issues found after initial testing with ganache
- In eth:
    - In auction.go: Add method to get constants
    - Update README to use ganache instead of buidlerevm as local blockchain
      for testing
    - Update env variables and test vectors to pass the tests with the
      deployment in the ganache testnet.
    - Use ethereum keys derived from paths (hdwallet) in testing to avoid
      hardcoding private keys and generate the same keys from a mnemonic used
      in the ganache tesnet.
2020-11-04 12:06:28 +01:00
Eduard S
79b105b497 Update importers of eth, review eth/README.md 2020-10-26 16:09:40 +01:00
Eduard S
827e917fa0 Update missing parts, improve til, and more
- Node
	- Updated configuration to initialize the interface to all the smart
	  contracts
- Common
	- Moved BlockData and BatchData types to common so that they can be
	  shared among: historydb, til and synchronizer
	- Remove hash.go (it was never used)
	- Remove slot.go (it was never used)
	- Remove smartcontractparams.go (it was never used, and appropriate
	  structs are defined in `eth/`)
	- Comment state / status method until requirements of this method are
	  properly defined, and move it to Synchronizer
- Synchronizer
	- Simplify `Sync` routine to only sync one block per call, and return
	  useful information.
	- Use BlockData and BatchData from common
	- Check that events belong to the expected block hash
	- In L1Batch, query L1UserTxs from HistoryDB
	- Fill ERC20 token information
	- Test AddTokens with test.Client
- HistryDB
	- Use BlockData and BatchData from common
	- Add `GetAllTokens` method
	- Uncomment and update GetL1UserTxs (with corresponding tests)
- Til
	- Rename all instances of RegisterToken to AddToken (to follow the smart
	  contract implementation naming)
	- Use BlockData and BatchData from common
		- Move testL1CoordinatorTxs and testL2Txs to a separate struct
		  from BatchData in Context
	- Start Context with BatchNum = 1 (which the protocol defines to be the
	  first batchNum)
	- In every Batch, set StateRoot and ExitRoot to a non-nil big.Int
	  (zero).
	- In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not
	  used, set it to 0; so that no *big.Int is nil.
	- In L1UserTx, don't set BatchNum, because when L1UserTxs are created
	  and obtained by the synchronizer, the BatchNum is not known yet (it's
	  a synchronizer job to set it)
	- In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
2020-10-22 15:20:51 +02:00
Eduard S
a2bda1890d Simplify eth code 2020-10-21 11:56:55 +02:00
Eduard S
0277210c39 Extend ethclient test, implement new TxID spec
- Implement new TxID spec that distinguishes L1UserTx and L1CoordinatorTx
- Replace some type []*Foo by []Foo
- Fix HistoryDB & L2DB bug: in case of error, a rollback was applied and the returned error was nil
- Reorder inserts in historydb.NewHistoryDB() to follow foreign key dependencies
- Add initial synchronizer test with test.Client (for now, only tested l1UserTxs, blocks, addToken)
- Update L1UserTx event in test.Client
2020-10-07 16:41:51 +02:00
Eduard S
9e4f020c2b Merge pull request #171 from hermeznetwork/feature/processtx-l2tx-nonces
Add StateDB compute nonces on ProcessTx L2Tx
2020-10-02 11:59:31 +02:00
arnaucube
3374a4754d Add StateDB compute nonces on ProcessTx L2Tx
Add StateDB compute nonces on ProcessTx L2Tx, and update StateDB for
type TypeSynchronizer, TypeTxSelector, TypeBatchBuilder
2020-10-02 11:54:58 +02:00
Eduard S
0fc4930652 Update test ethclient and coordinator 2020-10-01 11:34:59 +02:00
a_bennassar
c6f70f3177 Feature/merge history l2 tables (#156)
* WIP rebase

* Combine both SQL DBs

* API and DB refactor
2020-09-29 18:27:07 +02:00
Eduard S
852cd762d3 Add reverts, forge in test ethClient, update auction 2020-09-15 12:47:27 +02:00
Eduard S
475614cc3a Implement first iteration of node
The coordinator implementation has been refactored to allow all the goroutines
to be handled from the node.
2020-09-10 11:36:58 +02:00
arnaucube
8390e4a784 Add initial structure Coordinator Forge Sequence 2020-08-14 14:06:28 +02:00