Commit Graph

124 Commits

Author SHA1 Message Date
Eduard S
c829eb99dc Remove unecessary slice indexing 2021-02-19 11:00:36 +01:00
Eduard S
f0e79f3d55 Update coordinator to work better under real net
- cli / node
    - Update handler of SIGINT so that after 3 SIGINTs, the process terminates
      unconditionally
- coordinator
    - Store stats without pointer
    - In all functions that send a variable via channel, check for context done
      to avoid deadlock (due to no process reading from the channel, which has
      no queue) when the node is stopped.
    - Abstract `canForge` so that it can be used outside of the `Coordinator`
    - In `canForge` check the blockNumber in current and next slot.
    - Update tests due to smart contract changes in slot handling, and minimum
      bid defaults
    - TxManager
        - Add consts, vars and stats to allow evaluating `canForge`
        - Add `canForge` method (not used yet)
        - Store batch and nonces status (last success and last pending)
        - Track nonces internally instead of relying on the ethereum node (this
          is required to work with ganache when there are pending txs)
        - Handle the (common) case of the receipt not being found after the tx
          is sent.
        - Don't start the main loop until we get an initial messae fo the stats
          and vars (so that in the loop the stats and vars are set to
          synchronizer values)
        - When a tx fails, check and discard all the failed transactions before
          sending the message to stop the pipeline.  This will avoid sending
          consecutive messages of stop the pipeline when multiple txs are
          detected to be failed consecutively.  Also, future txs of the same
          pipeline after a discarded txs are discarded, and their nonces reused.
        - Robust handling of nonces:
            - If geth returns nonce is too low, increase it
            - If geth returns nonce too hight, decrease it
            - If geth returns underpriced, increase gas price
            - If geth returns replace underpriced, increase gas price
        - Add support for resending transactions after a timeout
        - Store `BatchInfos` in a queue
    - Pipeline
        - When an error is found, stop forging batches and send a message to the
          coordinator to stop the pipeline with information of the failed batch
          number so that in a restart, non-failed batches are not repated.
        - When doing a reset of the stateDB, if possible reset from the local
          checkpoint instead of resetting from the synchronizer.  This allows
          resetting from a batch that is valid but not yet sent / synced.
    - Every time a pipeline is started, assign it a number from a counter.  This
      allows the TxManager to ignore batches from stopped pipelines, via a
      message sent by the coordinator.
    - Avoid forging when we haven't reached the rollup genesis block number.
    - Add config parameter `StartSlotBlocksDelay`: StartSlotBlocksDelay is the
      number of blocks of delay to wait before starting the pipeline when we
      reach a slot in which we can forge.
    - When detecting a reorg, only reset the pipeline if the batch from which
      the pipeline started changed and wasn't sent by us.
    - Add config parameter `ScheduleBatchBlocksAheadCheck`:
      ScheduleBatchBlocksAheadCheck is the number of blocks ahead in which the
      forger address is checked to be allowed to forge (apart from checking the
      next block), used to decide when to stop scheduling new batches (by
      stopping the pipeline).  For example, if we are at block 10 and
      ScheduleBatchBlocksAheadCheck is 5, eventhough at block 11 we canForge,
      the pipeline will be stopped if we can't forge at block 15.  This value
      should be the expected number of blocks it takes between scheduling a
      batch and having it mined.
    - Add config parameter `SendBatchBlocksMarginCheck`:
      SendBatchBlocksMarginCheck is the number of margin blocks ahead in which
      the coordinator is also checked to be allowed to forge, apart from the
      next block; used to decide when to stop sending batches to the smart
      contract.  For example, if we are at block 10 and
      SendBatchBlocksMarginCheck is 5, eventhough at block 11 we canForge, the
      batch will be discarded if we can't forge at block 15.
    - Add config parameter `TxResendTimeout`: TxResendTimeout is the timeout
      after which a non-mined ethereum transaction will be resent (reusing the
      nonce) with a newly calculated gas price
    - Add config parameter `MaxGasPrice`: MaxGasPrice is the maximum gas price
      allowed for ethereum transactions
    - Add config parameter `NoReuseNonce`: NoReuseNonce disables reusing nonces
      of pending transactions for new replacement transactions.  This is useful
      for testing with Ganache.
    - Extend BatchInfo with more useful information for debugging

- eth / ethereum client
    - Add necessary methods to create the auth object for transactions manually
      so that we can set the nonce, gas price, gas limit, etc manually
    - Update `RollupForgeBatch` to take an auth object as input (so that the
      coordinator can set parameters manually)
- synchronizer
    - In stats, add `NextSlot`
    - In stats, store full last batch instead of just last batch number
    - Instead of calculating a nextSlot from scratch every time, update the
      current struct (only updating the forger info if we are Synced)
    - Afer every processed batch, check that the calculated StateDB MTRoot
      matches the StateRoot found in the forgeBatch event.
2021-02-16 16:10:23 +01:00
arnaucube
63151a285c Migrate all packages to use Float40
Migrate all packages to use Float40 & Add more test vectors at common
2021-02-12 14:27:07 +01:00
Arnau B
ac1fd9acf7 Add semaphore for API queries to SQL 2021-02-10 13:36:17 +01:00
arnaucube
c1cd37913f Update TxID to use eth Keccak256
- Update TxID to use eth Keccak256
- Added more tests to L2Tx TxID calculation to check compatibility with
js version from https://github.com/hermeznetwork/hermezjs/pull/57
2021-02-01 16:01:55 +01:00
arnaucube
aade6220d4 Update TxID to avoid collisions on DB (fix #503) 2021-01-29 11:38:22 +01:00
arnaucube
a899f43914 Add TokenID to L2Tx
L2Tx.TokenID is not on the data obtained by the Synchronizer from the
blockchain, but is set by the TxProcessor when processing the
transactions in the StateDB.
2021-01-27 12:40:09 +01:00
Eduard S
ec194d5066 Set l1tx.EffectiveFromIdx in TxProcessor and til 2021-01-13 18:20:47 +01:00
Arnau B
50fc607306 Add idx for txs that creat account, step1 2021-01-13 18:16:32 +01:00
ToniRamirezM
170da500db Remove big.Int from state API 2021-01-12 18:37:56 +01:00
ToniRamirezM
ce6a271448 Remove bigints from API 2021-01-12 11:05:13 +01:00
a_bennassar
65a1057980 Merge pull request #473 from hermeznetwork/feature/buckets-update
Update buckets withdrawals on API
2021-01-11 15:43:00 +01:00
ToniRamirezM
0bb69c3f3e Update buckets withdrawals on API 2021-01-11 15:21:50 +01:00
Eduard S
2ae936f2bd Merge pull request #469 from hermeznetwork/feature/swagger-imprvmnts
Swagger improvements
2021-01-08 17:57:26 +01:00
Arnau B
b4d8e68730 Swagger improvements 2021-01-08 15:47:44 +01:00
Eduard S
423b67b707 Merge pull request #468 from hermeznetwork/fix/repeated-bids
Fix repeated items when coordinator is updated
2021-01-07 17:06:16 +01:00
Arnau B
c5e3fdad75 Fix repeated items when coordinator is updated 2021-01-07 16:30:36 +01:00
ToniRamirezM
e2d243a9b5 Sanitize varchars that may not be UTF-8 2021-01-04 13:06:53 +01:00
a_bennassar
926943cb71 Merge pull request #447 from hermeznetwork/feature/change-queries
Change queries that produced undesired meddler logs
2021-01-04 10:31:21 +01:00
a_bennassar
65abdd408a Merge pull request #442 from hermeznetwork/feature/new-metrics-tests
Added new metrics tests
2021-01-04 10:23:17 +01:00
ToniRamirezM
3e70551ac5 Added new metrics tests 2020-12-31 12:29:49 +01:00
ToniRamirezM
64b214061c Change queries that produced undesired meddler logs 2020-12-30 19:12:53 +01:00
Arnau B
a32f75db78 empty array instead of 404 2020-12-30 12:41:53 +01:00
Arnau B
6cd292b937 Include addrs in exit responses 2020-12-29 17:35:20 +01:00
Eduard S
890b94a41a Merge pull request #428 from hermeznetwork/fix/api-next-forgers
Fix next forgers api response
2020-12-29 15:32:02 +01:00
Arnau B
6058d3c41a Fix next forgers api response 2020-12-29 15:17:55 +01:00
Eduard S
02c2ee155a Fix item_id order in forged l1UserTxs
Also, make sure that all SQL queries that return slices are sorted
2020-12-29 12:28:53 +01:00
Eduard S
8267d007c9 Sync ForgerCommitment and use it in coord
Previously the coordinator was erroneously using Slot.BatchesLen to determine
when anyone can forge.  The correct behaviour is implmenented with the boolean
flag `ForgerCommitment`, that is set to true only when there's a batch before
the deadline within a slot.

Delete Slot.BatchesLen, and the synchronization code of this value from the
Synchronizer, as this is not needed
2020-12-23 17:06:20 +01:00
arnaucube
150597c282 Parametrize ChainID 2020-12-23 15:23:48 +01:00
Eduard S
adc044001f Merge pull request #412 from hermeznetwork/fix/sdb-nonce
Fix StateDB & Til Nonce increment
2020-12-23 12:47:40 +01:00
arnaucube
eea44b3990 Fix StateDB & Til Nonce increment 2020-12-23 12:32:41 +01:00
Eduard S
fb48ca4e58 Merge pull request #411 from hermeznetwork/feature/forged-txs-in-batchesapi
Add forged txs in batch endpoints response
2020-12-23 12:07:45 +01:00
Arnau B
6e7f860d61 Add forged txs in batch endpoints response 2020-12-23 11:26:23 +01:00
Arnau B
bc5fa92898 Add bootCoord to coord table, get coord by forgerAddr 2020-12-23 11:03:21 +01:00
arnaucube
4b10549822 mv of babyjub.PublicKey to babyjub.PublicKeyComp
Update usage of `*babyjub.PublicKey` to `babyjub.PublicKeyComp`
- when the key is not defined, internally is used `babyjub.EmptyBJJComp`, which is a `[32]byte` of zeroes of type `babyjub.PublicKeyComp`
- the API continues returning `nil` when the key is not defined
2020-12-22 11:29:47 +01:00
laisolizq
bc29222341 Fix historyDB GetBestBidCoordinator 2020-12-18 17:47:41 +01:00
Eduard S
8a59296cb8 Fix order of L1UserTxs
When the synchronizer queries the unforged L1UserTxs, sort them by position

Swap the order of calls setL1UserTxEffectiveAmounts and addBatch in
AddBlockSCData because otherwise, for reasons I dont understand, the item_id of
the txs doesn't follow the position of the txs.
2020-12-17 15:48:01 +01:00
Eduard S
12fba96eee Merge pull request #366 from hermeznetwork/fix/api-exit-response
Fix format of the MTP for exit endpoint
2020-12-17 11:02:31 +01:00
Arnau B
8083e4fec3 Fix format of the MTP for exit endpoint 2020-12-16 20:09:17 +01:00
Eduard S
7a30294688 Update txs constructors and helpers
For each tx, move the logic of setting the Type and TxID to separate functions,
so that they can be called when necessary.

In synchronizer, set all the required fields using the `SetID` and `SetType`
for l2txs when needed.  This is necessary because the `ProcessTxs()` works with
`common.PoolL2Tx`, which misses some fields from `common.L2Tx`, and because
`ProcessTxs()` needs the Type to be set, but at the same time `ProcessTxs()`
sets the Nonce, which is required for the `TxID`.
2020-12-16 15:25:55 +01:00
Arnau B
c7b7a85b85 Fix queries when no rows 2020-12-15 16:15:42 +01:00
Eduard S
20b8d0561f Update synchronizer and DB with last contracts updates
- API
	- When updating network info, handle cases where no batches exists and
	  where no forgers exists
- cli/node
	- Update `cfg.buidler.toml` config file to a working version
- common
	- Add new smart contract structs and extend some existing ones to
	  reflect updates regarding events from the smart contracts
- SQL
	- Add new tables and extend existing ones to reflect updates regarding
	  events from the smart contracts
- db/historydb
	- Add functions to insert new smart contract events data
	- Fix unclosed rows that led to inconsistent sql driver state (replace
	  NamedQuery by NamedExec).  This fixes the error:
	  `pq: unexpected Parse response 'C'`
- db/l2db
	- Close rows after usage
- eth
	- In Rollup event, introduce a new UpdateBucketsParameter when there's a
	  SafeMode event, with `SafeMode = true`
- synchronizer
	- synchronize new events
	- avoid calling `auction.CanForge` before the genesisBlock to avoid
	  getting a revert.
2020-12-10 17:45:17 +01:00
arnaucube
9d0b278d50 Update missing LoadAmount to DepositAmount
Related to 445f26ec9f
2020-12-09 18:34:01 +01:00
Arnau B
e1d34e3796 Add amount success and load amount success 2020-12-09 16:39:18 +01:00
Arnau B
445f26ec9f Rename load amount to deposit amount 2020-12-04 16:13:41 +01:00
Eduard S
897d7d5319 Merge pull request #326 from hermeznetwork/feature/effective-amount-update
Feature/effective amount update
2020-12-04 12:33:30 +01:00
Arnau B
5fd258ee07 Use flags for effective amounts in SQL schema 2020-12-04 12:17:09 +01:00
Eduard S
329d2796c6 Update some eth types, update eth dependencies to new changes 2020-12-03 18:05:33 +01:00
Eduard S
665e0829ec Set EffectiveAmount and EffectiveLoadAmount of forged L1UserTxs 2020-11-30 16:25:54 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00