Commit Graph

315 Commits

Author SHA1 Message Date
Eduard S
6260dfedad Fix eth events query and sync inconsistent state
- kvdb
	- Fix path in Last when doing `setNew`
	- Only close if db != nil, and after closing, always set db to nil
		- This will avoid a panic in the case where the db is closed but
		  there's an error soon after, and a future call tries to close
		  again.  This is because pebble.Close() will panic if the db is
		  already closed.
	- Avoid calling pebble methods when a the Storage interface already
	  implements that method (like Close).
- statedb
	- In test, avoid calling KVDB method if the same method is available for
	  the StateDB (like MakeCheckpoint, CurrentBatch).
- eth
	- In *EventByBlock methods, take blockHash as input argument and use it
	  when querying the event logs.  Previously the blockHash was only taken
	  from the logs results *only if* there was any log.  This caused the
	  following issue: if there was no logs, it was not possible to know if
	  the result was from the expected block or an uncle block!  By querying
	  logs by blockHash we make sure that even if there are no logs, they
	  are from the right block.
	  	- Note that now the function can either be called with a
		  blockNum or blockHash, but not both at the same time.
- sync
	- If there's an error during call to Sync call resetState, which
	  internally resets the stateDB to avoid stale checkpoints (and a
	  corresponding invalid increase in the StateDB batchNum).
	- During a Sync, after very batch processed, make sure that the StateDB
	  currentBatch corresponds to the batchNum in the smart contract
	  log/event.
2021-02-05 16:06:17 +01:00
Eduard S
6590c47a9a Add Last db view in kvdb and statedb
Last db view is an opened pebble db which always contains a checkpoint from the
last batch.  Methods to access this last batch are thread safe so that views of
the last checkpoint can be made anywhere and with a consistent view of the
state.
2021-02-04 18:02:53 +01:00
a_bennassar
ca7fa8ae2e Merge pull request #523 from hermeznetwork/feature/pooll2tx-info
Add PoolL2Tx.Info about the status of the tx
2021-02-03 13:58:29 +01:00
arnaucube
a8f6891aea Add PoolL2Tx.Info about the status of the tx
PoolL2Tx.Info contains information about the status & State of the
transaction. As for example, if the Tx has not been selected in the last
batch due not enough Balance at the Sender account, this reason would
appear at this parameter.
This will help the client (wallet, batchexplorer, etc) to reason why a
L2Tx is not selected in the forged batches.
2021-02-03 13:03:30 +01:00
arnau
7ce60f0200 Merge pull request #521 from hermeznetwork/feature/robuststatedb
Improve stateDB reliability
2021-02-03 11:03:28 +01:00
Eduard S
8517e6afa0 Improve stateDB reliability
- Close StateDB when stopping the node
- Lock the StateDB when doing checkpoints to avoid multiple instances of
  oppening the pebble DB at the same time.
2021-02-01 16:59:12 +01:00
arnaucube
c1cd37913f Update TxID to use eth Keccak256
- Update TxID to use eth Keccak256
- Added more tests to L2Tx TxID calculation to check compatibility with
js version from https://github.com/hermeznetwork/hermezjs/pull/57
2021-02-01 16:01:55 +01:00
arnau
8cc23b9fb2 Merge pull request #513 from hermeznetwork/test/idx
Fix incorrect initial CurrentIdx in kvdb
2021-01-29 13:27:03 +01:00
Eduard S
d7c56afa5c Fix incorrect initial CurrentIdx in kvdb 2021-01-29 12:41:26 +01:00
arnaucube
aade6220d4 Update TxID to avoid collisions on DB (fix #503) 2021-01-29 11:38:22 +01:00
arnaucube
a899f43914 Add TokenID to L2Tx
L2Tx.TokenID is not on the data obtained by the Synchronizer from the
blockchain, but is set by the TxProcessor when processing the
transactions in the StateDB.
2021-01-27 12:40:09 +01:00
arnau
3801d0bee6 Merge pull request #498 from hermeznetwork/feature/update-txman
Update coordinator to work better under real net
2021-01-20 17:33:57 +01:00
Eduard S
70482605c4 Update coordinator to work better under real net
- cli / node
    - Update handler of SIGINT so that after 3 SIGINTs, the process terminates
      unconditionally
- coordinator
    - Store stats without pointer
    - In all functions that send a variable via channel, check for context done
      to avoid deadlock (due to no process reading from the channel, which has
      no queue) when the node is stopped.
    - Abstract `canForge` so that it can be used outside of the `Coordinator`
    - In `canForge` check the blockNumber in current and next slot.
    - Update tests due to smart contract changes in slot handling, and minimum
      bid defaults
    - TxManager
        - Add consts, vars and stats to allow evaluating `canForge`
        - Add `canForge` method (not used yet)
        - Store batch and nonces status (last success and last pending)
        - Track nonces internally instead of relying on the ethereum node (this
          is required to work with ganache when there are pending txs)
        - Handle the (common) case of the receipt not being found after the tx
          is sent.
        - Don't start the main loop until we get an initial messae fo the stats
          and vars (so that in the loop the stats and vars are set to
          synchronizer values)
- eth / ethereum client
    - Add necessary methods to create the auth object for transactions manually
      so that we can set the nonce, gas price, gas limit, etc manually
    - Update `RollupForgeBatch` to take an auth object as input (so that the
      coordinator can set parameters manually)
- synchronizer
    - In stats, add `NextSlot`
2021-01-20 17:29:02 +01:00
arnaucube
7f46c3028e Add tests connecting TxSelector, BatchBuilder, ZKInputs, ProofServer
- Add tests connecting TxSelector, BatchBuilder, ZKInputs, ProofServer
- Added test to check that the signatures of the PoolL2Txs from the L2DB
pool can be verified, to check that the parameters of the PoolL2Tx match
the original parameters signed before inserting them into the L2DB
2021-01-18 17:52:14 +01:00
Eduard S
828b8177c2 In pool l2tx, store effective in aux column 2021-01-18 13:21:56 +01:00
Eduard S
ec194d5066 Set l1tx.EffectiveFromIdx in TxProcessor and til 2021-01-13 18:20:47 +01:00
Arnau B
50fc607306 Add idx for txs that creat account, step1 2021-01-13 18:16:32 +01:00
ToniRamirezM
170da500db Remove big.Int from state API 2021-01-12 18:37:56 +01:00
a_bennassar
a5643c16f6 Merge pull request #476 from hermeznetwork/feature/api-big-ints-removal
Remove bigints from API
2021-01-12 15:35:16 +01:00
arnaucube
0bea7bcfea Add ZKI tests Til last batches MinimumFlow0 &other
- Add ZKI tests Til last batches MinimumFlow0
- Fix zki.CurrentNumBatch
- Fix StateDB BatchNum log
2021-01-12 13:10:42 +01:00
ToniRamirezM
ce6a271448 Remove bigints from API 2021-01-12 11:05:13 +01:00
a_bennassar
65a1057980 Merge pull request #473 from hermeznetwork/feature/buckets-update
Update buckets withdrawals on API
2021-01-11 15:43:00 +01:00
ToniRamirezM
0bb69c3f3e Update buckets withdrawals on API 2021-01-11 15:21:50 +01:00
Eduard S
2ae936f2bd Merge pull request #469 from hermeznetwork/feature/swagger-imprvmnts
Swagger improvements
2021-01-08 17:57:26 +01:00
Arnau B
b4d8e68730 Swagger improvements 2021-01-08 15:47:44 +01:00
Eduard S
423b67b707 Merge pull request #468 from hermeznetwork/fix/repeated-bids
Fix repeated items when coordinator is updated
2021-01-07 17:06:16 +01:00
Arnau B
c5e3fdad75 Fix repeated items when coordinator is updated 2021-01-07 16:30:36 +01:00
Eduard S
a79cb4edfd Update node and apply some fixes
- Node
	- Load Coordinator Fee Account from config
		- Sign the AccountCreationMsg to generate the
		  AccountCreationAuth
		- Resolve #465
	- Wait for synchronizer termination before stopping coordinator to avoid
	  getting stuck when closing in the following case:
		- The coordinator stops reading the synchronizer msg channel,
		  and the node gets stuck sending a message to that channel.
- Common
	- Move account creation auth signature code to common.
	- Update RollupConstInputSHAConstantBytes
- Coordinator
	- Set batch status in the debug file
	- Propagate SCVariables on reorg
	- Pipeline: Get SCVariables updates
		- Resolve #457
	- Fix off by 1 error in Pipeline.shouldL1L2Batch() (which shouldn't have
	  caused any problem, but it was not right)
- KVDB
	- Delete future checkpoints after reset
	- In `ResetFromSynchronizer`, remove all checkpoints first, and follow
	  the same logic as `reset()`.
- Cli
	- Add command to generate a BabyJubJub key pair (to be used for the
	  Coordinator Fee Account)
- Node
	- Adjust example config `Coordinator.L1BatchTimeoutPerc` to avoid
	  missing the L1Batch deadline with the following setup:
		- a block is mined every 2 seconds
		- single proof server that takes 2 seconds to calculate a proof
- TxProcessor
	- Close temporary pebble used for the exit tree after usage.
		- Resolve #463
2021-01-07 16:09:28 +01:00
arnaucube
bf5ab12d0e Update Account.PublicKey to Account.BJJ
Update account.PublicKey to account.BJJ for naming consistency with the rest of the code
2021-01-05 15:34:50 +01:00
arnaucube
962b9e4712 Add checks to TxSelector & other:
- Upgrade go-merkletree version to include the last changes of Pebble
that fixes the cgo issues (which should fix #453), from:
c2b05f12d7
- TxSelector
	- Remove parameter batchNum for GetL2TxSelection & GetL1L2TxSelection
	- Add checks of ToBJJ & ToEthAddr when ToIdx>255
	- Avoid getting the sender account twice to get the TokenID of a l2tx
	- Add test to check that selected L2Txs are sorted by Nonce
	- Discard L2Tx that return error at ProcessL2Txs
- executed `go mod tidy`
2021-01-05 15:28:43 +01:00
Eduard S
6fa4a2e799 Merge pull request #450 from hermeznetwork/feature/l2db-maxtxs
Add max txs limit to tx pool
2021-01-05 12:42:22 +01:00
Arnau B
622e01e9af Add max txs limit to tx pool 2021-01-04 18:38:25 +01:00
ToniRamirezM
e2d243a9b5 Sanitize varchars that may not be UTF-8 2021-01-04 13:06:53 +01:00
a_bennassar
926943cb71 Merge pull request #447 from hermeznetwork/feature/change-queries
Change queries that produced undesired meddler logs
2021-01-04 10:31:21 +01:00
a_bennassar
65abdd408a Merge pull request #442 from hermeznetwork/feature/new-metrics-tests
Added new metrics tests
2021-01-04 10:23:17 +01:00
arnaucube
79b3de7178 Update TxSelector to return CoordIdxs used & other
- StateDB
  - Update GetIdxByEthAddrBJJ to return ErrToIdxNotFound when idx not found, so can be checked at upper levels
- TxSelector
  - rm CoordIdxsDB that is no longer needed (also related methods)
  - add `getCoordIdx` method to get the Coordinator Idx for a given TokenID
  - Update coordinator account creation related to new TokenIDs from L2Txs
  - Reorganize GetL1L2TxSelection
  - return CoordIdxs used in the selection
- Update go-merkletree version which avoids marshaling Siblings to json
with 'null' value in case of empty array
2020-12-31 13:16:41 +01:00
ToniRamirezM
3e70551ac5 Added new metrics tests 2020-12-31 12:29:49 +01:00
Eduard S
e5fc403451 Add cli command to wipe the SQL DB
- Refactor the migrations code so that packr is only called (via an init
  function in `db/utils.go`).
- When loading the migrations, make sure there is at least one migration,
  otherwise panic (this would happen if the node is built incorrectly)
2020-12-31 10:34:32 +01:00
arnau
2e51860c37 Merge pull request #437 from hermeznetwork/feature/ethprivkey
Load ethereum private key
2020-12-31 10:33:44 +01:00
ToniRamirezM
64b214061c Change queries that produced undesired meddler logs 2020-12-30 19:12:53 +01:00
Arnau B
a32f75db78 empty array instead of 404 2020-12-30 12:41:53 +01:00
a_bennassar
c2dd982d26 Merge pull request #436 from hermeznetwork/feature/exit-proof-siblings
Update ExitProofs-Siblings padding
2020-12-30 11:13:22 +01:00
Eduard S
42791181a1 Load ethereum private key
Load an ethereum keystore when the node is started in coordinator mode.  The
private key corresponding to the forger address must be imported into the
keystore before running the node in coordinator mode.  You can see an examples
in `cli/node/load-sk-example.sh`.
2020-12-30 10:48:14 +01:00
arnaucube
ca5870e3ba Update ExitProofs-Siblings padding 2020-12-30 10:44:34 +01:00
Eduard S
88f924e9dd Merge pull request #435 from hermeznetwork/feature/addrs-in-exit-endpoints
Include addrs in exit responses
2020-12-29 18:01:46 +01:00
Arnau B
6cd292b937 Include addrs in exit responses 2020-12-29 17:35:20 +01:00
Eduard S
8c6dc6e3f8 Merge pull request #433 from hermeznetwork/feature/kvdb
Abstract KVDB from StateDB
2020-12-29 17:21:01 +01:00
arnaucube
68bfbff269 Abstract KVDB from StateDB
- KVDB contains the Checkpoint & Resets system
- StateDB uses KVDB and adds all the StateDB related methods
2020-12-29 16:48:01 +01:00
Eduard S
890b94a41a Merge pull request #428 from hermeznetwork/fix/api-next-forgers
Fix next forgers api response
2020-12-29 15:32:02 +01:00
Arnau B
6058d3c41a Fix next forgers api response 2020-12-29 15:17:55 +01:00