Commit Graph

100 Commits

Author SHA1 Message Date
arnaucube
0bea7bcfea Add ZKI tests Til last batches MinimumFlow0 &other
- Add ZKI tests Til last batches MinimumFlow0
- Fix zki.CurrentNumBatch
- Fix StateDB BatchNum log
2021-01-12 13:10:42 +01:00
arnaucube
bf5ab12d0e Update Account.PublicKey to Account.BJJ
Update account.PublicKey to account.BJJ for naming consistency with the rest of the code
2021-01-05 15:34:50 +01:00
arnaucube
962b9e4712 Add checks to TxSelector & other:
- Upgrade go-merkletree version to include the last changes of Pebble
that fixes the cgo issues (which should fix #453), from:
c2b05f12d7
- TxSelector
	- Remove parameter batchNum for GetL2TxSelection & GetL1L2TxSelection
	- Add checks of ToBJJ & ToEthAddr when ToIdx>255
	- Avoid getting the sender account twice to get the TokenID of a l2tx
	- Add test to check that selected L2Txs are sorted by Nonce
	- Discard L2Tx that return error at ProcessL2Txs
- executed `go mod tidy`
2021-01-05 15:28:43 +01:00
arnaucube
79b3de7178 Update TxSelector to return CoordIdxs used & other
- StateDB
  - Update GetIdxByEthAddrBJJ to return ErrToIdxNotFound when idx not found, so can be checked at upper levels
- TxSelector
  - rm CoordIdxsDB that is no longer needed (also related methods)
  - add `getCoordIdx` method to get the Coordinator Idx for a given TokenID
  - Update coordinator account creation related to new TokenIDs from L2Txs
  - Reorganize GetL1L2TxSelection
  - return CoordIdxs used in the selection
- Update go-merkletree version which avoids marshaling Siblings to json
with 'null' value in case of empty array
2020-12-31 13:16:41 +01:00
arnau
2e51860c37 Merge pull request #437 from hermeznetwork/feature/ethprivkey
Load ethereum private key
2020-12-31 10:33:44 +01:00
Eduard S
42791181a1 Load ethereum private key
Load an ethereum keystore when the node is started in coordinator mode.  The
private key corresponding to the forger address must be imported into the
keystore before running the node in coordinator mode.  You can see an examples
in `cli/node/load-sk-example.sh`.
2020-12-30 10:48:14 +01:00
arnaucube
ca5870e3ba Update ExitProofs-Siblings padding 2020-12-30 10:44:34 +01:00
arnaucube
68bfbff269 Abstract KVDB from StateDB
- KVDB contains the Checkpoint & Resets system
- StateDB uses KVDB and adds all the StateDB related methods
2020-12-29 16:48:01 +01:00
arnaucube
0cf1ed217b Abstract TxProcessor from StateDB
- Abstract TxProcessor from StateDB
- Upgrade to last version of go-merkletree for the key-value DB usage
2020-12-29 11:14:38 +01:00
arnaucube
cf199fd832 Fix ZKI NTx for MaxTx
On the Circuits uses the name `nTx` (and on the tests `maxTx`), but
Contracts `maxTx`. On Go we had both variables, but really only used
one, now at Go we use `MaxTx` to refer to the maximum number of
transactions.
2020-12-28 11:26:47 +01:00
Eduard S
a0c8ace38d Add missing tracerr.Wrap 2020-12-24 13:39:27 +01:00
Eduard S
2205fcadbc Delete old checkpoints in stateDB automatically
Introduce a constructor parameter for the StateDB called `keep`, which tells
how many checkpoints to keep.  When doing a new checkpoint, if the number of
existing checkpoints exeeds `keep`, the oldest ones will be deleted.
2020-12-24 13:35:47 +01:00
arnaucube
150597c282 Parametrize ChainID 2020-12-23 15:23:48 +01:00
arnaucube
eea44b3990 Fix StateDB & Til Nonce increment 2020-12-23 12:32:41 +01:00
arnaucube
982a423bf3 TxSelector update txs processing logic 2020-12-23 11:36:41 +01:00
arnaucube
2583d35adf Update ZKInputs for empty batches
- Add TestZKInputsEmpty circom test vectors
- Fix ZKInputsHash OldLastIdx bytearray
2020-12-23 11:09:43 +01:00
Eduard S
6a990376b4 Implement Pipeline.prepareForgeBatchArgs()
- Implement Pipeline.prepareForgeBatchArgs()
- Add a minimal stress test for the coordinator (that also runs the
  synchronizer)
- Update txprocessor.ProcessTxs() to return valid results for batches without
  transactions
  	- Add the boilerplate for the corresponding test, leaving as TODO the
	  zkInput values
- Update prover client to use the same point format as proof server (projective)
- Update interface of TxSelector.GetCoordIdxs to also return the authorizations
  to create accounts that go with the l1CoordinatorTxs.
2020-12-22 17:27:34 +01:00
arnaucube
4b10549822 mv of babyjub.PublicKey to babyjub.PublicKeyComp
Update usage of `*babyjub.PublicKey` to `babyjub.PublicKeyComp`
- when the key is not defined, internally is used `babyjub.EmptyBJJComp`, which is a `[32]byte` of zeroes of type `babyjub.PublicKeyComp`
- the API continues returning `nil` when the key is not defined
2020-12-22 11:29:47 +01:00
arnaucube
0bcf57603c Effective amounts add missing checks 2020-12-17 17:51:23 +01:00
arnau
6dd766be4d Merge pull request #359 from hermeznetwork/feature/integration25
Extend Coordinator tests
2020-12-16 18:07:44 +01:00
Eduard S
6a9b5ce420 Extend Coordinator tests 2020-12-16 13:13:40 +01:00
arnaucube
0955016049 Update StateDB computeEffectiveAmounts
For the case of tx.EthAddr!=tx.FromIdx.EthAddr
2020-12-16 11:55:53 +01:00
arnaucube
593a477e6c ZKInputs generation ISFinalAccFee update
- Update IntermediateState FinalAccFee values to fit in the Circom circuit expected inputs
- Add ZKInputs generation test with transactions generated by Til
(TestZKInputs6)
- BatchBuilder: remove redundant MakeCheckpoint call
2020-12-15 11:49:09 +01:00
arnaucube
a4584499a4 StateDB Exit & ForceExit correct processing & ZKI
- Update tx processor of Exit & ForceExit types
- Update ZKInputs generation related to Exit transactions
- Add TestZKInputs5 which adds tests generation of ZKInputs compatible
with Circom circuits including Exit (L2) & ForceExit (L1) transaction types
- Add StateDB test to check max DepositAmount case
2020-12-11 12:39:05 +01:00
arnaucube
e7c8328df1 Fix process L1Tx for correct ZKInputs generation
- StateDB fix process txs of type CreateAccountDepositTransfer & DepositTransfer
- Add TestZKInputs3 & TestZKInputs4 which:
    - have a compatible check in js version to check Circom circuits
compatibility
    - adds test of tx type of CreateAccountDepositTransfer &
DepositTransfer
2020-12-10 12:59:01 +01:00
Eduard S
f229f49f0e Merge pull request #334 from hermeznetwork/feature/test-zkinputs2
Add zki.ISAccFeeOut generation for circuits compat
2020-12-09 15:34:07 +01:00
arnaucube
c162563376 Add zki.ISAccFeeOut generation for circuits compat
- add IntermediateStates AccFeeOut correct generation for circuits
compatibility at ZKInputs
- Fix formatAccumulatedFees
- organize ZKInputs tests to check compatibility with Circom circuits
- Add TestZKInputs2
    - which checks ZKInputs for 4 L1 txs + 2 L2Txs with fees
    - tested with Hermez Circom circuits
2020-12-09 12:47:00 +01:00
Eduard S
900d1fb6ce Integrate purger to node
- Common
	- Add `IdxNonce` type used to track nonces in accounts to invalidate
	  l2txs in the pool
- Config
	- Update coordinator config will all the new configuration parameters
	  used in the coordinator
- Coordinator
	- Introduce the `Purger` to track how often to purge and do the job when
	  needed according to a configuration.
	- Implement the methods to invalidate l2txs transactions due to l2txs
	  selection in batches.  For now these functions are not used in favour
	  of the `Purger` methods, which check ALL the l2txs in the pool.
	- Call Invalidation and Purging methods of the purger both when the
	  node is forging (in the pipeline when starting a new batch) and when
	  the node is not forging (in coordinator when being notified about a
	  new synced block)
- L2DB:
	- Implement `GetPendingUniqueFromIdxs` to get all the unique idxs from
	  pending transactions (used to get their nonces and then invalidate
	  txs)
	- Redo `CheckNonces` with a single SQL query and using `common.IdxNonce`
	  instead of `common.Account`
- StateDB:
	- Expose GetIdx to check errors when invalidating pool txs
- Synchronizer:
	- Test forged L1UserTxs processed by TxProcessor
	- Improve checks of Effective values
- TxSelector:
	- Expose the internal LocalStateDB in order to check account nonces in
	  the coordinator when not forging.
2020-12-09 12:43:16 +01:00
arnaucube
c1fd0d6626 ZKInputs with Fee txs compatible with circom
- Separate ZKInputs tests at StateDB package
- Small fix at ZKInputs generation FeeIdxs length
- Fees related parameters at ZKInputs works properly
- Add ZKInputs generation test with fees (`TestZKInputs1`), which output
has been tested with circom to ensure compatibility
2020-12-07 12:54:18 +01:00
Arnau B
445f26ec9f Rename load amount to deposit amount 2020-12-04 16:13:41 +01:00
arnaucube
80cbac24f9 StateDB ZKInputs generation compatible with circom
- Add SignatureConstant to L1Tx.TxCompressedData
- ZKInputs generation works initial version working and compatible with circom Hermez circuit
    - Lots of small updates for circuits compatibility
- Update version (with new changes) of `go-iden3-crypto` & `go-merkletree`
2020-12-03 13:16:51 +01:00
Eduard S
482c94d374 Advance coordinator implementation
- Common
	- Move ErrTODO and ErrDone to common for usage where needed.
- Coordinator
	- Move prover types to prover package
	- Handle reorgs, stopping the pipeline when necessary
	- Handle ethereum transaction errors by stopping the pipeline
	- In case of ethereum transaction revert, check for known revert causes
	  (more revert causes can be added to handle more cases)
	- Fix skipped transactions in TxManager confirmation logic
	- Cancel and wait for provers to be ready
	- Connect L2DB to:
		- purge l2txs due to timeout
		- mark l2txs at the different states
	- Connect HistoryDB to query L1UserTxs to forge in an L1Batch
- L2DB
	- Skip update functions when the input slices have no values (to avoid a
	  query with no values that results in an SQL error)
- StateDB
	- In LocalStateDB, fix Reset when mt == nil
- Prover (new package)
	- Rename the interface to Prover
	- Rename the mock struct to Mock
	- Extend Prover interface methods to provide everything required by the
	  coordinator
	- Begin implementing required http client code to interact with server
	  proof (not tested)
- Synchronizer:
	- Add LastForgeL1TxsNum to Stats
- Test/Client
	- Update Auction logic to track slots in which there's no forge during
	  the time before the deadline (following the solidity implementation)
2020-12-03 13:00:06 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
arnaucube
dde9250429 Update Common & StateDB & ZKInputs to last protocol version
- Add InvalidData flag to L1Tx
- Add BytesDataAvailability to L1Tx
- Update ZKInputs & HashGlobalInputs to last spec of the protocol
(massive migrations)
- TxProcessor check correctness of L1Txs

Compatible with hermeznetwork/commonjs v0.0.4
(c345239bba)
2020-11-30 11:26:15 +01:00
arnaucube
d3a38a3ee1 Add methods for ZKInputs IntermStates generation
- Add L1Tx TxCompressedData method
- Add PoolL2Tx TxCompressedDataV2 method
- Update ProcessTxs logic
- Add ZKInputs Intermediate States & Fee parameters calculation
2020-11-23 17:28:26 +01:00
arnaucube
ce772b1d19 Add HashGlobalInputs for ZKInputs
Add HashGlobalInputs for ZKInputs compatible with js & circom circuits version.

Compatible with hermeznetwork/commonjs at version: c6a1448db5bae4cda839ce36c1f35d8defccc9cd
(c6a1448db5)
2020-11-17 16:33:29 +01:00
Eduard S
457ff94bfa Test synchronizer reorg 2020-11-16 11:35:15 +01:00
arnaucube
a9f112c4ce Cmptbility test for Root StateDB Accounts tree
Checked with commonjs version:
c8414bd205
2020-11-10 17:45:42 +01:00
arnaucube
900494fd8e StateDB group Fee txs and ZKInputs gen of Fee txs 2020-11-06 16:41:03 +01:00
Eduard S
5b6639a947 Add DebugAPI to Node, fix StateDB
- Allow starting the DebugAPI from the node via config
- In StateDB:
    - Make checkpoints when ProcessTxs() succeeds
    - Remove extra hardcoded `statedb` path that was redundant
    - Replace hardcoded `[:4]` by `[:]` when parsing idx, which failed because
      idx is 6 bytes length now.
- Extra: In node, use waitgroup instead of `stoppedXXX` channels to wait for
syncrhonizer goroutines to finish.
2020-11-06 13:00:11 +01:00
arnaucube
7af6371f1d ZKInputs update:
- Add ZKInputs parser to JSON
- Update ProcessTxs to last go-merkletree version changes
- Add StateDB test to generate ZKInputs to check inside a circom circuit
- Small fix at Til CoordinatorTx.LoadAmount=0
2020-11-04 16:40:55 +01:00
Eduard S
e6fb0a03de Reorganize smart contract types, udate eth tests, etc.
- Move smart contract constants and structs for variables to
  common/{ethrollup.go, ethauction.go, ethwdelayer.go}:
    - This removes repeated code of the structs for variables
    - Allows reusing the constants and variables from all modules without
      import cycles
- Remove unused common/scvars.go
- In common.BlockData, split data from each smart contract into a sepparate
  field (Rollup, Auction, WDelayer).  This affects the structures that til uses
  as output, and HistoryDB in the AddBlockSCData.
- In Synchronizer:
    - Pass starting block of each smart contract as config, instead of
      incorrectly using the genesis block found in the acution constant (which
      has a very different meaning)
    - Use variable structs from common instead of an internal copy
    - Synchronize more stuff (resolve some TODOs)
    - Fix some issues found after initial testing with ganache
- In eth:
    - In auction.go: Add method to get constants
    - Update README to use ganache instead of buidlerevm as local blockchain
      for testing
    - Update env variables and test vectors to pass the tests with the
      deployment in the ganache testnet.
    - Use ethereum keys derived from paths (hdwallet) in testing to avoid
      hardcoding private keys and generate the same keys from a mnemonic used
      in the ganache tesnet.
2020-11-04 12:06:28 +01:00
Eduard S
fe23361037 Update fee tables 2020-11-03 11:26:23 +01:00
arnaucube
f65a6047b4 Add Til set w minimum flow, small fixes at StateDB 2020-11-02 12:47:34 +01:00
arnaucube
2338b6b90b ProcessTxs fees collct update & tests, and other:
- Update StateDB ProcessTxs fees collection
- Update ProcessTxs tests to last StateDB & Til changes
- Til Blockchain L2Txs remove Nonce generation, update tests accordingly
- Update Til Set to test fees
2020-10-28 17:21:12 +01:00
Eduard S
4eb2972b1b Add collectedFees in TxProcessor, update sync test
- In Synchronizer test, test all tx types
2020-10-28 14:24:14 +01:00
Eduard S
558de2737e Test L2Txs, ExitTree in synchronizer 2020-10-27 17:59:18 +01:00
arnaucube
aaa71a6b11 StateDB store Idxs with TokenID at the db-key 2020-10-27 17:24:10 +01:00
arnaucube
e09092732e Group ProcessTxs output 2020-10-27 13:48:07 +01:00
arnaucube
0ff3fb5ae7 ProcessTxs return an array of created accounts 2020-10-27 12:50:16 +01:00