Commit Graph

92 Commits

Author SHA1 Message Date
arnaucube
0cf1ed217b Abstract TxProcessor from StateDB
- Abstract TxProcessor from StateDB
- Upgrade to last version of go-merkletree for the key-value DB usage
2020-12-29 11:14:38 +01:00
arnaucube
cf199fd832 Fix ZKI NTx for MaxTx
On the Circuits uses the name `nTx` (and on the tests `maxTx`), but
Contracts `maxTx`. On Go we had both variables, but really only used
one, now at Go we use `MaxTx` to refer to the maximum number of
transactions.
2020-12-28 11:26:47 +01:00
Eduard S
a0c8ace38d Add missing tracerr.Wrap 2020-12-24 13:39:27 +01:00
Eduard S
2205fcadbc Delete old checkpoints in stateDB automatically
Introduce a constructor parameter for the StateDB called `keep`, which tells
how many checkpoints to keep.  When doing a new checkpoint, if the number of
existing checkpoints exeeds `keep`, the oldest ones will be deleted.
2020-12-24 13:35:47 +01:00
arnaucube
150597c282 Parametrize ChainID 2020-12-23 15:23:48 +01:00
arnaucube
eea44b3990 Fix StateDB & Til Nonce increment 2020-12-23 12:32:41 +01:00
arnaucube
982a423bf3 TxSelector update txs processing logic 2020-12-23 11:36:41 +01:00
arnaucube
2583d35adf Update ZKInputs for empty batches
- Add TestZKInputsEmpty circom test vectors
- Fix ZKInputsHash OldLastIdx bytearray
2020-12-23 11:09:43 +01:00
Eduard S
6a990376b4 Implement Pipeline.prepareForgeBatchArgs()
- Implement Pipeline.prepareForgeBatchArgs()
- Add a minimal stress test for the coordinator (that also runs the
  synchronizer)
- Update txprocessor.ProcessTxs() to return valid results for batches without
  transactions
  	- Add the boilerplate for the corresponding test, leaving as TODO the
	  zkInput values
- Update prover client to use the same point format as proof server (projective)
- Update interface of TxSelector.GetCoordIdxs to also return the authorizations
  to create accounts that go with the l1CoordinatorTxs.
2020-12-22 17:27:34 +01:00
arnaucube
4b10549822 mv of babyjub.PublicKey to babyjub.PublicKeyComp
Update usage of `*babyjub.PublicKey` to `babyjub.PublicKeyComp`
- when the key is not defined, internally is used `babyjub.EmptyBJJComp`, which is a `[32]byte` of zeroes of type `babyjub.PublicKeyComp`
- the API continues returning `nil` when the key is not defined
2020-12-22 11:29:47 +01:00
arnaucube
0bcf57603c Effective amounts add missing checks 2020-12-17 17:51:23 +01:00
arnau
6dd766be4d Merge pull request #359 from hermeznetwork/feature/integration25
Extend Coordinator tests
2020-12-16 18:07:44 +01:00
Eduard S
6a9b5ce420 Extend Coordinator tests 2020-12-16 13:13:40 +01:00
arnaucube
0955016049 Update StateDB computeEffectiveAmounts
For the case of tx.EthAddr!=tx.FromIdx.EthAddr
2020-12-16 11:55:53 +01:00
arnaucube
593a477e6c ZKInputs generation ISFinalAccFee update
- Update IntermediateState FinalAccFee values to fit in the Circom circuit expected inputs
- Add ZKInputs generation test with transactions generated by Til
(TestZKInputs6)
- BatchBuilder: remove redundant MakeCheckpoint call
2020-12-15 11:49:09 +01:00
arnaucube
a4584499a4 StateDB Exit & ForceExit correct processing & ZKI
- Update tx processor of Exit & ForceExit types
- Update ZKInputs generation related to Exit transactions
- Add TestZKInputs5 which adds tests generation of ZKInputs compatible
with Circom circuits including Exit (L2) & ForceExit (L1) transaction types
- Add StateDB test to check max DepositAmount case
2020-12-11 12:39:05 +01:00
arnaucube
e7c8328df1 Fix process L1Tx for correct ZKInputs generation
- StateDB fix process txs of type CreateAccountDepositTransfer & DepositTransfer
- Add TestZKInputs3 & TestZKInputs4 which:
    - have a compatible check in js version to check Circom circuits
compatibility
    - adds test of tx type of CreateAccountDepositTransfer &
DepositTransfer
2020-12-10 12:59:01 +01:00
Eduard S
f229f49f0e Merge pull request #334 from hermeznetwork/feature/test-zkinputs2
Add zki.ISAccFeeOut generation for circuits compat
2020-12-09 15:34:07 +01:00
arnaucube
c162563376 Add zki.ISAccFeeOut generation for circuits compat
- add IntermediateStates AccFeeOut correct generation for circuits
compatibility at ZKInputs
- Fix formatAccumulatedFees
- organize ZKInputs tests to check compatibility with Circom circuits
- Add TestZKInputs2
    - which checks ZKInputs for 4 L1 txs + 2 L2Txs with fees
    - tested with Hermez Circom circuits
2020-12-09 12:47:00 +01:00
Eduard S
900d1fb6ce Integrate purger to node
- Common
	- Add `IdxNonce` type used to track nonces in accounts to invalidate
	  l2txs in the pool
- Config
	- Update coordinator config will all the new configuration parameters
	  used in the coordinator
- Coordinator
	- Introduce the `Purger` to track how often to purge and do the job when
	  needed according to a configuration.
	- Implement the methods to invalidate l2txs transactions due to l2txs
	  selection in batches.  For now these functions are not used in favour
	  of the `Purger` methods, which check ALL the l2txs in the pool.
	- Call Invalidation and Purging methods of the purger both when the
	  node is forging (in the pipeline when starting a new batch) and when
	  the node is not forging (in coordinator when being notified about a
	  new synced block)
- L2DB:
	- Implement `GetPendingUniqueFromIdxs` to get all the unique idxs from
	  pending transactions (used to get their nonces and then invalidate
	  txs)
	- Redo `CheckNonces` with a single SQL query and using `common.IdxNonce`
	  instead of `common.Account`
- StateDB:
	- Expose GetIdx to check errors when invalidating pool txs
- Synchronizer:
	- Test forged L1UserTxs processed by TxProcessor
	- Improve checks of Effective values
- TxSelector:
	- Expose the internal LocalStateDB in order to check account nonces in
	  the coordinator when not forging.
2020-12-09 12:43:16 +01:00
arnaucube
c1fd0d6626 ZKInputs with Fee txs compatible with circom
- Separate ZKInputs tests at StateDB package
- Small fix at ZKInputs generation FeeIdxs length
- Fees related parameters at ZKInputs works properly
- Add ZKInputs generation test with fees (`TestZKInputs1`), which output
has been tested with circom to ensure compatibility
2020-12-07 12:54:18 +01:00
Arnau B
445f26ec9f Rename load amount to deposit amount 2020-12-04 16:13:41 +01:00
arnaucube
80cbac24f9 StateDB ZKInputs generation compatible with circom
- Add SignatureConstant to L1Tx.TxCompressedData
- ZKInputs generation works initial version working and compatible with circom Hermez circuit
    - Lots of small updates for circuits compatibility
- Update version (with new changes) of `go-iden3-crypto` & `go-merkletree`
2020-12-03 13:16:51 +01:00
Eduard S
482c94d374 Advance coordinator implementation
- Common
	- Move ErrTODO and ErrDone to common for usage where needed.
- Coordinator
	- Move prover types to prover package
	- Handle reorgs, stopping the pipeline when necessary
	- Handle ethereum transaction errors by stopping the pipeline
	- In case of ethereum transaction revert, check for known revert causes
	  (more revert causes can be added to handle more cases)
	- Fix skipped transactions in TxManager confirmation logic
	- Cancel and wait for provers to be ready
	- Connect L2DB to:
		- purge l2txs due to timeout
		- mark l2txs at the different states
	- Connect HistoryDB to query L1UserTxs to forge in an L1Batch
- L2DB
	- Skip update functions when the input slices have no values (to avoid a
	  query with no values that results in an SQL error)
- StateDB
	- In LocalStateDB, fix Reset when mt == nil
- Prover (new package)
	- Rename the interface to Prover
	- Rename the mock struct to Mock
	- Extend Prover interface methods to provide everything required by the
	  coordinator
	- Begin implementing required http client code to interact with server
	  proof (not tested)
- Synchronizer:
	- Add LastForgeL1TxsNum to Stats
- Test/Client
	- Update Auction logic to track slots in which there's no forge during
	  the time before the deadline (following the solidity implementation)
2020-12-03 13:00:06 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
arnaucube
dde9250429 Update Common & StateDB & ZKInputs to last protocol version
- Add InvalidData flag to L1Tx
- Add BytesDataAvailability to L1Tx
- Update ZKInputs & HashGlobalInputs to last spec of the protocol
(massive migrations)
- TxProcessor check correctness of L1Txs

Compatible with hermeznetwork/commonjs v0.0.4
(c345239bba)
2020-11-30 11:26:15 +01:00
arnaucube
d3a38a3ee1 Add methods for ZKInputs IntermStates generation
- Add L1Tx TxCompressedData method
- Add PoolL2Tx TxCompressedDataV2 method
- Update ProcessTxs logic
- Add ZKInputs Intermediate States & Fee parameters calculation
2020-11-23 17:28:26 +01:00
arnaucube
ce772b1d19 Add HashGlobalInputs for ZKInputs
Add HashGlobalInputs for ZKInputs compatible with js & circom circuits version.

Compatible with hermeznetwork/commonjs at version: c6a1448db5bae4cda839ce36c1f35d8defccc9cd
(c6a1448db5)
2020-11-17 16:33:29 +01:00
Eduard S
457ff94bfa Test synchronizer reorg 2020-11-16 11:35:15 +01:00
arnaucube
a9f112c4ce Cmptbility test for Root StateDB Accounts tree
Checked with commonjs version:
c8414bd205
2020-11-10 17:45:42 +01:00
arnaucube
900494fd8e StateDB group Fee txs and ZKInputs gen of Fee txs 2020-11-06 16:41:03 +01:00
Eduard S
5b6639a947 Add DebugAPI to Node, fix StateDB
- Allow starting the DebugAPI from the node via config
- In StateDB:
    - Make checkpoints when ProcessTxs() succeeds
    - Remove extra hardcoded `statedb` path that was redundant
    - Replace hardcoded `[:4]` by `[:]` when parsing idx, which failed because
      idx is 6 bytes length now.
- Extra: In node, use waitgroup instead of `stoppedXXX` channels to wait for
syncrhonizer goroutines to finish.
2020-11-06 13:00:11 +01:00
arnaucube
7af6371f1d ZKInputs update:
- Add ZKInputs parser to JSON
- Update ProcessTxs to last go-merkletree version changes
- Add StateDB test to generate ZKInputs to check inside a circom circuit
- Small fix at Til CoordinatorTx.LoadAmount=0
2020-11-04 16:40:55 +01:00
Eduard S
e6fb0a03de Reorganize smart contract types, udate eth tests, etc.
- Move smart contract constants and structs for variables to
  common/{ethrollup.go, ethauction.go, ethwdelayer.go}:
    - This removes repeated code of the structs for variables
    - Allows reusing the constants and variables from all modules without
      import cycles
- Remove unused common/scvars.go
- In common.BlockData, split data from each smart contract into a sepparate
  field (Rollup, Auction, WDelayer).  This affects the structures that til uses
  as output, and HistoryDB in the AddBlockSCData.
- In Synchronizer:
    - Pass starting block of each smart contract as config, instead of
      incorrectly using the genesis block found in the acution constant (which
      has a very different meaning)
    - Use variable structs from common instead of an internal copy
    - Synchronize more stuff (resolve some TODOs)
    - Fix some issues found after initial testing with ganache
- In eth:
    - In auction.go: Add method to get constants
    - Update README to use ganache instead of buidlerevm as local blockchain
      for testing
    - Update env variables and test vectors to pass the tests with the
      deployment in the ganache testnet.
    - Use ethereum keys derived from paths (hdwallet) in testing to avoid
      hardcoding private keys and generate the same keys from a mnemonic used
      in the ganache tesnet.
2020-11-04 12:06:28 +01:00
Eduard S
fe23361037 Update fee tables 2020-11-03 11:26:23 +01:00
arnaucube
f65a6047b4 Add Til set w minimum flow, small fixes at StateDB 2020-11-02 12:47:34 +01:00
arnaucube
2338b6b90b ProcessTxs fees collct update & tests, and other:
- Update StateDB ProcessTxs fees collection
- Update ProcessTxs tests to last StateDB & Til changes
- Til Blockchain L2Txs remove Nonce generation, update tests accordingly
- Update Til Set to test fees
2020-10-28 17:21:12 +01:00
Eduard S
4eb2972b1b Add collectedFees in TxProcessor, update sync test
- In Synchronizer test, test all tx types
2020-10-28 14:24:14 +01:00
Eduard S
558de2737e Test L2Txs, ExitTree in synchronizer 2020-10-27 17:59:18 +01:00
arnaucube
aaa71a6b11 StateDB store Idxs with TokenID at the db-key 2020-10-27 17:24:10 +01:00
arnaucube
e09092732e Group ProcessTxs output 2020-10-27 13:48:07 +01:00
arnaucube
0ff3fb5ae7 ProcessTxs return an array of created accounts 2020-10-27 12:50:16 +01:00
arnaucube
8f24aa93c9 Add ProcessTxs send fees to Coordinator accounts 2020-10-27 10:28:32 +01:00
arnaucube
1e71d7f19e Add fees subtraction on TxProcessors 2020-10-23 18:13:12 +02:00
arnaucube
0ac6c0ecbb Add TxTypeTransferToEthAddr&ToBJJ to StateDB & Til
Add TxTypeTransferToEthAddr&ToBJJ to StateDB & Til
resolves #203, resolves #209
2020-10-23 11:42:39 +02:00
Arnau B
006bbf3321 Implement tx pool endpoints 2020-10-21 16:34:54 +02:00
arnaucube
265bdf23e3 Rename Transakcio to Til for a easier usage
Rename Transakcio to Til for a easier usage, also change til.TestContext
to til.Context, and til.NewTestContext to til.NewContext.
2020-10-21 15:17:22 +02:00
arnau
6a8b454368 Merge pull request #211 from hermeznetwork/feature/update-general
Feature/update general
2020-10-21 13:06:09 +02:00
Eduard S
54562c0367 Implement CalcFeeAmount, Resolve #140 2020-10-21 12:25:15 +02:00
arnau
edb466c996 Merge pull request #208 from hermeznetwork/feature/debugapi
Extend statedb and use prefixes, add debugapi
2020-10-21 11:39:42 +02:00