Commit Graph

79 Commits

Author SHA1 Message Date
arnaucube
0955016049 Update StateDB computeEffectiveAmounts
For the case of tx.EthAddr!=tx.FromIdx.EthAddr
2020-12-16 11:55:53 +01:00
arnaucube
593a477e6c ZKInputs generation ISFinalAccFee update
- Update IntermediateState FinalAccFee values to fit in the Circom circuit expected inputs
- Add ZKInputs generation test with transactions generated by Til
(TestZKInputs6)
- BatchBuilder: remove redundant MakeCheckpoint call
2020-12-15 11:49:09 +01:00
arnaucube
a4584499a4 StateDB Exit & ForceExit correct processing & ZKI
- Update tx processor of Exit & ForceExit types
- Update ZKInputs generation related to Exit transactions
- Add TestZKInputs5 which adds tests generation of ZKInputs compatible
with Circom circuits including Exit (L2) & ForceExit (L1) transaction types
- Add StateDB test to check max DepositAmount case
2020-12-11 12:39:05 +01:00
arnaucube
e7c8328df1 Fix process L1Tx for correct ZKInputs generation
- StateDB fix process txs of type CreateAccountDepositTransfer & DepositTransfer
- Add TestZKInputs3 & TestZKInputs4 which:
    - have a compatible check in js version to check Circom circuits
compatibility
    - adds test of tx type of CreateAccountDepositTransfer &
DepositTransfer
2020-12-10 12:59:01 +01:00
Eduard S
f229f49f0e Merge pull request #334 from hermeznetwork/feature/test-zkinputs2
Add zki.ISAccFeeOut generation for circuits compat
2020-12-09 15:34:07 +01:00
arnaucube
c162563376 Add zki.ISAccFeeOut generation for circuits compat
- add IntermediateStates AccFeeOut correct generation for circuits
compatibility at ZKInputs
- Fix formatAccumulatedFees
- organize ZKInputs tests to check compatibility with Circom circuits
- Add TestZKInputs2
    - which checks ZKInputs for 4 L1 txs + 2 L2Txs with fees
    - tested with Hermez Circom circuits
2020-12-09 12:47:00 +01:00
Eduard S
900d1fb6ce Integrate purger to node
- Common
	- Add `IdxNonce` type used to track nonces in accounts to invalidate
	  l2txs in the pool
- Config
	- Update coordinator config will all the new configuration parameters
	  used in the coordinator
- Coordinator
	- Introduce the `Purger` to track how often to purge and do the job when
	  needed according to a configuration.
	- Implement the methods to invalidate l2txs transactions due to l2txs
	  selection in batches.  For now these functions are not used in favour
	  of the `Purger` methods, which check ALL the l2txs in the pool.
	- Call Invalidation and Purging methods of the purger both when the
	  node is forging (in the pipeline when starting a new batch) and when
	  the node is not forging (in coordinator when being notified about a
	  new synced block)
- L2DB:
	- Implement `GetPendingUniqueFromIdxs` to get all the unique idxs from
	  pending transactions (used to get their nonces and then invalidate
	  txs)
	- Redo `CheckNonces` with a single SQL query and using `common.IdxNonce`
	  instead of `common.Account`
- StateDB:
	- Expose GetIdx to check errors when invalidating pool txs
- Synchronizer:
	- Test forged L1UserTxs processed by TxProcessor
	- Improve checks of Effective values
- TxSelector:
	- Expose the internal LocalStateDB in order to check account nonces in
	  the coordinator when not forging.
2020-12-09 12:43:16 +01:00
arnaucube
c1fd0d6626 ZKInputs with Fee txs compatible with circom
- Separate ZKInputs tests at StateDB package
- Small fix at ZKInputs generation FeeIdxs length
- Fees related parameters at ZKInputs works properly
- Add ZKInputs generation test with fees (`TestZKInputs1`), which output
has been tested with circom to ensure compatibility
2020-12-07 12:54:18 +01:00
Arnau B
445f26ec9f Rename load amount to deposit amount 2020-12-04 16:13:41 +01:00
arnaucube
80cbac24f9 StateDB ZKInputs generation compatible with circom
- Add SignatureConstant to L1Tx.TxCompressedData
- ZKInputs generation works initial version working and compatible with circom Hermez circuit
    - Lots of small updates for circuits compatibility
- Update version (with new changes) of `go-iden3-crypto` & `go-merkletree`
2020-12-03 13:16:51 +01:00
Eduard S
482c94d374 Advance coordinator implementation
- Common
	- Move ErrTODO and ErrDone to common for usage where needed.
- Coordinator
	- Move prover types to prover package
	- Handle reorgs, stopping the pipeline when necessary
	- Handle ethereum transaction errors by stopping the pipeline
	- In case of ethereum transaction revert, check for known revert causes
	  (more revert causes can be added to handle more cases)
	- Fix skipped transactions in TxManager confirmation logic
	- Cancel and wait for provers to be ready
	- Connect L2DB to:
		- purge l2txs due to timeout
		- mark l2txs at the different states
	- Connect HistoryDB to query L1UserTxs to forge in an L1Batch
- L2DB
	- Skip update functions when the input slices have no values (to avoid a
	  query with no values that results in an SQL error)
- StateDB
	- In LocalStateDB, fix Reset when mt == nil
- Prover (new package)
	- Rename the interface to Prover
	- Rename the mock struct to Mock
	- Extend Prover interface methods to provide everything required by the
	  coordinator
	- Begin implementing required http client code to interact with server
	  proof (not tested)
- Synchronizer:
	- Add LastForgeL1TxsNum to Stats
- Test/Client
	- Update Auction logic to track slots in which there's no forge during
	  the time before the deadline (following the solidity implementation)
2020-12-03 13:00:06 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
arnaucube
dde9250429 Update Common & StateDB & ZKInputs to last protocol version
- Add InvalidData flag to L1Tx
- Add BytesDataAvailability to L1Tx
- Update ZKInputs & HashGlobalInputs to last spec of the protocol
(massive migrations)
- TxProcessor check correctness of L1Txs

Compatible with hermeznetwork/commonjs v0.0.4
(c345239bba)
2020-11-30 11:26:15 +01:00
arnaucube
d3a38a3ee1 Add methods for ZKInputs IntermStates generation
- Add L1Tx TxCompressedData method
- Add PoolL2Tx TxCompressedDataV2 method
- Update ProcessTxs logic
- Add ZKInputs Intermediate States & Fee parameters calculation
2020-11-23 17:28:26 +01:00
arnaucube
ce772b1d19 Add HashGlobalInputs for ZKInputs
Add HashGlobalInputs for ZKInputs compatible with js & circom circuits version.

Compatible with hermeznetwork/commonjs at version: c6a1448db5bae4cda839ce36c1f35d8defccc9cd
(c6a1448db5)
2020-11-17 16:33:29 +01:00
Eduard S
457ff94bfa Test synchronizer reorg 2020-11-16 11:35:15 +01:00
arnaucube
a9f112c4ce Cmptbility test for Root StateDB Accounts tree
Checked with commonjs version:
c8414bd205
2020-11-10 17:45:42 +01:00
arnaucube
900494fd8e StateDB group Fee txs and ZKInputs gen of Fee txs 2020-11-06 16:41:03 +01:00
Eduard S
5b6639a947 Add DebugAPI to Node, fix StateDB
- Allow starting the DebugAPI from the node via config
- In StateDB:
    - Make checkpoints when ProcessTxs() succeeds
    - Remove extra hardcoded `statedb` path that was redundant
    - Replace hardcoded `[:4]` by `[:]` when parsing idx, which failed because
      idx is 6 bytes length now.
- Extra: In node, use waitgroup instead of `stoppedXXX` channels to wait for
syncrhonizer goroutines to finish.
2020-11-06 13:00:11 +01:00
arnaucube
7af6371f1d ZKInputs update:
- Add ZKInputs parser to JSON
- Update ProcessTxs to last go-merkletree version changes
- Add StateDB test to generate ZKInputs to check inside a circom circuit
- Small fix at Til CoordinatorTx.LoadAmount=0
2020-11-04 16:40:55 +01:00
Eduard S
e6fb0a03de Reorganize smart contract types, udate eth tests, etc.
- Move smart contract constants and structs for variables to
  common/{ethrollup.go, ethauction.go, ethwdelayer.go}:
    - This removes repeated code of the structs for variables
    - Allows reusing the constants and variables from all modules without
      import cycles
- Remove unused common/scvars.go
- In common.BlockData, split data from each smart contract into a sepparate
  field (Rollup, Auction, WDelayer).  This affects the structures that til uses
  as output, and HistoryDB in the AddBlockSCData.
- In Synchronizer:
    - Pass starting block of each smart contract as config, instead of
      incorrectly using the genesis block found in the acution constant (which
      has a very different meaning)
    - Use variable structs from common instead of an internal copy
    - Synchronize more stuff (resolve some TODOs)
    - Fix some issues found after initial testing with ganache
- In eth:
    - In auction.go: Add method to get constants
    - Update README to use ganache instead of buidlerevm as local blockchain
      for testing
    - Update env variables and test vectors to pass the tests with the
      deployment in the ganache testnet.
    - Use ethereum keys derived from paths (hdwallet) in testing to avoid
      hardcoding private keys and generate the same keys from a mnemonic used
      in the ganache tesnet.
2020-11-04 12:06:28 +01:00
Eduard S
fe23361037 Update fee tables 2020-11-03 11:26:23 +01:00
arnaucube
f65a6047b4 Add Til set w minimum flow, small fixes at StateDB 2020-11-02 12:47:34 +01:00
arnaucube
2338b6b90b ProcessTxs fees collct update & tests, and other:
- Update StateDB ProcessTxs fees collection
- Update ProcessTxs tests to last StateDB & Til changes
- Til Blockchain L2Txs remove Nonce generation, update tests accordingly
- Update Til Set to test fees
2020-10-28 17:21:12 +01:00
Eduard S
4eb2972b1b Add collectedFees in TxProcessor, update sync test
- In Synchronizer test, test all tx types
2020-10-28 14:24:14 +01:00
Eduard S
558de2737e Test L2Txs, ExitTree in synchronizer 2020-10-27 17:59:18 +01:00
arnaucube
aaa71a6b11 StateDB store Idxs with TokenID at the db-key 2020-10-27 17:24:10 +01:00
arnaucube
e09092732e Group ProcessTxs output 2020-10-27 13:48:07 +01:00
arnaucube
0ff3fb5ae7 ProcessTxs return an array of created accounts 2020-10-27 12:50:16 +01:00
arnaucube
8f24aa93c9 Add ProcessTxs send fees to Coordinator accounts 2020-10-27 10:28:32 +01:00
arnaucube
1e71d7f19e Add fees subtraction on TxProcessors 2020-10-23 18:13:12 +02:00
arnaucube
0ac6c0ecbb Add TxTypeTransferToEthAddr&ToBJJ to StateDB & Til
Add TxTypeTransferToEthAddr&ToBJJ to StateDB & Til
resolves #203, resolves #209
2020-10-23 11:42:39 +02:00
Arnau B
006bbf3321 Implement tx pool endpoints 2020-10-21 16:34:54 +02:00
arnaucube
265bdf23e3 Rename Transakcio to Til for a easier usage
Rename Transakcio to Til for a easier usage, also change til.TestContext
to til.Context, and til.NewTestContext to til.NewContext.
2020-10-21 15:17:22 +02:00
arnau
6a8b454368 Merge pull request #211 from hermeznetwork/feature/update-general
Feature/update general
2020-10-21 13:06:09 +02:00
Eduard S
54562c0367 Implement CalcFeeAmount, Resolve #140 2020-10-21 12:25:15 +02:00
arnau
edb466c996 Merge pull request #208 from hermeznetwork/feature/debugapi
Extend statedb and use prefixes, add debugapi
2020-10-21 11:39:42 +02:00
arnaucube
e89f89102c Add L1 Queues system for tx generation in Trnskcio 2020-10-20 17:43:57 +02:00
Eduard S
44c70dfbcb Replace cp -r in StateDB by checkpoints, Resolve #93 2020-10-20 14:52:46 +02:00
Eduard S
88ddbbcddc Delete tmp folders from tests, Resolve #105 2020-10-20 13:13:10 +02:00
Eduard S
96ceb014bf Implement an eficient memory storage, Resolve #120 2020-10-20 13:13:05 +02:00
Eduard S
6afbda5302 Extend statedb and use prefixes, add debugapi
In statedb:
- Store all values using prefixes for keys to allow iteration
- Add methods MTGetRoot, GetAccounts

Implement debugapi, an http server with debugging endpoints:
- debugAPI.GET("sdb/batchnum", a.handleCurrentBatch)
- debugAPI.GET("sdb/mtroot", a.handleMTRoot)
- debugAPI.GET("sdb/accounts", a.handleAccounts)
- debugAPI.GET("sdb/accounts/:Idx", a.handleAccount)
2020-10-19 13:53:39 +02:00
arnaucube
a6006ebde7 Add transakcio Token Register instructions parser 2020-10-15 18:06:11 +02:00
arnaucube
17ff917a20 Add transakcio coordinator tx instruction feature
Transakcio:
- Transaction generation to fail when Transfer to an account that don't exist (not created yet)
- Add CreateAccountDepositCoordinator instruction feature
- Updated lang.go instruction type parser approach for more simplicity of code
- Remove `*testing.T` from transactions generation methods, use `log.Fatal` instead
2020-10-14 17:17:28 +02:00
arnaucube
1a2f769b5d Add TxSel Nonce sort. Fix surplus from refactors
- Add TxSel Nonce sort
- Fix surplus from refactors
- StateDB reuse computation of ToIdx across Synchronizer, TxSelector,
BatchBuilder
2020-10-14 11:38:40 +02:00
arnaucube
431df55120 Add transakcio set type define, add set load [...]
Add transakcio set type definition, add set loading, move transakcio to
package, adapt branch to last master updates (fix compile due new common
types & git conflicts).
Update tests to pass the test, pending to adapt to new Transakcio
interface.
2020-10-09 20:00:43 +02:00
Arnau B
5097939b12 Avoid using pointers in common 2020-10-08 16:42:16 +02:00
Eduard S
b14495cfcc Replace all []*Foo by []Foo in sql db return values
- Implement SlicePtrsToSlice and use it in all `meddler.QueryAll` sql db functions to always return []Foo instead of []*Foo
2020-10-07 17:03:39 +02:00
Eduard S
0277210c39 Extend ethclient test, implement new TxID spec
- Implement new TxID spec that distinguishes L1UserTx and L1CoordinatorTx
- Replace some type []*Foo by []Foo
- Fix HistoryDB & L2DB bug: in case of error, a rollback was applied and the returned error was nil
- Reorder inserts in historydb.NewHistoryDB() to follow foreign key dependencies
- Add initial synchronizer test with test.Client (for now, only tested l1UserTxs, blocks, addToken)
- Update L1UserTx event in test.Client
2020-10-07 16:41:51 +02:00
a_bennassar
49366e3fa4 Feature/null refactor (#173)
* WIP: rebase

* Make nullable fields use pointers
2020-10-06 10:34:49 +02:00