Commit Graph

40 Commits

Author SHA1 Message Date
arnaucube
0cf1ed217b Abstract TxProcessor from StateDB
- Abstract TxProcessor from StateDB
- Upgrade to last version of go-merkletree for the key-value DB usage
2020-12-29 11:14:38 +01:00
Eduard S
a0c8ace38d Add missing tracerr.Wrap 2020-12-24 13:39:27 +01:00
Eduard S
2205fcadbc Delete old checkpoints in stateDB automatically
Introduce a constructor parameter for the StateDB called `keep`, which tells
how many checkpoints to keep.  When doing a new checkpoint, if the number of
existing checkpoints exeeds `keep`, the oldest ones will be deleted.
2020-12-24 13:35:47 +01:00
arnaucube
150597c282 Parametrize ChainID 2020-12-23 15:23:48 +01:00
arnaucube
982a423bf3 TxSelector update txs processing logic 2020-12-23 11:36:41 +01:00
Eduard S
6a9b5ce420 Extend Coordinator tests 2020-12-16 13:13:40 +01:00
Eduard S
900d1fb6ce Integrate purger to node
- Common
	- Add `IdxNonce` type used to track nonces in accounts to invalidate
	  l2txs in the pool
- Config
	- Update coordinator config will all the new configuration parameters
	  used in the coordinator
- Coordinator
	- Introduce the `Purger` to track how often to purge and do the job when
	  needed according to a configuration.
	- Implement the methods to invalidate l2txs transactions due to l2txs
	  selection in batches.  For now these functions are not used in favour
	  of the `Purger` methods, which check ALL the l2txs in the pool.
	- Call Invalidation and Purging methods of the purger both when the
	  node is forging (in the pipeline when starting a new batch) and when
	  the node is not forging (in coordinator when being notified about a
	  new synced block)
- L2DB:
	- Implement `GetPendingUniqueFromIdxs` to get all the unique idxs from
	  pending transactions (used to get their nonces and then invalidate
	  txs)
	- Redo `CheckNonces` with a single SQL query and using `common.IdxNonce`
	  instead of `common.Account`
- StateDB:
	- Expose GetIdx to check errors when invalidating pool txs
- Synchronizer:
	- Test forged L1UserTxs processed by TxProcessor
	- Improve checks of Effective values
- TxSelector:
	- Expose the internal LocalStateDB in order to check account nonces in
	  the coordinator when not forging.
2020-12-09 12:43:16 +01:00
Eduard S
482c94d374 Advance coordinator implementation
- Common
	- Move ErrTODO and ErrDone to common for usage where needed.
- Coordinator
	- Move prover types to prover package
	- Handle reorgs, stopping the pipeline when necessary
	- Handle ethereum transaction errors by stopping the pipeline
	- In case of ethereum transaction revert, check for known revert causes
	  (more revert causes can be added to handle more cases)
	- Fix skipped transactions in TxManager confirmation logic
	- Cancel and wait for provers to be ready
	- Connect L2DB to:
		- purge l2txs due to timeout
		- mark l2txs at the different states
	- Connect HistoryDB to query L1UserTxs to forge in an L1Batch
- L2DB
	- Skip update functions when the input slices have no values (to avoid a
	  query with no values that results in an SQL error)
- StateDB
	- In LocalStateDB, fix Reset when mt == nil
- Prover (new package)
	- Rename the interface to Prover
	- Rename the mock struct to Mock
	- Extend Prover interface methods to provide everything required by the
	  coordinator
	- Begin implementing required http client code to interact with server
	  proof (not tested)
- Synchronizer:
	- Add LastForgeL1TxsNum to Stats
- Test/Client
	- Update Auction logic to track slots in which there's no forge during
	  the time before the deadline (following the solidity implementation)
2020-12-03 13:00:06 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
Eduard S
457ff94bfa Test synchronizer reorg 2020-11-16 11:35:15 +01:00
arnaucube
900494fd8e StateDB group Fee txs and ZKInputs gen of Fee txs 2020-11-06 16:41:03 +01:00
Eduard S
5b6639a947 Add DebugAPI to Node, fix StateDB
- Allow starting the DebugAPI from the node via config
- In StateDB:
    - Make checkpoints when ProcessTxs() succeeds
    - Remove extra hardcoded `statedb` path that was redundant
    - Replace hardcoded `[:4]` by `[:]` when parsing idx, which failed because
      idx is 6 bytes length now.
- Extra: In node, use waitgroup instead of `stoppedXXX` channels to wait for
syncrhonizer goroutines to finish.
2020-11-06 13:00:11 +01:00
arnaucube
f65a6047b4 Add Til set w minimum flow, small fixes at StateDB 2020-11-02 12:47:34 +01:00
arnaucube
aaa71a6b11 StateDB store Idxs with TokenID at the db-key 2020-10-27 17:24:10 +01:00
arnau
6a8b454368 Merge pull request #211 from hermeznetwork/feature/update-general
Feature/update general
2020-10-21 13:06:09 +02:00
Eduard S
44c70dfbcb Replace cp -r in StateDB by checkpoints, Resolve #93 2020-10-20 14:52:46 +02:00
Eduard S
6afbda5302 Extend statedb and use prefixes, add debugapi
In statedb:
- Store all values using prefixes for keys to allow iteration
- Add methods MTGetRoot, GetAccounts

Implement debugapi, an http server with debugging endpoints:
- debugAPI.GET("sdb/batchnum", a.handleCurrentBatch)
- debugAPI.GET("sdb/mtroot", a.handleMTRoot)
- debugAPI.GET("sdb/accounts", a.handleAccounts)
- debugAPI.GET("sdb/accounts/:Idx", a.handleAccount)
2020-10-19 13:53:39 +02:00
Arnau B
5097939b12 Avoid using pointers in common 2020-10-08 16:42:16 +02:00
a_bennassar
49366e3fa4 Feature/null refactor (#173)
* WIP: rebase

* Make nullable fields use pointers
2020-10-06 10:34:49 +02:00
arnaucube
3374a4754d Add StateDB compute nonces on ProcessTx L2Tx
Add StateDB compute nonces on ProcessTx L2Tx, and update StateDB for
type TypeSynchronizer, TypeTxSelector, TypeBatchBuilder
2020-10-02 11:54:58 +02:00
arnaucube
9bb4a4ec1b Add TxID calculation & New{Layer}Tx Type
Add TxID calculation & New{Layer}Tx Type

New{Layer}Tx methods that compute the `TxID` & `TxType` values from the
transaction values:
- NewL1Tx
- NewL2Tx
- NewPoolL2Tx

Add TxID Scanner & Valuer for database/sql

HistoryDB & L2DB & API packages tests will need to be addapted to the
TestTransaction generation once is done.
2020-09-30 15:37:13 +02:00
arnaucube
06703aecad Idx to 6 bytes (48 bits) 2020-09-22 18:01:14 +02:00
arnaucube
934b6120ae Add StateDB set & get Idx by EthAddr & BJJ methods 2020-09-21 22:02:20 +02:00
Eduard S
9f83fcf520 Review synchronizer, rearange some stuff, add notes and TODOs. 2020-09-21 14:33:07 +02:00
arnaucube
69fe471f11 Add StateTree leafs & ExtTree data ZKI calculation 2020-09-10 17:58:08 +02:00
arnaucube
aa0bde61d2 Add StateDB ZKInputs generation for L1 & L2 Txs 2020-09-09 14:40:55 +02:00
arnaucube
b1454d441c StateDB intermediate state reset when opening DB
StateDB intermediate state reset when opening DB to force getting always last
Checkpoint at last BatchNum, avoiding inconsistent intermediate state.
2020-09-07 12:19:15 +02:00
arnaucube
fdaf2c9ce2 Update log package with fields & file log
Update log package with fields & file log, and update constants in the
repository code to be MixedCase.

resolves #91, resolves #92, resolves #95
2020-09-02 10:56:48 +02:00
arnaucube
cbbcb65c8c Add lint checks: gofmt, goimports, golint
- gofmt - Gofmt checks whether code was gofmt-ed. By default this tool runs
  with -s option to check for code simplification
- goimports - Goimports does everything that gofmt does. Additionally it checks
  unused imports
- golint - Golint differs from gofmt. Gofmt reformats Go source code, whereas
  golint prints out style mistakes
    - checks the uncommented exported functions & types

Update the code to fix the lint checks.
2020-08-31 12:32:05 +02:00
arnaucube
9309722dfc Update StateDB to new & ExitInfo struct & BatchNum
Update StateDB to new & ExitInfo struct & BatchNum
Also a small fix at txselector & log packages
2020-08-26 18:28:53 +02:00
arnaucube
0d2004721c Add statedb ExitTree implementation 2020-08-25 12:09:49 +02:00
arnaucube
b4044a2ef7 Add abstraction method of processTxs to StateDB
- Update GHA lint.yml increasing timeout time to avoid GHA Lint errors
- Update common.BatchNum & common.Idx & common.Nonce usage in StateDB
- Add abstraction method of processTxs to StateDB
    - Which will be used by Synchronizer & BatchBuilder
2020-08-24 13:06:51 +02:00
arnaucube
12aa31e46b Unify StateDB MT{Create/Update}Account
Previously as the txprocessor.go methods were specific for the BatchBuilder,
the MTCreateAccount & CreateAccount and MTUpdateAccount & UpdateAccount were
also designed to be used by BatchBuilder and TxSelector depending on the
MerkleTree usage calling one kind of method or anotherone.

But now that this methods are being called directly by the StateDB (through the
methods in txprocessors.go), to allow also the methods usage from the
Synchronizer, there can not be the MT and no-MT methods separated, so this
commit unifies MTCreateAccount with CreateAccount, and MTUpdateAccount with
UpdateAccount, which internally will update the MerkleTree depending if the
specific StateDB in usage has the MerkleTree defined or not.
2020-08-20 15:53:16 +02:00
arnaucube
cfa441e1ac Migrate TxProcessors from BatchBuilder to StateDB
Migrate TxProcessors from BatchBuilder to StateDB in order to be used by
BatchBuilder & Synchronizer
2020-08-20 12:06:28 +02:00
arnaucube
aa003063f6 Update BatchBuilder & TxSelector to new StateDB 2020-08-17 13:38:48 +02:00
arnaucube
205db8e4d3 Implement StateDB Checkpoints & Resets system 2020-08-17 11:27:33 +02:00
arnaucube
a1c339c918 Add BatchQueue implementation & minor updates 2020-08-13 18:25:50 +02:00
arnaucube
60fa76b4d2 Integrate TxSelector with StateDB 2020-08-13 13:07:30 +02:00
arnau
edb8974a5a Apply fix method description typos
Co-authored-by: Eduard S. <eduard@iden3.io>
2020-08-10 16:46:15 +02:00
arnaucube
d8cb7298e0 Add StateDB & LocalStateDB 2020-08-07 10:59:30 +02:00