Commit Graph

743 Commits

Author SHA1 Message Date
laisolizq
4ad67a3d18 Update prover & add test 2020-12-09 16:07:45 +01:00
Eduard S
f229f49f0e Merge pull request #334 from hermeznetwork/feature/test-zkinputs2
Add zki.ISAccFeeOut generation for circuits compat
2020-12-09 15:34:07 +01:00
arnau
347d18878e Merge pull request #332 from hermeznetwork/feature/integration21
Integrate purger to node
2020-12-09 14:55:03 +01:00
Arnau B
039c7b29da Fix time sensitive test 2020-12-09 14:12:10 +01:00
Arnau B
a50ddbe4e0 Fix hez addr format for exits 2020-12-09 14:04:15 +01:00
Arnau B
87501276ce Fix errors when fromItem is not provided 2020-12-09 13:59:10 +01:00
arnaucube
c162563376 Add zki.ISAccFeeOut generation for circuits compat
- add IntermediateStates AccFeeOut correct generation for circuits
compatibility at ZKInputs
- Fix formatAccumulatedFees
- organize ZKInputs tests to check compatibility with Circom circuits
- Add TestZKInputs2
    - which checks ZKInputs for 4 L1 txs + 2 L2Txs with fees
    - tested with Hermez Circom circuits
2020-12-09 12:47:00 +01:00
Eduard S
900d1fb6ce Integrate purger to node
- Common
	- Add `IdxNonce` type used to track nonces in accounts to invalidate
	  l2txs in the pool
- Config
	- Update coordinator config will all the new configuration parameters
	  used in the coordinator
- Coordinator
	- Introduce the `Purger` to track how often to purge and do the job when
	  needed according to a configuration.
	- Implement the methods to invalidate l2txs transactions due to l2txs
	  selection in batches.  For now these functions are not used in favour
	  of the `Purger` methods, which check ALL the l2txs in the pool.
	- Call Invalidation and Purging methods of the purger both when the
	  node is forging (in the pipeline when starting a new batch) and when
	  the node is not forging (in coordinator when being notified about a
	  new synced block)
- L2DB:
	- Implement `GetPendingUniqueFromIdxs` to get all the unique idxs from
	  pending transactions (used to get their nonces and then invalidate
	  txs)
	- Redo `CheckNonces` with a single SQL query and using `common.IdxNonce`
	  instead of `common.Account`
- StateDB:
	- Expose GetIdx to check errors when invalidating pool txs
- Synchronizer:
	- Test forged L1UserTxs processed by TxProcessor
	- Improve checks of Effective values
- TxSelector:
	- Expose the internal LocalStateDB in order to check account nonces in
	  the coordinator when not forging.
2020-12-09 12:43:16 +01:00
Eduard S
8de7fe537a Merge pull request #330 from hermeznetwork/feature/zki4-fees
ZKInputs with Fee txs compatible with circom
2020-12-07 16:57:40 +01:00
arnaucube
c1fd0d6626 ZKInputs with Fee txs compatible with circom
- Separate ZKInputs tests at StateDB package
- Small fix at ZKInputs generation FeeIdxs length
- Fees related parameters at ZKInputs works properly
- Add ZKInputs generation test with fees (`TestZKInputs1`), which output
has been tested with circom to ensure compatibility
2020-12-07 12:54:18 +01:00
arnau
4771cf5084 Merge pull request #329 from hermeznetwork/feature/deposit-amount
Rename load amount to deposit amount
2020-12-07 12:21:26 +01:00
Arnau B
445f26ec9f Rename load amount to deposit amount 2020-12-04 16:13:41 +01:00
laisolizq
2dd6d82dbe Merge pull request #328 from hermeznetwork/feature/ganache-bug-workaround
syncrhonizer fix, ganache bug workaround
2020-12-04 12:58:46 +01:00
Eduard S
1370bcca03 Use a non-zero from address in eth calls 2020-12-04 12:44:29 +01:00
Eduard S
a4a193c7f2 Fix missing EthBlockNum in DB for Coordinator 2020-12-04 12:35:52 +01:00
Eduard S
897d7d5319 Merge pull request #326 from hermeznetwork/feature/effective-amount-update
Feature/effective amount update
2020-12-04 12:33:30 +01:00
Arnau B
5fd258ee07 Use flags for effective amounts in SQL schema 2020-12-04 12:17:09 +01:00
a_bennassar
0602332d88 Merge pull request #325 from hermeznetwork/feature/ethclient-up-ev
Update ethclient
2020-12-04 11:58:52 +01:00
laisolizq
75c6d8f14e Updates API to new changes 2020-12-04 11:22:12 +01:00
Eduard S
329d2796c6 Update some eth types, update eth dependencies to new changes 2020-12-03 18:05:33 +01:00
Arnau B
3e3eb3ffe6 Update API mock server URL for swagger UI 2020-12-03 17:47:16 +01:00
laisolizq
18031343ec Update ethclient 2020-12-03 14:19:38 +01:00
Eduard S
0bacd6052d Merge pull request #323 from hermeznetwork/feature/zkinputs3
StateDB ZKInputs generation compatible with circom
2020-12-03 13:28:35 +01:00
arnaucube
80cbac24f9 StateDB ZKInputs generation compatible with circom
- Add SignatureConstant to L1Tx.TxCompressedData
- ZKInputs generation works initial version working and compatible with circom Hermez circuit
    - Lots of small updates for circuits compatibility
- Update version (with new changes) of `go-iden3-crypto` & `go-merkletree`
2020-12-03 13:16:51 +01:00
arnau
d306f4890a Merge pull request #322 from hermeznetwork/feature/integration20
Advance coordinator implementation
2020-12-03 13:16:07 +01:00
Eduard S
482c94d374 Advance coordinator implementation
- Common
	- Move ErrTODO and ErrDone to common for usage where needed.
- Coordinator
	- Move prover types to prover package
	- Handle reorgs, stopping the pipeline when necessary
	- Handle ethereum transaction errors by stopping the pipeline
	- In case of ethereum transaction revert, check for known revert causes
	  (more revert causes can be added to handle more cases)
	- Fix skipped transactions in TxManager confirmation logic
	- Cancel and wait for provers to be ready
	- Connect L2DB to:
		- purge l2txs due to timeout
		- mark l2txs at the different states
	- Connect HistoryDB to query L1UserTxs to forge in an L1Batch
- L2DB
	- Skip update functions when the input slices have no values (to avoid a
	  query with no values that results in an SQL error)
- StateDB
	- In LocalStateDB, fix Reset when mt == nil
- Prover (new package)
	- Rename the interface to Prover
	- Rename the mock struct to Mock
	- Extend Prover interface methods to provide everything required by the
	  coordinator
	- Begin implementing required http client code to interact with server
	  proof (not tested)
- Synchronizer:
	- Add LastForgeL1TxsNum to Stats
- Test/Client
	- Update Auction logic to track slots in which there's no forge during
	  the time before the deadline (following the solidity implementation)
2020-12-03 13:00:06 +01:00
arnau
f58e79d21b Merge pull request #321 from hermeznetwork/feature/til-gen-from-instructions
Allow using til with instructions
2020-12-03 11:38:09 +01:00
Arnau B
d9121bf5a6 Allow using til with instructions 2020-12-02 16:01:17 +01:00
a_bennassar
7155743820 Merge pull request #316 from hermeznetwork/feature/set-effectives
Set EffectiveAmount and EffectiveLoadAmount of forged L1UserTxs
2020-11-30 16:54:03 +01:00
Eduard S
665e0829ec Set EffectiveAmount and EffectiveLoadAmount of forged L1UserTxs 2020-11-30 16:25:54 +01:00
Eduard S
16d04de489 Merge pull request #315 from hermeznetwork/feature/rm-ethvar
Rm unused ETHCLIENT_DIAL_URL
2020-11-30 15:59:16 +01:00
arnaucube
2d272548b2 Rm unused ETHCLIENT_DIAL_URL 2020-11-30 14:15:02 +01:00
a_bennassar
a8c9b7a604 Merge pull request #304 from hermeznetwork/feature/tracerr1
Wrap all errors with tracerr
2020-11-30 13:08:42 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
Eduard S
879af66a2a Use tracerr call traces in log 2020-11-30 12:58:05 +01:00
Eduard S
aaeec88149 Merge pull request #306 from hermeznetwork/feature/massive-migrations-update
Update Common & StateDB & ZKInputs to last protocol version
2020-11-30 12:55:42 +01:00
arnaucube
dde9250429 Update Common & StateDB & ZKInputs to last protocol version
- Add InvalidData flag to L1Tx
- Add BytesDataAvailability to L1Tx
- Update ZKInputs & HashGlobalInputs to last spec of the protocol
(massive migrations)
- TxProcessor check correctness of L1Txs

Compatible with hermeznetwork/commonjs v0.0.4
(c345239bba)
2020-11-30 11:26:15 +01:00
Eduard S
f30e470028 Merge pull request #302 from hermeznetwork/feature/swagger-examples
Improve swagger doc
2020-11-27 12:18:44 +01:00
Arnau B
5991752e30 Improve swagger doc 2020-11-27 10:45:21 +01:00
arnau
0c83417881 Merge pull request #301 from hermeznetwork/feature/integration19
Update coordinator, call all api update functions
2020-11-26 18:10:29 +01:00
Eduard S
8f1cf2f145 Update coordinator, call all api update functions
- Common:
	- Rename Block.EthBlockNum to Block.Num to avoid unneeded repetition
- API:
	- Add UpdateNetworkInfoBlock to update just block information, to be
	  used when the node is not yet synchronized
- Node:
	- Call API.UpdateMetrics and UpdateRecommendedFee in a loop, with
	  configurable time intervals
- Synchronizer:
	- When mapping events by TxHash, use an array to support the possibility
	  of multiple calls of the same function happening in the same
	  transaction (for example, a smart contract in a single transaction
	  could call withdraw with delay twice, which would generate 2 withdraw
	  events, and 2 deposit events).
	- In Stats, keep entire LastBlock instead of just the blockNum
	- In Stats, add lastL1BatchBlock
	- Test Stats and SCVars
- Coordinator:
	- Enable writing the BatchInfo in every step of the pipeline to disk
	  (with JSON text files) for debugging purposes.
	- Move the Pipeline functionality from the Coordinator to its own struct
	  (Pipeline)
	- Implement shouldL1lL2Batch
	- In TxManager, implement logic to perform several attempts when doing
	  ethereum node RPC calls before considering the error. (Both for calls
	  to forgeBatch and transaction receipt)
	- In TxManager, reorganize the flow and note the specific points in
	  which actions are made when err != nil
- HistoryDB:
	- Implement GetLastL1BatchBlockNum: returns the blockNum of the latest
	  forged l1Batch, to help the coordinator decide when to forge an
	  L1Batch.
- EthereumClient and test.Client:
	- Update EthBlockByNumber to return the last block when the passed
	  number is -1.
2020-11-26 18:06:30 +01:00
Eduard S
4d830fdc89 Merge pull request #300 from hermeznetwork/feature/l1coord-update
Update L1 coordinator Txs
2020-11-25 16:41:23 +01:00
Eduard S
58c9be3644 Merge pull request #299 from hermeznetwork/fix/til-bigint
Til use bigint for amount & loadamount values
2020-11-25 11:38:14 +01:00
laisolizq
7ceaf4c32b Update L1 coordinator Txs 2020-11-25 11:04:23 +01:00
arnaucube
ea7e2546c7 Til use bigint for amount & loadamount values 2020-11-24 21:38:54 +01:00
a_bennassar
fd4e7ac315 Merge pull request #295 from hermeznetwork/feature/integration17
Redo coordinator structure, connect API to node
2020-11-24 19:07:35 +01:00
Eduard S
ebecc5c671 Check all errors in eth tests 2020-11-24 18:59:36 +01:00
Eduard S
3b99953007 Redo coordinator structure, connect API to node
- API:
	- Modify the constructor so that hardcoded rollup constants don't need
	  to be passed (introduce a `Config` and use `configAPI` internally)
- Common:
	- Update rollup constants with proper *big.Int when required
	- Add BidCoordinator and Slot structs used by the HistoryDB and
	  Synchronizer.
	- Add helper methods to AuctionConstants
	- AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL
	  table: `default_slot_set_bid_slot_num`), which indicates at which
	  slotNum does the `DefaultSlotSetBid` specified starts applying.
- Config:
	- Move coordinator exclusive configuration from the node config to the
	  coordinator config
- Coordinator:
	- Reorganize the code towards having the goroutines started and stopped
	  from the coordinator itself instead of the node.
	- Remove all stop and stopped channels, and use context.Context and
	  sync.WaitGroup instead.
	- Remove BatchInfo setters and assing variables directly
	- In ServerProof and ServerProofPool use context instead stop channel.
	- Use message passing to notify the coordinator about sync updates and
	  reorgs
	- Introduce the Pipeline, which can be started and stopped by the
	  Coordinator
	- Introduce the TxManager, which manages ethereum transactions (the
	  TxManager is also in charge of making the forge call to the rollup
	  smart contract).  The TxManager keeps ethereum transactions and:
	  	1. Waits for the transaction to be accepted
		2. Waits for the transaction to be confirmed for N blocks
	- In forge logic, first prepare a batch and then wait for an available
	  server proof to have all work ready once the proof server is ready.
	- Remove the `isForgeSequence` method which was querying the smart
	  contract, and instead use notifications sent by the Synchronizer to
	  figure out if it's forging time.
	- Update test (which is a minimal test to manually see if the
	  coordinator starts)
- HistoryDB:
	- Add method to get the number of batches in a slot (used to detect when
	  a slot has passed the bid winner forging deadline)
	- Add method to get the best bid and associated coordinator of a slot
	  (used to detect the forgerAddress that can forge the slot)
- General:
	- Rename some instances of `currentBlock` to `lastBlock` to be more
	  clear.
- Node:
	- Connect the API to the node and call the methods to update cached
	  state when the sync advances blocks.
	- Call methods to update Coordinator state when the sync advances blocks
	  and finds reorgs.
- Synchronizer:
	- Add Auction field in the Stats, which contain the current slot with
	  info about highest bidder and other related info required to know who
	  can forge in the current block.
	- Better organization of cached state:
		- On Sync, update the internal cached state
		- On Init or Reorg, load the state from HistoryDB into the
		  internal cached state.
2020-11-24 18:59:30 +01:00
Eduard S
29f6d975d1 Merge pull request #293 from hermeznetwork/feature/swagger-examples
Improve swagger doc
2020-11-24 12:30:40 +01:00
Arnau B
86029ca7de Improve swagger doc 2020-11-24 11:39:29 +01:00