Commit Graph

575 Commits

Author SHA1 Message Date
laisolizq
75c6d8f14e Updates API to new changes 2020-12-04 11:22:12 +01:00
Eduard S
329d2796c6 Update some eth types, update eth dependencies to new changes 2020-12-03 18:05:33 +01:00
Arnau B
3e3eb3ffe6 Update API mock server URL for swagger UI 2020-12-03 17:47:16 +01:00
laisolizq
18031343ec Update ethclient 2020-12-03 14:19:38 +01:00
Eduard S
0bacd6052d Merge pull request #323 from hermeznetwork/feature/zkinputs3
StateDB ZKInputs generation compatible with circom
2020-12-03 13:28:35 +01:00
arnaucube
80cbac24f9 StateDB ZKInputs generation compatible with circom
- Add SignatureConstant to L1Tx.TxCompressedData
- ZKInputs generation works initial version working and compatible with circom Hermez circuit
    - Lots of small updates for circuits compatibility
- Update version (with new changes) of `go-iden3-crypto` & `go-merkletree`
2020-12-03 13:16:51 +01:00
arnau
d306f4890a Merge pull request #322 from hermeznetwork/feature/integration20
Advance coordinator implementation
2020-12-03 13:16:07 +01:00
Eduard S
482c94d374 Advance coordinator implementation
- Common
	- Move ErrTODO and ErrDone to common for usage where needed.
- Coordinator
	- Move prover types to prover package
	- Handle reorgs, stopping the pipeline when necessary
	- Handle ethereum transaction errors by stopping the pipeline
	- In case of ethereum transaction revert, check for known revert causes
	  (more revert causes can be added to handle more cases)
	- Fix skipped transactions in TxManager confirmation logic
	- Cancel and wait for provers to be ready
	- Connect L2DB to:
		- purge l2txs due to timeout
		- mark l2txs at the different states
	- Connect HistoryDB to query L1UserTxs to forge in an L1Batch
- L2DB
	- Skip update functions when the input slices have no values (to avoid a
	  query with no values that results in an SQL error)
- StateDB
	- In LocalStateDB, fix Reset when mt == nil
- Prover (new package)
	- Rename the interface to Prover
	- Rename the mock struct to Mock
	- Extend Prover interface methods to provide everything required by the
	  coordinator
	- Begin implementing required http client code to interact with server
	  proof (not tested)
- Synchronizer:
	- Add LastForgeL1TxsNum to Stats
- Test/Client
	- Update Auction logic to track slots in which there's no forge during
	  the time before the deadline (following the solidity implementation)
2020-12-03 13:00:06 +01:00
arnau
f58e79d21b Merge pull request #321 from hermeznetwork/feature/til-gen-from-instructions
Allow using til with instructions
2020-12-03 11:38:09 +01:00
Arnau B
d9121bf5a6 Allow using til with instructions 2020-12-02 16:01:17 +01:00
a_bennassar
7155743820 Merge pull request #316 from hermeznetwork/feature/set-effectives
Set EffectiveAmount and EffectiveLoadAmount of forged L1UserTxs
2020-11-30 16:54:03 +01:00
Eduard S
665e0829ec Set EffectiveAmount and EffectiveLoadAmount of forged L1UserTxs 2020-11-30 16:25:54 +01:00
Eduard S
16d04de489 Merge pull request #315 from hermeznetwork/feature/rm-ethvar
Rm unused ETHCLIENT_DIAL_URL
2020-11-30 15:59:16 +01:00
arnaucube
2d272548b2 Rm unused ETHCLIENT_DIAL_URL 2020-11-30 14:15:02 +01:00
a_bennassar
a8c9b7a604 Merge pull request #304 from hermeznetwork/feature/tracerr1
Wrap all errors with tracerr
2020-11-30 13:08:42 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
Eduard S
879af66a2a Use tracerr call traces in log 2020-11-30 12:58:05 +01:00
Eduard S
aaeec88149 Merge pull request #306 from hermeznetwork/feature/massive-migrations-update
Update Common & StateDB & ZKInputs to last protocol version
2020-11-30 12:55:42 +01:00
arnaucube
dde9250429 Update Common & StateDB & ZKInputs to last protocol version
- Add InvalidData flag to L1Tx
- Add BytesDataAvailability to L1Tx
- Update ZKInputs & HashGlobalInputs to last spec of the protocol
(massive migrations)
- TxProcessor check correctness of L1Txs

Compatible with hermeznetwork/commonjs v0.0.4
(c345239bba)
2020-11-30 11:26:15 +01:00
Eduard S
f30e470028 Merge pull request #302 from hermeznetwork/feature/swagger-examples
Improve swagger doc
2020-11-27 12:18:44 +01:00
Arnau B
5991752e30 Improve swagger doc 2020-11-27 10:45:21 +01:00
arnau
0c83417881 Merge pull request #301 from hermeznetwork/feature/integration19
Update coordinator, call all api update functions
2020-11-26 18:10:29 +01:00
Eduard S
8f1cf2f145 Update coordinator, call all api update functions
- Common:
	- Rename Block.EthBlockNum to Block.Num to avoid unneeded repetition
- API:
	- Add UpdateNetworkInfoBlock to update just block information, to be
	  used when the node is not yet synchronized
- Node:
	- Call API.UpdateMetrics and UpdateRecommendedFee in a loop, with
	  configurable time intervals
- Synchronizer:
	- When mapping events by TxHash, use an array to support the possibility
	  of multiple calls of the same function happening in the same
	  transaction (for example, a smart contract in a single transaction
	  could call withdraw with delay twice, which would generate 2 withdraw
	  events, and 2 deposit events).
	- In Stats, keep entire LastBlock instead of just the blockNum
	- In Stats, add lastL1BatchBlock
	- Test Stats and SCVars
- Coordinator:
	- Enable writing the BatchInfo in every step of the pipeline to disk
	  (with JSON text files) for debugging purposes.
	- Move the Pipeline functionality from the Coordinator to its own struct
	  (Pipeline)
	- Implement shouldL1lL2Batch
	- In TxManager, implement logic to perform several attempts when doing
	  ethereum node RPC calls before considering the error. (Both for calls
	  to forgeBatch and transaction receipt)
	- In TxManager, reorganize the flow and note the specific points in
	  which actions are made when err != nil
- HistoryDB:
	- Implement GetLastL1BatchBlockNum: returns the blockNum of the latest
	  forged l1Batch, to help the coordinator decide when to forge an
	  L1Batch.
- EthereumClient and test.Client:
	- Update EthBlockByNumber to return the last block when the passed
	  number is -1.
2020-11-26 18:06:30 +01:00
Eduard S
4d830fdc89 Merge pull request #300 from hermeznetwork/feature/l1coord-update
Update L1 coordinator Txs
2020-11-25 16:41:23 +01:00
Eduard S
58c9be3644 Merge pull request #299 from hermeznetwork/fix/til-bigint
Til use bigint for amount & loadamount values
2020-11-25 11:38:14 +01:00
laisolizq
7ceaf4c32b Update L1 coordinator Txs 2020-11-25 11:04:23 +01:00
arnaucube
ea7e2546c7 Til use bigint for amount & loadamount values 2020-11-24 21:38:54 +01:00
a_bennassar
fd4e7ac315 Merge pull request #295 from hermeznetwork/feature/integration17
Redo coordinator structure, connect API to node
2020-11-24 19:07:35 +01:00
Eduard S
ebecc5c671 Check all errors in eth tests 2020-11-24 18:59:36 +01:00
Eduard S
3b99953007 Redo coordinator structure, connect API to node
- API:
	- Modify the constructor so that hardcoded rollup constants don't need
	  to be passed (introduce a `Config` and use `configAPI` internally)
- Common:
	- Update rollup constants with proper *big.Int when required
	- Add BidCoordinator and Slot structs used by the HistoryDB and
	  Synchronizer.
	- Add helper methods to AuctionConstants
	- AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL
	  table: `default_slot_set_bid_slot_num`), which indicates at which
	  slotNum does the `DefaultSlotSetBid` specified starts applying.
- Config:
	- Move coordinator exclusive configuration from the node config to the
	  coordinator config
- Coordinator:
	- Reorganize the code towards having the goroutines started and stopped
	  from the coordinator itself instead of the node.
	- Remove all stop and stopped channels, and use context.Context and
	  sync.WaitGroup instead.
	- Remove BatchInfo setters and assing variables directly
	- In ServerProof and ServerProofPool use context instead stop channel.
	- Use message passing to notify the coordinator about sync updates and
	  reorgs
	- Introduce the Pipeline, which can be started and stopped by the
	  Coordinator
	- Introduce the TxManager, which manages ethereum transactions (the
	  TxManager is also in charge of making the forge call to the rollup
	  smart contract).  The TxManager keeps ethereum transactions and:
	  	1. Waits for the transaction to be accepted
		2. Waits for the transaction to be confirmed for N blocks
	- In forge logic, first prepare a batch and then wait for an available
	  server proof to have all work ready once the proof server is ready.
	- Remove the `isForgeSequence` method which was querying the smart
	  contract, and instead use notifications sent by the Synchronizer to
	  figure out if it's forging time.
	- Update test (which is a minimal test to manually see if the
	  coordinator starts)
- HistoryDB:
	- Add method to get the number of batches in a slot (used to detect when
	  a slot has passed the bid winner forging deadline)
	- Add method to get the best bid and associated coordinator of a slot
	  (used to detect the forgerAddress that can forge the slot)
- General:
	- Rename some instances of `currentBlock` to `lastBlock` to be more
	  clear.
- Node:
	- Connect the API to the node and call the methods to update cached
	  state when the sync advances blocks.
	- Call methods to update Coordinator state when the sync advances blocks
	  and finds reorgs.
- Synchronizer:
	- Add Auction field in the Stats, which contain the current slot with
	  info about highest bidder and other related info required to know who
	  can forge in the current block.
	- Better organization of cached state:
		- On Sync, update the internal cached state
		- On Init or Reorg, load the state from HistoryDB into the
		  internal cached state.
2020-11-24 18:59:30 +01:00
Eduard S
29f6d975d1 Merge pull request #293 from hermeznetwork/feature/swagger-examples
Improve swagger doc
2020-11-24 12:30:40 +01:00
Arnau B
86029ca7de Improve swagger doc 2020-11-24 11:39:29 +01:00
Eduard S
486c70e935 Merge pull request #297 from hermeznetwork/feature/ethclient-update
Update ethclient & L1CoordinatorTx
2020-11-24 10:31:15 +01:00
Eduard S
132691bb43 Merge pull request #296 from hermeznetwork/feature/zkinputs2
Add methods for ZKInputs IntermStates generation
2020-11-23 18:11:45 +01:00
laisolizq
f775af8890 Update ethclient & L1CoordinatorTx 2020-11-23 17:40:45 +01:00
arnaucube
d3a38a3ee1 Add methods for ZKInputs IntermStates generation
- Add L1Tx TxCompressedData method
- Add PoolL2Tx TxCompressedDataV2 method
- Update ProcessTxs logic
- Add ZKInputs Intermediate States & Fee parameters calculation
2020-11-23 17:28:26 +01:00
a_bennassar
bf88eb60b8 Merge pull request #292 from hermeznetwork/feature/fixes
Update SQL types, improve SQL *big.Int
2020-11-18 15:15:10 +01:00
Eduard S
738fa0ac30 Update SQL types, improve SQL *big.Int
- In SQL Tables, set rollup_vars.forge_l1_timeout to BIGINT (instead of BYTEA)
- In "bigint" encoding/decoding used with meddler, store the *big.Int as raw
  bytes, instead of base64 to be more space efficient
2020-11-18 13:26:46 +01:00
Eduard S
0e7d3e72fd Merge pull request #287 from hermeznetwork/feature/api-test-til
Add til to API
2020-11-17 17:59:03 +01:00
Eduard S
98aa5d791f Merge pull request #289 from hermeznetwork/feature/zki-hashGlobal
Add HashGlobalInputs for ZKInputs
2020-11-17 17:54:58 +01:00
Arnau B
ce1218e28a Integrate til into api 2020-11-17 17:50:07 +01:00
arnaucube
ce772b1d19 Add HashGlobalInputs for ZKInputs
Add HashGlobalInputs for ZKInputs compatible with js & circom circuits version.

Compatible with hermeznetwork/commonjs at version: c6a1448db5bae4cda839ce36c1f35d8defccc9cd
(c6a1448db5)
2020-11-17 16:33:29 +01:00
laisolizq
b3b23473b6 Add til api test first commit 2020-11-17 11:24:19 +01:00
laisolizq
4aa686797d Merge pull request #284 from hermeznetwork/feature/api-db-test-til-tx
HistoryDB TXs til test
2020-11-17 10:34:26 +01:00
a_bennassar
e0a4b656fe Merge pull request #279 from hermeznetwork/feature/l2db-test-til
Add til test l2db
2020-11-16 18:13:50 +01:00
ToniRamirezM
93d67ff699 HistoryDB TXs til test 2020-11-16 16:27:18 +01:00
laisolizq
1925110fda Add til test l2db 2020-11-16 13:35:24 +01:00
a_bennassar
f5c92418f2 Merge pull request #280 from hermeznetwork/feature/integration15
Add Sync stats, and report them in DebugAPI
2020-11-16 12:13:00 +01:00
Eduard S
6c0d48f4a2 Add Sync stats, and report them in DebugAPI 2020-11-16 11:48:36 +01:00
a_bennassar
7294d766cd Merge pull request #278 from hermeznetwork/feature/integration14
Test synchronizer reorg
2020-11-16 11:46:38 +01:00