Commit Graph

167 Commits

Author SHA1 Message Date
arnaucube
a8f6891aea Add PoolL2Tx.Info about the status of the tx
PoolL2Tx.Info contains information about the status & State of the
transaction. As for example, if the Tx has not been selected in the last
batch due not enough Balance at the Sender account, this reason would
appear at this parameter.
This will help the client (wallet, batchexplorer, etc) to reason why a
L2Tx is not selected in the forged batches.
2021-02-03 13:03:30 +01:00
Eduard S
510d5c548c Merge pull request #520 from hermeznetwork/feature/txid-keccak256
Update TxID to use eth Keccak256
2021-02-01 16:33:37 +01:00
Alberto Elias
cea3af4291 Merge pull request #518 from hermeznetwork/fix/api-recommended-fee
Fix api resopnse format for recommended fee
2021-02-01 15:04:15 +00:00
arnaucube
c1cd37913f Update TxID to use eth Keccak256
- Update TxID to use eth Keccak256
- Added more tests to L2Tx TxID calculation to check compatibility with
js version from https://github.com/hermeznetwork/hermezjs/pull/57
2021-02-01 16:01:55 +01:00
Arnau B
dd1657847e Fix api resopnse format for recommended fee 2021-02-01 13:40:24 +01:00
Eduard S
90fe98d15c Update contracts and get verifiersLen from SC 2021-01-29 17:46:14 +01:00
arnaucube
aade6220d4 Update TxID to avoid collisions on DB (fix #503) 2021-01-29 11:38:22 +01:00
arnaucube
a899f43914 Add TokenID to L2Tx
L2Tx.TokenID is not on the data obtained by the Synchronizer from the
blockchain, but is set by the TxProcessor when processing the
transactions in the StateDB.
2021-01-27 12:40:09 +01:00
Eduard S
ec194d5066 Set l1tx.EffectiveFromIdx in TxProcessor and til 2021-01-13 18:20:47 +01:00
Eduard S
75f068baa8 Fix ChainID in zkInputs empty TxCmpData
Test TxProcessor zkInputs against a real proof server
2021-01-13 12:41:00 +01:00
ToniRamirezM
170da500db Remove big.Int from state API 2021-01-12 18:37:56 +01:00
arnaucube
b59710a149 ZKInput with L2Txs compatible with circom circuits
- Til
  - update Til users BJJ key generation for better js tests
    compatibility
- Common
  - PoolL2Tx to L2Tx use AuxToIdx in case that ToIdx is 0
  - Update ZKInputs parameter descriptions
- TxProcessor
  - Fix AccumulatedFees in case that there is no CoordIdx for that token
  - Fix zki.NewExit usage
  - Use same order for AccumulatedFees & FeeIdx & FeePlanTokens
  - Add Nonce usage to ExitLeafs
  - Update TestZKInput6 and check its compatibility with circom Hermez
circuits
2021-01-11 11:23:04 +01:00
Eduard S
a79cb4edfd Update node and apply some fixes
- Node
	- Load Coordinator Fee Account from config
		- Sign the AccountCreationMsg to generate the
		  AccountCreationAuth
		- Resolve #465
	- Wait for synchronizer termination before stopping coordinator to avoid
	  getting stuck when closing in the following case:
		- The coordinator stops reading the synchronizer msg channel,
		  and the node gets stuck sending a message to that channel.
- Common
	- Move account creation auth signature code to common.
	- Update RollupConstInputSHAConstantBytes
- Coordinator
	- Set batch status in the debug file
	- Propagate SCVariables on reorg
	- Pipeline: Get SCVariables updates
		- Resolve #457
	- Fix off by 1 error in Pipeline.shouldL1L2Batch() (which shouldn't have
	  caused any problem, but it was not right)
- KVDB
	- Delete future checkpoints after reset
	- In `ResetFromSynchronizer`, remove all checkpoints first, and follow
	  the same logic as `reset()`.
- Cli
	- Add command to generate a BabyJubJub key pair (to be used for the
	  Coordinator Fee Account)
- Node
	- Adjust example config `Coordinator.L1BatchTimeoutPerc` to avoid
	  missing the L1Batch deadline with the following setup:
		- a block is mined every 2 seconds
		- single proof server that takes 2 seconds to calculate a proof
- TxProcessor
	- Close temporary pebble used for the exit tree after usage.
		- Resolve #463
2021-01-07 16:09:28 +01:00
arnaucube
bf5ab12d0e Update Account.PublicKey to Account.BJJ
Update account.PublicKey to account.BJJ for naming consistency with the rest of the code
2021-01-05 15:34:50 +01:00
arnaucube
48bdec1397 Fix TxProcessor TokenID use &updates at Til&Common
- Til
  - tests that were using `til.SetBlockchainMinimumFlow0` have been
updated, as now the L1CoordinatorTxs are placed simulating TxSelector
creation (when needed, not before)
  - Add `EthSk` to `tc.User` to allow to sign `AccoutCreationAuths` at
the tests flows
- TxProcessor
  - for L2Txs get TokenID from tx.FromIdx Account.TokenID instead than
  - update tests vectors for new values
from tx.TokenID (which not always is set)
- Common
  - move TxIDsFromL2Txs & TxIDsFromPoolL2Txs from `coordinator` to
`common` to allow usage from other packages
2021-01-04 15:28:04 +01:00
Eduard S
18c854fbaa Add circuit configuration to node config
- Remove MaxL1CoordTxs parameter from the TxSelector because this parameter
  doesn't exist
- Use ChainID in l1tx byte encoding
- Pass txprocessor configuration to batch builder via an existing parameter
2020-12-31 10:44:19 +01:00
arnaucube
ca5870e3ba Update ExitProofs-Siblings padding 2020-12-30 10:44:34 +01:00
Eduard S
890b94a41a Merge pull request #428 from hermeznetwork/fix/api-next-forgers
Fix next forgers api response
2020-12-29 15:32:02 +01:00
Arnau B
6058d3c41a Fix next forgers api response 2020-12-29 15:17:55 +01:00
Eduard S
025b56c8c3 Merge pull request #425 from hermeznetwork/fix/zki-ntx
Fix ZKI NTx for MaxTx
2020-12-28 12:17:51 +01:00
arnaucube
cf199fd832 Fix ZKI NTx for MaxTx
On the Circuits uses the name `nTx` (and on the tests `maxTx`), but
Contracts `maxTx`. On Go we had both variables, but really only used
one, now at Go we use `MaxTx` to refer to the maximum number of
transactions.
2020-12-28 11:26:47 +01:00
arnaucube
ca010275e1 AccountCreationAuth compatible with js implementation 2020-12-28 11:16:00 +01:00
arnaucube
8615a462ab Update AccountCreationAuth & fix auth.HashToSign
- Fix AccountCreationAuth.HashToSign (was using `[]byte("0x...")`, which
uses the bytes of the string. Now uses the bytearray of the compressed
BJJ public key (compatible with js implementation))
- Update AccountCreationAuth to last specification (add to hash
parameters ChainID & HermezAddress)
- Add missing test to AccountCreationAuth
2020-12-24 10:41:35 +01:00
Eduard S
8267d007c9 Sync ForgerCommitment and use it in coord
Previously the coordinator was erroneously using Slot.BatchesLen to determine
when anyone can forge.  The correct behaviour is implmenented with the boolean
flag `ForgerCommitment`, that is set to true only when there's a batch before
the deadline within a slot.

Delete Slot.BatchesLen, and the synchronization code of this value from the
Synchronizer, as this is not needed
2020-12-23 17:06:20 +01:00
arnaucube
150597c282 Parametrize ChainID 2020-12-23 15:23:48 +01:00
arnaucube
2583d35adf Update ZKInputs for empty batches
- Add TestZKInputsEmpty circom test vectors
- Fix ZKInputsHash OldLastIdx bytearray
2020-12-23 11:09:43 +01:00
arnaucube
4b10549822 mv of babyjub.PublicKey to babyjub.PublicKeyComp
Update usage of `*babyjub.PublicKey` to `babyjub.PublicKeyComp`
- when the key is not defined, internally is used `babyjub.EmptyBJJComp`, which is a `[32]byte` of zeroes of type `babyjub.PublicKeyComp`
- the API continues returning `nil` when the key is not defined
2020-12-22 11:29:47 +01:00
Eduard S
b1a8384f27 Run go-instrument-errors to wrap errors 2020-12-18 13:39:42 +01:00
Eduard S
7a30294688 Update txs constructors and helpers
For each tx, move the logic of setting the Type and TxID to separate functions,
so that they can be called when necessary.

In synchronizer, set all the required fields using the `SetID` and `SetType`
for l2txs when needed.  This is necessary because the `ProcessTxs()` works with
`common.PoolL2Tx`, which misses some fields from `common.L2Tx`, and because
`ProcessTxs()` needs the Type to be set, but at the same time `ProcessTxs()`
sets the Nonce, which is required for the `TxID`.
2020-12-16 15:25:55 +01:00
Eduard S
d15721d28a Merge pull request #350 from hermeznetwork/feature/zkinputs6
Update ZKInputs generation:
2020-12-15 15:44:31 +01:00
Eduard S
a7351992cd Make coordinator more responsive
- API:
	- Replace `emergencyModeStaringTime` by `emercengyModeStartingBlock`
- Synchronizer:
	- Track emergency mode starting block
- cli/node
	- Add working coordinator config
- coordinator:
	- Retry handler for synchronizer stats in case of error (instead of
	  waiting for the next block to try again)
	- On init, trigger an initial call to the handler for synced block
	  before waiting for the synchronizer, to force the coordinator to start
	  its logic even if there's no new block right after the node has been
	  started (very useful for running in testnet where the frequency of
	  blocks is variable)
	- Merge Msg for synced block and updated vars into one: `MsgSyncBlock`.
2020-12-15 15:15:25 +01:00
arnaucube
593a477e6c ZKInputs generation ISFinalAccFee update
- Update IntermediateState FinalAccFee values to fit in the Circom circuit expected inputs
- Add ZKInputs generation test with transactions generated by Til
(TestZKInputs6)
- BatchBuilder: remove redundant MakeCheckpoint call
2020-12-15 11:49:09 +01:00
Eduard S
20b8d0561f Update synchronizer and DB with last contracts updates
- API
	- When updating network info, handle cases where no batches exists and
	  where no forgers exists
- cli/node
	- Update `cfg.buidler.toml` config file to a working version
- common
	- Add new smart contract structs and extend some existing ones to
	  reflect updates regarding events from the smart contracts
- SQL
	- Add new tables and extend existing ones to reflect updates regarding
	  events from the smart contracts
- db/historydb
	- Add functions to insert new smart contract events data
	- Fix unclosed rows that led to inconsistent sql driver state (replace
	  NamedQuery by NamedExec).  This fixes the error:
	  `pq: unexpected Parse response 'C'`
- db/l2db
	- Close rows after usage
- eth
	- In Rollup event, introduce a new UpdateBucketsParameter when there's a
	  SafeMode event, with `SafeMode = true`
- synchronizer
	- synchronize new events
	- avoid calling `auction.CanForge` before the genesisBlock to avoid
	  getting a revert.
2020-12-10 17:45:17 +01:00
arnaucube
9d0b278d50 Update missing LoadAmount to DepositAmount
Related to 445f26ec9f
2020-12-09 18:34:01 +01:00
Eduard S
900d1fb6ce Integrate purger to node
- Common
	- Add `IdxNonce` type used to track nonces in accounts to invalidate
	  l2txs in the pool
- Config
	- Update coordinator config will all the new configuration parameters
	  used in the coordinator
- Coordinator
	- Introduce the `Purger` to track how often to purge and do the job when
	  needed according to a configuration.
	- Implement the methods to invalidate l2txs transactions due to l2txs
	  selection in batches.  For now these functions are not used in favour
	  of the `Purger` methods, which check ALL the l2txs in the pool.
	- Call Invalidation and Purging methods of the purger both when the
	  node is forging (in the pipeline when starting a new batch) and when
	  the node is not forging (in coordinator when being notified about a
	  new synced block)
- L2DB:
	- Implement `GetPendingUniqueFromIdxs` to get all the unique idxs from
	  pending transactions (used to get their nonces and then invalidate
	  txs)
	- Redo `CheckNonces` with a single SQL query and using `common.IdxNonce`
	  instead of `common.Account`
- StateDB:
	- Expose GetIdx to check errors when invalidating pool txs
- Synchronizer:
	- Test forged L1UserTxs processed by TxProcessor
	- Improve checks of Effective values
- TxSelector:
	- Expose the internal LocalStateDB in order to check account nonces in
	  the coordinator when not forging.
2020-12-09 12:43:16 +01:00
arnaucube
c1fd0d6626 ZKInputs with Fee txs compatible with circom
- Separate ZKInputs tests at StateDB package
- Small fix at ZKInputs generation FeeIdxs length
- Fees related parameters at ZKInputs works properly
- Add ZKInputs generation test with fees (`TestZKInputs1`), which output
has been tested with circom to ensure compatibility
2020-12-07 12:54:18 +01:00
Arnau B
445f26ec9f Rename load amount to deposit amount 2020-12-04 16:13:41 +01:00
laisolizq
75c6d8f14e Updates API to new changes 2020-12-04 11:22:12 +01:00
Eduard S
329d2796c6 Update some eth types, update eth dependencies to new changes 2020-12-03 18:05:33 +01:00
laisolizq
18031343ec Update ethclient 2020-12-03 14:19:38 +01:00
arnaucube
80cbac24f9 StateDB ZKInputs generation compatible with circom
- Add SignatureConstant to L1Tx.TxCompressedData
- ZKInputs generation works initial version working and compatible with circom Hermez circuit
    - Lots of small updates for circuits compatibility
- Update version (with new changes) of `go-iden3-crypto` & `go-merkletree`
2020-12-03 13:16:51 +01:00
arnau
d306f4890a Merge pull request #322 from hermeznetwork/feature/integration20
Advance coordinator implementation
2020-12-03 13:16:07 +01:00
Eduard S
482c94d374 Advance coordinator implementation
- Common
	- Move ErrTODO and ErrDone to common for usage where needed.
- Coordinator
	- Move prover types to prover package
	- Handle reorgs, stopping the pipeline when necessary
	- Handle ethereum transaction errors by stopping the pipeline
	- In case of ethereum transaction revert, check for known revert causes
	  (more revert causes can be added to handle more cases)
	- Fix skipped transactions in TxManager confirmation logic
	- Cancel and wait for provers to be ready
	- Connect L2DB to:
		- purge l2txs due to timeout
		- mark l2txs at the different states
	- Connect HistoryDB to query L1UserTxs to forge in an L1Batch
- L2DB
	- Skip update functions when the input slices have no values (to avoid a
	  query with no values that results in an SQL error)
- StateDB
	- In LocalStateDB, fix Reset when mt == nil
- Prover (new package)
	- Rename the interface to Prover
	- Rename the mock struct to Mock
	- Extend Prover interface methods to provide everything required by the
	  coordinator
	- Begin implementing required http client code to interact with server
	  proof (not tested)
- Synchronizer:
	- Add LastForgeL1TxsNum to Stats
- Test/Client
	- Update Auction logic to track slots in which there's no forge during
	  the time before the deadline (following the solidity implementation)
2020-12-03 13:00:06 +01:00
Eduard S
665e0829ec Set EffectiveAmount and EffectiveLoadAmount of forged L1UserTxs 2020-11-30 16:25:54 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
arnaucube
dde9250429 Update Common & StateDB & ZKInputs to last protocol version
- Add InvalidData flag to L1Tx
- Add BytesDataAvailability to L1Tx
- Update ZKInputs & HashGlobalInputs to last spec of the protocol
(massive migrations)
- TxProcessor check correctness of L1Txs

Compatible with hermeznetwork/commonjs v0.0.4
(c345239bba)
2020-11-30 11:26:15 +01:00
arnau
0c83417881 Merge pull request #301 from hermeznetwork/feature/integration19
Update coordinator, call all api update functions
2020-11-26 18:10:29 +01:00
Eduard S
8f1cf2f145 Update coordinator, call all api update functions
- Common:
	- Rename Block.EthBlockNum to Block.Num to avoid unneeded repetition
- API:
	- Add UpdateNetworkInfoBlock to update just block information, to be
	  used when the node is not yet synchronized
- Node:
	- Call API.UpdateMetrics and UpdateRecommendedFee in a loop, with
	  configurable time intervals
- Synchronizer:
	- When mapping events by TxHash, use an array to support the possibility
	  of multiple calls of the same function happening in the same
	  transaction (for example, a smart contract in a single transaction
	  could call withdraw with delay twice, which would generate 2 withdraw
	  events, and 2 deposit events).
	- In Stats, keep entire LastBlock instead of just the blockNum
	- In Stats, add lastL1BatchBlock
	- Test Stats and SCVars
- Coordinator:
	- Enable writing the BatchInfo in every step of the pipeline to disk
	  (with JSON text files) for debugging purposes.
	- Move the Pipeline functionality from the Coordinator to its own struct
	  (Pipeline)
	- Implement shouldL1lL2Batch
	- In TxManager, implement logic to perform several attempts when doing
	  ethereum node RPC calls before considering the error. (Both for calls
	  to forgeBatch and transaction receipt)
	- In TxManager, reorganize the flow and note the specific points in
	  which actions are made when err != nil
- HistoryDB:
	- Implement GetLastL1BatchBlockNum: returns the blockNum of the latest
	  forged l1Batch, to help the coordinator decide when to forge an
	  L1Batch.
- EthereumClient and test.Client:
	- Update EthBlockByNumber to return the last block when the passed
	  number is -1.
2020-11-26 18:06:30 +01:00
Eduard S
4d830fdc89 Merge pull request #300 from hermeznetwork/feature/l1coord-update
Update L1 coordinator Txs
2020-11-25 16:41:23 +01:00
laisolizq
7ceaf4c32b Update L1 coordinator Txs 2020-11-25 11:04:23 +01:00