Commit Graph

54 Commits

Author SHA1 Message Date
arnaubennassar
4acfeb0ad9 Add API metric for time to forge L1 tx 2021-02-25 13:06:26 +01:00
arnaubennassar
ed9bfdffa9 Diferentiate SQL connections by read write 2021-02-24 15:12:43 +01:00
Eduard S
d16fba72a7 In API recommended fee, use minFeeUSD as min value 2021-02-23 17:20:16 +01:00
Eduard S
3706ddb2fb Add minPriceUSD in L2DB, check maxTxs atomically
- Add config parameter `Coordinator.L2DB.MinPriceUSD` which allows rejecting
  txs to the pool that have a fee lower than the minimum.
- In pool tx insertion, checking the number of pending txs atomically with the
  insertion to avoid data races leading to more than MaxTxs pending txs in the
  pool.
2021-02-22 16:45:15 +01:00
Eduard S
67b2b7da4b Delete pending txs by external mark, store tx IP
- In tx_pool, add a column called `external_delete` that can be set to true
  externally.  Regularly, the coordinator will delete all pending txs with this
  column set to true.  The interval for this action is set via the new config
  parameter `Coordinator.PurgeByExtDelInterval`.
- In tx_pool, add a column for the client ip that sent the transaction.  The
  api fills this value using the ClientIP method from gin.Context, which should
  work even under a reverse-proxy.
2021-02-19 16:00:45 +01:00
Eduard S
2a77dac9c1 Merge pull request #536 from hermeznetwork/feature/sql-semaphore
Add semaphore for API queries to SQL
2021-02-10 13:47:27 +01:00
Arnau B
ac1fd9acf7 Add semaphore for API queries to SQL 2021-02-10 13:36:17 +01:00
Eduard S
2bf3b843ed Fix l2db reorg of forging l2txs 2021-02-09 17:13:29 +01:00
arnaucube
a8f6891aea Add PoolL2Tx.Info about the status of the tx
PoolL2Tx.Info contains information about the status & State of the
transaction. As for example, if the Tx has not been selected in the last
batch due not enough Balance at the Sender account, this reason would
appear at this parameter.
This will help the client (wallet, batchexplorer, etc) to reason why a
L2Tx is not selected in the forged batches.
2021-02-03 13:03:30 +01:00
arnaucube
7f46c3028e Add tests connecting TxSelector, BatchBuilder, ZKInputs, ProofServer
- Add tests connecting TxSelector, BatchBuilder, ZKInputs, ProofServer
- Added test to check that the signatures of the PoolL2Txs from the L2DB
pool can be verified, to check that the parameters of the PoolL2Tx match
the original parameters signed before inserting them into the L2DB
2021-01-18 17:52:14 +01:00
Eduard S
828b8177c2 In pool l2tx, store effective in aux column 2021-01-18 13:21:56 +01:00
Arnau B
622e01e9af Add max txs limit to tx pool 2021-01-04 18:38:25 +01:00
arnaucube
8615a462ab Update AccountCreationAuth & fix auth.HashToSign
- Fix AccountCreationAuth.HashToSign (was using `[]byte("0x...")`, which
uses the bytes of the string. Now uses the bytearray of the compressed
BJJ public key (compatible with js implementation))
- Update AccountCreationAuth to last specification (add to hash
parameters ChainID & HermezAddress)
- Add missing test to AccountCreationAuth
2020-12-24 10:41:35 +01:00
arnaucube
150597c282 Parametrize ChainID 2020-12-23 15:23:48 +01:00
Eduard S
a8ac35059a Test purger, fix some nonces
- Test all the purger functions
- Fix nonces set by til (previously til started with nonce 1 for pool l2txs,
  but the correct implementation is to start with nonce 0)
- Rename L2DB.CheckNonces to L2DB.invalidateOldNoncesQuery
- Rename L2DB.checkNoncesQuery to L2DB.InvalidateOldNonces

Related https://github.com/hermeznetwork/hermez-node/issues/392 (Fix checkNoncesQuery)
Resolve https://github.com/hermeznetwork/hermez-node/issues/396
2020-12-22 12:00:12 +01:00
arnaucube
4b10549822 mv of babyjub.PublicKey to babyjub.PublicKeyComp
Update usage of `*babyjub.PublicKey` to `babyjub.PublicKeyComp`
- when the key is not defined, internally is used `babyjub.EmptyBJJComp`, which is a `[32]byte` of zeroes of type `babyjub.PublicKeyComp`
- the API continues returning `nil` when the key is not defined
2020-12-22 11:29:47 +01:00
Eduard S
20b8d0561f Update synchronizer and DB with last contracts updates
- API
	- When updating network info, handle cases where no batches exists and
	  where no forgers exists
- cli/node
	- Update `cfg.buidler.toml` config file to a working version
- common
	- Add new smart contract structs and extend some existing ones to
	  reflect updates regarding events from the smart contracts
- SQL
	- Add new tables and extend existing ones to reflect updates regarding
	  events from the smart contracts
- db/historydb
	- Add functions to insert new smart contract events data
	- Fix unclosed rows that led to inconsistent sql driver state (replace
	  NamedQuery by NamedExec).  This fixes the error:
	  `pq: unexpected Parse response 'C'`
- db/l2db
	- Close rows after usage
- eth
	- In Rollup event, introduce a new UpdateBucketsParameter when there's a
	  SafeMode event, with `SafeMode = true`
- synchronizer
	- synchronize new events
	- avoid calling `auction.CanForge` before the genesisBlock to avoid
	  getting a revert.
2020-12-10 17:45:17 +01:00
laisolizq
d5f579aec5 Merge pull request #336 from hermeznetwork/fix/api1
Fix/api1
2020-12-09 16:08:01 +01:00
Arnau B
039c7b29da Fix time sensitive test 2020-12-09 14:12:10 +01:00
Eduard S
900d1fb6ce Integrate purger to node
- Common
	- Add `IdxNonce` type used to track nonces in accounts to invalidate
	  l2txs in the pool
- Config
	- Update coordinator config will all the new configuration parameters
	  used in the coordinator
- Coordinator
	- Introduce the `Purger` to track how often to purge and do the job when
	  needed according to a configuration.
	- Implement the methods to invalidate l2txs transactions due to l2txs
	  selection in batches.  For now these functions are not used in favour
	  of the `Purger` methods, which check ALL the l2txs in the pool.
	- Call Invalidation and Purging methods of the purger both when the
	  node is forging (in the pipeline when starting a new batch) and when
	  the node is not forging (in coordinator when being notified about a
	  new synced block)
- L2DB:
	- Implement `GetPendingUniqueFromIdxs` to get all the unique idxs from
	  pending transactions (used to get their nonces and then invalidate
	  txs)
	- Redo `CheckNonces` with a single SQL query and using `common.IdxNonce`
	  instead of `common.Account`
- StateDB:
	- Expose GetIdx to check errors when invalidating pool txs
- Synchronizer:
	- Test forged L1UserTxs processed by TxProcessor
	- Improve checks of Effective values
- TxSelector:
	- Expose the internal LocalStateDB in order to check account nonces in
	  the coordinator when not forging.
2020-12-09 12:43:16 +01:00
Eduard S
482c94d374 Advance coordinator implementation
- Common
	- Move ErrTODO and ErrDone to common for usage where needed.
- Coordinator
	- Move prover types to prover package
	- Handle reorgs, stopping the pipeline when necessary
	- Handle ethereum transaction errors by stopping the pipeline
	- In case of ethereum transaction revert, check for known revert causes
	  (more revert causes can be added to handle more cases)
	- Fix skipped transactions in TxManager confirmation logic
	- Cancel and wait for provers to be ready
	- Connect L2DB to:
		- purge l2txs due to timeout
		- mark l2txs at the different states
	- Connect HistoryDB to query L1UserTxs to forge in an L1Batch
- L2DB
	- Skip update functions when the input slices have no values (to avoid a
	  query with no values that results in an SQL error)
- StateDB
	- In LocalStateDB, fix Reset when mt == nil
- Prover (new package)
	- Rename the interface to Prover
	- Rename the mock struct to Mock
	- Extend Prover interface methods to provide everything required by the
	  coordinator
	- Begin implementing required http client code to interact with server
	  proof (not tested)
- Synchronizer:
	- Add LastForgeL1TxsNum to Stats
- Test/Client
	- Update Auction logic to track slots in which there's no forge during
	  the time before the deadline (following the solidity implementation)
2020-12-03 13:00:06 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
a_bennassar
e0a4b656fe Merge pull request #279 from hermeznetwork/feature/l2db-test-til
Add til test l2db
2020-11-16 18:13:50 +01:00
laisolizq
1925110fda Add til test l2db 2020-11-16 13:35:24 +01:00
Eduard S
e731b79e96 Fix exit table, set delayed_withdrawn in exits
- In exit table, `instant_withdrawn`, `delayed_withdraw_request`, and
  `delayed_withdrawn` were referencing batch_num.  But these actions happen
  outside a batch, so they should reference a block_num.
- Process delayed withdrawns:
    - In Synchronizer, first match a Rollup delayed withdrawn request, with the
      WDelayer deposit (via TxHash), and store the owner and token associated
      with the delayed withdrawn.
    - In HistoryDB: store the owner and token of a delayed withdrawal request
      in the exit_tree, and set delayed_withdrawn when the withdraw is done in
      the WDelayer.
- Update dependency of sqlx to master
    - Last release of sqlx is from 2018 October, and it doesn't support
      `NamedQuery` with a slice of structs, which is used in this commit.
2020-11-13 16:53:16 +01:00
Arnau B
bed25d424a Use til for batches and blocks at historydb 2020-11-11 14:00:19 +01:00
laisolizq
22f0a45118 ItemID to uint64 & check timestamp 2020-11-06 10:40:57 +01:00
Eduard S
6e4b9b4b70 Update Synchronizer (node) implementation
- node:
    - Extend config to add initial variables of the smart contracts used as
      defaults before they are changed via events.
    - In stopped channels, set size 1 so that panics are not witheld until the
      node stops completely.
- common:
    - In Smart Contract variables, comment:
      - `WDelayerVariables.HermezRollupAddress` because it's not needed.
      - `RollupVariables.Buckets` because there are no events for it, and for
        now it's not used.
- historydb:
    - Add functions to get and set smart contract variables.
- db:
    - Add `Rollback` function in `utils.go` to reduce boilerplate in sql
      transaction rollbacks in defers in db functions.
    - Update `rollup_vars` and `auction_vars` (renamed from `consensus_vars`)
      table, and add `wdelayer_vars` table.
- synchronizer:
    - Synchronize WDelayer
    - Handle SC variables properly
- test/ethclient:
    - Add essential implementation of WDelayer
2020-11-04 15:12:51 +01:00
Arnau B
233ecc4704 Refactor api exits 2020-11-02 14:00:42 +01:00
Arnau B
8c9d13ffa5 Refactor account creation auth endpoints 2020-11-02 13:58:42 +01:00
Arnau B
cbeca0f76e Insert ETH as HistoryDB token 2020-11-02 12:09:09 +01:00
Arnau B
a329d894d2 Refactor api txs 2020-10-28 12:16:35 +01:00
arnaucube
1203ab3865 TxSelector include L1CoordinatorTxs in selections 2020-10-27 17:25:41 +01:00
laisolizq
bc34f3456b API Update tokens endpoint 2020-10-27 13:23:01 +01:00
Arnau B
ede6c33a18 Impl account creation auth endpoints 2020-10-23 13:05:20 +02:00
Arnau B
006bbf3321 Implement tx pool endpoints 2020-10-21 16:34:54 +02:00
arnaucube
d7a1ba19b9 Fix PoolL2Tx.RqTxCompressedData 2020-10-14 18:30:58 +02:00
Arnau B
5097939b12 Avoid using pointers in common 2020-10-08 16:42:16 +02:00
Eduard S
b14495cfcc Replace all []*Foo by []Foo in sql db return values
- Implement SlicePtrsToSlice and use it in all `meddler.QueryAll` sql db functions to always return []Foo instead of []*Foo
2020-10-07 17:03:39 +02:00
Eduard S
0277210c39 Extend ethclient test, implement new TxID spec
- Implement new TxID spec that distinguishes L1UserTx and L1CoordinatorTx
- Replace some type []*Foo by []Foo
- Fix HistoryDB & L2DB bug: in case of error, a rollback was applied and the returned error was nil
- Reorder inserts in historydb.NewHistoryDB() to follow foreign key dependencies
- Add initial synchronizer test with test.Client (for now, only tested l1UserTxs, blocks, addToken)
- Update L1UserTx event in test.Client
2020-10-07 16:41:51 +02:00
a_bennassar
49366e3fa4 Feature/null refactor (#173)
* WIP: rebase

* Make nullable fields use pointers
2020-10-06 10:34:49 +02:00
a_bennassar
c6f70f3177 Feature/merge history l2 tables (#156)
* WIP rebase

* Combine both SQL DBs

* API and DB refactor
2020-09-29 18:27:07 +02:00
ToniRamirezM
ac677694fe Helper methods for Synchronizer and TX refactor 2020-09-23 17:05:38 +02:00
Arnau B
85fe885265 Add GET histroy-transactions endpoint 2020-09-22 13:21:28 +02:00
Eduard S
35a558f6c9 Merge pull request #135 from hermeznetwork/feature/synchronizer-sc
Smart Contracts Data Synchronization
2020-09-21 14:40:10 +02:00
Eduard S
9f83fcf520 Review synchronizer, rearange some stuff, add notes and TODOs. 2020-09-21 14:33:07 +02:00
Arnau B
94e1f11a98 Implement L2DB 2020-09-18 10:17:51 +02:00
arnaucube
41715e47ba TxSel integrated with L2DB, Add L1CoordTx creation
- GetL2TxSelection & GetL1L2TxSelection integrated with dbs
- Create L1CoordinatorTx of type CreateAccountDeposit when a L2
requires it (and the AccountCreationAuth exists)
2020-09-16 12:40:56 +02:00
arnaucube
fd1e9c25ee Add linter checks to GHA & Fix code to pass lints
Add linter checks to GHA & Fix code to pass lints.
The linters added are:
- whitespace: Tool for detection of leading and trailing whitespace
- gosec: Inspects source code for security problems
- gci: Gci control golang package import order and make it always deterministic
- misspell: Finds commonly misspelled English words in comments
- gomnd: An analyzer to detect magic numbers

The file utils/utils.go is excluded from the checks of gomnd, as uses magic numbers through the code
2020-08-28 09:58:39 +02:00
Arnau B
c108739c24 Add usefull methods for reorg 2020-08-25 14:15:12 +02:00