Commit Graph

85 Commits

Author SHA1 Message Date
Arnau B
8083e4fec3 Fix format of the MTP for exit endpoint 2020-12-16 20:09:17 +01:00
Arnau B
c7b7a85b85 Fix queries when no rows 2020-12-15 16:15:42 +01:00
Eduard S
20b8d0561f Update synchronizer and DB with last contracts updates
- API
	- When updating network info, handle cases where no batches exists and
	  where no forgers exists
- cli/node
	- Update `cfg.buidler.toml` config file to a working version
- common
	- Add new smart contract structs and extend some existing ones to
	  reflect updates regarding events from the smart contracts
- SQL
	- Add new tables and extend existing ones to reflect updates regarding
	  events from the smart contracts
- db/historydb
	- Add functions to insert new smart contract events data
	- Fix unclosed rows that led to inconsistent sql driver state (replace
	  NamedQuery by NamedExec).  This fixes the error:
	  `pq: unexpected Parse response 'C'`
- db/l2db
	- Close rows after usage
- eth
	- In Rollup event, introduce a new UpdateBucketsParameter when there's a
	  SafeMode event, with `SafeMode = true`
- synchronizer
	- synchronize new events
	- avoid calling `auction.CanForge` before the genesisBlock to avoid
	  getting a revert.
2020-12-10 17:45:17 +01:00
arnaucube
9d0b278d50 Update missing LoadAmount to DepositAmount
Related to 445f26ec9f
2020-12-09 18:34:01 +01:00
Arnau B
e1d34e3796 Add amount success and load amount success 2020-12-09 16:39:18 +01:00
Arnau B
445f26ec9f Rename load amount to deposit amount 2020-12-04 16:13:41 +01:00
Eduard S
897d7d5319 Merge pull request #326 from hermeznetwork/feature/effective-amount-update
Feature/effective amount update
2020-12-04 12:33:30 +01:00
Arnau B
5fd258ee07 Use flags for effective amounts in SQL schema 2020-12-04 12:17:09 +01:00
Eduard S
329d2796c6 Update some eth types, update eth dependencies to new changes 2020-12-03 18:05:33 +01:00
Eduard S
665e0829ec Set EffectiveAmount and EffectiveLoadAmount of forged L1UserTxs 2020-11-30 16:25:54 +01:00
Eduard S
ced42634da Wrap all errors with tracerr 2020-11-30 12:58:36 +01:00
arnaucube
dde9250429 Update Common & StateDB & ZKInputs to last protocol version
- Add InvalidData flag to L1Tx
- Add BytesDataAvailability to L1Tx
- Update ZKInputs & HashGlobalInputs to last spec of the protocol
(massive migrations)
- TxProcessor check correctness of L1Txs

Compatible with hermeznetwork/commonjs v0.0.4
(c345239bba)
2020-11-30 11:26:15 +01:00
Eduard S
8f1cf2f145 Update coordinator, call all api update functions
- Common:
	- Rename Block.EthBlockNum to Block.Num to avoid unneeded repetition
- API:
	- Add UpdateNetworkInfoBlock to update just block information, to be
	  used when the node is not yet synchronized
- Node:
	- Call API.UpdateMetrics and UpdateRecommendedFee in a loop, with
	  configurable time intervals
- Synchronizer:
	- When mapping events by TxHash, use an array to support the possibility
	  of multiple calls of the same function happening in the same
	  transaction (for example, a smart contract in a single transaction
	  could call withdraw with delay twice, which would generate 2 withdraw
	  events, and 2 deposit events).
	- In Stats, keep entire LastBlock instead of just the blockNum
	- In Stats, add lastL1BatchBlock
	- Test Stats and SCVars
- Coordinator:
	- Enable writing the BatchInfo in every step of the pipeline to disk
	  (with JSON text files) for debugging purposes.
	- Move the Pipeline functionality from the Coordinator to its own struct
	  (Pipeline)
	- Implement shouldL1lL2Batch
	- In TxManager, implement logic to perform several attempts when doing
	  ethereum node RPC calls before considering the error. (Both for calls
	  to forgeBatch and transaction receipt)
	- In TxManager, reorganize the flow and note the specific points in
	  which actions are made when err != nil
- HistoryDB:
	- Implement GetLastL1BatchBlockNum: returns the blockNum of the latest
	  forged l1Batch, to help the coordinator decide when to forge an
	  L1Batch.
- EthereumClient and test.Client:
	- Update EthBlockByNumber to return the last block when the passed
	  number is -1.
2020-11-26 18:06:30 +01:00
Eduard S
3b99953007 Redo coordinator structure, connect API to node
- API:
	- Modify the constructor so that hardcoded rollup constants don't need
	  to be passed (introduce a `Config` and use `configAPI` internally)
- Common:
	- Update rollup constants with proper *big.Int when required
	- Add BidCoordinator and Slot structs used by the HistoryDB and
	  Synchronizer.
	- Add helper methods to AuctionConstants
	- AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL
	  table: `default_slot_set_bid_slot_num`), which indicates at which
	  slotNum does the `DefaultSlotSetBid` specified starts applying.
- Config:
	- Move coordinator exclusive configuration from the node config to the
	  coordinator config
- Coordinator:
	- Reorganize the code towards having the goroutines started and stopped
	  from the coordinator itself instead of the node.
	- Remove all stop and stopped channels, and use context.Context and
	  sync.WaitGroup instead.
	- Remove BatchInfo setters and assing variables directly
	- In ServerProof and ServerProofPool use context instead stop channel.
	- Use message passing to notify the coordinator about sync updates and
	  reorgs
	- Introduce the Pipeline, which can be started and stopped by the
	  Coordinator
	- Introduce the TxManager, which manages ethereum transactions (the
	  TxManager is also in charge of making the forge call to the rollup
	  smart contract).  The TxManager keeps ethereum transactions and:
	  	1. Waits for the transaction to be accepted
		2. Waits for the transaction to be confirmed for N blocks
	- In forge logic, first prepare a batch and then wait for an available
	  server proof to have all work ready once the proof server is ready.
	- Remove the `isForgeSequence` method which was querying the smart
	  contract, and instead use notifications sent by the Synchronizer to
	  figure out if it's forging time.
	- Update test (which is a minimal test to manually see if the
	  coordinator starts)
- HistoryDB:
	- Add method to get the number of batches in a slot (used to detect when
	  a slot has passed the bid winner forging deadline)
	- Add method to get the best bid and associated coordinator of a slot
	  (used to detect the forgerAddress that can forge the slot)
- General:
	- Rename some instances of `currentBlock` to `lastBlock` to be more
	  clear.
- Node:
	- Connect the API to the node and call the methods to update cached
	  state when the sync advances blocks.
	- Call methods to update Coordinator state when the sync advances blocks
	  and finds reorgs.
- Synchronizer:
	- Add Auction field in the Stats, which contain the current slot with
	  info about highest bidder and other related info required to know who
	  can forge in the current block.
	- Better organization of cached state:
		- On Sync, update the internal cached state
		- On Init or Reorg, load the state from HistoryDB into the
		  internal cached state.
2020-11-24 18:59:30 +01:00
Arnau B
ce1218e28a Integrate til into api 2020-11-17 17:50:07 +01:00
laisolizq
b3b23473b6 Add til api test first commit 2020-11-17 11:24:19 +01:00
laisolizq
4aa686797d Merge pull request #284 from hermeznetwork/feature/api-db-test-til-tx
HistoryDB TXs til test
2020-11-17 10:34:26 +01:00
ToniRamirezM
93d67ff699 HistoryDB TXs til test 2020-11-16 16:27:18 +01:00
Eduard S
457ff94bfa Test synchronizer reorg 2020-11-16 11:35:15 +01:00
Eduard S
e731b79e96 Fix exit table, set delayed_withdrawn in exits
- In exit table, `instant_withdrawn`, `delayed_withdraw_request`, and
  `delayed_withdrawn` were referencing batch_num.  But these actions happen
  outside a batch, so they should reference a block_num.
- Process delayed withdrawns:
    - In Synchronizer, first match a Rollup delayed withdrawn request, with the
      WDelayer deposit (via TxHash), and store the owner and token associated
      with the delayed withdrawn.
    - In HistoryDB: store the owner and token of a delayed withdrawal request
      in the exit_tree, and set delayed_withdrawn when the withdraw is done in
      the WDelayer.
- Update dependency of sqlx to master
    - Last release of sqlx is from 2018 October, and it doesn't support
      `NamedQuery` with a slice of structs, which is used in this commit.
2020-11-13 16:53:16 +01:00
arnau
8cc165f562 Merge pull request #275 from hermeznetwork/feature/api-db-tests-til
Feature/api db tests til
2020-11-11 15:54:41 +01:00
Arnau B
bed25d424a Use til for batches and blocks at historydb 2020-11-11 14:00:19 +01:00
laisolizq
43e9bee9db API state endpoint test 2020-11-10 13:47:18 +01:00
Toni Ramírez
6ec7a0b55d Merge pull request #271 from hermeznetwork/feature/api-state-fee
API State RecommendedFee
2020-11-09 18:03:50 +01:00
laisolizq
9edf9d2168 API State RecommendedFee 2020-11-09 17:57:23 +01:00
a_bennassar
e8148e66ff Merge pull request #269 from hermeznetwork/feature/integration12
Process withdrawals in HistoryDB, test withdrawals and vars in sync
2020-11-09 17:27:04 +01:00
Eduard S
9f5af0aa07 Process withdrawals in HistoryDB, test withdrawals and vars in sync 2020-11-09 17:13:53 +01:00
a_bennassar
a1b9720e77 Merge pull request #270 from hermeznetwork/feature/api-state-metrics
API State Metrics
2020-11-09 16:54:25 +01:00
ToniRamirezM
55a78fbd72 API State Metrics 2020-11-09 16:50:15 +01:00
Eduard S
8190755d49 Merge pull request #268 from hermeznetwork/feature/wrapped-in-struct
Refactor api pagination
2020-11-09 15:32:28 +01:00
laisolizq
cc6446e019 API add get state 2020-11-09 13:01:28 +01:00
Arnau B
d2e9196fba Refactor api pagination 2020-11-09 12:58:09 +01:00
arnau
fd9b247120 Merge pull request #261 from hermeznetwork/feature/til-extra
Move logic to extend til blocks to til
2020-11-06 11:46:49 +01:00
laisolizq
22f0a45118 ItemID to uint64 & check timestamp 2020-11-06 10:40:57 +01:00
ToniRamirezM
b840ceb57a Account API 2020-11-06 09:30:32 +01:00
Eduard S
5ebf46e045 Move logic to extend til blocks to til
Previously, the synchronizer test was extending the output from til to
precalculate many fields to compare it with the synchronizer and BD output.
Since this is useful outside of the syncrhonizer testing: move this
functionality to til via a function that extends the output
(til.Context.FillBlocksExtra).

Also, add new functionality: calculate fee idxs dynamically by setting a user
name, and calculate collected fees.
2020-11-05 18:09:12 +01:00
laisolizq
5012b82914 API add slots endpoints 2020-11-05 15:59:27 +01:00
Eduard S
6e4b9b4b70 Update Synchronizer (node) implementation
- node:
    - Extend config to add initial variables of the smart contracts used as
      defaults before they are changed via events.
    - In stopped channels, set size 1 so that panics are not witheld until the
      node stops completely.
- common:
    - In Smart Contract variables, comment:
      - `WDelayerVariables.HermezRollupAddress` because it's not needed.
      - `RollupVariables.Buckets` because there are no events for it, and for
        now it's not used.
- historydb:
    - Add functions to get and set smart contract variables.
- db:
    - Add `Rollback` function in `utils.go` to reduce boilerplate in sql
      transaction rollbacks in defers in db functions.
    - Update `rollup_vars` and `auction_vars` (renamed from `consensus_vars`)
      table, and add `wdelayer_vars` table.
- synchronizer:
    - Synchronize WDelayer
    - Handle SC variables properly
- test/ethclient:
    - Add essential implementation of WDelayer
2020-11-04 15:12:51 +01:00
arnau
11dbf67377 Merge pull request #250 from hermeznetwork/feature/integration9
Reorganize smart contract types, udate eth tests, etc.
2020-11-04 12:36:42 +01:00
Eduard S
e6fb0a03de Reorganize smart contract types, udate eth tests, etc.
- Move smart contract constants and structs for variables to
  common/{ethrollup.go, ethauction.go, ethwdelayer.go}:
    - This removes repeated code of the structs for variables
    - Allows reusing the constants and variables from all modules without
      import cycles
- Remove unused common/scvars.go
- In common.BlockData, split data from each smart contract into a sepparate
  field (Rollup, Auction, WDelayer).  This affects the structures that til uses
  as output, and HistoryDB in the AddBlockSCData.
- In Synchronizer:
    - Pass starting block of each smart contract as config, instead of
      incorrectly using the genesis block found in the acution constant (which
      has a very different meaning)
    - Use variable structs from common instead of an internal copy
    - Synchronize more stuff (resolve some TODOs)
    - Fix some issues found after initial testing with ganache
- In eth:
    - In auction.go: Add method to get constants
    - Update README to use ganache instead of buidlerevm as local blockchain
      for testing
    - Update env variables and test vectors to pass the tests with the
      deployment in the ganache testnet.
    - Use ethereum keys derived from paths (hdwallet) in testing to avoid
      hardcoding private keys and generate the same keys from a mnemonic used
      in the ganache tesnet.
2020-11-04 12:06:28 +01:00
laisolizq
04d969c274 API add pending withdraws filter on GET /exits 2020-11-03 16:12:58 +01:00
Arnau B
233ecc4704 Refactor api exits 2020-11-02 14:00:42 +01:00
Arnau B
cbeca0f76e Insert ETH as HistoryDB token 2020-11-02 12:09:09 +01:00
laisolizq
aa6cb6f818 API add bids endpoint 2020-10-28 18:11:52 +01:00
Arnau B
a329d894d2 Refactor api txs 2020-10-28 12:16:35 +01:00
laisolizq
decc4996ee Merge pull request #235 from hermeznetwork/feature/api-coordinator-refactor
API Coordinator Refactor
2020-10-28 10:20:44 +01:00
Eduard S
558de2737e Test L2Txs, ExitTree in synchronizer 2020-10-27 17:59:18 +01:00
ToniRamirezM
b521a77fc5 API Coordinator Refactor 2020-10-27 17:44:22 +01:00
laisolizq
bc34f3456b API Update tokens endpoint 2020-10-27 13:23:01 +01:00
a_bennassar
b320069e75 Merge pull request #229 from hermeznetwork/feature/api-getstate
Add struct get state endpoint
2020-10-27 12:30:40 +01:00