- Add new command to the cli/node: `serveapi` that alows serving the API just
by connecting to the PostgreSQL database. The mode flag should me passed in
order to select whether we are connecting to a synchronizer database or a
coordinator database. If `coord` is chosen as mode, the coordinator
endpoints can be activated in order to allow inserting l2txs and
authorizations into the L2DB.
Summary of the implementation details
- New SQL table with 3 columns (plus `item_id` pk). The table only contains a
single row with `item_id` = 1. Columns:
- state: historydb.StateAPI in JSON. This is the struct that is served via
the `/state` API endpoint. The node will periodically update this struct
and store it int he DB. The api server will query it from the DB to
serve it.
- config: historydb.NodeConfig in JSON. This struct contains node
configuration parameters that the API needs to be aware of. It's updated
once every time the node starts.
- constants: historydb.Constants in JSON. This struct contains all the
hermez network constants gathered via the ethereum client by the node.
It's written once every time the node starts.
- The HistoryDB contains methods to get and update each one of these columns
individually.
- The HistoryDB contains all methods that query the DB and prepare objects that
will appear in the StateAPI endpoint.
- The configuration used in for the `serveapi` cli/node command is defined in
`config.APIServer`, and is a subset of `node.Config` in order to allow
reusing the same configuration file of the node if desired.
- A new object is introduced in the api: `StateAPIUpdater`, which contains all
the necessary information to update the StateAPI in the DB periodically by
the node.
- Moved the types `SCConsts`, `SCVariables` and `SCVariablesPtr` from
`syncrhonizer` to `common` for convenience.
- Add config parameter `Coordinator.L2DB.MinPriceUSD` which allows rejecting
txs to the pool that have a fee lower than the minimum.
- In pool tx insertion, checking the number of pending txs atomically with the
insertion to avoid data races leading to more than MaxTxs pending txs in the
pool.
- In tx_pool, add a column called `external_delete` that can be set to true
externally. Regularly, the coordinator will delete all pending txs with this
column set to true. The interval for this action is set via the new config
parameter `Coordinator.PurgeByExtDelInterval`.
- In tx_pool, add a column for the client ip that sent the transaction. The
api fills this value using the ClientIP method from gin.Context, which should
work even under a reverse-proxy.
PoolL2Tx.Info contains information about the status & State of the
transaction. As for example, if the Tx has not been selected in the last
batch due not enough Balance at the Sender account, this reason would
appear at this parameter.
This will help the client (wallet, batchexplorer, etc) to reason why a
L2Tx is not selected in the forged batches.
- API:
- Replace `emergencyModeStaringTime` by `emercengyModeStartingBlock`
- Synchronizer:
- Track emergency mode starting block
- cli/node
- Add working coordinator config
- coordinator:
- Retry handler for synchronizer stats in case of error (instead of
waiting for the next block to try again)
- On init, trigger an initial call to the handler for synced block
before waiting for the synchronizer, to force the coordinator to start
its logic even if there's no new block right after the node has been
started (very useful for running in testnet where the frequency of
blocks is variable)
- Merge Msg for synced block and updated vars into one: `MsgSyncBlock`.
- API
- When updating network info, handle cases where no batches exists and
where no forgers exists
- cli/node
- Update `cfg.buidler.toml` config file to a working version
- common
- Add new smart contract structs and extend some existing ones to
reflect updates regarding events from the smart contracts
- SQL
- Add new tables and extend existing ones to reflect updates regarding
events from the smart contracts
- db/historydb
- Add functions to insert new smart contract events data
- Fix unclosed rows that led to inconsistent sql driver state (replace
NamedQuery by NamedExec). This fixes the error:
`pq: unexpected Parse response 'C'`
- db/l2db
- Close rows after usage
- eth
- In Rollup event, introduce a new UpdateBucketsParameter when there's a
SafeMode event, with `SafeMode = true`
- synchronizer
- synchronize new events
- avoid calling `auction.CanForge` before the genesisBlock to avoid
getting a revert.
- Add InvalidData flag to L1Tx
- Add BytesDataAvailability to L1Tx
- Update ZKInputs & HashGlobalInputs to last spec of the protocol
(massive migrations)
- TxProcessor check correctness of L1Txs
Compatible with hermeznetwork/commonjs v0.0.4
(c345239bba)
- API:
- Modify the constructor so that hardcoded rollup constants don't need
to be passed (introduce a `Config` and use `configAPI` internally)
- Common:
- Update rollup constants with proper *big.Int when required
- Add BidCoordinator and Slot structs used by the HistoryDB and
Synchronizer.
- Add helper methods to AuctionConstants
- AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL
table: `default_slot_set_bid_slot_num`), which indicates at which
slotNum does the `DefaultSlotSetBid` specified starts applying.
- Config:
- Move coordinator exclusive configuration from the node config to the
coordinator config
- Coordinator:
- Reorganize the code towards having the goroutines started and stopped
from the coordinator itself instead of the node.
- Remove all stop and stopped channels, and use context.Context and
sync.WaitGroup instead.
- Remove BatchInfo setters and assing variables directly
- In ServerProof and ServerProofPool use context instead stop channel.
- Use message passing to notify the coordinator about sync updates and
reorgs
- Introduce the Pipeline, which can be started and stopped by the
Coordinator
- Introduce the TxManager, which manages ethereum transactions (the
TxManager is also in charge of making the forge call to the rollup
smart contract). The TxManager keeps ethereum transactions and:
1. Waits for the transaction to be accepted
2. Waits for the transaction to be confirmed for N blocks
- In forge logic, first prepare a batch and then wait for an available
server proof to have all work ready once the proof server is ready.
- Remove the `isForgeSequence` method which was querying the smart
contract, and instead use notifications sent by the Synchronizer to
figure out if it's forging time.
- Update test (which is a minimal test to manually see if the
coordinator starts)
- HistoryDB:
- Add method to get the number of batches in a slot (used to detect when
a slot has passed the bid winner forging deadline)
- Add method to get the best bid and associated coordinator of a slot
(used to detect the forgerAddress that can forge the slot)
- General:
- Rename some instances of `currentBlock` to `lastBlock` to be more
clear.
- Node:
- Connect the API to the node and call the methods to update cached
state when the sync advances blocks.
- Call methods to update Coordinator state when the sync advances blocks
and finds reorgs.
- Synchronizer:
- Add Auction field in the Stats, which contain the current slot with
info about highest bidder and other related info required to know who
can forge in the current block.
- Better organization of cached state:
- On Sync, update the internal cached state
- On Init or Reorg, load the state from HistoryDB into the
internal cached state.
- In SQL Tables, set rollup_vars.forge_l1_timeout to BIGINT (instead of BYTEA)
- In "bigint" encoding/decoding used with meddler, store the *big.Int as raw
bytes, instead of base64 to be more space efficient
- In exit table, `instant_withdrawn`, `delayed_withdraw_request`, and
`delayed_withdrawn` were referencing batch_num. But these actions happen
outside a batch, so they should reference a block_num.
- Process delayed withdrawns:
- In Synchronizer, first match a Rollup delayed withdrawn request, with the
WDelayer deposit (via TxHash), and store the owner and token associated
with the delayed withdrawn.
- In HistoryDB: store the owner and token of a delayed withdrawal request
in the exit_tree, and set delayed_withdrawn when the withdraw is done in
the WDelayer.
- Update dependency of sqlx to master
- Last release of sqlx is from 2018 October, and it doesn't support
`NamedQuery` with a slice of structs, which is used in this commit.
Previously, the synchronizer test was extending the output from til to
precalculate many fields to compare it with the synchronizer and BD output.
Since this is useful outside of the syncrhonizer testing: move this
functionality to til via a function that extends the output
(til.Context.FillBlocksExtra).
Also, add new functionality: calculate fee idxs dynamically by setting a user
name, and calculate collected fees.
- node:
- Extend config to add initial variables of the smart contracts used as
defaults before they are changed via events.
- In stopped channels, set size 1 so that panics are not witheld until the
node stops completely.
- common:
- In Smart Contract variables, comment:
- `WDelayerVariables.HermezRollupAddress` because it's not needed.
- `RollupVariables.Buckets` because there are no events for it, and for
now it's not used.
- historydb:
- Add functions to get and set smart contract variables.
- db:
- Add `Rollback` function in `utils.go` to reduce boilerplate in sql
transaction rollbacks in defers in db functions.
- Update `rollup_vars` and `auction_vars` (renamed from `consensus_vars`)
table, and add `wdelayer_vars` table.
- synchronizer:
- Synchronize WDelayer
- Handle SC variables properly
- test/ethclient:
- Add essential implementation of WDelayer
- Test L1UserTxs in Synchronizer
- Test Batches in Synchronizer
- Add last_idx in `TABLE batch` in HistoryDB
- Minor updates in til to satisfy blockchain constraints
Eduard S
arnaubennassar
arnaucube
laisolizq
ToniRamirezM