You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1872 lines
58 KiB

Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Redo coordinator structure, connect API to node - API: - Modify the constructor so that hardcoded rollup constants don't need to be passed (introduce a `Config` and use `configAPI` internally) - Common: - Update rollup constants with proper *big.Int when required - Add BidCoordinator and Slot structs used by the HistoryDB and Synchronizer. - Add helper methods to AuctionConstants - AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL table: `default_slot_set_bid_slot_num`), which indicates at which slotNum does the `DefaultSlotSetBid` specified starts applying. - Config: - Move coordinator exclusive configuration from the node config to the coordinator config - Coordinator: - Reorganize the code towards having the goroutines started and stopped from the coordinator itself instead of the node. - Remove all stop and stopped channels, and use context.Context and sync.WaitGroup instead. - Remove BatchInfo setters and assing variables directly - In ServerProof and ServerProofPool use context instead stop channel. - Use message passing to notify the coordinator about sync updates and reorgs - Introduce the Pipeline, which can be started and stopped by the Coordinator - Introduce the TxManager, which manages ethereum transactions (the TxManager is also in charge of making the forge call to the rollup smart contract). The TxManager keeps ethereum transactions and: 1. Waits for the transaction to be accepted 2. Waits for the transaction to be confirmed for N blocks - In forge logic, first prepare a batch and then wait for an available server proof to have all work ready once the proof server is ready. - Remove the `isForgeSequence` method which was querying the smart contract, and instead use notifications sent by the Synchronizer to figure out if it's forging time. - Update test (which is a minimal test to manually see if the coordinator starts) - HistoryDB: - Add method to get the number of batches in a slot (used to detect when a slot has passed the bid winner forging deadline) - Add method to get the best bid and associated coordinator of a slot (used to detect the forgerAddress that can forge the slot) - General: - Rename some instances of `currentBlock` to `lastBlock` to be more clear. - Node: - Connect the API to the node and call the methods to update cached state when the sync advances blocks. - Call methods to update Coordinator state when the sync advances blocks and finds reorgs. - Synchronizer: - Add Auction field in the Stats, which contain the current slot with info about highest bidder and other related info required to know who can forge in the current block. - Better organization of cached state: - On Sync, update the internal cached state - On Init or Reorg, load the state from HistoryDB into the internal cached state.
3 years ago
Update coordinator, call all api update functions - Common: - Rename Block.EthBlockNum to Block.Num to avoid unneeded repetition - API: - Add UpdateNetworkInfoBlock to update just block information, to be used when the node is not yet synchronized - Node: - Call API.UpdateMetrics and UpdateRecommendedFee in a loop, with configurable time intervals - Synchronizer: - When mapping events by TxHash, use an array to support the possibility of multiple calls of the same function happening in the same transaction (for example, a smart contract in a single transaction could call withdraw with delay twice, which would generate 2 withdraw events, and 2 deposit events). - In Stats, keep entire LastBlock instead of just the blockNum - In Stats, add lastL1BatchBlock - Test Stats and SCVars - Coordinator: - Enable writing the BatchInfo in every step of the pipeline to disk (with JSON text files) for debugging purposes. - Move the Pipeline functionality from the Coordinator to its own struct (Pipeline) - Implement shouldL1lL2Batch - In TxManager, implement logic to perform several attempts when doing ethereum node RPC calls before considering the error. (Both for calls to forgeBatch and transaction receipt) - In TxManager, reorganize the flow and note the specific points in which actions are made when err != nil - HistoryDB: - Implement GetLastL1BatchBlockNum: returns the blockNum of the latest forged l1Batch, to help the coordinator decide when to forge an L1Batch. - EthereumClient and test.Client: - Update EthBlockByNumber to return the last block when the passed number is -1.
3 years ago
Update coordinator, call all api update functions - Common: - Rename Block.EthBlockNum to Block.Num to avoid unneeded repetition - API: - Add UpdateNetworkInfoBlock to update just block information, to be used when the node is not yet synchronized - Node: - Call API.UpdateMetrics and UpdateRecommendedFee in a loop, with configurable time intervals - Synchronizer: - When mapping events by TxHash, use an array to support the possibility of multiple calls of the same function happening in the same transaction (for example, a smart contract in a single transaction could call withdraw with delay twice, which would generate 2 withdraw events, and 2 deposit events). - In Stats, keep entire LastBlock instead of just the blockNum - In Stats, add lastL1BatchBlock - Test Stats and SCVars - Coordinator: - Enable writing the BatchInfo in every step of the pipeline to disk (with JSON text files) for debugging purposes. - Move the Pipeline functionality from the Coordinator to its own struct (Pipeline) - Implement shouldL1lL2Batch - In TxManager, implement logic to perform several attempts when doing ethereum node RPC calls before considering the error. (Both for calls to forgeBatch and transaction receipt) - In TxManager, reorganize the flow and note the specific points in which actions are made when err != nil - HistoryDB: - Implement GetLastL1BatchBlockNum: returns the blockNum of the latest forged l1Batch, to help the coordinator decide when to forge an L1Batch. - EthereumClient and test.Client: - Update EthBlockByNumber to return the last block when the passed number is -1.
3 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Redo coordinator structure, connect API to node - API: - Modify the constructor so that hardcoded rollup constants don't need to be passed (introduce a `Config` and use `configAPI` internally) - Common: - Update rollup constants with proper *big.Int when required - Add BidCoordinator and Slot structs used by the HistoryDB and Synchronizer. - Add helper methods to AuctionConstants - AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL table: `default_slot_set_bid_slot_num`), which indicates at which slotNum does the `DefaultSlotSetBid` specified starts applying. - Config: - Move coordinator exclusive configuration from the node config to the coordinator config - Coordinator: - Reorganize the code towards having the goroutines started and stopped from the coordinator itself instead of the node. - Remove all stop and stopped channels, and use context.Context and sync.WaitGroup instead. - Remove BatchInfo setters and assing variables directly - In ServerProof and ServerProofPool use context instead stop channel. - Use message passing to notify the coordinator about sync updates and reorgs - Introduce the Pipeline, which can be started and stopped by the Coordinator - Introduce the TxManager, which manages ethereum transactions (the TxManager is also in charge of making the forge call to the rollup smart contract). The TxManager keeps ethereum transactions and: 1. Waits for the transaction to be accepted 2. Waits for the transaction to be confirmed for N blocks - In forge logic, first prepare a batch and then wait for an available server proof to have all work ready once the proof server is ready. - Remove the `isForgeSequence` method which was querying the smart contract, and instead use notifications sent by the Synchronizer to figure out if it's forging time. - Update test (which is a minimal test to manually see if the coordinator starts) - HistoryDB: - Add method to get the number of batches in a slot (used to detect when a slot has passed the bid winner forging deadline) - Add method to get the best bid and associated coordinator of a slot (used to detect the forgerAddress that can forge the slot) - General: - Rename some instances of `currentBlock` to `lastBlock` to be more clear. - Node: - Connect the API to the node and call the methods to update cached state when the sync advances blocks. - Call methods to update Coordinator state when the sync advances blocks and finds reorgs. - Synchronizer: - Add Auction field in the Stats, which contain the current slot with info about highest bidder and other related info required to know who can forge in the current block. - Better organization of cached state: - On Sync, update the internal cached state - On Init or Reorg, load the state from HistoryDB into the internal cached state.
3 years ago
Redo coordinator structure, connect API to node - API: - Modify the constructor so that hardcoded rollup constants don't need to be passed (introduce a `Config` and use `configAPI` internally) - Common: - Update rollup constants with proper *big.Int when required - Add BidCoordinator and Slot structs used by the HistoryDB and Synchronizer. - Add helper methods to AuctionConstants - AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL table: `default_slot_set_bid_slot_num`), which indicates at which slotNum does the `DefaultSlotSetBid` specified starts applying. - Config: - Move coordinator exclusive configuration from the node config to the coordinator config - Coordinator: - Reorganize the code towards having the goroutines started and stopped from the coordinator itself instead of the node. - Remove all stop and stopped channels, and use context.Context and sync.WaitGroup instead. - Remove BatchInfo setters and assing variables directly - In ServerProof and ServerProofPool use context instead stop channel. - Use message passing to notify the coordinator about sync updates and reorgs - Introduce the Pipeline, which can be started and stopped by the Coordinator - Introduce the TxManager, which manages ethereum transactions (the TxManager is also in charge of making the forge call to the rollup smart contract). The TxManager keeps ethereum transactions and: 1. Waits for the transaction to be accepted 2. Waits for the transaction to be confirmed for N blocks - In forge logic, first prepare a batch and then wait for an available server proof to have all work ready once the proof server is ready. - Remove the `isForgeSequence` method which was querying the smart contract, and instead use notifications sent by the Synchronizer to figure out if it's forging time. - Update test (which is a minimal test to manually see if the coordinator starts) - HistoryDB: - Add method to get the number of batches in a slot (used to detect when a slot has passed the bid winner forging deadline) - Add method to get the best bid and associated coordinator of a slot (used to detect the forgerAddress that can forge the slot) - General: - Rename some instances of `currentBlock` to `lastBlock` to be more clear. - Node: - Connect the API to the node and call the methods to update cached state when the sync advances blocks. - Call methods to update Coordinator state when the sync advances blocks and finds reorgs. - Synchronizer: - Add Auction field in the Stats, which contain the current slot with info about highest bidder and other related info required to know who can forge in the current block. - Better organization of cached state: - On Sync, update the internal cached state - On Init or Reorg, load the state from HistoryDB into the internal cached state.
3 years ago
Redo coordinator structure, connect API to node - API: - Modify the constructor so that hardcoded rollup constants don't need to be passed (introduce a `Config` and use `configAPI` internally) - Common: - Update rollup constants with proper *big.Int when required - Add BidCoordinator and Slot structs used by the HistoryDB and Synchronizer. - Add helper methods to AuctionConstants - AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL table: `default_slot_set_bid_slot_num`), which indicates at which slotNum does the `DefaultSlotSetBid` specified starts applying. - Config: - Move coordinator exclusive configuration from the node config to the coordinator config - Coordinator: - Reorganize the code towards having the goroutines started and stopped from the coordinator itself instead of the node. - Remove all stop and stopped channels, and use context.Context and sync.WaitGroup instead. - Remove BatchInfo setters and assing variables directly - In ServerProof and ServerProofPool use context instead stop channel. - Use message passing to notify the coordinator about sync updates and reorgs - Introduce the Pipeline, which can be started and stopped by the Coordinator - Introduce the TxManager, which manages ethereum transactions (the TxManager is also in charge of making the forge call to the rollup smart contract). The TxManager keeps ethereum transactions and: 1. Waits for the transaction to be accepted 2. Waits for the transaction to be confirmed for N blocks - In forge logic, first prepare a batch and then wait for an available server proof to have all work ready once the proof server is ready. - Remove the `isForgeSequence` method which was querying the smart contract, and instead use notifications sent by the Synchronizer to figure out if it's forging time. - Update test (which is a minimal test to manually see if the coordinator starts) - HistoryDB: - Add method to get the number of batches in a slot (used to detect when a slot has passed the bid winner forging deadline) - Add method to get the best bid and associated coordinator of a slot (used to detect the forgerAddress that can forge the slot) - General: - Rename some instances of `currentBlock` to `lastBlock` to be more clear. - Node: - Connect the API to the node and call the methods to update cached state when the sync advances blocks. - Call methods to update Coordinator state when the sync advances blocks and finds reorgs. - Synchronizer: - Add Auction field in the Stats, which contain the current slot with info about highest bidder and other related info required to know who can forge in the current block. - Better organization of cached state: - On Sync, update the internal cached state - On Init or Reorg, load the state from HistoryDB into the internal cached state.
3 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Redo coordinator structure, connect API to node - API: - Modify the constructor so that hardcoded rollup constants don't need to be passed (introduce a `Config` and use `configAPI` internally) - Common: - Update rollup constants with proper *big.Int when required - Add BidCoordinator and Slot structs used by the HistoryDB and Synchronizer. - Add helper methods to AuctionConstants - AuctionVariables: Add column `DefaultSlotSetBidSlotNum` (in the SQL table: `default_slot_set_bid_slot_num`), which indicates at which slotNum does the `DefaultSlotSetBid` specified starts applying. - Config: - Move coordinator exclusive configuration from the node config to the coordinator config - Coordinator: - Reorganize the code towards having the goroutines started and stopped from the coordinator itself instead of the node. - Remove all stop and stopped channels, and use context.Context and sync.WaitGroup instead. - Remove BatchInfo setters and assing variables directly - In ServerProof and ServerProofPool use context instead stop channel. - Use message passing to notify the coordinator about sync updates and reorgs - Introduce the Pipeline, which can be started and stopped by the Coordinator - Introduce the TxManager, which manages ethereum transactions (the TxManager is also in charge of making the forge call to the rollup smart contract). The TxManager keeps ethereum transactions and: 1. Waits for the transaction to be accepted 2. Waits for the transaction to be confirmed for N blocks - In forge logic, first prepare a batch and then wait for an available server proof to have all work ready once the proof server is ready. - Remove the `isForgeSequence` method which was querying the smart contract, and instead use notifications sent by the Synchronizer to figure out if it's forging time. - Update test (which is a minimal test to manually see if the coordinator starts) - HistoryDB: - Add method to get the number of batches in a slot (used to detect when a slot has passed the bid winner forging deadline) - Add method to get the best bid and associated coordinator of a slot (used to detect the forgerAddress that can forge the slot) - General: - Rename some instances of `currentBlock` to `lastBlock` to be more clear. - Node: - Connect the API to the node and call the methods to update cached state when the sync advances blocks. - Call methods to update Coordinator state when the sync advances blocks and finds reorgs. - Synchronizer: - Add Auction field in the Stats, which contain the current slot with info about highest bidder and other related info required to know who can forge in the current block. - Better organization of cached state: - On Sync, update the internal cached state - On Init or Reorg, load the state from HistoryDB into the internal cached state.
3 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update coordinator, call all api update functions - Common: - Rename Block.EthBlockNum to Block.Num to avoid unneeded repetition - API: - Add UpdateNetworkInfoBlock to update just block information, to be used when the node is not yet synchronized - Node: - Call API.UpdateMetrics and UpdateRecommendedFee in a loop, with configurable time intervals - Synchronizer: - When mapping events by TxHash, use an array to support the possibility of multiple calls of the same function happening in the same transaction (for example, a smart contract in a single transaction could call withdraw with delay twice, which would generate 2 withdraw events, and 2 deposit events). - In Stats, keep entire LastBlock instead of just the blockNum - In Stats, add lastL1BatchBlock - Test Stats and SCVars - Coordinator: - Enable writing the BatchInfo in every step of the pipeline to disk (with JSON text files) for debugging purposes. - Move the Pipeline functionality from the Coordinator to its own struct (Pipeline) - Implement shouldL1lL2Batch - In TxManager, implement logic to perform several attempts when doing ethereum node RPC calls before considering the error. (Both for calls to forgeBatch and transaction receipt) - In TxManager, reorganize the flow and note the specific points in which actions are made when err != nil - HistoryDB: - Implement GetLastL1BatchBlockNum: returns the blockNum of the latest forged l1Batch, to help the coordinator decide when to forge an L1Batch. - EthereumClient and test.Client: - Update EthBlockByNumber to return the last block when the passed number is -1.
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
3 years ago
  1. package historydb
  2. import (
  3. "database/sql"
  4. "errors"
  5. "fmt"
  6. "math"
  7. "math/big"
  8. ethCommon "github.com/ethereum/go-ethereum/common"
  9. "github.com/hermeznetwork/hermez-node/common"
  10. "github.com/hermeznetwork/hermez-node/db"
  11. "github.com/hermeznetwork/tracerr"
  12. "github.com/iden3/go-iden3-crypto/babyjub"
  13. "github.com/jmoiron/sqlx"
  14. //nolint:errcheck // driver for postgres DB
  15. _ "github.com/lib/pq"
  16. "github.com/russross/meddler"
  17. )
  18. const (
  19. // OrderAsc indicates ascending order when using pagination
  20. OrderAsc = "ASC"
  21. // OrderDesc indicates descending order when using pagination
  22. OrderDesc = "DESC"
  23. )
  24. // TODO(Edu): Document here how HistoryDB is kept consistent
  25. // HistoryDB persist the historic of the rollup
  26. type HistoryDB struct {
  27. db *sqlx.DB
  28. }
  29. // NewHistoryDB initialize the DB
  30. func NewHistoryDB(db *sqlx.DB) *HistoryDB {
  31. return &HistoryDB{db: db}
  32. }
  33. // DB returns a pointer to the L2DB.db. This method should be used only for
  34. // internal testing purposes.
  35. func (hdb *HistoryDB) DB() *sqlx.DB {
  36. return hdb.db
  37. }
  38. // AddBlock insert a block into the DB
  39. func (hdb *HistoryDB) AddBlock(block *common.Block) error { return hdb.addBlock(hdb.db, block) }
  40. func (hdb *HistoryDB) addBlock(d meddler.DB, block *common.Block) error {
  41. return tracerr.Wrap(meddler.Insert(d, "block", block))
  42. }
  43. // AddBlocks inserts blocks into the DB
  44. func (hdb *HistoryDB) AddBlocks(blocks []common.Block) error {
  45. return tracerr.Wrap(hdb.addBlocks(hdb.db, blocks))
  46. }
  47. func (hdb *HistoryDB) addBlocks(d meddler.DB, blocks []common.Block) error {
  48. return tracerr.Wrap(db.BulkInsert(
  49. d,
  50. `INSERT INTO block (
  51. eth_block_num,
  52. timestamp,
  53. hash
  54. ) VALUES %s;`,
  55. blocks[:],
  56. ))
  57. }
  58. // GetBlock retrieve a block from the DB, given a block number
  59. func (hdb *HistoryDB) GetBlock(blockNum int64) (*common.Block, error) {
  60. block := &common.Block{}
  61. err := meddler.QueryRow(
  62. hdb.db, block,
  63. "SELECT * FROM block WHERE eth_block_num = $1;", blockNum,
  64. )
  65. return block, tracerr.Wrap(err)
  66. }
  67. // GetAllBlocks retrieve all blocks from the DB
  68. func (hdb *HistoryDB) GetAllBlocks() ([]common.Block, error) {
  69. var blocks []*common.Block
  70. err := meddler.QueryAll(
  71. hdb.db, &blocks,
  72. "SELECT * FROM block;",
  73. )
  74. return db.SlicePtrsToSlice(blocks).([]common.Block), tracerr.Wrap(err)
  75. }
  76. // GetBlocks retrieve blocks from the DB, given a range of block numbers defined by from and to
  77. func (hdb *HistoryDB) GetBlocks(from, to int64) ([]common.Block, error) {
  78. var blocks []*common.Block
  79. err := meddler.QueryAll(
  80. hdb.db, &blocks,
  81. "SELECT * FROM block WHERE $1 <= eth_block_num AND eth_block_num < $2;",
  82. from, to,
  83. )
  84. return db.SlicePtrsToSlice(blocks).([]common.Block), tracerr.Wrap(err)
  85. }
  86. // GetLastBlock retrieve the block with the highest block number from the DB
  87. func (hdb *HistoryDB) GetLastBlock() (*common.Block, error) {
  88. block := &common.Block{}
  89. err := meddler.QueryRow(
  90. hdb.db, block, "SELECT * FROM block ORDER BY eth_block_num DESC LIMIT 1;",
  91. )
  92. return block, tracerr.Wrap(err)
  93. }
  94. // AddBatch insert a Batch into the DB
  95. func (hdb *HistoryDB) AddBatch(batch *common.Batch) error { return hdb.addBatch(hdb.db, batch) }
  96. func (hdb *HistoryDB) addBatch(d meddler.DB, batch *common.Batch) error {
  97. // Calculate total collected fees in USD
  98. // Get IDs of collected tokens for fees
  99. tokenIDs := []common.TokenID{}
  100. for id := range batch.CollectedFees {
  101. tokenIDs = append(tokenIDs, id)
  102. }
  103. // Get USD value of the tokens
  104. type tokenPrice struct {
  105. ID common.TokenID `meddler:"token_id"`
  106. USD *float64 `meddler:"usd"`
  107. Decimals int `meddler:"decimals"`
  108. }
  109. var tokenPrices []*tokenPrice
  110. if len(tokenIDs) > 0 {
  111. query, args, err := sqlx.In(
  112. "SELECT token_id, usd, decimals FROM token WHERE token_id IN (?);",
  113. tokenIDs,
  114. )
  115. if err != nil {
  116. return tracerr.Wrap(err)
  117. }
  118. query = hdb.db.Rebind(query)
  119. if err := meddler.QueryAll(
  120. hdb.db, &tokenPrices, query, args...,
  121. ); err != nil {
  122. return tracerr.Wrap(err)
  123. }
  124. }
  125. // Calculate total collected
  126. var total float64
  127. for _, tokenPrice := range tokenPrices {
  128. if tokenPrice.USD == nil {
  129. continue
  130. }
  131. f := new(big.Float).SetInt(batch.CollectedFees[tokenPrice.ID])
  132. amount, _ := f.Float64()
  133. total += *tokenPrice.USD * (amount / math.Pow(10, float64(tokenPrice.Decimals))) //nolint decimals have to be ^10
  134. }
  135. batch.TotalFeesUSD = &total
  136. // Insert to DB
  137. return tracerr.Wrap(meddler.Insert(d, "batch", batch))
  138. }
  139. // AddBatches insert Bids into the DB
  140. func (hdb *HistoryDB) AddBatches(batches []common.Batch) error {
  141. return tracerr.Wrap(hdb.addBatches(hdb.db, batches))
  142. }
  143. func (hdb *HistoryDB) addBatches(d meddler.DB, batches []common.Batch) error {
  144. for i := 0; i < len(batches); i++ {
  145. if err := hdb.addBatch(d, &batches[i]); err != nil {
  146. return tracerr.Wrap(err)
  147. }
  148. }
  149. return nil
  150. }
  151. // GetBatchAPI return the batch with the given batchNum
  152. func (hdb *HistoryDB) GetBatchAPI(batchNum common.BatchNum) (*BatchAPI, error) {
  153. batch := &BatchAPI{}
  154. return batch, tracerr.Wrap(meddler.QueryRow(
  155. hdb.db, batch,
  156. `SELECT batch.*, block.timestamp, block.hash,
  157. COALESCE ((SELECT COUNT(*) FROM tx WHERE batch_num = batch.batch_num), 0) AS forged_txs
  158. FROM batch INNER JOIN block ON batch.eth_block_num = block.eth_block_num
  159. WHERE batch_num = $1;`, batchNum,
  160. ))
  161. }
  162. // GetBatchesAPI return the batches applying the given filters
  163. func (hdb *HistoryDB) GetBatchesAPI(
  164. minBatchNum, maxBatchNum, slotNum *uint,
  165. forgerAddr *ethCommon.Address,
  166. fromItem, limit *uint, order string,
  167. ) ([]BatchAPI, uint64, error) {
  168. var query string
  169. var args []interface{}
  170. queryStr := `SELECT batch.*, block.timestamp, block.hash,
  171. COALESCE ((SELECT COUNT(*) FROM tx WHERE batch_num = batch.batch_num), 0) AS forged_txs,
  172. count(*) OVER() AS total_items
  173. FROM batch INNER JOIN block ON batch.eth_block_num = block.eth_block_num `
  174. // Apply filters
  175. nextIsAnd := false
  176. // minBatchNum filter
  177. if minBatchNum != nil {
  178. if nextIsAnd {
  179. queryStr += "AND "
  180. } else {
  181. queryStr += "WHERE "
  182. }
  183. queryStr += "batch.batch_num > ? "
  184. args = append(args, minBatchNum)
  185. nextIsAnd = true
  186. }
  187. // maxBatchNum filter
  188. if maxBatchNum != nil {
  189. if nextIsAnd {
  190. queryStr += "AND "
  191. } else {
  192. queryStr += "WHERE "
  193. }
  194. queryStr += "batch.batch_num < ? "
  195. args = append(args, maxBatchNum)
  196. nextIsAnd = true
  197. }
  198. // slotNum filter
  199. if slotNum != nil {
  200. if nextIsAnd {
  201. queryStr += "AND "
  202. } else {
  203. queryStr += "WHERE "
  204. }
  205. queryStr += "batch.slot_num = ? "
  206. args = append(args, slotNum)
  207. nextIsAnd = true
  208. }
  209. // forgerAddr filter
  210. if forgerAddr != nil {
  211. if nextIsAnd {
  212. queryStr += "AND "
  213. } else {
  214. queryStr += "WHERE "
  215. }
  216. queryStr += "batch.forger_addr = ? "
  217. args = append(args, forgerAddr)
  218. nextIsAnd = true
  219. }
  220. // pagination
  221. if fromItem != nil {
  222. if nextIsAnd {
  223. queryStr += "AND "
  224. } else {
  225. queryStr += "WHERE "
  226. }
  227. if order == OrderAsc {
  228. queryStr += "batch.item_id >= ? "
  229. } else {
  230. queryStr += "batch.item_id <= ? "
  231. }
  232. args = append(args, fromItem)
  233. }
  234. queryStr += "ORDER BY batch.item_id "
  235. if order == OrderAsc {
  236. queryStr += " ASC "
  237. } else {
  238. queryStr += " DESC "
  239. }
  240. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  241. query = hdb.db.Rebind(queryStr)
  242. // log.Debug(query)
  243. batchPtrs := []*BatchAPI{}
  244. if err := meddler.QueryAll(hdb.db, &batchPtrs, query, args...); err != nil {
  245. return nil, 0, tracerr.Wrap(err)
  246. }
  247. batches := db.SlicePtrsToSlice(batchPtrs).([]BatchAPI)
  248. if len(batches) == 0 {
  249. return nil, 0, tracerr.Wrap(sql.ErrNoRows)
  250. }
  251. return batches, batches[0].TotalItems - uint64(len(batches)), nil
  252. }
  253. // GetAllBatches retrieve all batches from the DB
  254. func (hdb *HistoryDB) GetAllBatches() ([]common.Batch, error) {
  255. var batches []*common.Batch
  256. err := meddler.QueryAll(
  257. hdb.db, &batches,
  258. `SELECT batch.batch_num, batch.eth_block_num, batch.forger_addr, batch.fees_collected,
  259. batch.fee_idxs_coordinator, batch.state_root, batch.num_accounts, batch.last_idx, batch.exit_root,
  260. batch.forge_l1_txs_num, batch.slot_num, batch.total_fees_usd FROM batch;`,
  261. )
  262. return db.SlicePtrsToSlice(batches).([]common.Batch), tracerr.Wrap(err)
  263. }
  264. // GetBatches retrieve batches from the DB, given a range of batch numbers defined by from and to
  265. func (hdb *HistoryDB) GetBatches(from, to common.BatchNum) ([]common.Batch, error) {
  266. var batches []*common.Batch
  267. err := meddler.QueryAll(
  268. hdb.db, &batches,
  269. "SELECT * FROM batch WHERE $1 <= batch_num AND batch_num < $2;",
  270. from, to,
  271. )
  272. return db.SlicePtrsToSlice(batches).([]common.Batch), tracerr.Wrap(err)
  273. }
  274. // GetFirstBatchBlockNumBySlot returns the ethereum block number of the first
  275. // batch within a slot
  276. func (hdb *HistoryDB) GetFirstBatchBlockNumBySlot(slotNum int64) (int64, error) {
  277. row := hdb.db.QueryRow(
  278. `SELECT eth_block_num FROM batch
  279. WHERE slot_num = $1 ORDER BY batch_num ASC LIMIT 1;`, slotNum,
  280. )
  281. var blockNum int64
  282. return blockNum, tracerr.Wrap(row.Scan(&blockNum))
  283. }
  284. // GetLastBatchNum returns the BatchNum of the latest forged batch
  285. func (hdb *HistoryDB) GetLastBatchNum() (common.BatchNum, error) {
  286. row := hdb.db.QueryRow("SELECT batch_num FROM batch ORDER BY batch_num DESC LIMIT 1;")
  287. var batchNum common.BatchNum
  288. return batchNum, tracerr.Wrap(row.Scan(&batchNum))
  289. }
  290. // GetLastL1BatchBlockNum returns the blockNum of the latest forged l1Batch
  291. func (hdb *HistoryDB) GetLastL1BatchBlockNum() (int64, error) {
  292. row := hdb.db.QueryRow(`SELECT eth_block_num FROM batch
  293. WHERE forge_l1_txs_num IS NOT NULL
  294. ORDER BY batch_num DESC LIMIT 1;`)
  295. var blockNum int64
  296. return blockNum, tracerr.Wrap(row.Scan(&blockNum))
  297. }
  298. // GetLastL1TxsNum returns the greatest ForgeL1TxsNum in the DB from forged
  299. // batches. If there's no batch in the DB (nil, nil) is returned.
  300. func (hdb *HistoryDB) GetLastL1TxsNum() (*int64, error) {
  301. row := hdb.db.QueryRow("SELECT MAX(forge_l1_txs_num) FROM batch;")
  302. lastL1TxsNum := new(int64)
  303. return lastL1TxsNum, tracerr.Wrap(row.Scan(&lastL1TxsNum))
  304. }
  305. // Reorg deletes all the information that was added into the DB after the
  306. // lastValidBlock. If lastValidBlock is negative, all block information is
  307. // deleted.
  308. func (hdb *HistoryDB) Reorg(lastValidBlock int64) error {
  309. var err error
  310. if lastValidBlock < 0 {
  311. _, err = hdb.db.Exec("DELETE FROM block;")
  312. } else {
  313. _, err = hdb.db.Exec("DELETE FROM block WHERE eth_block_num > $1;", lastValidBlock)
  314. }
  315. return tracerr.Wrap(err)
  316. }
  317. // AddBids insert Bids into the DB
  318. func (hdb *HistoryDB) AddBids(bids []common.Bid) error { return hdb.addBids(hdb.db, bids) }
  319. func (hdb *HistoryDB) addBids(d meddler.DB, bids []common.Bid) error {
  320. if len(bids) == 0 {
  321. return nil
  322. }
  323. // TODO: check the coordinator info
  324. return tracerr.Wrap(db.BulkInsert(
  325. d,
  326. "INSERT INTO bid (slot_num, bid_value, eth_block_num, bidder_addr) VALUES %s;",
  327. bids[:],
  328. ))
  329. }
  330. // GetAllBids retrieve all bids from the DB
  331. func (hdb *HistoryDB) GetAllBids() ([]common.Bid, error) {
  332. var bids []*common.Bid
  333. err := meddler.QueryAll(
  334. hdb.db, &bids,
  335. `SELECT bid.slot_num, bid.bid_value, bid.eth_block_num, bid.bidder_addr FROM bid;`,
  336. )
  337. return db.SlicePtrsToSlice(bids).([]common.Bid), tracerr.Wrap(err)
  338. }
  339. // GetBestBidAPI returns the best bid in specific slot by slotNum
  340. func (hdb *HistoryDB) GetBestBidAPI(slotNum *int64) (BidAPI, error) {
  341. bid := &BidAPI{}
  342. err := meddler.QueryRow(
  343. hdb.db, bid, `SELECT bid.*, block.timestamp, coordinator.forger_addr, coordinator.url
  344. FROM bid INNER JOIN block ON bid.eth_block_num = block.eth_block_num
  345. INNER JOIN coordinator ON bid.bidder_addr = coordinator.bidder_addr
  346. WHERE slot_num = $1 ORDER BY item_id DESC LIMIT 1;`, slotNum,
  347. )
  348. return *bid, tracerr.Wrap(err)
  349. }
  350. // GetBestBidCoordinator returns the forger address of the highest bidder in a slot by slotNum
  351. func (hdb *HistoryDB) GetBestBidCoordinator(slotNum int64) (*common.BidCoordinator, error) {
  352. bidCoord := &common.BidCoordinator{}
  353. err := meddler.QueryRow(
  354. hdb.db, bidCoord,
  355. `SELECT (
  356. SELECT default_slot_set_bid
  357. FROM auction_vars
  358. WHERE default_slot_set_bid_slot_num <= $1
  359. ORDER BY eth_block_num DESC LIMIT 1
  360. ),
  361. bid.slot_num, bid.bid_value, bid.bidder_addr,
  362. coordinator.forger_addr, coordinator.url
  363. FROM bid
  364. INNER JOIN coordinator ON bid.bidder_addr = coordinator.bidder_addr
  365. WHERE bid.slot_num = $1 ORDER BY bid.item_id DESC LIMIT 1;`,
  366. slotNum)
  367. return bidCoord, tracerr.Wrap(err)
  368. }
  369. // GetBestBidsAPI returns the best bid in specific slot by slotNum
  370. func (hdb *HistoryDB) GetBestBidsAPI(
  371. minSlotNum, maxSlotNum *int64,
  372. bidderAddr *ethCommon.Address,
  373. limit *uint, order string,
  374. ) ([]BidAPI, uint64, error) {
  375. var query string
  376. var args []interface{}
  377. queryStr := `SELECT b.*, block.timestamp, coordinator.forger_addr, coordinator.url,
  378. COUNT(*) OVER() AS total_items FROM (
  379. SELECT slot_num, MAX(item_id) as maxitem
  380. FROM bid GROUP BY slot_num
  381. )
  382. AS x INNER JOIN bid AS b ON b.item_id = x.maxitem
  383. INNER JOIN block ON b.eth_block_num = block.eth_block_num
  384. INNER JOIN coordinator ON b.bidder_addr = coordinator.bidder_addr
  385. WHERE (b.slot_num >= ? AND b.slot_num <= ?)`
  386. args = append(args, minSlotNum)
  387. args = append(args, maxSlotNum)
  388. // Apply filters
  389. if bidderAddr != nil {
  390. queryStr += " AND b.bidder_addr = ? "
  391. args = append(args, bidderAddr)
  392. }
  393. queryStr += " ORDER BY b.slot_num "
  394. if order == OrderAsc {
  395. queryStr += "ASC "
  396. } else {
  397. queryStr += "DESC "
  398. }
  399. if limit != nil {
  400. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  401. }
  402. query = hdb.db.Rebind(queryStr)
  403. bidPtrs := []*BidAPI{}
  404. if err := meddler.QueryAll(hdb.db, &bidPtrs, query, args...); err != nil {
  405. return nil, 0, tracerr.Wrap(err)
  406. }
  407. // log.Debug(query)
  408. bids := db.SlicePtrsToSlice(bidPtrs).([]BidAPI)
  409. if len(bids) == 0 {
  410. return nil, 0, tracerr.Wrap(sql.ErrNoRows)
  411. }
  412. return bids, bids[0].TotalItems - uint64(len(bids)), nil
  413. }
  414. // GetBidsAPI return the bids applying the given filters
  415. func (hdb *HistoryDB) GetBidsAPI(
  416. slotNum *int64, forgerAddr *ethCommon.Address,
  417. fromItem, limit *uint, order string,
  418. ) ([]BidAPI, uint64, error) {
  419. var query string
  420. var args []interface{}
  421. queryStr := `SELECT bid.*, block.timestamp, coordinator.forger_addr, coordinator.url,
  422. COUNT(*) OVER() AS total_items
  423. FROM bid INNER JOIN block ON bid.eth_block_num = block.eth_block_num
  424. INNER JOIN coordinator ON bid.bidder_addr = coordinator.bidder_addr `
  425. // Apply filters
  426. nextIsAnd := false
  427. // slotNum filter
  428. if slotNum != nil {
  429. if nextIsAnd {
  430. queryStr += "AND "
  431. } else {
  432. queryStr += "WHERE "
  433. }
  434. queryStr += "bid.slot_num = ? "
  435. args = append(args, slotNum)
  436. nextIsAnd = true
  437. }
  438. // slotNum filter
  439. if forgerAddr != nil {
  440. if nextIsAnd {
  441. queryStr += "AND "
  442. } else {
  443. queryStr += "WHERE "
  444. }
  445. queryStr += "bid.bidder_addr = ? "
  446. args = append(args, forgerAddr)
  447. nextIsAnd = true
  448. }
  449. if fromItem != nil {
  450. if nextIsAnd {
  451. queryStr += "AND "
  452. } else {
  453. queryStr += "WHERE "
  454. }
  455. if order == OrderAsc {
  456. queryStr += "bid.item_id >= ? "
  457. } else {
  458. queryStr += "bid.item_id <= ? "
  459. }
  460. args = append(args, fromItem)
  461. }
  462. // pagination
  463. queryStr += "ORDER BY bid.item_id "
  464. if order == OrderAsc {
  465. queryStr += "ASC "
  466. } else {
  467. queryStr += "DESC "
  468. }
  469. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  470. query, argsQ, err := sqlx.In(queryStr, args...)
  471. if err != nil {
  472. return nil, 0, tracerr.Wrap(err)
  473. }
  474. query = hdb.db.Rebind(query)
  475. bids := []*BidAPI{}
  476. if err := meddler.QueryAll(hdb.db, &bids, query, argsQ...); err != nil {
  477. return nil, 0, tracerr.Wrap(err)
  478. }
  479. if len(bids) == 0 {
  480. return nil, 0, tracerr.Wrap(sql.ErrNoRows)
  481. }
  482. return db.SlicePtrsToSlice(bids).([]BidAPI), bids[0].TotalItems - uint64(len(bids)), nil
  483. }
  484. // AddCoordinators insert Coordinators into the DB
  485. func (hdb *HistoryDB) AddCoordinators(coordinators []common.Coordinator) error {
  486. return tracerr.Wrap(hdb.addCoordinators(hdb.db, coordinators))
  487. }
  488. func (hdb *HistoryDB) addCoordinators(d meddler.DB, coordinators []common.Coordinator) error {
  489. if len(coordinators) == 0 {
  490. return nil
  491. }
  492. return tracerr.Wrap(db.BulkInsert(
  493. d,
  494. "INSERT INTO coordinator (bidder_addr, forger_addr, eth_block_num, url) VALUES %s;",
  495. coordinators[:],
  496. ))
  497. }
  498. // AddExitTree insert Exit tree into the DB
  499. func (hdb *HistoryDB) AddExitTree(exitTree []common.ExitInfo) error {
  500. return tracerr.Wrap(hdb.addExitTree(hdb.db, exitTree))
  501. }
  502. func (hdb *HistoryDB) addExitTree(d meddler.DB, exitTree []common.ExitInfo) error {
  503. if len(exitTree) == 0 {
  504. return nil
  505. }
  506. return tracerr.Wrap(db.BulkInsert(
  507. d,
  508. "INSERT INTO exit_tree (batch_num, account_idx, merkle_proof, balance, "+
  509. "instant_withdrawn, delayed_withdraw_request, delayed_withdrawn) VALUES %s;",
  510. exitTree[:],
  511. ))
  512. }
  513. func (hdb *HistoryDB) updateExitTree(d sqlx.Ext, blockNum int64,
  514. rollupWithdrawals []common.WithdrawInfo, wDelayerWithdrawals []common.WDelayerTransfer) error {
  515. if len(rollupWithdrawals) == 0 && len(wDelayerWithdrawals) == 0 {
  516. return nil
  517. }
  518. type withdrawal struct {
  519. BatchNum int64 `db:"batch_num"`
  520. AccountIdx int64 `db:"account_idx"`
  521. InstantWithdrawn *int64 `db:"instant_withdrawn"`
  522. DelayedWithdrawRequest *int64 `db:"delayed_withdraw_request"`
  523. DelayedWithdrawn *int64 `db:"delayed_withdrawn"`
  524. Owner *ethCommon.Address `db:"owner"`
  525. Token *ethCommon.Address `db:"token"`
  526. }
  527. withdrawals := make([]withdrawal, len(rollupWithdrawals)+len(wDelayerWithdrawals))
  528. for i := range rollupWithdrawals {
  529. info := &rollupWithdrawals[i]
  530. withdrawals[i] = withdrawal{
  531. BatchNum: int64(info.NumExitRoot),
  532. AccountIdx: int64(info.Idx),
  533. }
  534. if info.InstantWithdraw {
  535. withdrawals[i].InstantWithdrawn = &blockNum
  536. } else {
  537. withdrawals[i].DelayedWithdrawRequest = &blockNum
  538. withdrawals[i].Owner = &info.Owner
  539. withdrawals[i].Token = &info.Token
  540. }
  541. }
  542. for i := range wDelayerWithdrawals {
  543. info := &wDelayerWithdrawals[i]
  544. withdrawals[len(rollupWithdrawals)+i] = withdrawal{
  545. DelayedWithdrawn: &blockNum,
  546. Owner: &info.Owner,
  547. Token: &info.Token,
  548. }
  549. }
  550. // In VALUES we set an initial row of NULLs to set the types of each
  551. // variable passed as argument
  552. const query string = `
  553. UPDATE exit_tree e SET
  554. instant_withdrawn = d.instant_withdrawn,
  555. delayed_withdraw_request = CASE
  556. WHEN e.delayed_withdraw_request IS NOT NULL THEN e.delayed_withdraw_request
  557. ELSE d.delayed_withdraw_request
  558. END,
  559. delayed_withdrawn = d.delayed_withdrawn,
  560. owner = d.owner,
  561. token = d.token
  562. FROM (VALUES
  563. (NULL::::BIGINT, NULL::::BIGINT, NULL::::BIGINT, NULL::::BIGINT, NULL::::BIGINT, NULL::::BYTEA, NULL::::BYTEA),
  564. (:batch_num,
  565. :account_idx,
  566. :instant_withdrawn,
  567. :delayed_withdraw_request,
  568. :delayed_withdrawn,
  569. :owner,
  570. :token)
  571. ) as d (batch_num, account_idx, instant_withdrawn, delayed_withdraw_request, delayed_withdrawn, owner, token)
  572. WHERE
  573. (d.batch_num IS NOT NULL AND e.batch_num = d.batch_num AND e.account_idx = d.account_idx) OR
  574. (d.delayed_withdrawn IS NOT NULL AND e.delayed_withdrawn IS NULL AND e.owner = d.owner AND e.token = d.token);
  575. `
  576. if len(withdrawals) > 0 {
  577. if _, err := sqlx.NamedExec(d, query, withdrawals); err != nil {
  578. return tracerr.Wrap(err)
  579. }
  580. }
  581. return nil
  582. }
  583. // AddToken insert a token into the DB
  584. func (hdb *HistoryDB) AddToken(token *common.Token) error {
  585. return tracerr.Wrap(meddler.Insert(hdb.db, "token", token))
  586. }
  587. // AddTokens insert tokens into the DB
  588. func (hdb *HistoryDB) AddTokens(tokens []common.Token) error { return hdb.addTokens(hdb.db, tokens) }
  589. func (hdb *HistoryDB) addTokens(d meddler.DB, tokens []common.Token) error {
  590. if len(tokens) == 0 {
  591. return nil
  592. }
  593. return tracerr.Wrap(db.BulkInsert(
  594. d,
  595. `INSERT INTO token (
  596. token_id,
  597. eth_block_num,
  598. eth_addr,
  599. name,
  600. symbol,
  601. decimals
  602. ) VALUES %s;`,
  603. tokens[:],
  604. ))
  605. }
  606. // UpdateTokenValue updates the USD value of a token
  607. func (hdb *HistoryDB) UpdateTokenValue(tokenSymbol string, value float64) error {
  608. _, err := hdb.db.Exec(
  609. "UPDATE token SET usd = $1 WHERE symbol = $2;",
  610. value, tokenSymbol,
  611. )
  612. return tracerr.Wrap(err)
  613. }
  614. // GetToken returns a token from the DB given a TokenID
  615. func (hdb *HistoryDB) GetToken(tokenID common.TokenID) (*TokenWithUSD, error) {
  616. token := &TokenWithUSD{}
  617. err := meddler.QueryRow(
  618. hdb.db, token, `SELECT * FROM token WHERE token_id = $1;`, tokenID,
  619. )
  620. return token, tracerr.Wrap(err)
  621. }
  622. // GetAllTokens returns all tokens from the DB
  623. func (hdb *HistoryDB) GetAllTokens() ([]TokenWithUSD, error) {
  624. var tokens []*TokenWithUSD
  625. err := meddler.QueryAll(
  626. hdb.db, &tokens,
  627. "SELECT * FROM token ORDER BY token_id;",
  628. )
  629. return db.SlicePtrsToSlice(tokens).([]TokenWithUSD), tracerr.Wrap(err)
  630. }
  631. // GetTokens returns a list of tokens from the DB
  632. func (hdb *HistoryDB) GetTokens(
  633. ids []common.TokenID, symbols []string, name string, fromItem,
  634. limit *uint, order string,
  635. ) ([]TokenWithUSD, uint64, error) {
  636. var query string
  637. var args []interface{}
  638. queryStr := `SELECT * , COUNT(*) OVER() AS total_items FROM token `
  639. // Apply filters
  640. nextIsAnd := false
  641. if len(ids) > 0 {
  642. queryStr += "WHERE token_id IN (?) "
  643. nextIsAnd = true
  644. args = append(args, ids)
  645. }
  646. if len(symbols) > 0 {
  647. if nextIsAnd {
  648. queryStr += "AND "
  649. } else {
  650. queryStr += "WHERE "
  651. }
  652. queryStr += "symbol IN (?) "
  653. args = append(args, symbols)
  654. nextIsAnd = true
  655. }
  656. if name != "" {
  657. if nextIsAnd {
  658. queryStr += "AND "
  659. } else {
  660. queryStr += "WHERE "
  661. }
  662. queryStr += "name ~ ? "
  663. args = append(args, name)
  664. nextIsAnd = true
  665. }
  666. if fromItem != nil {
  667. if nextIsAnd {
  668. queryStr += "AND "
  669. } else {
  670. queryStr += "WHERE "
  671. }
  672. if order == OrderAsc {
  673. queryStr += "item_id >= ? "
  674. } else {
  675. queryStr += "item_id <= ? "
  676. }
  677. args = append(args, fromItem)
  678. }
  679. // pagination
  680. queryStr += "ORDER BY item_id "
  681. if order == OrderAsc {
  682. queryStr += "ASC "
  683. } else {
  684. queryStr += "DESC "
  685. }
  686. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  687. query, argsQ, err := sqlx.In(queryStr, args...)
  688. if err != nil {
  689. return nil, 0, tracerr.Wrap(err)
  690. }
  691. query = hdb.db.Rebind(query)
  692. tokens := []*TokenWithUSD{}
  693. if err := meddler.QueryAll(hdb.db, &tokens, query, argsQ...); err != nil {
  694. return nil, 0, tracerr.Wrap(err)
  695. }
  696. if len(tokens) == 0 {
  697. return nil, 0, tracerr.Wrap(sql.ErrNoRows)
  698. }
  699. return db.SlicePtrsToSlice(tokens).([]TokenWithUSD), uint64(len(tokens)) - tokens[0].TotalItems, nil
  700. }
  701. // GetTokenSymbols returns all the token symbols from the DB
  702. func (hdb *HistoryDB) GetTokenSymbols() ([]string, error) {
  703. var tokenSymbols []string
  704. rows, err := hdb.db.Query("SELECT symbol FROM token;")
  705. if err != nil {
  706. return nil, tracerr.Wrap(err)
  707. }
  708. defer db.RowsClose(rows)
  709. sym := new(string)
  710. for rows.Next() {
  711. err = rows.Scan(sym)
  712. if err != nil {
  713. return nil, tracerr.Wrap(err)
  714. }
  715. tokenSymbols = append(tokenSymbols, *sym)
  716. }
  717. return tokenSymbols, nil
  718. }
  719. // AddAccounts insert accounts into the DB
  720. func (hdb *HistoryDB) AddAccounts(accounts []common.Account) error {
  721. return tracerr.Wrap(hdb.addAccounts(hdb.db, accounts))
  722. }
  723. func (hdb *HistoryDB) addAccounts(d meddler.DB, accounts []common.Account) error {
  724. if len(accounts) == 0 {
  725. return nil
  726. }
  727. return tracerr.Wrap(db.BulkInsert(
  728. d,
  729. `INSERT INTO account (
  730. idx,
  731. token_id,
  732. batch_num,
  733. bjj,
  734. eth_addr
  735. ) VALUES %s;`,
  736. accounts[:],
  737. ))
  738. }
  739. // GetAllAccounts returns a list of accounts from the DB
  740. func (hdb *HistoryDB) GetAllAccounts() ([]common.Account, error) {
  741. var accs []*common.Account
  742. err := meddler.QueryAll(
  743. hdb.db, &accs,
  744. "SELECT * FROM account ORDER BY idx;",
  745. )
  746. return db.SlicePtrsToSlice(accs).([]common.Account), tracerr.Wrap(err)
  747. }
  748. // AddL1Txs inserts L1 txs to the DB. USD and DepositAmountUSD will be set automatically before storing the tx.
  749. // If the tx is originated by a coordinator, BatchNum must be provided. If it's originated by a user,
  750. // BatchNum should be null, and the value will be setted by a trigger when a batch forges the tx.
  751. // EffectiveAmount and EffectiveDepositAmount are seted with default values by the DB.
  752. func (hdb *HistoryDB) AddL1Txs(l1txs []common.L1Tx) error {
  753. return tracerr.Wrap(hdb.addL1Txs(hdb.db, l1txs))
  754. }
  755. // addL1Txs inserts L1 txs to the DB. USD and DepositAmountUSD will be set automatically before storing the tx.
  756. // If the tx is originated by a coordinator, BatchNum must be provided. If it's originated by a user,
  757. // BatchNum should be null, and the value will be setted by a trigger when a batch forges the tx.
  758. // EffectiveAmount and EffectiveDepositAmount are seted with default values by the DB.
  759. func (hdb *HistoryDB) addL1Txs(d meddler.DB, l1txs []common.L1Tx) error {
  760. txs := []txWrite{}
  761. for i := 0; i < len(l1txs); i++ {
  762. af := new(big.Float).SetInt(l1txs[i].Amount)
  763. amountFloat, _ := af.Float64()
  764. laf := new(big.Float).SetInt(l1txs[i].DepositAmount)
  765. depositAmountFloat, _ := laf.Float64()
  766. txs = append(txs, txWrite{
  767. // Generic
  768. IsL1: true,
  769. TxID: l1txs[i].TxID,
  770. Type: l1txs[i].Type,
  771. Position: l1txs[i].Position,
  772. FromIdx: &l1txs[i].FromIdx,
  773. ToIdx: l1txs[i].ToIdx,
  774. Amount: l1txs[i].Amount,
  775. AmountFloat: amountFloat,
  776. TokenID: l1txs[i].TokenID,
  777. BatchNum: l1txs[i].BatchNum,
  778. EthBlockNum: l1txs[i].EthBlockNum,
  779. // L1
  780. ToForgeL1TxsNum: l1txs[i].ToForgeL1TxsNum,
  781. UserOrigin: &l1txs[i].UserOrigin,
  782. FromEthAddr: &l1txs[i].FromEthAddr,
  783. FromBJJ: &l1txs[i].FromBJJ,
  784. DepositAmount: l1txs[i].DepositAmount,
  785. DepositAmountFloat: &depositAmountFloat,
  786. })
  787. }
  788. return tracerr.Wrap(hdb.addTxs(d, txs))
  789. }
  790. // AddL2Txs inserts L2 txs to the DB. TokenID, USD and FeeUSD will be set automatically before storing the tx.
  791. func (hdb *HistoryDB) AddL2Txs(l2txs []common.L2Tx) error {
  792. return tracerr.Wrap(hdb.addL2Txs(hdb.db, l2txs))
  793. }
  794. // addL2Txs inserts L2 txs to the DB. TokenID, USD and FeeUSD will be set automatically before storing the tx.
  795. func (hdb *HistoryDB) addL2Txs(d meddler.DB, l2txs []common.L2Tx) error {
  796. txs := []txWrite{}
  797. for i := 0; i < len(l2txs); i++ {
  798. f := new(big.Float).SetInt(l2txs[i].Amount)
  799. amountFloat, _ := f.Float64()
  800. txs = append(txs, txWrite{
  801. // Generic
  802. IsL1: false,
  803. TxID: l2txs[i].TxID,
  804. Type: l2txs[i].Type,
  805. Position: l2txs[i].Position,
  806. FromIdx: &l2txs[i].FromIdx,
  807. ToIdx: l2txs[i].ToIdx,
  808. Amount: l2txs[i].Amount,
  809. AmountFloat: amountFloat,
  810. BatchNum: &l2txs[i].BatchNum,
  811. EthBlockNum: l2txs[i].EthBlockNum,
  812. // L2
  813. Fee: &l2txs[i].Fee,
  814. Nonce: &l2txs[i].Nonce,
  815. })
  816. }
  817. return tracerr.Wrap(hdb.addTxs(d, txs))
  818. }
  819. func (hdb *HistoryDB) addTxs(d meddler.DB, txs []txWrite) error {
  820. if len(txs) == 0 {
  821. return nil
  822. }
  823. return tracerr.Wrap(db.BulkInsert(
  824. d,
  825. `INSERT INTO tx (
  826. is_l1,
  827. id,
  828. type,
  829. position,
  830. from_idx,
  831. to_idx,
  832. amount,
  833. amount_f,
  834. token_id,
  835. batch_num,
  836. eth_block_num,
  837. to_forge_l1_txs_num,
  838. user_origin,
  839. from_eth_addr,
  840. from_bjj,
  841. deposit_amount,
  842. deposit_amount_f,
  843. fee,
  844. nonce
  845. ) VALUES %s;`,
  846. txs[:],
  847. ))
  848. }
  849. // // GetTxs returns a list of txs from the DB
  850. // func (hdb *HistoryDB) GetTxs() ([]common.Tx, error) {
  851. // var txs []*common.Tx
  852. // err := meddler.QueryAll(
  853. // hdb.db, &txs,
  854. // `SELECT * FROM tx
  855. // ORDER BY (batch_num, position) ASC`,
  856. // )
  857. // return db.SlicePtrsToSlice(txs).([]common.Tx), err
  858. // }
  859. // GetHistoryTx returns a tx from the DB given a TxID
  860. func (hdb *HistoryDB) GetHistoryTx(txID common.TxID) (*TxAPI, error) {
  861. // Warning: amount_success and deposit_amount_success have true as default for
  862. // performance reasons. The expected default value is false (when txs are unforged)
  863. // this case is handled at the function func (tx TxAPI) MarshalJSON() ([]byte, error)
  864. tx := &TxAPI{}
  865. err := meddler.QueryRow(
  866. hdb.db, tx, `SELECT tx.item_id, tx.is_l1, tx.id, tx.type, tx.position,
  867. hez_idx(tx.from_idx, token.symbol) AS from_idx, tx.from_eth_addr, tx.from_bjj,
  868. hez_idx(tx.to_idx, token.symbol) AS to_idx, tx.to_eth_addr, tx.to_bjj,
  869. tx.amount, tx.amount_success, tx.token_id, tx.amount_usd,
  870. tx.batch_num, tx.eth_block_num, tx.to_forge_l1_txs_num, tx.user_origin,
  871. tx.deposit_amount, tx.deposit_amount_usd, tx.deposit_amount_success, tx.fee, tx.fee_usd, tx.nonce,
  872. token.token_id, token.item_id AS token_item_id, token.eth_block_num AS token_block,
  873. token.eth_addr, token.name, token.symbol, token.decimals, token.usd,
  874. token.usd_update, block.timestamp
  875. FROM tx INNER JOIN token ON tx.token_id = token.token_id
  876. INNER JOIN block ON tx.eth_block_num = block.eth_block_num
  877. WHERE tx.id = $1;`, txID,
  878. )
  879. return tx, tracerr.Wrap(err)
  880. }
  881. // GetHistoryTxs returns a list of txs from the DB using the HistoryTx struct
  882. // and pagination info
  883. func (hdb *HistoryDB) GetHistoryTxs(
  884. ethAddr *ethCommon.Address, bjj *babyjub.PublicKeyComp,
  885. tokenID *common.TokenID, idx *common.Idx, batchNum *uint, txType *common.TxType,
  886. fromItem, limit *uint, order string,
  887. ) ([]TxAPI, uint64, error) {
  888. // Warning: amount_success and deposit_amount_success have true as default for
  889. // performance reasons. The expected default value is false (when txs are unforged)
  890. // this case is handled at the function func (tx TxAPI) MarshalJSON() ([]byte, error)
  891. if ethAddr != nil && bjj != nil {
  892. return nil, 0, tracerr.Wrap(errors.New("ethAddr and bjj are incompatible"))
  893. }
  894. var query string
  895. var args []interface{}
  896. queryStr := `SELECT tx.item_id, tx.is_l1, tx.id, tx.type, tx.position,
  897. hez_idx(tx.from_idx, token.symbol) AS from_idx, tx.from_eth_addr, tx.from_bjj,
  898. hez_idx(tx.to_idx, token.symbol) AS to_idx, tx.to_eth_addr, tx.to_bjj,
  899. tx.amount, tx.amount_success, tx.token_id, tx.amount_usd,
  900. tx.batch_num, tx.eth_block_num, tx.to_forge_l1_txs_num, tx.user_origin,
  901. tx.deposit_amount, tx.deposit_amount_usd, tx.deposit_amount_success, tx.fee, tx.fee_usd, tx.nonce,
  902. token.token_id, token.item_id AS token_item_id, token.eth_block_num AS token_block,
  903. token.eth_addr, token.name, token.symbol, token.decimals, token.usd,
  904. token.usd_update, block.timestamp, count(*) OVER() AS total_items
  905. FROM tx INNER JOIN token ON tx.token_id = token.token_id
  906. INNER JOIN block ON tx.eth_block_num = block.eth_block_num `
  907. // Apply filters
  908. nextIsAnd := false
  909. // ethAddr filter
  910. if ethAddr != nil {
  911. queryStr += "WHERE (tx.from_eth_addr = ? OR tx.to_eth_addr = ?) "
  912. nextIsAnd = true
  913. args = append(args, ethAddr, ethAddr)
  914. } else if bjj != nil { // bjj filter
  915. queryStr += "WHERE (tx.from_bjj = ? OR tx.to_bjj = ?) "
  916. nextIsAnd = true
  917. args = append(args, bjj, bjj)
  918. }
  919. // tokenID filter
  920. if tokenID != nil {
  921. if nextIsAnd {
  922. queryStr += "AND "
  923. } else {
  924. queryStr += "WHERE "
  925. }
  926. queryStr += "tx.token_id = ? "
  927. args = append(args, tokenID)
  928. nextIsAnd = true
  929. }
  930. // idx filter
  931. if idx != nil {
  932. if nextIsAnd {
  933. queryStr += "AND "
  934. } else {
  935. queryStr += "WHERE "
  936. }
  937. queryStr += "(tx.from_idx = ? OR tx.to_idx = ?) "
  938. args = append(args, idx, idx)
  939. nextIsAnd = true
  940. }
  941. // batchNum filter
  942. if batchNum != nil {
  943. if nextIsAnd {
  944. queryStr += "AND "
  945. } else {
  946. queryStr += "WHERE "
  947. }
  948. queryStr += "tx.batch_num = ? "
  949. args = append(args, batchNum)
  950. nextIsAnd = true
  951. }
  952. // txType filter
  953. if txType != nil {
  954. if nextIsAnd {
  955. queryStr += "AND "
  956. } else {
  957. queryStr += "WHERE "
  958. }
  959. queryStr += "tx.type = ? "
  960. args = append(args, txType)
  961. nextIsAnd = true
  962. }
  963. if fromItem != nil {
  964. if nextIsAnd {
  965. queryStr += "AND "
  966. } else {
  967. queryStr += "WHERE "
  968. }
  969. if order == OrderAsc {
  970. queryStr += "tx.item_id >= ? "
  971. } else {
  972. queryStr += "tx.item_id <= ? "
  973. }
  974. args = append(args, fromItem)
  975. nextIsAnd = true
  976. }
  977. if nextIsAnd {
  978. queryStr += "AND "
  979. } else {
  980. queryStr += "WHERE "
  981. }
  982. queryStr += "tx.batch_num IS NOT NULL "
  983. // pagination
  984. queryStr += "ORDER BY tx.item_id "
  985. if order == OrderAsc {
  986. queryStr += " ASC "
  987. } else {
  988. queryStr += " DESC "
  989. }
  990. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  991. query = hdb.db.Rebind(queryStr)
  992. // log.Debug(query)
  993. txsPtrs := []*TxAPI{}
  994. if err := meddler.QueryAll(hdb.db, &txsPtrs, query, args...); err != nil {
  995. return nil, 0, tracerr.Wrap(err)
  996. }
  997. txs := db.SlicePtrsToSlice(txsPtrs).([]TxAPI)
  998. if len(txs) == 0 {
  999. return nil, 0, tracerr.Wrap(sql.ErrNoRows)
  1000. }
  1001. return txs, txs[0].TotalItems - uint64(len(txs)), nil
  1002. }
  1003. // GetAllExits returns all exit from the DB
  1004. func (hdb *HistoryDB) GetAllExits() ([]common.ExitInfo, error) {
  1005. var exits []*common.ExitInfo
  1006. err := meddler.QueryAll(
  1007. hdb.db, &exits,
  1008. `SELECT exit_tree.batch_num, exit_tree.account_idx, exit_tree.merkle_proof,
  1009. exit_tree.balance, exit_tree.instant_withdrawn, exit_tree.delayed_withdraw_request,
  1010. exit_tree.delayed_withdrawn FROM exit_tree;`,
  1011. )
  1012. return db.SlicePtrsToSlice(exits).([]common.ExitInfo), tracerr.Wrap(err)
  1013. }
  1014. // GetExitAPI returns a exit from the DB
  1015. func (hdb *HistoryDB) GetExitAPI(batchNum *uint, idx *common.Idx) (*ExitAPI, error) {
  1016. exit := &ExitAPI{}
  1017. err := meddler.QueryRow(
  1018. hdb.db, exit, `SELECT exit_tree.item_id, exit_tree.batch_num,
  1019. hez_idx(exit_tree.account_idx, token.symbol) AS account_idx,
  1020. exit_tree.merkle_proof, exit_tree.balance, exit_tree.instant_withdrawn,
  1021. exit_tree.delayed_withdraw_request, exit_tree.delayed_withdrawn,
  1022. token.token_id, token.item_id AS token_item_id,
  1023. token.eth_block_num AS token_block, token.eth_addr, token.name, token.symbol,
  1024. token.decimals, token.usd, token.usd_update
  1025. FROM exit_tree INNER JOIN account ON exit_tree.account_idx = account.idx
  1026. INNER JOIN token ON account.token_id = token.token_id
  1027. WHERE exit_tree.batch_num = $1 AND exit_tree.account_idx = $2;`, batchNum, idx,
  1028. )
  1029. return exit, tracerr.Wrap(err)
  1030. }
  1031. // GetExitsAPI returns a list of exits from the DB and pagination info
  1032. func (hdb *HistoryDB) GetExitsAPI(
  1033. ethAddr *ethCommon.Address, bjj *babyjub.PublicKeyComp, tokenID *common.TokenID,
  1034. idx *common.Idx, batchNum *uint, onlyPendingWithdraws *bool,
  1035. fromItem, limit *uint, order string,
  1036. ) ([]ExitAPI, uint64, error) {
  1037. if ethAddr != nil && bjj != nil {
  1038. return nil, 0, tracerr.Wrap(errors.New("ethAddr and bjj are incompatible"))
  1039. }
  1040. var query string
  1041. var args []interface{}
  1042. queryStr := `SELECT exit_tree.item_id, exit_tree.batch_num,
  1043. hez_idx(exit_tree.account_idx, token.symbol) AS account_idx,
  1044. exit_tree.merkle_proof, exit_tree.balance, exit_tree.instant_withdrawn,
  1045. exit_tree.delayed_withdraw_request, exit_tree.delayed_withdrawn,
  1046. token.token_id, token.item_id AS token_item_id,
  1047. token.eth_block_num AS token_block, token.eth_addr, token.name, token.symbol,
  1048. token.decimals, token.usd, token.usd_update, COUNT(*) OVER() AS total_items
  1049. FROM exit_tree INNER JOIN account ON exit_tree.account_idx = account.idx
  1050. INNER JOIN token ON account.token_id = token.token_id `
  1051. // Apply filters
  1052. nextIsAnd := false
  1053. // ethAddr filter
  1054. if ethAddr != nil {
  1055. queryStr += "WHERE account.eth_addr = ? "
  1056. nextIsAnd = true
  1057. args = append(args, ethAddr)
  1058. } else if bjj != nil { // bjj filter
  1059. queryStr += "WHERE account.bjj = ? "
  1060. nextIsAnd = true
  1061. args = append(args, bjj)
  1062. }
  1063. // tokenID filter
  1064. if tokenID != nil {
  1065. if nextIsAnd {
  1066. queryStr += "AND "
  1067. } else {
  1068. queryStr += "WHERE "
  1069. }
  1070. queryStr += "account.token_id = ? "
  1071. args = append(args, tokenID)
  1072. nextIsAnd = true
  1073. }
  1074. // idx filter
  1075. if idx != nil {
  1076. if nextIsAnd {
  1077. queryStr += "AND "
  1078. } else {
  1079. queryStr += "WHERE "
  1080. }
  1081. queryStr += "exit_tree.account_idx = ? "
  1082. args = append(args, idx)
  1083. nextIsAnd = true
  1084. }
  1085. // batchNum filter
  1086. if batchNum != nil {
  1087. if nextIsAnd {
  1088. queryStr += "AND "
  1089. } else {
  1090. queryStr += "WHERE "
  1091. }
  1092. queryStr += "exit_tree.batch_num = ? "
  1093. args = append(args, batchNum)
  1094. nextIsAnd = true
  1095. }
  1096. // onlyPendingWithdraws
  1097. if onlyPendingWithdraws != nil {
  1098. if *onlyPendingWithdraws {
  1099. if nextIsAnd {
  1100. queryStr += "AND "
  1101. } else {
  1102. queryStr += "WHERE "
  1103. }
  1104. queryStr += "(exit_tree.instant_withdrawn IS NULL AND exit_tree.delayed_withdrawn IS NULL) "
  1105. nextIsAnd = true
  1106. }
  1107. }
  1108. if fromItem != nil {
  1109. if nextIsAnd {
  1110. queryStr += "AND "
  1111. } else {
  1112. queryStr += "WHERE "
  1113. }
  1114. if order == OrderAsc {
  1115. queryStr += "exit_tree.item_id >= ? "
  1116. } else {
  1117. queryStr += "exit_tree.item_id <= ? "
  1118. }
  1119. args = append(args, fromItem)
  1120. // nextIsAnd = true
  1121. }
  1122. // pagination
  1123. queryStr += "ORDER BY exit_tree.item_id "
  1124. if order == OrderAsc {
  1125. queryStr += " ASC "
  1126. } else {
  1127. queryStr += " DESC "
  1128. }
  1129. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  1130. query = hdb.db.Rebind(queryStr)
  1131. // log.Debug(query)
  1132. exits := []*ExitAPI{}
  1133. if err := meddler.QueryAll(hdb.db, &exits, query, args...); err != nil {
  1134. return nil, 0, tracerr.Wrap(err)
  1135. }
  1136. if len(exits) == 0 {
  1137. return nil, 0, tracerr.Wrap(sql.ErrNoRows)
  1138. }
  1139. return db.SlicePtrsToSlice(exits).([]ExitAPI), exits[0].TotalItems - uint64(len(exits)), nil
  1140. }
  1141. // GetAllL1UserTxs returns all L1UserTxs from the DB
  1142. func (hdb *HistoryDB) GetAllL1UserTxs() ([]common.L1Tx, error) {
  1143. var txs []*common.L1Tx
  1144. err := meddler.QueryAll(
  1145. hdb.db, &txs, // Note that '\x' gets parsed as a big.Int with value = 0
  1146. `SELECT tx.id, tx.to_forge_l1_txs_num, tx.position, tx.user_origin,
  1147. tx.from_idx, tx.from_eth_addr, tx.from_bjj, tx.to_idx, tx.token_id,
  1148. tx.amount, (CASE WHEN tx.batch_num IS NULL THEN NULL WHEN tx.amount_success THEN tx.amount ELSE '\x' END) AS effective_amount,
  1149. tx.deposit_amount, (CASE WHEN tx.batch_num IS NULL THEN NULL WHEN tx.deposit_amount_success THEN tx.deposit_amount ELSE '\x' END) AS effective_deposit_amount,
  1150. tx.eth_block_num, tx.type, tx.batch_num
  1151. FROM tx WHERE is_l1 = TRUE AND user_origin = TRUE;`,
  1152. )
  1153. return db.SlicePtrsToSlice(txs).([]common.L1Tx), tracerr.Wrap(err)
  1154. }
  1155. // GetAllL1CoordinatorTxs returns all L1CoordinatorTxs from the DB
  1156. func (hdb *HistoryDB) GetAllL1CoordinatorTxs() ([]common.L1Tx, error) {
  1157. var txs []*common.L1Tx
  1158. // Since the query specifies that only coordinator txs are returned, it's safe to assume
  1159. // that returned txs will always have effective amounts
  1160. err := meddler.QueryAll(
  1161. hdb.db, &txs,
  1162. `SELECT tx.id, tx.to_forge_l1_txs_num, tx.position, tx.user_origin,
  1163. tx.from_idx, tx.from_eth_addr, tx.from_bjj, tx.to_idx, tx.token_id,
  1164. tx.amount, tx.amount AS effective_amount,
  1165. tx.deposit_amount, tx.deposit_amount AS effective_deposit_amount,
  1166. tx.eth_block_num, tx.type, tx.batch_num
  1167. FROM tx WHERE is_l1 = TRUE AND user_origin = FALSE;`,
  1168. )
  1169. return db.SlicePtrsToSlice(txs).([]common.L1Tx), tracerr.Wrap(err)
  1170. }
  1171. // GetAllL2Txs returns all L2Txs from the DB
  1172. func (hdb *HistoryDB) GetAllL2Txs() ([]common.L2Tx, error) {
  1173. var txs []*common.L2Tx
  1174. err := meddler.QueryAll(
  1175. hdb.db, &txs,
  1176. `SELECT tx.id, tx.batch_num, tx.position,
  1177. tx.from_idx, tx.to_idx, tx.amount, tx.fee, tx.nonce,
  1178. tx.type, tx.eth_block_num
  1179. FROM tx WHERE is_l1 = FALSE;`,
  1180. )
  1181. return db.SlicePtrsToSlice(txs).([]common.L2Tx), tracerr.Wrap(err)
  1182. }
  1183. // GetUnforgedL1UserTxs gets L1 User Txs to be forged in the L1Batch with toForgeL1TxsNum.
  1184. func (hdb *HistoryDB) GetUnforgedL1UserTxs(toForgeL1TxsNum int64) ([]common.L1Tx, error) {
  1185. var txs []*common.L1Tx
  1186. err := meddler.QueryAll(
  1187. hdb.db, &txs, // only L1 user txs can have batch_num set to null
  1188. `SELECT tx.id, tx.to_forge_l1_txs_num, tx.position, tx.user_origin,
  1189. tx.from_idx, tx.from_eth_addr, tx.from_bjj, tx.to_idx, tx.token_id,
  1190. tx.amount, NULL AS effective_amount,
  1191. tx.deposit_amount, NULL AS effective_deposit_amount,
  1192. tx.eth_block_num, tx.type, tx.batch_num
  1193. FROM tx WHERE batch_num IS NULL AND to_forge_l1_txs_num = $1
  1194. ORDER BY position;`,
  1195. toForgeL1TxsNum,
  1196. )
  1197. return db.SlicePtrsToSlice(txs).([]common.L1Tx), tracerr.Wrap(err)
  1198. }
  1199. // TODO: Think about chaning all the queries that return a last value, to queries that return the next valid value.
  1200. // GetLastTxsPosition for a given to_forge_l1_txs_num
  1201. func (hdb *HistoryDB) GetLastTxsPosition(toForgeL1TxsNum int64) (int, error) {
  1202. row := hdb.db.QueryRow(
  1203. "SELECT position FROM tx WHERE to_forge_l1_txs_num = $1 ORDER BY position DESC;",
  1204. toForgeL1TxsNum,
  1205. )
  1206. var lastL1TxsPosition int
  1207. return lastL1TxsPosition, tracerr.Wrap(row.Scan(&lastL1TxsPosition))
  1208. }
  1209. // GetSCVars returns the rollup, auction and wdelayer smart contracts variables at their last update.
  1210. func (hdb *HistoryDB) GetSCVars() (*common.RollupVariables, *common.AuctionVariables,
  1211. *common.WDelayerVariables, error) {
  1212. var rollup common.RollupVariables
  1213. var auction common.AuctionVariables
  1214. var wDelayer common.WDelayerVariables
  1215. if err := meddler.QueryRow(hdb.db, &rollup,
  1216. "SELECT * FROM rollup_vars ORDER BY eth_block_num DESC LIMIT 1;"); err != nil {
  1217. return nil, nil, nil, tracerr.Wrap(err)
  1218. }
  1219. if err := meddler.QueryRow(hdb.db, &auction,
  1220. "SELECT * FROM auction_vars ORDER BY eth_block_num DESC LIMIT 1;"); err != nil {
  1221. return nil, nil, nil, tracerr.Wrap(err)
  1222. }
  1223. if err := meddler.QueryRow(hdb.db, &wDelayer,
  1224. "SELECT * FROM wdelayer_vars ORDER BY eth_block_num DESC LIMIT 1;"); err != nil {
  1225. return nil, nil, nil, tracerr.Wrap(err)
  1226. }
  1227. return &rollup, &auction, &wDelayer, nil
  1228. }
  1229. func (hdb *HistoryDB) setRollupVars(d meddler.DB, rollup *common.RollupVariables) error {
  1230. return tracerr.Wrap(meddler.Insert(d, "rollup_vars", rollup))
  1231. }
  1232. func (hdb *HistoryDB) setAuctionVars(d meddler.DB, auction *common.AuctionVariables) error {
  1233. return tracerr.Wrap(meddler.Insert(d, "auction_vars", auction))
  1234. }
  1235. func (hdb *HistoryDB) setWDelayerVars(d meddler.DB, wDelayer *common.WDelayerVariables) error {
  1236. return tracerr.Wrap(meddler.Insert(d, "wdelayer_vars", wDelayer))
  1237. }
  1238. func (hdb *HistoryDB) addBucketUpdates(d meddler.DB, bucketUpdates []common.BucketUpdate) error {
  1239. if len(bucketUpdates) == 0 {
  1240. return nil
  1241. }
  1242. return tracerr.Wrap(db.BulkInsert(
  1243. d,
  1244. `INSERT INTO bucket_update (
  1245. eth_block_num,
  1246. num_bucket,
  1247. block_stamp,
  1248. withdrawals
  1249. ) VALUES %s;`,
  1250. bucketUpdates[:],
  1251. ))
  1252. }
  1253. // GetAllBucketUpdates retrieves all the bucket updates
  1254. func (hdb *HistoryDB) GetAllBucketUpdates() ([]common.BucketUpdate, error) {
  1255. var bucketUpdates []*common.BucketUpdate
  1256. err := meddler.QueryAll(
  1257. hdb.db, &bucketUpdates,
  1258. "SELECT * FROM bucket_update;",
  1259. )
  1260. return db.SlicePtrsToSlice(bucketUpdates).([]common.BucketUpdate), tracerr.Wrap(err)
  1261. }
  1262. func (hdb *HistoryDB) addTokenExchanges(d meddler.DB, tokenExchanges []common.TokenExchange) error {
  1263. if len(tokenExchanges) == 0 {
  1264. return nil
  1265. }
  1266. return tracerr.Wrap(db.BulkInsert(
  1267. d,
  1268. `INSERT INTO token_exchange (
  1269. eth_block_num,
  1270. eth_addr,
  1271. value_usd
  1272. ) VALUES %s;`,
  1273. tokenExchanges[:],
  1274. ))
  1275. }
  1276. // GetAllTokenExchanges retrieves all the token exchanges
  1277. func (hdb *HistoryDB) GetAllTokenExchanges() ([]common.TokenExchange, error) {
  1278. var tokenExchanges []*common.TokenExchange
  1279. err := meddler.QueryAll(
  1280. hdb.db, &tokenExchanges,
  1281. "SELECT * FROM token_exchange;",
  1282. )
  1283. return db.SlicePtrsToSlice(tokenExchanges).([]common.TokenExchange), tracerr.Wrap(err)
  1284. }
  1285. func (hdb *HistoryDB) addEscapeHatchWithdrawals(d meddler.DB,
  1286. escapeHatchWithdrawals []common.WDelayerEscapeHatchWithdrawal) error {
  1287. if len(escapeHatchWithdrawals) == 0 {
  1288. return nil
  1289. }
  1290. return tracerr.Wrap(db.BulkInsert(
  1291. d,
  1292. `INSERT INTO escape_hatch_withdrawal (
  1293. eth_block_num,
  1294. who_addr,
  1295. to_addr,
  1296. token_addr,
  1297. amount
  1298. ) VALUES %s;`,
  1299. escapeHatchWithdrawals[:],
  1300. ))
  1301. }
  1302. // GetAllEscapeHatchWithdrawals retrieves all the escape hatch withdrawals
  1303. func (hdb *HistoryDB) GetAllEscapeHatchWithdrawals() ([]common.WDelayerEscapeHatchWithdrawal, error) {
  1304. var escapeHatchWithdrawals []*common.WDelayerEscapeHatchWithdrawal
  1305. err := meddler.QueryAll(
  1306. hdb.db, &escapeHatchWithdrawals,
  1307. "SELECT * FROM escape_hatch_withdrawal;",
  1308. )
  1309. return db.SlicePtrsToSlice(escapeHatchWithdrawals).([]common.WDelayerEscapeHatchWithdrawal),
  1310. tracerr.Wrap(err)
  1311. }
  1312. // SetInitialSCVars sets the initial state of rollup, auction, wdelayer smart
  1313. // contract variables. This initial state is stored linked to block 0, which
  1314. // always exist in the DB and is used to store initialization data that always
  1315. // exist in the smart contracts.
  1316. func (hdb *HistoryDB) SetInitialSCVars(rollup *common.RollupVariables,
  1317. auction *common.AuctionVariables, wDelayer *common.WDelayerVariables) error {
  1318. txn, err := hdb.db.Beginx()
  1319. if err != nil {
  1320. return tracerr.Wrap(err)
  1321. }
  1322. defer func() {
  1323. if err != nil {
  1324. db.Rollback(txn)
  1325. }
  1326. }()
  1327. // Force EthBlockNum to be 0 because it's the block used to link data
  1328. // that belongs to the creation of the smart contracts
  1329. rollup.EthBlockNum = 0
  1330. auction.EthBlockNum = 0
  1331. wDelayer.EthBlockNum = 0
  1332. auction.DefaultSlotSetBidSlotNum = 0
  1333. if err := hdb.setRollupVars(txn, rollup); err != nil {
  1334. return tracerr.Wrap(err)
  1335. }
  1336. if err := hdb.setAuctionVars(txn, auction); err != nil {
  1337. return tracerr.Wrap(err)
  1338. }
  1339. if err := hdb.setWDelayerVars(txn, wDelayer); err != nil {
  1340. return tracerr.Wrap(err)
  1341. }
  1342. return tracerr.Wrap(txn.Commit())
  1343. }
  1344. // setL1UserTxEffectiveAmounts sets the EffectiveAmount and EffectiveDepositAmount
  1345. // of the given l1UserTxs (with an UPDATE)
  1346. func (hdb *HistoryDB) setL1UserTxEffectiveAmounts(d sqlx.Ext, txs []common.L1Tx) error {
  1347. if len(txs) == 0 {
  1348. return nil
  1349. }
  1350. // Effective amounts are stored as success flags in the DB, with true value by default
  1351. // to reduce the amount of updates. Therefore, only amounts that became uneffective should be
  1352. // updated to become false
  1353. type txUpdate struct {
  1354. ID common.TxID `db:"id"`
  1355. AmountSuccess bool `db:"amount_success"`
  1356. DepositAmountSuccess bool `db:"deposit_amount_success"`
  1357. }
  1358. txUpdates := []txUpdate{}
  1359. equal := func(a *big.Int, b *big.Int) bool {
  1360. return a.Cmp(b) == 0
  1361. }
  1362. for i := range txs {
  1363. amountSuccess := equal(txs[i].Amount, txs[i].EffectiveAmount)
  1364. depositAmountSuccess := equal(txs[i].DepositAmount, txs[i].EffectiveDepositAmount)
  1365. if !amountSuccess || !depositAmountSuccess {
  1366. txUpdates = append(txUpdates, txUpdate{
  1367. ID: txs[i].TxID,
  1368. AmountSuccess: amountSuccess,
  1369. DepositAmountSuccess: depositAmountSuccess,
  1370. })
  1371. }
  1372. }
  1373. const query string = `
  1374. UPDATE tx SET
  1375. amount_success = tx_update.amount_success,
  1376. deposit_amount_success = tx_update.deposit_amount_success
  1377. FROM (VALUES
  1378. (NULL::::BYTEA, NULL::::BOOL, NULL::::BOOL),
  1379. (:id, :amount_success, :deposit_amount_success)
  1380. ) as tx_update (id, amount_success, deposit_amount_success)
  1381. WHERE tx.id = tx_update.id;
  1382. `
  1383. if len(txUpdates) > 0 {
  1384. if _, err := sqlx.NamedExec(d, query, txUpdates); err != nil {
  1385. return tracerr.Wrap(err)
  1386. }
  1387. }
  1388. return nil
  1389. }
  1390. // AddBlockSCData stores all the information of a block retrieved by the
  1391. // Synchronizer. Blocks should be inserted in order, leaving no gaps because
  1392. // the pagination system of the API/DB depends on this. Within blocks, all
  1393. // items should also be in the correct order (Accounts, Tokens, Txs, etc.)
  1394. func (hdb *HistoryDB) AddBlockSCData(blockData *common.BlockData) (err error) {
  1395. txn, err := hdb.db.Beginx()
  1396. if err != nil {
  1397. return tracerr.Wrap(err)
  1398. }
  1399. defer func() {
  1400. if err != nil {
  1401. db.Rollback(txn)
  1402. }
  1403. }()
  1404. // Add block
  1405. if err := hdb.addBlock(txn, &blockData.Block); err != nil {
  1406. return tracerr.Wrap(err)
  1407. }
  1408. // Add Coordinators
  1409. if err := hdb.addCoordinators(txn, blockData.Auction.Coordinators); err != nil {
  1410. return tracerr.Wrap(err)
  1411. }
  1412. // Add Bids
  1413. if err := hdb.addBids(txn, blockData.Auction.Bids); err != nil {
  1414. return tracerr.Wrap(err)
  1415. }
  1416. // Add Tokens
  1417. if err := hdb.addTokens(txn, blockData.Rollup.AddedTokens); err != nil {
  1418. return tracerr.Wrap(err)
  1419. }
  1420. // Prepare user L1 txs to be added.
  1421. // They must be added before the batch that will forge them (which can be in the same block)
  1422. // and after the account that will be sent to (also can be in the same block).
  1423. // Note: insert order is not relevant since item_id will be updated by a DB trigger when
  1424. // the batch that forges those txs is inserted
  1425. userL1s := make(map[common.BatchNum][]common.L1Tx)
  1426. for i := range blockData.Rollup.L1UserTxs {
  1427. batchThatForgesIsInTheBlock := false
  1428. for _, batch := range blockData.Rollup.Batches {
  1429. if batch.Batch.ForgeL1TxsNum != nil &&
  1430. *batch.Batch.ForgeL1TxsNum == *blockData.Rollup.L1UserTxs[i].ToForgeL1TxsNum {
  1431. // Tx is forged in this block. It's guaranteed that:
  1432. // * the first batch of the block won't forge user L1 txs that have been added in this block
  1433. // * batch nums are sequential therefore it's safe to add the tx at batch.BatchNum -1
  1434. batchThatForgesIsInTheBlock = true
  1435. addAtBatchNum := batch.Batch.BatchNum - 1
  1436. userL1s[addAtBatchNum] = append(userL1s[addAtBatchNum], blockData.Rollup.L1UserTxs[i])
  1437. break
  1438. }
  1439. }
  1440. if !batchThatForgesIsInTheBlock {
  1441. // User artificial batchNum 0 to add txs that are not forge in this block
  1442. // after all the accounts of the block have been added
  1443. userL1s[0] = append(userL1s[0], blockData.Rollup.L1UserTxs[i])
  1444. }
  1445. }
  1446. // Add Batches
  1447. for i := range blockData.Rollup.Batches {
  1448. batch := &blockData.Rollup.Batches[i]
  1449. // Add Batch: this will trigger an update on the DB
  1450. // that will set the batch num of forged L1 txs in this batch
  1451. if err = hdb.addBatch(txn, &batch.Batch); err != nil {
  1452. return tracerr.Wrap(err)
  1453. }
  1454. // Set the EffectiveAmount and EffectiveDepositAmount of all the
  1455. // L1UserTxs that have been forged in this batch
  1456. if err = hdb.setL1UserTxEffectiveAmounts(txn, batch.L1UserTxs); err != nil {
  1457. return tracerr.Wrap(err)
  1458. }
  1459. // Add accounts
  1460. if err := hdb.addAccounts(txn, batch.CreatedAccounts); err != nil {
  1461. return tracerr.Wrap(err)
  1462. }
  1463. // Add forged l1 coordinator Txs
  1464. if err := hdb.addL1Txs(txn, batch.L1CoordinatorTxs); err != nil {
  1465. return tracerr.Wrap(err)
  1466. }
  1467. // Add l2 Txs
  1468. if err := hdb.addL2Txs(txn, batch.L2Txs); err != nil {
  1469. return tracerr.Wrap(err)
  1470. }
  1471. // Add user L1 txs that will be forged in next batch
  1472. if userlL1s, ok := userL1s[batch.Batch.BatchNum]; ok {
  1473. if err := hdb.addL1Txs(txn, userlL1s); err != nil {
  1474. return tracerr.Wrap(err)
  1475. }
  1476. }
  1477. // Add exit tree
  1478. if err := hdb.addExitTree(txn, batch.ExitTree); err != nil {
  1479. return tracerr.Wrap(err)
  1480. }
  1481. }
  1482. // Add user L1 txs that won't be forged in this block
  1483. if userL1sNotForgedInThisBlock, ok := userL1s[0]; ok {
  1484. if err := hdb.addL1Txs(txn, userL1sNotForgedInThisBlock); err != nil {
  1485. return tracerr.Wrap(err)
  1486. }
  1487. }
  1488. // Set SC Vars if there was an update
  1489. if blockData.Rollup.Vars != nil {
  1490. if err := hdb.setRollupVars(txn, blockData.Rollup.Vars); err != nil {
  1491. return tracerr.Wrap(err)
  1492. }
  1493. }
  1494. if blockData.Auction.Vars != nil {
  1495. if err := hdb.setAuctionVars(txn, blockData.Auction.Vars); err != nil {
  1496. return tracerr.Wrap(err)
  1497. }
  1498. }
  1499. if blockData.WDelayer.Vars != nil {
  1500. if err := hdb.setWDelayerVars(txn, blockData.WDelayer.Vars); err != nil {
  1501. return tracerr.Wrap(err)
  1502. }
  1503. }
  1504. // Update withdrawals in exit tree table
  1505. if err := hdb.updateExitTree(txn, blockData.Block.Num,
  1506. blockData.Rollup.Withdrawals, blockData.WDelayer.Withdrawals); err != nil {
  1507. return tracerr.Wrap(err)
  1508. }
  1509. // Add Escape Hatch Withdrawals
  1510. if err := hdb.addEscapeHatchWithdrawals(txn,
  1511. blockData.WDelayer.EscapeHatchWithdrawals); err != nil {
  1512. return tracerr.Wrap(err)
  1513. }
  1514. // Add Buckets withdrawals updates
  1515. if err := hdb.addBucketUpdates(txn, blockData.Rollup.UpdateBucketWithdraw); err != nil {
  1516. return tracerr.Wrap(err)
  1517. }
  1518. // Add Token exchange updates
  1519. if err := hdb.addTokenExchanges(txn, blockData.Rollup.TokenExchanges); err != nil {
  1520. return tracerr.Wrap(err)
  1521. }
  1522. return tracerr.Wrap(txn.Commit())
  1523. }
  1524. // GetCoordinatorAPI returns a coordinator by its bidderAddr
  1525. func (hdb *HistoryDB) GetCoordinatorAPI(bidderAddr ethCommon.Address) (*CoordinatorAPI, error) {
  1526. coordinator := &CoordinatorAPI{}
  1527. err := meddler.QueryRow(hdb.db, coordinator, "SELECT * FROM coordinator WHERE bidder_addr = $1;", bidderAddr)
  1528. return coordinator, tracerr.Wrap(err)
  1529. }
  1530. // GetCoordinatorsAPI returns a list of coordinators from the DB and pagination info
  1531. func (hdb *HistoryDB) GetCoordinatorsAPI(
  1532. bidderAddr, forgerAddr *ethCommon.Address,
  1533. fromItem, limit *uint, order string,
  1534. ) ([]CoordinatorAPI, uint64, error) {
  1535. var query string
  1536. var args []interface{}
  1537. queryStr := `SELECT coordinator.*,
  1538. COUNT(*) OVER() AS total_items
  1539. FROM coordinator `
  1540. // Apply filters
  1541. nextIsAnd := false
  1542. if bidderAddr != nil {
  1543. queryStr += "WHERE bidder_addr = ? "
  1544. nextIsAnd = true
  1545. args = append(args, bidderAddr)
  1546. }
  1547. if forgerAddr != nil {
  1548. if nextIsAnd {
  1549. queryStr += "AND "
  1550. } else {
  1551. queryStr += "WHERE "
  1552. }
  1553. queryStr += "forger_addr = ? "
  1554. nextIsAnd = true
  1555. args = append(args, forgerAddr)
  1556. }
  1557. if fromItem != nil {
  1558. if nextIsAnd {
  1559. queryStr += "AND "
  1560. } else {
  1561. queryStr += "WHERE "
  1562. }
  1563. if order == OrderAsc {
  1564. queryStr += "coordinator.item_id >= ? "
  1565. } else {
  1566. queryStr += "coordinator.item_id <= ? "
  1567. }
  1568. args = append(args, fromItem)
  1569. }
  1570. // pagination
  1571. queryStr += "ORDER BY coordinator.item_id "
  1572. if order == OrderAsc {
  1573. queryStr += " ASC "
  1574. } else {
  1575. queryStr += " DESC "
  1576. }
  1577. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  1578. query = hdb.db.Rebind(queryStr)
  1579. coordinators := []*CoordinatorAPI{}
  1580. if err := meddler.QueryAll(hdb.db, &coordinators, query, args...); err != nil {
  1581. return nil, 0, tracerr.Wrap(err)
  1582. }
  1583. if len(coordinators) == 0 {
  1584. return nil, 0, tracerr.Wrap(sql.ErrNoRows)
  1585. }
  1586. return db.SlicePtrsToSlice(coordinators).([]CoordinatorAPI),
  1587. coordinators[0].TotalItems - uint64(len(coordinators)), nil
  1588. }
  1589. // AddAuctionVars insert auction vars into the DB
  1590. func (hdb *HistoryDB) AddAuctionVars(auctionVars *common.AuctionVariables) error {
  1591. return tracerr.Wrap(meddler.Insert(hdb.db, "auction_vars", auctionVars))
  1592. }
  1593. // GetAuctionVars returns auction variables
  1594. func (hdb *HistoryDB) GetAuctionVars() (*common.AuctionVariables, error) {
  1595. auctionVars := &common.AuctionVariables{}
  1596. err := meddler.QueryRow(
  1597. hdb.db, auctionVars, `SELECT * FROM auction_vars;`,
  1598. )
  1599. return auctionVars, tracerr.Wrap(err)
  1600. }
  1601. // GetAccountAPI returns an account by its index
  1602. func (hdb *HistoryDB) GetAccountAPI(idx common.Idx) (*AccountAPI, error) {
  1603. account := &AccountAPI{}
  1604. err := meddler.QueryRow(hdb.db, account, `SELECT account.item_id, hez_idx(account.idx,
  1605. token.symbol) as idx, account.batch_num, account.bjj, account.eth_addr,
  1606. token.token_id, token.item_id AS token_item_id, token.eth_block_num AS token_block,
  1607. token.eth_addr as token_eth_addr, token.name, token.symbol, token.decimals, token.usd, token.usd_update
  1608. FROM account INNER JOIN token ON account.token_id = token.token_id WHERE idx = $1;`, idx)
  1609. if err != nil {
  1610. return nil, tracerr.Wrap(err)
  1611. }
  1612. return account, nil
  1613. }
  1614. // GetAccountsAPI returns a list of accounts from the DB and pagination info
  1615. func (hdb *HistoryDB) GetAccountsAPI(
  1616. tokenIDs []common.TokenID, ethAddr *ethCommon.Address,
  1617. bjj *babyjub.PublicKeyComp, fromItem, limit *uint, order string,
  1618. ) ([]AccountAPI, uint64, error) {
  1619. if ethAddr != nil && bjj != nil {
  1620. return nil, 0, tracerr.Wrap(errors.New("ethAddr and bjj are incompatible"))
  1621. }
  1622. var query string
  1623. var args []interface{}
  1624. queryStr := `SELECT account.item_id, hez_idx(account.idx, token.symbol) as idx, account.batch_num,
  1625. account.bjj, account.eth_addr, token.token_id, token.item_id AS token_item_id, token.eth_block_num AS token_block,
  1626. token.eth_addr as token_eth_addr, token.name, token.symbol, token.decimals, token.usd, token.usd_update,
  1627. COUNT(*) OVER() AS total_items
  1628. FROM account INNER JOIN token ON account.token_id = token.token_id `
  1629. // Apply filters
  1630. nextIsAnd := false
  1631. // ethAddr filter
  1632. if ethAddr != nil {
  1633. queryStr += "WHERE account.eth_addr = ? "
  1634. nextIsAnd = true
  1635. args = append(args, ethAddr)
  1636. } else if bjj != nil { // bjj filter
  1637. queryStr += "WHERE account.bjj = ? "
  1638. nextIsAnd = true
  1639. args = append(args, bjj)
  1640. }
  1641. // tokenID filter
  1642. if len(tokenIDs) > 0 {
  1643. if nextIsAnd {
  1644. queryStr += "AND "
  1645. } else {
  1646. queryStr += "WHERE "
  1647. }
  1648. queryStr += "account.token_id IN (?) "
  1649. args = append(args, tokenIDs)
  1650. nextIsAnd = true
  1651. }
  1652. if fromItem != nil {
  1653. if nextIsAnd {
  1654. queryStr += "AND "
  1655. } else {
  1656. queryStr += "WHERE "
  1657. }
  1658. if order == OrderAsc {
  1659. queryStr += "account.item_id >= ? "
  1660. } else {
  1661. queryStr += "account.item_id <= ? "
  1662. }
  1663. args = append(args, fromItem)
  1664. }
  1665. // pagination
  1666. queryStr += "ORDER BY account.item_id "
  1667. if order == OrderAsc {
  1668. queryStr += " ASC "
  1669. } else {
  1670. queryStr += " DESC "
  1671. }
  1672. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  1673. query, argsQ, err := sqlx.In(queryStr, args...)
  1674. if err != nil {
  1675. return nil, 0, tracerr.Wrap(err)
  1676. }
  1677. query = hdb.db.Rebind(query)
  1678. accounts := []*AccountAPI{}
  1679. if err := meddler.QueryAll(hdb.db, &accounts, query, argsQ...); err != nil {
  1680. return nil, 0, tracerr.Wrap(err)
  1681. }
  1682. if len(accounts) == 0 {
  1683. return nil, 0, tracerr.Wrap(sql.ErrNoRows)
  1684. }
  1685. return db.SlicePtrsToSlice(accounts).([]AccountAPI),
  1686. accounts[0].TotalItems - uint64(len(accounts)), nil
  1687. }
  1688. // GetMetrics returns metrics
  1689. func (hdb *HistoryDB) GetMetrics(lastBatchNum common.BatchNum) (*Metrics, error) {
  1690. metricsTotals := &MetricsTotals{}
  1691. metrics := &Metrics{}
  1692. err := meddler.QueryRow(
  1693. hdb.db, metricsTotals, `SELECT COUNT(tx.*) as total_txs,
  1694. COALESCE (MIN(tx.batch_num), 0) as batch_num
  1695. FROM tx INNER JOIN block ON tx.eth_block_num = block.eth_block_num
  1696. WHERE block.timestamp >= NOW() - INTERVAL '24 HOURS';`)
  1697. if err != nil {
  1698. return nil, tracerr.Wrap(err)
  1699. }
  1700. metrics.TransactionsPerSecond = float64(metricsTotals.TotalTransactions / (24 * 60 * 60))
  1701. if (lastBatchNum - metricsTotals.FirstBatchNum) > 0 {
  1702. metrics.TransactionsPerBatch = float64(int64(metricsTotals.TotalTransactions) /
  1703. int64(lastBatchNum-metricsTotals.FirstBatchNum))
  1704. } else {
  1705. metrics.TransactionsPerBatch = float64(0)
  1706. }
  1707. err = meddler.QueryRow(
  1708. hdb.db, metricsTotals, `SELECT COUNT(*) AS total_batches,
  1709. COALESCE (SUM(total_fees_usd), 0) AS total_fees FROM batch
  1710. WHERE batch_num > $1;`, metricsTotals.FirstBatchNum)
  1711. if err != nil {
  1712. return nil, tracerr.Wrap(err)
  1713. }
  1714. if metricsTotals.TotalBatches > 0 {
  1715. metrics.BatchFrequency = float64((24 * 60 * 60) / metricsTotals.TotalBatches)
  1716. } else {
  1717. metrics.BatchFrequency = 0
  1718. }
  1719. if metricsTotals.TotalTransactions > 0 {
  1720. metrics.AvgTransactionFee = metricsTotals.TotalFeesUSD / float64(metricsTotals.TotalTransactions)
  1721. } else {
  1722. metrics.AvgTransactionFee = 0
  1723. }
  1724. err = meddler.QueryRow(
  1725. hdb.db, metrics,
  1726. `SELECT COUNT(*) AS total_bjjs, COUNT(DISTINCT(bjj)) AS total_accounts FROM account;`)
  1727. if err != nil {
  1728. return nil, tracerr.Wrap(err)
  1729. }
  1730. return metrics, nil
  1731. }
  1732. // GetAvgTxFee returns average transaction fee of the last 1h
  1733. func (hdb *HistoryDB) GetAvgTxFee() (float64, error) {
  1734. metricsTotals := &MetricsTotals{}
  1735. err := meddler.QueryRow(
  1736. hdb.db, metricsTotals, `SELECT COUNT(tx.*) as total_txs,
  1737. COALESCE (MIN(tx.batch_num), 0) as batch_num
  1738. FROM tx INNER JOIN block ON tx.eth_block_num = block.eth_block_num
  1739. WHERE block.timestamp >= NOW() - INTERVAL '1 HOURS';`)
  1740. if err != nil {
  1741. return 0, tracerr.Wrap(err)
  1742. }
  1743. err = meddler.QueryRow(
  1744. hdb.db, metricsTotals, `SELECT COUNT(*) AS total_batches,
  1745. COALESCE (SUM(total_fees_usd), 0) AS total_fees FROM batch
  1746. WHERE batch_num > $1;`, metricsTotals.FirstBatchNum)
  1747. if err != nil {
  1748. return 0, tracerr.Wrap(err)
  1749. }
  1750. var avgTransactionFee float64
  1751. if metricsTotals.TotalTransactions > 0 {
  1752. avgTransactionFee = metricsTotals.TotalFeesUSD / float64(metricsTotals.TotalTransactions)
  1753. } else {
  1754. avgTransactionFee = 0
  1755. }
  1756. return avgTransactionFee, nil
  1757. }