You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

1256 lines
36 KiB

Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
Update missing parts, improve til, and more - Node - Updated configuration to initialize the interface to all the smart contracts - Common - Moved BlockData and BatchData types to common so that they can be shared among: historydb, til and synchronizer - Remove hash.go (it was never used) - Remove slot.go (it was never used) - Remove smartcontractparams.go (it was never used, and appropriate structs are defined in `eth/`) - Comment state / status method until requirements of this method are properly defined, and move it to Synchronizer - Synchronizer - Simplify `Sync` routine to only sync one block per call, and return useful information. - Use BlockData and BatchData from common - Check that events belong to the expected block hash - In L1Batch, query L1UserTxs from HistoryDB - Fill ERC20 token information - Test AddTokens with test.Client - HistryDB - Use BlockData and BatchData from common - Add `GetAllTokens` method - Uncomment and update GetL1UserTxs (with corresponding tests) - Til - Rename all instances of RegisterToken to AddToken (to follow the smart contract implementation naming) - Use BlockData and BatchData from common - Move testL1CoordinatorTxs and testL2Txs to a separate struct from BatchData in Context - Start Context with BatchNum = 1 (which the protocol defines to be the first batchNum) - In every Batch, set StateRoot and ExitRoot to a non-nil big.Int (zero). - In all L1Txs, if LoadAmount is not used, set it to 0; if Amount is not used, set it to 0; so that no *big.Int is nil. - In L1UserTx, don't set BatchNum, because when L1UserTxs are created and obtained by the synchronizer, the BatchNum is not known yet (it's a synchronizer job to set it) - In L1UserTxs, set `UserOrigin` and set `ToForgeL1TxsNum`.
4 years ago
  1. package historydb
  2. import (
  3. "database/sql"
  4. "errors"
  5. "fmt"
  6. "math"
  7. "math/big"
  8. ethCommon "github.com/ethereum/go-ethereum/common"
  9. "github.com/hermeznetwork/hermez-node/common"
  10. "github.com/hermeznetwork/hermez-node/db"
  11. "github.com/hermeznetwork/hermez-node/log"
  12. "github.com/iden3/go-iden3-crypto/babyjub"
  13. "github.com/jmoiron/sqlx"
  14. //nolint:errcheck // driver for postgres DB
  15. _ "github.com/lib/pq"
  16. "github.com/russross/meddler"
  17. )
  18. const (
  19. // OrderAsc indicates ascending order when using pagination
  20. OrderAsc = "ASC"
  21. // OrderDesc indicates descending order when using pagination
  22. OrderDesc = "DESC"
  23. )
  24. // TODO(Edu): Document here how HistoryDB is kept consistent
  25. // HistoryDB persist the historic of the rollup
  26. type HistoryDB struct {
  27. db *sqlx.DB
  28. }
  29. // NewHistoryDB initialize the DB
  30. func NewHistoryDB(db *sqlx.DB) *HistoryDB {
  31. return &HistoryDB{db: db}
  32. }
  33. // DB returns a pointer to the L2DB.db. This method should be used only for
  34. // internal testing purposes.
  35. func (hdb *HistoryDB) DB() *sqlx.DB {
  36. return hdb.db
  37. }
  38. // AddBlock insert a block into the DB
  39. func (hdb *HistoryDB) AddBlock(block *common.Block) error { return hdb.addBlock(hdb.db, block) }
  40. func (hdb *HistoryDB) addBlock(d meddler.DB, block *common.Block) error {
  41. return meddler.Insert(d, "block", block)
  42. }
  43. // AddBlocks inserts blocks into the DB
  44. func (hdb *HistoryDB) AddBlocks(blocks []common.Block) error {
  45. return hdb.addBlocks(hdb.db, blocks)
  46. }
  47. func (hdb *HistoryDB) addBlocks(d meddler.DB, blocks []common.Block) error {
  48. return db.BulkInsert(
  49. d,
  50. `INSERT INTO block (
  51. eth_block_num,
  52. timestamp,
  53. hash
  54. ) VALUES %s;`,
  55. blocks[:],
  56. )
  57. }
  58. // GetBlock retrieve a block from the DB, given a block number
  59. func (hdb *HistoryDB) GetBlock(blockNum int64) (*common.Block, error) {
  60. block := &common.Block{}
  61. err := meddler.QueryRow(
  62. hdb.db, block,
  63. "SELECT * FROM block WHERE eth_block_num = $1;", blockNum,
  64. )
  65. return block, err
  66. }
  67. // GetAllBlocks retrieve all blocks from the DB
  68. func (hdb *HistoryDB) GetAllBlocks() ([]common.Block, error) {
  69. var blocks []*common.Block
  70. err := meddler.QueryAll(
  71. hdb.db, &blocks,
  72. "SELECT * FROM block;",
  73. )
  74. return db.SlicePtrsToSlice(blocks).([]common.Block), err
  75. }
  76. // GetBlocks retrieve blocks from the DB, given a range of block numbers defined by from and to
  77. func (hdb *HistoryDB) GetBlocks(from, to int64) ([]common.Block, error) {
  78. var blocks []*common.Block
  79. err := meddler.QueryAll(
  80. hdb.db, &blocks,
  81. "SELECT * FROM block WHERE $1 <= eth_block_num AND eth_block_num < $2;",
  82. from, to,
  83. )
  84. return db.SlicePtrsToSlice(blocks).([]common.Block), err
  85. }
  86. // GetLastBlock retrieve the block with the highest block number from the DB
  87. func (hdb *HistoryDB) GetLastBlock() (*common.Block, error) {
  88. block := &common.Block{}
  89. err := meddler.QueryRow(
  90. hdb.db, block, "SELECT * FROM block ORDER BY eth_block_num DESC LIMIT 1;",
  91. )
  92. return block, err
  93. }
  94. // AddBatch insert a Batch into the DB
  95. func (hdb *HistoryDB) AddBatch(batch *common.Batch) error { return hdb.addBatch(hdb.db, batch) }
  96. func (hdb *HistoryDB) addBatch(d meddler.DB, batch *common.Batch) error {
  97. // Calculate total collected fees in USD
  98. // Get IDs of collected tokens for fees
  99. tokenIDs := []common.TokenID{}
  100. for id := range batch.CollectedFees {
  101. tokenIDs = append(tokenIDs, id)
  102. }
  103. // Get USD value of the tokens
  104. type tokenPrice struct {
  105. ID common.TokenID `meddler:"token_id"`
  106. USD *float64 `meddler:"usd"`
  107. Decimals int `meddler:"decimals"`
  108. }
  109. var tokenPrices []*tokenPrice
  110. if len(tokenIDs) > 0 {
  111. query, args, err := sqlx.In(
  112. "SELECT token_id, usd, decimals FROM token WHERE token_id IN (?)",
  113. tokenIDs,
  114. )
  115. if err != nil {
  116. return err
  117. }
  118. query = hdb.db.Rebind(query)
  119. if err := meddler.QueryAll(
  120. hdb.db, &tokenPrices, query, args...,
  121. ); err != nil {
  122. return err
  123. }
  124. }
  125. // Calculate total collected
  126. var total float64
  127. for _, tokenPrice := range tokenPrices {
  128. if tokenPrice.USD == nil {
  129. continue
  130. }
  131. f := new(big.Float).SetInt(batch.CollectedFees[tokenPrice.ID])
  132. amount, _ := f.Float64()
  133. total += *tokenPrice.USD * (amount / math.Pow(10, float64(tokenPrice.Decimals))) //nolint decimals have to be ^10
  134. }
  135. batch.TotalFeesUSD = &total
  136. // Insert to DB
  137. return meddler.Insert(d, "batch", batch)
  138. }
  139. // AddBatches insert Bids into the DB
  140. func (hdb *HistoryDB) AddBatches(batches []common.Batch) error {
  141. return hdb.addBatches(hdb.db, batches)
  142. }
  143. func (hdb *HistoryDB) addBatches(d meddler.DB, batches []common.Batch) error {
  144. for i := 0; i < len(batches); i++ {
  145. if err := hdb.addBatch(d, &batches[i]); err != nil {
  146. return err
  147. }
  148. }
  149. return nil
  150. }
  151. // GetBatchAPI return the batch with the given batchNum
  152. func (hdb *HistoryDB) GetBatchAPI(batchNum common.BatchNum) (*BatchAPI, error) {
  153. batch := &BatchAPI{}
  154. return batch, meddler.QueryRow(
  155. hdb.db, batch,
  156. `SELECT batch.*, block.timestamp, block.hash
  157. FROM batch INNER JOIN block ON batch.eth_block_num = block.eth_block_num
  158. WHERE batch_num = $1;`, batchNum,
  159. )
  160. }
  161. // GetBatchesAPI return the batches applying the given filters
  162. func (hdb *HistoryDB) GetBatchesAPI(
  163. minBatchNum, maxBatchNum, slotNum *uint,
  164. forgerAddr *ethCommon.Address,
  165. fromItem, limit *uint, order string,
  166. ) ([]BatchAPI, *db.Pagination, error) {
  167. var query string
  168. var args []interface{}
  169. queryStr := `SELECT batch.*, block.timestamp, block.hash,
  170. count(*) OVER() AS total_items, MIN(batch.item_id) OVER() AS first_item,
  171. MAX(batch.item_id) OVER() AS last_item
  172. FROM batch INNER JOIN block ON batch.eth_block_num = block.eth_block_num `
  173. // Apply filters
  174. nextIsAnd := false
  175. // minBatchNum filter
  176. if minBatchNum != nil {
  177. if nextIsAnd {
  178. queryStr += "AND "
  179. } else {
  180. queryStr += "WHERE "
  181. }
  182. queryStr += "batch.batch_num > ? "
  183. args = append(args, minBatchNum)
  184. nextIsAnd = true
  185. }
  186. // maxBatchNum filter
  187. if maxBatchNum != nil {
  188. if nextIsAnd {
  189. queryStr += "AND "
  190. } else {
  191. queryStr += "WHERE "
  192. }
  193. queryStr += "batch.batch_num < ? "
  194. args = append(args, maxBatchNum)
  195. nextIsAnd = true
  196. }
  197. // slotNum filter
  198. if slotNum != nil {
  199. if nextIsAnd {
  200. queryStr += "AND "
  201. } else {
  202. queryStr += "WHERE "
  203. }
  204. queryStr += "batch.slot_num = ? "
  205. args = append(args, slotNum)
  206. nextIsAnd = true
  207. }
  208. // forgerAddr filter
  209. if forgerAddr != nil {
  210. if nextIsAnd {
  211. queryStr += "AND "
  212. } else {
  213. queryStr += "WHERE "
  214. }
  215. queryStr += "batch.forger_addr = ? "
  216. args = append(args, forgerAddr)
  217. nextIsAnd = true
  218. }
  219. // pagination
  220. if fromItem != nil {
  221. if nextIsAnd {
  222. queryStr += "AND "
  223. } else {
  224. queryStr += "WHERE "
  225. }
  226. if order == OrderAsc {
  227. queryStr += "batch.item_id >= ? "
  228. } else {
  229. queryStr += "batch.item_id <= ? "
  230. }
  231. args = append(args, fromItem)
  232. }
  233. queryStr += "ORDER BY batch.item_id "
  234. if order == OrderAsc {
  235. queryStr += " ASC "
  236. } else {
  237. queryStr += " DESC "
  238. }
  239. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  240. query = hdb.db.Rebind(queryStr)
  241. log.Debug(query)
  242. batchPtrs := []*BatchAPI{}
  243. if err := meddler.QueryAll(hdb.db, &batchPtrs, query, args...); err != nil {
  244. return nil, nil, err
  245. }
  246. batches := db.SlicePtrsToSlice(batchPtrs).([]BatchAPI)
  247. if len(batches) == 0 {
  248. return nil, nil, sql.ErrNoRows
  249. }
  250. return batches, &db.Pagination{
  251. TotalItems: batches[0].TotalItems,
  252. FirstItem: batches[0].FirstItem,
  253. LastItem: batches[0].LastItem,
  254. }, nil
  255. }
  256. // GetAllBatches retrieve all batches from the DB
  257. func (hdb *HistoryDB) GetAllBatches() ([]common.Batch, error) {
  258. var batches []*common.Batch
  259. err := meddler.QueryAll(
  260. hdb.db, &batches,
  261. `SELECT batch.batch_num, batch.eth_block_num, batch.forger_addr, batch.fees_collected,
  262. batch.state_root, batch.num_accounts, batch.last_idx, batch.exit_root,
  263. batch.forge_l1_txs_num, batch.slot_num, batch.total_fees_usd FROM batch;`,
  264. )
  265. return db.SlicePtrsToSlice(batches).([]common.Batch), err
  266. }
  267. // GetBatches retrieve batches from the DB, given a range of batch numbers defined by from and to
  268. func (hdb *HistoryDB) GetBatches(from, to common.BatchNum) ([]common.Batch, error) {
  269. var batches []*common.Batch
  270. err := meddler.QueryAll(
  271. hdb.db, &batches,
  272. "SELECT * FROM batch WHERE $1 <= batch_num AND batch_num < $2;",
  273. from, to,
  274. )
  275. return db.SlicePtrsToSlice(batches).([]common.Batch), err
  276. }
  277. // GetLastBatchNum returns the BatchNum of the latest forged batch
  278. func (hdb *HistoryDB) GetLastBatchNum() (common.BatchNum, error) {
  279. row := hdb.db.QueryRow("SELECT batch_num FROM batch ORDER BY batch_num DESC LIMIT 1;")
  280. var batchNum common.BatchNum
  281. return batchNum, row.Scan(&batchNum)
  282. }
  283. // GetLastL1TxsNum returns the greatest ForgeL1TxsNum in the DB from forged
  284. // batches. If there's no batch in the DB (nil, nil) is returned.
  285. func (hdb *HistoryDB) GetLastL1TxsNum() (*int64, error) {
  286. row := hdb.db.QueryRow("SELECT MAX(forge_l1_txs_num) FROM batch;")
  287. lastL1TxsNum := new(int64)
  288. return lastL1TxsNum, row.Scan(&lastL1TxsNum)
  289. }
  290. // Reorg deletes all the information that was added into the DB after the
  291. // lastValidBlock. If lastValidBlock is negative, all block information is
  292. // deleted.
  293. func (hdb *HistoryDB) Reorg(lastValidBlock int64) error {
  294. var err error
  295. if lastValidBlock < 0 {
  296. _, err = hdb.db.Exec("DELETE FROM block;")
  297. } else {
  298. _, err = hdb.db.Exec("DELETE FROM block WHERE eth_block_num > $1;", lastValidBlock)
  299. }
  300. return err
  301. }
  302. // SyncPoD stores all the data that can be changed / added on a block in the PoD SC
  303. func (hdb *HistoryDB) SyncPoD(
  304. blockNum uint64,
  305. bids []common.Bid,
  306. coordinators []common.Coordinator,
  307. vars *common.AuctionVariables,
  308. ) error {
  309. return nil
  310. }
  311. // AddBids insert Bids into the DB
  312. func (hdb *HistoryDB) AddBids(bids []common.Bid) error { return hdb.addBids(hdb.db, bids) }
  313. func (hdb *HistoryDB) addBids(d meddler.DB, bids []common.Bid) error {
  314. // TODO: check the coordinator info
  315. return db.BulkInsert(
  316. d,
  317. "INSERT INTO bid (slot_num, bid_value, eth_block_num, bidder_addr) VALUES %s;",
  318. bids[:],
  319. )
  320. }
  321. // GetAllBids retrieve all bids from the DB
  322. func (hdb *HistoryDB) GetAllBids() ([]common.Bid, error) {
  323. var bids []*common.Bid
  324. err := meddler.QueryAll(
  325. hdb.db, &bids,
  326. `SELECT bid.slot_num, bid.bid_value, bid.eth_block_num, bid.bidder_addr FROM bid;`,
  327. )
  328. return db.SlicePtrsToSlice(bids).([]common.Bid), err
  329. }
  330. // GetBidsAPI return the bids applying the given filters
  331. func (hdb *HistoryDB) GetBidsAPI(slotNum *uint, forgerAddr *ethCommon.Address, fromItem, limit *uint, order string) ([]BidAPI, *db.Pagination, error) {
  332. var query string
  333. var args []interface{}
  334. queryStr := `SELECT bid.*, block.timestamp, coordinator.forger_addr, coordinator.url,
  335. COUNT(*) OVER() AS total_items, MIN(bid.item_id) OVER() AS first_item,
  336. MAX(bid.item_id) OVER() AS last_item FROM bid
  337. INNER JOIN block ON bid.eth_block_num = block.eth_block_num
  338. INNER JOIN coordinator ON bid.bidder_addr = coordinator.bidder_addr `
  339. // Apply filters
  340. nextIsAnd := false
  341. // slotNum filter
  342. if slotNum != nil {
  343. if nextIsAnd {
  344. queryStr += "AND "
  345. } else {
  346. queryStr += "WHERE "
  347. }
  348. queryStr += "bid.slot_num = ? "
  349. args = append(args, slotNum)
  350. nextIsAnd = true
  351. }
  352. // slotNum filter
  353. if forgerAddr != nil {
  354. if nextIsAnd {
  355. queryStr += "AND "
  356. } else {
  357. queryStr += "WHERE "
  358. }
  359. queryStr += "bid.bidder_addr = ? "
  360. args = append(args, forgerAddr)
  361. nextIsAnd = true
  362. }
  363. if fromItem != nil {
  364. if nextIsAnd {
  365. queryStr += "AND "
  366. } else {
  367. queryStr += "WHERE "
  368. }
  369. if order == OrderAsc {
  370. queryStr += "bid.item_id >= ? "
  371. } else {
  372. queryStr += "bid.item_id <= ? "
  373. }
  374. args = append(args, fromItem)
  375. }
  376. // pagination
  377. queryStr += "ORDER BY bid.item_id "
  378. if order == OrderAsc {
  379. queryStr += "ASC "
  380. } else {
  381. queryStr += "DESC "
  382. }
  383. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  384. query, argsQ, err := sqlx.In(queryStr, args...)
  385. if err != nil {
  386. return nil, nil, err
  387. }
  388. query = hdb.db.Rebind(query)
  389. bids := []*BidAPI{}
  390. if err := meddler.QueryAll(hdb.db, &bids, query, argsQ...); err != nil {
  391. return nil, nil, err
  392. }
  393. if len(bids) == 0 {
  394. return nil, nil, sql.ErrNoRows
  395. }
  396. return db.SlicePtrsToSlice(bids).([]BidAPI), &db.Pagination{
  397. TotalItems: bids[0].TotalItems,
  398. FirstItem: bids[0].FirstItem,
  399. LastItem: bids[0].LastItem,
  400. }, nil
  401. }
  402. // AddCoordinators insert Coordinators into the DB
  403. func (hdb *HistoryDB) AddCoordinators(coordinators []common.Coordinator) error {
  404. return hdb.addCoordinators(hdb.db, coordinators)
  405. }
  406. func (hdb *HistoryDB) addCoordinators(d meddler.DB, coordinators []common.Coordinator) error {
  407. return db.BulkInsert(
  408. d,
  409. "INSERT INTO coordinator (bidder_addr, forger_addr, eth_block_num, url) VALUES %s;",
  410. coordinators[:],
  411. )
  412. }
  413. // AddExitTree insert Exit tree into the DB
  414. func (hdb *HistoryDB) AddExitTree(exitTree []common.ExitInfo) error {
  415. return hdb.addExitTree(hdb.db, exitTree)
  416. }
  417. func (hdb *HistoryDB) addExitTree(d meddler.DB, exitTree []common.ExitInfo) error {
  418. return db.BulkInsert(
  419. d,
  420. "INSERT INTO exit_tree (batch_num, account_idx, merkle_proof, balance, "+
  421. "instant_withdrawn, delayed_withdraw_request, delayed_withdrawn) VALUES %s;",
  422. exitTree[:],
  423. )
  424. }
  425. // AddToken insert a token into the DB
  426. func (hdb *HistoryDB) AddToken(token *common.Token) error {
  427. return meddler.Insert(hdb.db, "token", token)
  428. }
  429. // AddTokens insert tokens into the DB
  430. func (hdb *HistoryDB) AddTokens(tokens []common.Token) error { return hdb.addTokens(hdb.db, tokens) }
  431. func (hdb *HistoryDB) addTokens(d meddler.DB, tokens []common.Token) error {
  432. return db.BulkInsert(
  433. d,
  434. `INSERT INTO token (
  435. token_id,
  436. eth_block_num,
  437. eth_addr,
  438. name,
  439. symbol,
  440. decimals
  441. ) VALUES %s;`,
  442. tokens[:],
  443. )
  444. }
  445. // UpdateTokenValue updates the USD value of a token
  446. func (hdb *HistoryDB) UpdateTokenValue(tokenSymbol string, value float64) error {
  447. _, err := hdb.db.Exec(
  448. "UPDATE token SET usd = $1 WHERE symbol = $2;",
  449. value, tokenSymbol,
  450. )
  451. return err
  452. }
  453. // GetToken returns a token from the DB given a TokenID
  454. func (hdb *HistoryDB) GetToken(tokenID common.TokenID) (*TokenWithUSD, error) {
  455. token := &TokenWithUSD{}
  456. err := meddler.QueryRow(
  457. hdb.db, token, `SELECT * FROM token WHERE token_id = $1;`, tokenID,
  458. )
  459. return token, err
  460. }
  461. // GetAllTokens returns all tokens from the DB
  462. func (hdb *HistoryDB) GetAllTokens() ([]TokenWithUSD, error) {
  463. var tokens []*TokenWithUSD
  464. err := meddler.QueryAll(
  465. hdb.db, &tokens,
  466. "SELECT * FROM token ORDER BY token_id;",
  467. )
  468. return db.SlicePtrsToSlice(tokens).([]TokenWithUSD), err
  469. }
  470. // GetTokens returns a list of tokens from the DB
  471. func (hdb *HistoryDB) GetTokens(ids []common.TokenID, symbols []string, name string, fromItem, limit *uint, order string) ([]TokenWithUSD, *db.Pagination, error) {
  472. var query string
  473. var args []interface{}
  474. queryStr := `SELECT * , COUNT(*) OVER() AS total_items, MIN(token.item_id) OVER() AS first_item, MAX(token.item_id) OVER() AS last_item FROM token `
  475. // Apply filters
  476. nextIsAnd := false
  477. if len(ids) > 0 {
  478. queryStr += "WHERE token_id IN (?) "
  479. nextIsAnd = true
  480. args = append(args, ids)
  481. }
  482. if len(symbols) > 0 {
  483. if nextIsAnd {
  484. queryStr += "AND "
  485. } else {
  486. queryStr += "WHERE "
  487. }
  488. queryStr += "symbol IN (?) "
  489. args = append(args, symbols)
  490. nextIsAnd = true
  491. }
  492. if name != "" {
  493. if nextIsAnd {
  494. queryStr += "AND "
  495. } else {
  496. queryStr += "WHERE "
  497. }
  498. queryStr += "name ~ ? "
  499. args = append(args, name)
  500. nextIsAnd = true
  501. }
  502. if fromItem != nil {
  503. if nextIsAnd {
  504. queryStr += "AND "
  505. } else {
  506. queryStr += "WHERE "
  507. }
  508. if order == OrderAsc {
  509. queryStr += "item_id >= ? "
  510. } else {
  511. queryStr += "item_id <= ? "
  512. }
  513. args = append(args, fromItem)
  514. }
  515. // pagination
  516. queryStr += "ORDER BY item_id "
  517. if order == OrderAsc {
  518. queryStr += "ASC "
  519. } else {
  520. queryStr += "DESC "
  521. }
  522. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  523. query, argsQ, err := sqlx.In(queryStr, args...)
  524. if err != nil {
  525. return nil, nil, err
  526. }
  527. query = hdb.db.Rebind(query)
  528. tokens := []*TokenWithUSD{}
  529. if err := meddler.QueryAll(hdb.db, &tokens, query, argsQ...); err != nil {
  530. return nil, nil, err
  531. }
  532. if len(tokens) == 0 {
  533. return nil, nil, sql.ErrNoRows
  534. }
  535. return db.SlicePtrsToSlice(tokens).([]TokenWithUSD), &db.Pagination{
  536. TotalItems: tokens[0].TotalItems,
  537. FirstItem: tokens[0].FirstItem,
  538. LastItem: tokens[0].LastItem,
  539. }, nil
  540. }
  541. // GetTokenSymbols returns all the token symbols from the DB
  542. func (hdb *HistoryDB) GetTokenSymbols() ([]string, error) {
  543. var tokenSymbols []string
  544. rows, err := hdb.db.Query("SELECT symbol FROM token;")
  545. if err != nil {
  546. return nil, err
  547. }
  548. sym := new(string)
  549. for rows.Next() {
  550. err = rows.Scan(sym)
  551. if err != nil {
  552. return nil, err
  553. }
  554. tokenSymbols = append(tokenSymbols, *sym)
  555. }
  556. return tokenSymbols, nil
  557. }
  558. // AddAccounts insert accounts into the DB
  559. func (hdb *HistoryDB) AddAccounts(accounts []common.Account) error {
  560. return hdb.addAccounts(hdb.db, accounts)
  561. }
  562. func (hdb *HistoryDB) addAccounts(d meddler.DB, accounts []common.Account) error {
  563. return db.BulkInsert(
  564. d,
  565. `INSERT INTO account (
  566. idx,
  567. token_id,
  568. batch_num,
  569. bjj,
  570. eth_addr
  571. ) VALUES %s;`,
  572. accounts[:],
  573. )
  574. }
  575. // GetAccounts returns a list of accounts from the DB
  576. func (hdb *HistoryDB) GetAccounts() ([]common.Account, error) {
  577. var accs []*common.Account
  578. err := meddler.QueryAll(
  579. hdb.db, &accs,
  580. "SELECT * FROM account ORDER BY idx;",
  581. )
  582. return db.SlicePtrsToSlice(accs).([]common.Account), err
  583. }
  584. // AddL1Txs inserts L1 txs to the DB. USD and LoadAmountUSD will be set automatically before storing the tx.
  585. // If the tx is originated by a coordinator, BatchNum must be provided. If it's originated by a user,
  586. // BatchNum should be null, and the value will be setted by a trigger when a batch forges the tx.
  587. func (hdb *HistoryDB) AddL1Txs(l1txs []common.L1Tx) error { return hdb.addL1Txs(hdb.db, l1txs) }
  588. // addL1Txs inserts L1 txs to the DB. USD and LoadAmountUSD will be set automatically before storing the tx.
  589. // If the tx is originated by a coordinator, BatchNum must be provided. If it's originated by a user,
  590. // BatchNum should be null, and the value will be setted by a trigger when a batch forges the tx.
  591. func (hdb *HistoryDB) addL1Txs(d meddler.DB, l1txs []common.L1Tx) error {
  592. txs := []txWrite{}
  593. for i := 0; i < len(l1txs); i++ {
  594. af := new(big.Float).SetInt(l1txs[i].Amount)
  595. amountFloat, _ := af.Float64()
  596. laf := new(big.Float).SetInt(l1txs[i].LoadAmount)
  597. loadAmountFloat, _ := laf.Float64()
  598. txs = append(txs, txWrite{
  599. // Generic
  600. IsL1: true,
  601. TxID: l1txs[i].TxID,
  602. Type: l1txs[i].Type,
  603. Position: l1txs[i].Position,
  604. FromIdx: &l1txs[i].FromIdx,
  605. ToIdx: l1txs[i].ToIdx,
  606. Amount: l1txs[i].Amount,
  607. AmountFloat: amountFloat,
  608. TokenID: l1txs[i].TokenID,
  609. BatchNum: l1txs[i].BatchNum,
  610. EthBlockNum: l1txs[i].EthBlockNum,
  611. // L1
  612. ToForgeL1TxsNum: l1txs[i].ToForgeL1TxsNum,
  613. UserOrigin: &l1txs[i].UserOrigin,
  614. FromEthAddr: &l1txs[i].FromEthAddr,
  615. FromBJJ: l1txs[i].FromBJJ,
  616. LoadAmount: l1txs[i].LoadAmount,
  617. LoadAmountFloat: &loadAmountFloat,
  618. })
  619. }
  620. return hdb.addTxs(d, txs)
  621. }
  622. // AddL2Txs inserts L2 txs to the DB. TokenID, USD and FeeUSD will be set automatically before storing the tx.
  623. func (hdb *HistoryDB) AddL2Txs(l2txs []common.L2Tx) error { return hdb.addL2Txs(hdb.db, l2txs) }
  624. // addL2Txs inserts L2 txs to the DB. TokenID, USD and FeeUSD will be set automatically before storing the tx.
  625. func (hdb *HistoryDB) addL2Txs(d meddler.DB, l2txs []common.L2Tx) error {
  626. txs := []txWrite{}
  627. for i := 0; i < len(l2txs); i++ {
  628. f := new(big.Float).SetInt(l2txs[i].Amount)
  629. amountFloat, _ := f.Float64()
  630. txs = append(txs, txWrite{
  631. // Generic
  632. IsL1: false,
  633. TxID: l2txs[i].TxID,
  634. Type: l2txs[i].Type,
  635. Position: l2txs[i].Position,
  636. FromIdx: &l2txs[i].FromIdx,
  637. ToIdx: l2txs[i].ToIdx,
  638. Amount: l2txs[i].Amount,
  639. AmountFloat: amountFloat,
  640. BatchNum: &l2txs[i].BatchNum,
  641. EthBlockNum: l2txs[i].EthBlockNum,
  642. // L2
  643. Fee: &l2txs[i].Fee,
  644. Nonce: &l2txs[i].Nonce,
  645. })
  646. }
  647. return hdb.addTxs(d, txs)
  648. }
  649. func (hdb *HistoryDB) addTxs(d meddler.DB, txs []txWrite) error {
  650. return db.BulkInsert(
  651. d,
  652. `INSERT INTO tx (
  653. is_l1,
  654. id,
  655. type,
  656. position,
  657. from_idx,
  658. to_idx,
  659. amount,
  660. amount_f,
  661. token_id,
  662. batch_num,
  663. eth_block_num,
  664. to_forge_l1_txs_num,
  665. user_origin,
  666. from_eth_addr,
  667. from_bjj,
  668. load_amount,
  669. load_amount_f,
  670. fee,
  671. nonce
  672. ) VALUES %s;`,
  673. txs[:],
  674. )
  675. }
  676. // // GetTxs returns a list of txs from the DB
  677. // func (hdb *HistoryDB) GetTxs() ([]common.Tx, error) {
  678. // var txs []*common.Tx
  679. // err := meddler.QueryAll(
  680. // hdb.db, &txs,
  681. // `SELECT * FROM tx
  682. // ORDER BY (batch_num, position) ASC`,
  683. // )
  684. // return db.SlicePtrsToSlice(txs).([]common.Tx), err
  685. // }
  686. // GetHistoryTx returns a tx from the DB given a TxID
  687. func (hdb *HistoryDB) GetHistoryTx(txID common.TxID) (*TxAPI, error) {
  688. tx := &TxAPI{}
  689. err := meddler.QueryRow(
  690. hdb.db, tx, `SELECT tx.item_id, tx.is_l1, tx.id, tx.type, tx.position,
  691. hez_idx(tx.from_idx, token.symbol) AS from_idx, tx.from_eth_addr, tx.from_bjj,
  692. hez_idx(tx.to_idx, token.symbol) AS to_idx, tx.to_eth_addr, tx.to_bjj,
  693. tx.amount, tx.token_id, tx.amount_usd,
  694. tx.batch_num, tx.eth_block_num, tx.to_forge_l1_txs_num, tx.user_origin,
  695. tx.load_amount, tx.load_amount_usd, tx.fee, tx.fee_usd, tx.nonce,
  696. token.token_id, token.item_id AS token_item_id, token.eth_block_num AS token_block,
  697. token.eth_addr, token.name, token.symbol, token.decimals, token.usd,
  698. token.usd_update, block.timestamp
  699. FROM tx INNER JOIN token ON tx.token_id = token.token_id
  700. INNER JOIN block ON tx.eth_block_num = block.eth_block_num
  701. WHERE tx.id = $1;`, txID,
  702. )
  703. return tx, err
  704. }
  705. // GetHistoryTxs returns a list of txs from the DB using the HistoryTx struct
  706. // and pagination info
  707. func (hdb *HistoryDB) GetHistoryTxs(
  708. ethAddr *ethCommon.Address, bjj *babyjub.PublicKey,
  709. tokenID *common.TokenID, idx *common.Idx, batchNum *uint, txType *common.TxType,
  710. fromItem, limit *uint, order string,
  711. ) ([]TxAPI, *db.Pagination, error) {
  712. if ethAddr != nil && bjj != nil {
  713. return nil, nil, errors.New("ethAddr and bjj are incompatible")
  714. }
  715. var query string
  716. var args []interface{}
  717. queryStr := `SELECT tx.item_id, tx.is_l1, tx.id, tx.type, tx.position,
  718. hez_idx(tx.from_idx, token.symbol) AS from_idx, tx.from_eth_addr, tx.from_bjj,
  719. hez_idx(tx.to_idx, token.symbol) AS to_idx, tx.to_eth_addr, tx.to_bjj,
  720. tx.amount, tx.token_id, tx.amount_usd,
  721. tx.batch_num, tx.eth_block_num, tx.to_forge_l1_txs_num, tx.user_origin,
  722. tx.load_amount, tx.load_amount_usd, tx.fee, tx.fee_usd, tx.nonce,
  723. token.token_id, token.item_id AS token_item_id, token.eth_block_num AS token_block,
  724. token.eth_addr, token.name, token.symbol, token.decimals, token.usd,
  725. token.usd_update, block.timestamp, count(*) OVER() AS total_items,
  726. MIN(tx.item_id) OVER() AS first_item, MAX(tx.item_id) OVER() AS last_item
  727. FROM tx INNER JOIN token ON tx.token_id = token.token_id
  728. INNER JOIN block ON tx.eth_block_num = block.eth_block_num `
  729. // Apply filters
  730. nextIsAnd := false
  731. // ethAddr filter
  732. if ethAddr != nil {
  733. queryStr = `WITH acc AS
  734. (select idx from account where eth_addr = ?) ` + queryStr
  735. queryStr += ", acc WHERE (tx.from_idx IN(acc.idx) OR tx.to_idx IN(acc.idx)) "
  736. nextIsAnd = true
  737. args = append(args, ethAddr)
  738. } else if bjj != nil { // bjj filter
  739. queryStr = `WITH acc AS
  740. (select idx from account where bjj = ?) ` + queryStr
  741. queryStr += ", acc WHERE (tx.from_idx IN(acc.idx) OR tx.to_idx IN(acc.idx)) "
  742. nextIsAnd = true
  743. args = append(args, bjj)
  744. }
  745. // tokenID filter
  746. if tokenID != nil {
  747. if nextIsAnd {
  748. queryStr += "AND "
  749. } else {
  750. queryStr += "WHERE "
  751. }
  752. queryStr += "tx.token_id = ? "
  753. args = append(args, tokenID)
  754. nextIsAnd = true
  755. }
  756. // idx filter
  757. if idx != nil {
  758. if nextIsAnd {
  759. queryStr += "AND "
  760. } else {
  761. queryStr += "WHERE "
  762. }
  763. queryStr += "(tx.from_idx = ? OR tx.to_idx = ?) "
  764. args = append(args, idx, idx)
  765. nextIsAnd = true
  766. }
  767. // batchNum filter
  768. if batchNum != nil {
  769. if nextIsAnd {
  770. queryStr += "AND "
  771. } else {
  772. queryStr += "WHERE "
  773. }
  774. queryStr += "tx.batch_num = ? "
  775. args = append(args, batchNum)
  776. nextIsAnd = true
  777. }
  778. // txType filter
  779. if txType != nil {
  780. if nextIsAnd {
  781. queryStr += "AND "
  782. } else {
  783. queryStr += "WHERE "
  784. }
  785. queryStr += "tx.type = ? "
  786. args = append(args, txType)
  787. nextIsAnd = true
  788. }
  789. if fromItem != nil {
  790. if nextIsAnd {
  791. queryStr += "AND "
  792. } else {
  793. queryStr += "WHERE "
  794. }
  795. if order == OrderAsc {
  796. queryStr += "tx.item_id >= ? "
  797. } else {
  798. queryStr += "tx.item_id <= ? "
  799. }
  800. args = append(args, fromItem)
  801. nextIsAnd = true
  802. }
  803. if nextIsAnd {
  804. queryStr += "AND "
  805. } else {
  806. queryStr += "WHERE "
  807. }
  808. queryStr += "tx.batch_num IS NOT NULL "
  809. // pagination
  810. queryStr += "ORDER BY tx.item_id "
  811. if order == OrderAsc {
  812. queryStr += " ASC "
  813. } else {
  814. queryStr += " DESC "
  815. }
  816. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  817. query = hdb.db.Rebind(queryStr)
  818. log.Debug(query)
  819. txsPtrs := []*TxAPI{}
  820. if err := meddler.QueryAll(hdb.db, &txsPtrs, query, args...); err != nil {
  821. return nil, nil, err
  822. }
  823. txs := db.SlicePtrsToSlice(txsPtrs).([]TxAPI)
  824. if len(txs) == 0 {
  825. return nil, nil, sql.ErrNoRows
  826. }
  827. return txs, &db.Pagination{
  828. TotalItems: txs[0].TotalItems,
  829. FirstItem: txs[0].FirstItem,
  830. LastItem: txs[0].LastItem,
  831. }, nil
  832. }
  833. // GetAllExits returns all exit from the DB
  834. func (hdb *HistoryDB) GetAllExits() ([]common.ExitInfo, error) {
  835. var exits []*common.ExitInfo
  836. err := meddler.QueryAll(
  837. hdb.db, &exits,
  838. `SELECT exit_tree.batch_num, exit_tree.account_idx, exit_tree.merkle_proof,
  839. exit_tree.balance, exit_tree.instant_withdrawn, exit_tree.delayed_withdraw_request,
  840. exit_tree.delayed_withdrawn FROM exit_tree;`,
  841. )
  842. return db.SlicePtrsToSlice(exits).([]common.ExitInfo), err
  843. }
  844. // GetExitAPI returns a exit from the DB
  845. func (hdb *HistoryDB) GetExitAPI(batchNum *uint, idx *common.Idx) (*ExitAPI, error) {
  846. exit := &ExitAPI{}
  847. err := meddler.QueryRow(
  848. hdb.db, exit, `SELECT exit_tree.item_id, exit_tree.batch_num,
  849. hez_idx(exit_tree.account_idx, token.symbol) AS account_idx,
  850. exit_tree.merkle_proof, exit_tree.balance, exit_tree.instant_withdrawn,
  851. exit_tree.delayed_withdraw_request, exit_tree.delayed_withdrawn,
  852. token.token_id, token.item_id AS token_item_id,
  853. token.eth_block_num AS token_block, token.eth_addr, token.name, token.symbol,
  854. token.decimals, token.usd, token.usd_update
  855. FROM exit_tree INNER JOIN account ON exit_tree.account_idx = account.idx
  856. INNER JOIN token ON account.token_id = token.token_id
  857. WHERE exit_tree.batch_num = $1 AND exit_tree.account_idx = $2;`, batchNum, idx,
  858. )
  859. return exit, err
  860. }
  861. // GetExitsAPI returns a list of exits from the DB and pagination info
  862. func (hdb *HistoryDB) GetExitsAPI(
  863. ethAddr *ethCommon.Address, bjj *babyjub.PublicKey, tokenID *common.TokenID,
  864. idx *common.Idx, batchNum *uint, onlyPendingWithdraws *bool,
  865. fromItem, limit *uint, order string,
  866. ) ([]ExitAPI, *db.Pagination, error) {
  867. if ethAddr != nil && bjj != nil {
  868. return nil, nil, errors.New("ethAddr and bjj are incompatible")
  869. }
  870. var query string
  871. var args []interface{}
  872. queryStr := `SELECT exit_tree.item_id, exit_tree.batch_num,
  873. hez_idx(exit_tree.account_idx, token.symbol) AS account_idx,
  874. exit_tree.merkle_proof, exit_tree.balance, exit_tree.instant_withdrawn,
  875. exit_tree.delayed_withdraw_request, exit_tree.delayed_withdrawn,
  876. token.token_id, token.item_id AS token_item_id,
  877. token.eth_block_num AS token_block, token.eth_addr, token.name, token.symbol,
  878. token.decimals, token.usd, token.usd_update, COUNT(*) OVER() AS total_items,
  879. MIN(exit_tree.item_id) OVER() AS first_item, MAX(exit_tree.item_id) OVER() AS last_item
  880. FROM exit_tree INNER JOIN account ON exit_tree.account_idx = account.idx
  881. INNER JOIN token ON account.token_id = token.token_id `
  882. // Apply filters
  883. nextIsAnd := false
  884. // ethAddr filter
  885. if ethAddr != nil {
  886. queryStr += "WHERE account.eth_addr = ? "
  887. nextIsAnd = true
  888. args = append(args, ethAddr)
  889. } else if bjj != nil { // bjj filter
  890. queryStr += "WHERE account.bjj = ? "
  891. nextIsAnd = true
  892. args = append(args, bjj)
  893. }
  894. // tokenID filter
  895. if tokenID != nil {
  896. if nextIsAnd {
  897. queryStr += "AND "
  898. } else {
  899. queryStr += "WHERE "
  900. }
  901. queryStr += "account.token_id = ? "
  902. args = append(args, tokenID)
  903. nextIsAnd = true
  904. }
  905. // idx filter
  906. if idx != nil {
  907. if nextIsAnd {
  908. queryStr += "AND "
  909. } else {
  910. queryStr += "WHERE "
  911. }
  912. queryStr += "exit_tree.account_idx = ? "
  913. args = append(args, idx)
  914. nextIsAnd = true
  915. }
  916. // batchNum filter
  917. if batchNum != nil {
  918. if nextIsAnd {
  919. queryStr += "AND "
  920. } else {
  921. queryStr += "WHERE "
  922. }
  923. queryStr += "exit_tree.batch_num = ? "
  924. args = append(args, batchNum)
  925. nextIsAnd = true
  926. }
  927. // onlyPendingWithdraws
  928. if onlyPendingWithdraws != nil {
  929. if *onlyPendingWithdraws {
  930. if nextIsAnd {
  931. queryStr += "AND "
  932. } else {
  933. queryStr += "WHERE "
  934. }
  935. queryStr += "(exit_tree.instant_withdrawn IS NULL AND exit_tree.delayed_withdrawn IS NULL) "
  936. nextIsAnd = true
  937. }
  938. }
  939. if fromItem != nil {
  940. if nextIsAnd {
  941. queryStr += "AND "
  942. } else {
  943. queryStr += "WHERE "
  944. }
  945. if order == OrderAsc {
  946. queryStr += "exit_tree.item_id >= ? "
  947. } else {
  948. queryStr += "exit_tree.item_id <= ? "
  949. }
  950. args = append(args, fromItem)
  951. // nextIsAnd = true
  952. }
  953. // pagination
  954. queryStr += "ORDER BY exit_tree.item_id "
  955. if order == OrderAsc {
  956. queryStr += " ASC "
  957. } else {
  958. queryStr += " DESC "
  959. }
  960. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  961. query = hdb.db.Rebind(queryStr)
  962. // log.Debug(query)
  963. exits := []*ExitAPI{}
  964. if err := meddler.QueryAll(hdb.db, &exits, query, args...); err != nil {
  965. return nil, nil, err
  966. }
  967. if len(exits) == 0 {
  968. return nil, nil, sql.ErrNoRows
  969. }
  970. return db.SlicePtrsToSlice(exits).([]ExitAPI), &db.Pagination{
  971. TotalItems: exits[0].TotalItems,
  972. FirstItem: exits[0].FirstItem,
  973. LastItem: exits[0].LastItem,
  974. }, nil
  975. }
  976. // // GetTx returns a tx from the DB
  977. // func (hdb *HistoryDB) GetTx(txID common.TxID) (*common.Tx, error) {
  978. // tx := new(common.Tx)
  979. // return tx, meddler.QueryRow(
  980. // hdb.db, tx,
  981. // "SELECT * FROM tx WHERE id = $1;",
  982. // txID,
  983. // )
  984. // }
  985. // GetAllL1UserTxs returns all L1UserTxs from the DB
  986. func (hdb *HistoryDB) GetAllL1UserTxs() ([]common.L1Tx, error) {
  987. var txs []*common.L1Tx
  988. err := meddler.QueryAll(
  989. hdb.db, &txs,
  990. `SELECT tx.id, tx.to_forge_l1_txs_num, tx.position, tx.user_origin,
  991. tx.from_idx, tx.from_eth_addr, tx.from_bjj, tx.to_idx, tx.token_id, tx.amount,
  992. tx.load_amount, tx.eth_block_num, tx.type, tx.batch_num
  993. FROM tx WHERE is_l1 = TRUE AND user_origin = TRUE;`,
  994. )
  995. return db.SlicePtrsToSlice(txs).([]common.L1Tx), err
  996. }
  997. // GetAllL1CoordinatorTxs returns all L1CoordinatorTxs from the DB
  998. func (hdb *HistoryDB) GetAllL1CoordinatorTxs() ([]common.L1Tx, error) {
  999. var txs []*common.L1Tx
  1000. err := meddler.QueryAll(
  1001. hdb.db, &txs,
  1002. `SELECT tx.id, tx.to_forge_l1_txs_num, tx.position, tx.user_origin,
  1003. tx.from_idx, tx.from_eth_addr, tx.from_bjj, tx.to_idx, tx.token_id, tx.amount,
  1004. tx.load_amount, tx.eth_block_num, tx.type, tx.batch_num
  1005. FROM tx WHERE is_l1 = TRUE AND user_origin = FALSE;`,
  1006. )
  1007. return db.SlicePtrsToSlice(txs).([]common.L1Tx), err
  1008. }
  1009. // GetAllL2Txs returns all L2Txs from the DB
  1010. func (hdb *HistoryDB) GetAllL2Txs() ([]common.L2Tx, error) {
  1011. var txs []*common.L2Tx
  1012. err := meddler.QueryAll(
  1013. hdb.db, &txs,
  1014. `SELECT tx.id, tx.batch_num, tx.position,
  1015. tx.from_idx, tx.to_idx, tx.amount, tx.fee, tx.nonce,
  1016. tx.type, tx.eth_block_num
  1017. FROM tx WHERE is_l1 = FALSE;`,
  1018. )
  1019. return db.SlicePtrsToSlice(txs).([]common.L2Tx), err
  1020. }
  1021. // GetL1UserTxs gets L1 User Txs to be forged in the L1Batch with toForgeL1TxsNum.
  1022. func (hdb *HistoryDB) GetL1UserTxs(toForgeL1TxsNum int64) ([]common.L1Tx, error) {
  1023. var txs []*common.L1Tx
  1024. err := meddler.QueryAll(
  1025. hdb.db, &txs,
  1026. `SELECT tx.id, tx.to_forge_l1_txs_num, tx.position, tx.user_origin,
  1027. tx.from_idx, tx.from_eth_addr, tx.from_bjj, tx.to_idx, tx.token_id, tx.amount,
  1028. tx.load_amount, tx.eth_block_num, tx.type, tx.batch_num
  1029. FROM tx WHERE to_forge_l1_txs_num = $1 AND is_l1 = TRUE AND user_origin = TRUE;`,
  1030. toForgeL1TxsNum,
  1031. )
  1032. return db.SlicePtrsToSlice(txs).([]common.L1Tx), err
  1033. }
  1034. // TODO: Think about chaning all the queries that return a last value, to queries that return the next valid value.
  1035. // GetLastTxsPosition for a given to_forge_l1_txs_num
  1036. func (hdb *HistoryDB) GetLastTxsPosition(toForgeL1TxsNum int64) (int, error) {
  1037. row := hdb.db.QueryRow("SELECT MAX(position) FROM tx WHERE to_forge_l1_txs_num = $1;", toForgeL1TxsNum)
  1038. var lastL1TxsPosition int
  1039. return lastL1TxsPosition, row.Scan(&lastL1TxsPosition)
  1040. }
  1041. // AddBlockSCData stores all the information of a block retrieved by the
  1042. // Synchronizer. Blocks should be inserted in order, leaving no gaps because
  1043. // the pagination system of the API/DB depends on this. Within blocks, all
  1044. // items should also be in the correct order (Accounts, Tokens, Txs, etc.)
  1045. func (hdb *HistoryDB) AddBlockSCData(blockData *common.BlockData) (err error) {
  1046. txn, err := hdb.db.Begin()
  1047. if err != nil {
  1048. return err
  1049. }
  1050. defer func() {
  1051. if err != nil {
  1052. errRollback := txn.Rollback()
  1053. if errRollback != nil {
  1054. log.Errorw("Rollback", "err", errRollback)
  1055. }
  1056. }
  1057. }()
  1058. // Add block
  1059. err = hdb.addBlock(txn, &blockData.Block)
  1060. if err != nil {
  1061. return err
  1062. }
  1063. // Add Coordinators
  1064. if len(blockData.Auction.Coordinators) > 0 {
  1065. err = hdb.addCoordinators(txn, blockData.Auction.Coordinators)
  1066. if err != nil {
  1067. return err
  1068. }
  1069. }
  1070. // Add Bids
  1071. if len(blockData.Auction.Bids) > 0 {
  1072. err = hdb.addBids(txn, blockData.Auction.Bids)
  1073. if err != nil {
  1074. return err
  1075. }
  1076. }
  1077. // Add Tokens
  1078. if len(blockData.Rollup.AddedTokens) > 0 {
  1079. err = hdb.addTokens(txn, blockData.Rollup.AddedTokens)
  1080. if err != nil {
  1081. return err
  1082. }
  1083. }
  1084. // Add l1 Txs
  1085. if len(blockData.Rollup.L1UserTxs) > 0 {
  1086. err = hdb.addL1Txs(txn, blockData.Rollup.L1UserTxs)
  1087. if err != nil {
  1088. return err
  1089. }
  1090. }
  1091. // Add Batches
  1092. for i := range blockData.Rollup.Batches {
  1093. batch := &blockData.Rollup.Batches[i]
  1094. // Add Batch: this will trigger an update on the DB
  1095. // that will set the batch num of forged L1 txs in this batch
  1096. err = hdb.addBatch(txn, &batch.Batch)
  1097. if err != nil {
  1098. return err
  1099. }
  1100. // Add unforged l1 Txs
  1101. if batch.L1Batch {
  1102. if len(batch.L1CoordinatorTxs) > 0 {
  1103. err = hdb.addL1Txs(txn, batch.L1CoordinatorTxs)
  1104. if err != nil {
  1105. return err
  1106. }
  1107. }
  1108. }
  1109. // Add l2 Txs
  1110. if len(batch.L2Txs) > 0 {
  1111. err = hdb.addL2Txs(txn, batch.L2Txs)
  1112. if err != nil {
  1113. return err
  1114. }
  1115. }
  1116. // Add accounts
  1117. if len(batch.CreatedAccounts) > 0 {
  1118. err = hdb.addAccounts(txn, batch.CreatedAccounts)
  1119. if err != nil {
  1120. return err
  1121. }
  1122. }
  1123. // Add exit tree
  1124. if len(batch.ExitTree) > 0 {
  1125. err = hdb.addExitTree(txn, batch.ExitTree)
  1126. if err != nil {
  1127. return err
  1128. }
  1129. }
  1130. // TODO: INSERT CONTRACTS VARS
  1131. }
  1132. // TODO: Process withdrawals
  1133. return txn.Commit()
  1134. }
  1135. // GetCoordinatorAPI returns a coordinator by its bidderAddr
  1136. func (hdb *HistoryDB) GetCoordinatorAPI(bidderAddr ethCommon.Address) (*CoordinatorAPI, error) {
  1137. coordinator := &CoordinatorAPI{}
  1138. err := meddler.QueryRow(
  1139. hdb.db, coordinator, `SELECT * FROM coordinator WHERE bidder_addr = $1;`, bidderAddr,
  1140. )
  1141. return coordinator, err
  1142. }
  1143. // GetCoordinatorsAPI returns a list of coordinators from the DB and pagination info
  1144. func (hdb *HistoryDB) GetCoordinatorsAPI(fromItem, limit *uint, order string) ([]CoordinatorAPI, *db.Pagination, error) {
  1145. var query string
  1146. var args []interface{}
  1147. queryStr := `SELECT coordinator.*,
  1148. COUNT(*) OVER() AS total_items, MIN(coordinator.item_id) OVER() AS first_item, MAX(coordinator.item_id) OVER() AS last_item
  1149. FROM coordinator `
  1150. // Apply filters
  1151. if fromItem != nil {
  1152. queryStr += "WHERE "
  1153. if order == OrderAsc {
  1154. queryStr += "coordinator.item_id >= ? "
  1155. } else {
  1156. queryStr += "coordinator.item_id <= ? "
  1157. }
  1158. args = append(args, fromItem)
  1159. }
  1160. // pagination
  1161. queryStr += "ORDER BY coordinator.item_id "
  1162. if order == OrderAsc {
  1163. queryStr += " ASC "
  1164. } else {
  1165. queryStr += " DESC "
  1166. }
  1167. queryStr += fmt.Sprintf("LIMIT %d;", *limit)
  1168. query = hdb.db.Rebind(queryStr)
  1169. coordinators := []*CoordinatorAPI{}
  1170. if err := meddler.QueryAll(hdb.db, &coordinators, query, args...); err != nil {
  1171. return nil, nil, err
  1172. }
  1173. if len(coordinators) == 0 {
  1174. return nil, nil, sql.ErrNoRows
  1175. }
  1176. return db.SlicePtrsToSlice(coordinators).([]CoordinatorAPI), &db.Pagination{
  1177. TotalItems: coordinators[0].TotalItems,
  1178. FirstItem: coordinators[0].FirstItem,
  1179. LastItem: coordinators[0].LastItem,
  1180. }, nil
  1181. }