You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

535 lines
16 KiB

Fix eth events query and sync inconsistent state - kvdb - Fix path in Last when doing `setNew` - Only close if db != nil, and after closing, always set db to nil - This will avoid a panic in the case where the db is closed but there's an error soon after, and a future call tries to close again. This is because pebble.Close() will panic if the db is already closed. - Avoid calling pebble methods when a the Storage interface already implements that method (like Close). - statedb - In test, avoid calling KVDB method if the same method is available for the StateDB (like MakeCheckpoint, CurrentBatch). - eth - In *EventByBlock methods, take blockHash as input argument and use it when querying the event logs. Previously the blockHash was only taken from the logs results *only if* there was any log. This caused the following issue: if there was no logs, it was not possible to know if the result was from the expected block or an uncle block! By querying logs by blockHash we make sure that even if there are no logs, they are from the right block. - Note that now the function can either be called with a blockNum or blockHash, but not both at the same time. - sync - If there's an error during call to Sync call resetState, which internally resets the stateDB to avoid stale checkpoints (and a corresponding invalid increase in the StateDB batchNum). - During a Sync, after very batch processed, make sure that the StateDB currentBatch corresponds to the batchNum in the smart contract log/event.
3 years ago
Update coordinator to work better under real net - cli / node - Update handler of SIGINT so that after 3 SIGINTs, the process terminates unconditionally - coordinator - Store stats without pointer - In all functions that send a variable via channel, check for context done to avoid deadlock (due to no process reading from the channel, which has no queue) when the node is stopped. - Abstract `canForge` so that it can be used outside of the `Coordinator` - In `canForge` check the blockNumber in current and next slot. - Update tests due to smart contract changes in slot handling, and minimum bid defaults - TxManager - Add consts, vars and stats to allow evaluating `canForge` - Add `canForge` method (not used yet) - Store batch and nonces status (last success and last pending) - Track nonces internally instead of relying on the ethereum node (this is required to work with ganache when there are pending txs) - Handle the (common) case of the receipt not being found after the tx is sent. - Don't start the main loop until we get an initial messae fo the stats and vars (so that in the loop the stats and vars are set to synchronizer values) - When a tx fails, check and discard all the failed transactions before sending the message to stop the pipeline. This will avoid sending consecutive messages of stop the pipeline when multiple txs are detected to be failed consecutively. Also, future txs of the same pipeline after a discarded txs are discarded, and their nonces reused. - Robust handling of nonces: - If geth returns nonce is too low, increase it - If geth returns nonce too hight, decrease it - If geth returns underpriced, increase gas price - If geth returns replace underpriced, increase gas price - Add support for resending transactions after a timeout - Store `BatchInfos` in a queue - Pipeline - When an error is found, stop forging batches and send a message to the coordinator to stop the pipeline with information of the failed batch number so that in a restart, non-failed batches are not repated. - When doing a reset of the stateDB, if possible reset from the local checkpoint instead of resetting from the synchronizer. This allows resetting from a batch that is valid but not yet sent / synced. - Every time a pipeline is started, assign it a number from a counter. This allows the TxManager to ignore batches from stopped pipelines, via a message sent by the coordinator. - Avoid forging when we haven't reached the rollup genesis block number. - Add config parameter `StartSlotBlocksDelay`: StartSlotBlocksDelay is the number of blocks of delay to wait before starting the pipeline when we reach a slot in which we can forge. - When detecting a reorg, only reset the pipeline if the batch from which the pipeline started changed and wasn't sent by us. - Add config parameter `ScheduleBatchBlocksAheadCheck`: ScheduleBatchBlocksAheadCheck is the number of blocks ahead in which the forger address is checked to be allowed to forge (apart from checking the next block), used to decide when to stop scheduling new batches (by stopping the pipeline). For example, if we are at block 10 and ScheduleBatchBlocksAheadCheck is 5, eventhough at block 11 we canForge, the pipeline will be stopped if we can't forge at block 15. This value should be the expected number of blocks it takes between scheduling a batch and having it mined. - Add config parameter `SendBatchBlocksMarginCheck`: SendBatchBlocksMarginCheck is the number of margin blocks ahead in which the coordinator is also checked to be allowed to forge, apart from the next block; used to decide when to stop sending batches to the smart contract. For example, if we are at block 10 and SendBatchBlocksMarginCheck is 5, eventhough at block 11 we canForge, the batch will be discarded if we can't forge at block 15. - Add config parameter `TxResendTimeout`: TxResendTimeout is the timeout after which a non-mined ethereum transaction will be resent (reusing the nonce) with a newly calculated gas price - Add config parameter `MaxGasPrice`: MaxGasPrice is the maximum gas price allowed for ethereum transactions - Add config parameter `NoReuseNonce`: NoReuseNonce disables reusing nonces of pending transactions for new replacement transactions. This is useful for testing with Ganache. - Extend BatchInfo with more useful information for debugging - eth / ethereum client - Add necessary methods to create the auth object for transactions manually so that we can set the nonce, gas price, gas limit, etc manually - Update `RollupForgeBatch` to take an auth object as input (so that the coordinator can set parameters manually) - synchronizer - In stats, add `NextSlot` - In stats, store full last batch instead of just last batch number - Instead of calculating a nextSlot from scratch every time, update the current struct (only updating the forger info if we are Synced) - Afer every processed batch, check that the calculated StateDB MTRoot matches the StateRoot found in the forgeBatch event.
3 years ago
Update coordinator to work better under real net - cli / node - Update handler of SIGINT so that after 3 SIGINTs, the process terminates unconditionally - coordinator - Store stats without pointer - In all functions that send a variable via channel, check for context done to avoid deadlock (due to no process reading from the channel, which has no queue) when the node is stopped. - Abstract `canForge` so that it can be used outside of the `Coordinator` - In `canForge` check the blockNumber in current and next slot. - Update tests due to smart contract changes in slot handling, and minimum bid defaults - TxManager - Add consts, vars and stats to allow evaluating `canForge` - Add `canForge` method (not used yet) - Store batch and nonces status (last success and last pending) - Track nonces internally instead of relying on the ethereum node (this is required to work with ganache when there are pending txs) - Handle the (common) case of the receipt not being found after the tx is sent. - Don't start the main loop until we get an initial messae fo the stats and vars (so that in the loop the stats and vars are set to synchronizer values) - When a tx fails, check and discard all the failed transactions before sending the message to stop the pipeline. This will avoid sending consecutive messages of stop the pipeline when multiple txs are detected to be failed consecutively. Also, future txs of the same pipeline after a discarded txs are discarded, and their nonces reused. - Robust handling of nonces: - If geth returns nonce is too low, increase it - If geth returns nonce too hight, decrease it - If geth returns underpriced, increase gas price - If geth returns replace underpriced, increase gas price - Add support for resending transactions after a timeout - Store `BatchInfos` in a queue - Pipeline - When an error is found, stop forging batches and send a message to the coordinator to stop the pipeline with information of the failed batch number so that in a restart, non-failed batches are not repated. - When doing a reset of the stateDB, if possible reset from the local checkpoint instead of resetting from the synchronizer. This allows resetting from a batch that is valid but not yet sent / synced. - Every time a pipeline is started, assign it a number from a counter. This allows the TxManager to ignore batches from stopped pipelines, via a message sent by the coordinator. - Avoid forging when we haven't reached the rollup genesis block number. - Add config parameter `StartSlotBlocksDelay`: StartSlotBlocksDelay is the number of blocks of delay to wait before starting the pipeline when we reach a slot in which we can forge. - When detecting a reorg, only reset the pipeline if the batch from which the pipeline started changed and wasn't sent by us. - Add config parameter `ScheduleBatchBlocksAheadCheck`: ScheduleBatchBlocksAheadCheck is the number of blocks ahead in which the forger address is checked to be allowed to forge (apart from checking the next block), used to decide when to stop scheduling new batches (by stopping the pipeline). For example, if we are at block 10 and ScheduleBatchBlocksAheadCheck is 5, eventhough at block 11 we canForge, the pipeline will be stopped if we can't forge at block 15. This value should be the expected number of blocks it takes between scheduling a batch and having it mined. - Add config parameter `SendBatchBlocksMarginCheck`: SendBatchBlocksMarginCheck is the number of margin blocks ahead in which the coordinator is also checked to be allowed to forge, apart from the next block; used to decide when to stop sending batches to the smart contract. For example, if we are at block 10 and SendBatchBlocksMarginCheck is 5, eventhough at block 11 we canForge, the batch will be discarded if we can't forge at block 15. - Add config parameter `TxResendTimeout`: TxResendTimeout is the timeout after which a non-mined ethereum transaction will be resent (reusing the nonce) with a newly calculated gas price - Add config parameter `MaxGasPrice`: MaxGasPrice is the maximum gas price allowed for ethereum transactions - Add config parameter `NoReuseNonce`: NoReuseNonce disables reusing nonces of pending transactions for new replacement transactions. This is useful for testing with Ganache. - Extend BatchInfo with more useful information for debugging - eth / ethereum client - Add necessary methods to create the auth object for transactions manually so that we can set the nonce, gas price, gas limit, etc manually - Update `RollupForgeBatch` to take an auth object as input (so that the coordinator can set parameters manually) - synchronizer - In stats, add `NextSlot` - In stats, store full last batch instead of just last batch number - Instead of calculating a nextSlot from scratch every time, update the current struct (only updating the forger info if we are Synced) - Afer every processed batch, check that the calculated StateDB MTRoot matches the StateRoot found in the forgeBatch event.
3 years ago
Fix eth events query and sync inconsistent state - kvdb - Fix path in Last when doing `setNew` - Only close if db != nil, and after closing, always set db to nil - This will avoid a panic in the case where the db is closed but there's an error soon after, and a future call tries to close again. This is because pebble.Close() will panic if the db is already closed. - Avoid calling pebble methods when a the Storage interface already implements that method (like Close). - statedb - In test, avoid calling KVDB method if the same method is available for the StateDB (like MakeCheckpoint, CurrentBatch). - eth - In *EventByBlock methods, take blockHash as input argument and use it when querying the event logs. Previously the blockHash was only taken from the logs results *only if* there was any log. This caused the following issue: if there was no logs, it was not possible to know if the result was from the expected block or an uncle block! By querying logs by blockHash we make sure that even if there are no logs, they are from the right block. - Note that now the function can either be called with a blockNum or blockHash, but not both at the same time. - sync - If there's an error during call to Sync call resetState, which internally resets the stateDB to avoid stale checkpoints (and a corresponding invalid increase in the StateDB batchNum). - During a Sync, after very batch processed, make sure that the StateDB currentBatch corresponds to the batchNum in the smart contract log/event.
3 years ago
Fix eth events query and sync inconsistent state - kvdb - Fix path in Last when doing `setNew` - Only close if db != nil, and after closing, always set db to nil - This will avoid a panic in the case where the db is closed but there's an error soon after, and a future call tries to close again. This is because pebble.Close() will panic if the db is already closed. - Avoid calling pebble methods when a the Storage interface already implements that method (like Close). - statedb - In test, avoid calling KVDB method if the same method is available for the StateDB (like MakeCheckpoint, CurrentBatch). - eth - In *EventByBlock methods, take blockHash as input argument and use it when querying the event logs. Previously the blockHash was only taken from the logs results *only if* there was any log. This caused the following issue: if there was no logs, it was not possible to know if the result was from the expected block or an uncle block! By querying logs by blockHash we make sure that even if there are no logs, they are from the right block. - Note that now the function can either be called with a blockNum or blockHash, but not both at the same time. - sync - If there's an error during call to Sync call resetState, which internally resets the stateDB to avoid stale checkpoints (and a corresponding invalid increase in the StateDB batchNum). - During a Sync, after very batch processed, make sure that the StateDB currentBatch corresponds to the batchNum in the smart contract log/event.
3 years ago
  1. package statedb
  2. import (
  3. "errors"
  4. "fmt"
  5. "math/big"
  6. "github.com/hermeznetwork/hermez-node/common"
  7. "github.com/hermeznetwork/hermez-node/db/kvdb"
  8. "github.com/hermeznetwork/hermez-node/log"
  9. "github.com/hermeznetwork/tracerr"
  10. "github.com/iden3/go-merkletree"
  11. "github.com/iden3/go-merkletree/db"
  12. "github.com/iden3/go-merkletree/db/pebble"
  13. )
  14. var (
  15. // ErrStateDBWithoutMT is used when a method that requires a MerkleTree
  16. // is called in a StateDB that does not have a MerkleTree defined
  17. ErrStateDBWithoutMT = errors.New(
  18. "Can not call method to use MerkleTree in a StateDB without MerkleTree")
  19. // ErrAccountAlreadyExists is used when CreateAccount is called and the
  20. // Account already exists
  21. ErrAccountAlreadyExists = errors.New("Can not CreateAccount because Account already exists")
  22. // ErrIdxNotFound is used when trying to get the Idx from EthAddr or
  23. // EthAddr&ToBJJ
  24. ErrIdxNotFound = errors.New("Idx can not be found")
  25. // ErrGetIdxNoCase is used when trying to get the Idx from EthAddr &
  26. // BJJ with not compatible combination
  27. ErrGetIdxNoCase = errors.New(
  28. "Can not get Idx due unexpected combination of ethereum Address & BabyJubJub PublicKey")
  29. // PrefixKeyIdx is the key prefix for idx in the db
  30. PrefixKeyIdx = []byte("i:")
  31. // PrefixKeyAccHash is the key prefix for account hash in the db
  32. PrefixKeyAccHash = []byte("h:")
  33. // PrefixKeyMT is the key prefix for merkle tree in the db
  34. PrefixKeyMT = []byte("m:")
  35. // PrefixKeyAddr is the key prefix for address in the db
  36. PrefixKeyAddr = []byte("a:")
  37. // PrefixKeyAddrBJJ is the key prefix for address-babyjubjub in the db
  38. PrefixKeyAddrBJJ = []byte("ab:")
  39. )
  40. const (
  41. // TypeSynchronizer defines a StateDB used by the Synchronizer, that
  42. // generates the ExitTree when processing the txs
  43. TypeSynchronizer = "synchronizer"
  44. // TypeTxSelector defines a StateDB used by the TxSelector, without
  45. // computing ExitTree neither the ZKInputs
  46. TypeTxSelector = "txselector"
  47. // TypeBatchBuilder defines a StateDB used by the BatchBuilder, that
  48. // generates the ExitTree and the ZKInput when processing the txs
  49. TypeBatchBuilder = "batchbuilder"
  50. // MaxNLevels is the maximum value of NLevels for the merkle tree,
  51. // which comes from the fact that AccountIdx has 48 bits.
  52. MaxNLevels = 48
  53. )
  54. // TypeStateDB determines the type of StateDB
  55. type TypeStateDB string
  56. // Config of the StateDB
  57. type Config struct {
  58. // Path where the checkpoints will be stored
  59. Path string
  60. // Keep is the number of old checkpoints to keep. If 0, all
  61. // checkpoints are kept.
  62. Keep int
  63. // NoLast skips having an opened DB with a checkpoint to the last
  64. // batchNum for thread-safe reads.
  65. NoLast bool
  66. // Type of StateDB (
  67. Type TypeStateDB
  68. // NLevels is the number of merkle tree levels in case the Type uses a
  69. // merkle tree. If the Type doesn't use a merkle tree, NLevels should
  70. // be 0.
  71. NLevels int
  72. // At every checkpoint, check that there are no gaps between the
  73. // checkpoints
  74. noGapsCheck bool
  75. }
  76. // StateDB represents the StateDB object
  77. type StateDB struct {
  78. cfg Config
  79. db *kvdb.KVDB
  80. MT *merkletree.MerkleTree
  81. }
  82. // Last offers a subset of view methods of the StateDB that can be
  83. // called via the LastRead method of StateDB in a thread-safe manner to obtain
  84. // a consistent view to the last batch of the StateDB.
  85. type Last struct {
  86. db db.Storage
  87. }
  88. // GetAccount returns the account for the given Idx
  89. func (s *Last) GetAccount(idx common.Idx) (*common.Account, error) {
  90. return GetAccountInTreeDB(s.db, idx)
  91. }
  92. // GetCurrentBatch returns the current BatchNum stored in Last.db
  93. func (s *Last) GetCurrentBatch() (common.BatchNum, error) {
  94. cbBytes, err := s.db.Get(kvdb.KeyCurrentBatch)
  95. if tracerr.Unwrap(err) == db.ErrNotFound {
  96. return 0, nil
  97. } else if err != nil {
  98. return 0, tracerr.Wrap(err)
  99. }
  100. return common.BatchNumFromBytes(cbBytes)
  101. }
  102. // DB returns the underlying storage of Last
  103. func (s *Last) DB() db.Storage {
  104. return s.db
  105. }
  106. // GetAccounts returns all the accounts in the db. Use for debugging pruposes
  107. // only.
  108. func (s *Last) GetAccounts() ([]common.Account, error) {
  109. return getAccounts(s.db)
  110. }
  111. // NewStateDB creates a new StateDB, allowing to use an in-memory or in-disk
  112. // storage. Checkpoints older than the value defined by `keep` will be
  113. // deleted.
  114. // func NewStateDB(pathDB string, keep int, typ TypeStateDB, nLevels int) (*StateDB, error) {
  115. func NewStateDB(cfg Config) (*StateDB, error) {
  116. var kv *kvdb.KVDB
  117. var err error
  118. kv, err = kvdb.NewKVDB(kvdb.Config{Path: cfg.Path, Keep: cfg.Keep,
  119. NoGapsCheck: cfg.noGapsCheck, NoLast: cfg.NoLast})
  120. if err != nil {
  121. return nil, tracerr.Wrap(err)
  122. }
  123. var mt *merkletree.MerkleTree = nil
  124. if cfg.Type == TypeSynchronizer || cfg.Type == TypeBatchBuilder {
  125. mt, err = merkletree.NewMerkleTree(kv.StorageWithPrefix(PrefixKeyMT), cfg.NLevels)
  126. if err != nil {
  127. return nil, tracerr.Wrap(err)
  128. }
  129. }
  130. if cfg.Type == TypeTxSelector && cfg.NLevels != 0 {
  131. return nil, tracerr.Wrap(
  132. fmt.Errorf("invalid StateDB parameters: StateDB type==TypeStateDB can not have nLevels!=0"))
  133. }
  134. return &StateDB{
  135. cfg: cfg,
  136. db: kv,
  137. MT: mt,
  138. }, nil
  139. }
  140. // Type returns the StateDB configured Type
  141. func (s *StateDB) Type() TypeStateDB {
  142. return s.cfg.Type
  143. }
  144. // LastRead is a thread-safe method to query the last checkpoint of the StateDB
  145. // via the Last type methods
  146. func (s *StateDB) LastRead(fn func(sdbLast *Last) error) error {
  147. return s.db.LastRead(
  148. func(db *pebble.Storage) error {
  149. return fn(&Last{
  150. db: db,
  151. })
  152. },
  153. )
  154. }
  155. // LastGetAccount is a thread-safe method to query an account in the last
  156. // checkpoint of the StateDB.
  157. func (s *StateDB) LastGetAccount(idx common.Idx) (*common.Account, error) {
  158. var account *common.Account
  159. if err := s.LastRead(func(sdb *Last) error {
  160. var err error
  161. account, err = sdb.GetAccount(idx)
  162. return err
  163. }); err != nil {
  164. return nil, tracerr.Wrap(err)
  165. }
  166. return account, nil
  167. }
  168. // LastGetCurrentBatch is a thread-safe method to get the current BatchNum in
  169. // the last checkpoint of the StateDB.
  170. func (s *StateDB) LastGetCurrentBatch() (common.BatchNum, error) {
  171. var batchNum common.BatchNum
  172. if err := s.LastRead(func(sdb *Last) error {
  173. var err error
  174. batchNum, err = sdb.GetCurrentBatch()
  175. return err
  176. }); err != nil {
  177. return 0, tracerr.Wrap(err)
  178. }
  179. return batchNum, nil
  180. }
  181. // LastMTGetRoot returns the root of the underlying Merkle Tree in the last
  182. // checkpoint of the StateDB.
  183. func (s *StateDB) LastMTGetRoot() (*big.Int, error) {
  184. var root *big.Int
  185. if err := s.LastRead(func(sdb *Last) error {
  186. mt, err := merkletree.NewMerkleTree(sdb.DB().WithPrefix(PrefixKeyMT), s.cfg.NLevels)
  187. if err != nil {
  188. return tracerr.Wrap(err)
  189. }
  190. root = mt.Root().BigInt()
  191. return nil
  192. }); err != nil {
  193. return nil, tracerr.Wrap(err)
  194. }
  195. return root, nil
  196. }
  197. // MakeCheckpoint does a checkpoint at the given batchNum in the defined path.
  198. // Internally this advances & stores the current BatchNum, and then stores a
  199. // Checkpoint of the current state of the StateDB.
  200. func (s *StateDB) MakeCheckpoint() error {
  201. log.Debugw("Making StateDB checkpoint", "batch", s.CurrentBatch()+1, "type", s.cfg.Type)
  202. return s.db.MakeCheckpoint()
  203. }
  204. // CurrentBatch returns the current in-memory CurrentBatch of the StateDB.db
  205. func (s *StateDB) CurrentBatch() common.BatchNum {
  206. return s.db.CurrentBatch
  207. }
  208. // CurrentIdx returns the current in-memory CurrentIdx of the StateDB.db
  209. func (s *StateDB) CurrentIdx() common.Idx {
  210. return s.db.CurrentIdx
  211. }
  212. // getCurrentBatch returns the current BatchNum stored in the StateDB.db
  213. func (s *StateDB) getCurrentBatch() (common.BatchNum, error) {
  214. return s.db.GetCurrentBatch()
  215. }
  216. // GetCurrentIdx returns the stored Idx from the localStateDB, which is the
  217. // last Idx used for an Account in the localStateDB.
  218. func (s *StateDB) GetCurrentIdx() (common.Idx, error) {
  219. return s.db.GetCurrentIdx()
  220. }
  221. // SetCurrentIdx stores Idx in the StateDB
  222. func (s *StateDB) SetCurrentIdx(idx common.Idx) error {
  223. return s.db.SetCurrentIdx(idx)
  224. }
  225. // Reset resets the StateDB to the checkpoint at the given batchNum. Reset
  226. // does not delete the checkpoints between old current and the new current,
  227. // those checkpoints will remain in the storage, and eventually will be
  228. // deleted when MakeCheckpoint overwrites them.
  229. func (s *StateDB) Reset(batchNum common.BatchNum) error {
  230. log.Debugw("Making StateDB Reset", "batch", batchNum, "type", s.cfg.Type)
  231. if err := s.db.Reset(batchNum); err != nil {
  232. return tracerr.Wrap(err)
  233. }
  234. if s.MT != nil {
  235. // open the MT for the current s.db
  236. mt, err := merkletree.NewMerkleTree(s.db.StorageWithPrefix(PrefixKeyMT), s.MT.MaxLevels())
  237. if err != nil {
  238. return tracerr.Wrap(err)
  239. }
  240. s.MT = mt
  241. }
  242. return nil
  243. }
  244. // GetAccount returns the account for the given Idx
  245. func (s *StateDB) GetAccount(idx common.Idx) (*common.Account, error) {
  246. return GetAccountInTreeDB(s.db.DB(), idx)
  247. }
  248. func accountsIter(db db.Storage, fn func(a *common.Account) (bool, error)) error {
  249. idxDB := db.WithPrefix(PrefixKeyIdx)
  250. if err := idxDB.Iterate(func(k []byte, v []byte) (bool, error) {
  251. idx, err := common.IdxFromBytes(k)
  252. if err != nil {
  253. return false, tracerr.Wrap(err)
  254. }
  255. acc, err := GetAccountInTreeDB(db, idx)
  256. if err != nil {
  257. return false, tracerr.Wrap(err)
  258. }
  259. ok, err := fn(acc)
  260. if err != nil {
  261. return false, tracerr.Wrap(err)
  262. }
  263. return ok, nil
  264. }); err != nil {
  265. return tracerr.Wrap(err)
  266. }
  267. return nil
  268. }
  269. func getAccounts(db db.Storage) ([]common.Account, error) {
  270. accs := []common.Account{}
  271. if err := accountsIter(
  272. db,
  273. func(a *common.Account) (bool, error) {
  274. accs = append(accs, *a)
  275. return true, nil
  276. },
  277. ); err != nil {
  278. return nil, tracerr.Wrap(err)
  279. }
  280. return accs, nil
  281. }
  282. // TestGetAccounts returns all the accounts in the db. Use only in tests.
  283. // Outside tests getting all the accounts is discouraged because it's an
  284. // expensive operation, but if you must do it, use `LastRead()` method to get a
  285. // thread-safe and consistent view of the stateDB.
  286. func (s *StateDB) TestGetAccounts() ([]common.Account, error) {
  287. return getAccounts(s.db.DB())
  288. }
  289. // GetAccountInTreeDB is abstracted from StateDB to be used from StateDB and
  290. // from ExitTree. GetAccount returns the account for the given Idx
  291. func GetAccountInTreeDB(sto db.Storage, idx common.Idx) (*common.Account, error) {
  292. idxBytes, err := idx.Bytes()
  293. if err != nil {
  294. return nil, tracerr.Wrap(err)
  295. }
  296. vBytes, err := sto.Get(append(PrefixKeyIdx, idxBytes[:]...))
  297. if err != nil {
  298. return nil, tracerr.Wrap(err)
  299. }
  300. accBytes, err := sto.Get(append(PrefixKeyAccHash, vBytes...))
  301. if err != nil {
  302. return nil, tracerr.Wrap(err)
  303. }
  304. var b [32 * common.NLeafElems]byte
  305. copy(b[:], accBytes)
  306. account, err := common.AccountFromBytes(b)
  307. if err != nil {
  308. return nil, tracerr.Wrap(err)
  309. }
  310. account.Idx = idx
  311. return account, nil
  312. }
  313. // CreateAccount creates a new Account in the StateDB for the given Idx. If
  314. // StateDB.MT==nil, MerkleTree is not affected, otherwise updates the
  315. // MerkleTree, returning a CircomProcessorProof.
  316. func (s *StateDB) CreateAccount(idx common.Idx, account *common.Account) (
  317. *merkletree.CircomProcessorProof, error) {
  318. cpp, err := CreateAccountInTreeDB(s.db.DB(), s.MT, idx, account)
  319. if err != nil {
  320. return cpp, tracerr.Wrap(err)
  321. }
  322. // store idx by EthAddr & BJJ
  323. err = s.setIdxByEthAddrBJJ(idx, account.EthAddr, account.BJJ, account.TokenID)
  324. return cpp, tracerr.Wrap(err)
  325. }
  326. // CreateAccountInTreeDB is abstracted from StateDB to be used from StateDB and
  327. // from ExitTree. Creates a new Account in the StateDB for the given Idx. If
  328. // StateDB.MT==nil, MerkleTree is not affected, otherwise updates the
  329. // MerkleTree, returning a CircomProcessorProof.
  330. func CreateAccountInTreeDB(sto db.Storage, mt *merkletree.MerkleTree, idx common.Idx,
  331. account *common.Account) (*merkletree.CircomProcessorProof, error) {
  332. // store at the DB the key: v, and value: leaf.Bytes()
  333. v, err := account.HashValue()
  334. if err != nil {
  335. return nil, tracerr.Wrap(err)
  336. }
  337. accountBytes, err := account.Bytes()
  338. if err != nil {
  339. return nil, tracerr.Wrap(err)
  340. }
  341. // store the Leaf value
  342. tx, err := sto.NewTx()
  343. if err != nil {
  344. return nil, tracerr.Wrap(err)
  345. }
  346. idxBytes, err := idx.Bytes()
  347. if err != nil {
  348. return nil, tracerr.Wrap(err)
  349. }
  350. _, err = tx.Get(append(PrefixKeyIdx, idxBytes[:]...))
  351. if tracerr.Unwrap(err) != db.ErrNotFound {
  352. return nil, tracerr.Wrap(ErrAccountAlreadyExists)
  353. }
  354. err = tx.Put(append(PrefixKeyAccHash, v.Bytes()...), accountBytes[:])
  355. if err != nil {
  356. return nil, tracerr.Wrap(err)
  357. }
  358. err = tx.Put(append(PrefixKeyIdx, idxBytes[:]...), v.Bytes())
  359. if err != nil {
  360. return nil, tracerr.Wrap(err)
  361. }
  362. if err := tx.Commit(); err != nil {
  363. return nil, tracerr.Wrap(err)
  364. }
  365. if mt != nil {
  366. return mt.AddAndGetCircomProof(idx.BigInt(), v)
  367. }
  368. return nil, nil
  369. }
  370. // UpdateAccount updates the Account in the StateDB for the given Idx. If
  371. // StateDB.mt==nil, MerkleTree is not affected, otherwise updates the
  372. // MerkleTree, returning a CircomProcessorProof.
  373. func (s *StateDB) UpdateAccount(idx common.Idx, account *common.Account) (
  374. *merkletree.CircomProcessorProof, error) {
  375. return UpdateAccountInTreeDB(s.db.DB(), s.MT, idx, account)
  376. }
  377. // UpdateAccountInTreeDB is abstracted from StateDB to be used from StateDB and
  378. // from ExitTree. Updates the Account in the StateDB for the given Idx. If
  379. // StateDB.mt==nil, MerkleTree is not affected, otherwise updates the
  380. // MerkleTree, returning a CircomProcessorProof.
  381. func UpdateAccountInTreeDB(sto db.Storage, mt *merkletree.MerkleTree, idx common.Idx,
  382. account *common.Account) (*merkletree.CircomProcessorProof, error) {
  383. // store at the DB the key: v, and value: account.Bytes()
  384. v, err := account.HashValue()
  385. if err != nil {
  386. return nil, tracerr.Wrap(err)
  387. }
  388. accountBytes, err := account.Bytes()
  389. if err != nil {
  390. return nil, tracerr.Wrap(err)
  391. }
  392. tx, err := sto.NewTx()
  393. if err != nil {
  394. return nil, tracerr.Wrap(err)
  395. }
  396. err = tx.Put(append(PrefixKeyAccHash, v.Bytes()...), accountBytes[:])
  397. if err != nil {
  398. return nil, tracerr.Wrap(err)
  399. }
  400. idxBytes, err := idx.Bytes()
  401. if err != nil {
  402. return nil, tracerr.Wrap(err)
  403. }
  404. err = tx.Put(append(PrefixKeyIdx, idxBytes[:]...), v.Bytes())
  405. if err != nil {
  406. return nil, tracerr.Wrap(err)
  407. }
  408. if err := tx.Commit(); err != nil {
  409. return nil, tracerr.Wrap(err)
  410. }
  411. if mt != nil {
  412. proof, err := mt.Update(idx.BigInt(), v)
  413. return proof, tracerr.Wrap(err)
  414. }
  415. return nil, nil
  416. }
  417. // MTGetProof returns the CircomVerifierProof for a given Idx
  418. func (s *StateDB) MTGetProof(idx common.Idx) (*merkletree.CircomVerifierProof, error) {
  419. if s.MT == nil {
  420. return nil, tracerr.Wrap(ErrStateDBWithoutMT)
  421. }
  422. p, err := s.MT.GenerateSCVerifierProof(idx.BigInt(), s.MT.Root())
  423. if err != nil {
  424. return nil, tracerr.Wrap(err)
  425. }
  426. return p, nil
  427. }
  428. // Close the StateDB
  429. func (s *StateDB) Close() {
  430. s.db.Close()
  431. }
  432. // LocalStateDB represents the local StateDB which allows to make copies from
  433. // the synchronizer StateDB, and is used by the tx-selector and the
  434. // batch-builder. LocalStateDB is an in-memory storage.
  435. type LocalStateDB struct {
  436. *StateDB
  437. synchronizerStateDB *StateDB
  438. }
  439. // NewLocalStateDB returns a new LocalStateDB connected to the given
  440. // synchronizerDB. Checkpoints older than the value defined by `keep` will be
  441. // deleted.
  442. func NewLocalStateDB(cfg Config, synchronizerDB *StateDB) (*LocalStateDB, error) {
  443. cfg.noGapsCheck = true
  444. cfg.NoLast = true
  445. s, err := NewStateDB(cfg)
  446. if err != nil {
  447. return nil, tracerr.Wrap(err)
  448. }
  449. return &LocalStateDB{
  450. s,
  451. synchronizerDB,
  452. }, nil
  453. }
  454. // CheckpointExists returns true if the checkpoint exists
  455. func (l *LocalStateDB) CheckpointExists(batchNum common.BatchNum) (bool, error) {
  456. return l.db.CheckpointExists(batchNum)
  457. }
  458. // Reset performs a reset in the LocalStateDB. If fromSynchronizer is true, it
  459. // gets the state from LocalStateDB.synchronizerStateDB for the given batchNum.
  460. // If fromSynchronizer is false, get the state from LocalStateDB checkpoints.
  461. func (l *LocalStateDB) Reset(batchNum common.BatchNum, fromSynchronizer bool) error {
  462. if fromSynchronizer {
  463. log.Debugw("Making StateDB ResetFromSynchronizer", "batch", batchNum, "type", l.cfg.Type)
  464. if err := l.db.ResetFromSynchronizer(batchNum, l.synchronizerStateDB.db); err != nil {
  465. return tracerr.Wrap(err)
  466. }
  467. // open the MT for the current s.db
  468. if l.MT != nil {
  469. mt, err := merkletree.NewMerkleTree(l.db.StorageWithPrefix(PrefixKeyMT),
  470. l.MT.MaxLevels())
  471. if err != nil {
  472. return tracerr.Wrap(err)
  473. }
  474. l.MT = mt
  475. }
  476. return nil
  477. }
  478. // use checkpoint from LocalStateDB
  479. return l.StateDB.Reset(batchNum)
  480. }