258 Commits

Author SHA1 Message Date
Bobbin Threadbare
24f72c986b chore: update changelog 2024-02-14 11:52:40 -08:00
Andrey Khmuro
cd4525c7ad refactor: update repo to be compatible with Winterfell 0.8 (#275) 2024-02-14 11:52:40 -08:00
Philippe Laferrière
552d90429b Remove TieredSmt (#277) 2024-02-14 11:52:40 -08:00
Philippe Laferrière
119c7e2b6d SmtProof: add accessors (#276)
* add accessors

* fmt

* comments
2024-02-14 11:52:40 -08:00
Philippe Laferrière
45e7e78118 Clone (#274) 2024-02-14 11:52:40 -08:00
Philippe Laferrière
a9475b2a2d reexport (#273) 2024-02-14 11:52:40 -08:00
Philippe Laferrière
e55b3ed2ce Introduce SmtProof (#270)
* add conversion for `SmtLeaf`

* introduce `SmtProof` scaffolding

* implement `verify_membership()`

* SmtLeaf: knows its index

* `SmtLeaf::index`

* `SmtLeaf::get_value()` returns an Option

* fix `verify_membership()`

* impl `SmtProof::get`

* impl `into_parts()`

* `SmtProof::compute_root`

* use `SmtProof` in `Smt::open`

* `SmtLeaf` constructors

* Vec

* impl `Error` for `SmtLeafError`

* fix std Error

* move Word/Digest conversions to LeafIndex

* `SmtProof::new()` returns an error

* `SparseMerkleTree::path_and_leaf_to_opening`

* `SmtLeaf`: serializable/deserializable

* `SmtProof`: serializable/deserializable

* add tests for SmtLeaf serialization

* move `SmtLeaf` to submodule

* use constructors internally

* fix docs

* Add `Vec`

* add `Vec` to tests

* no_std use statements

* fmt

* `Errors`: make heading

* use `SMT_DEPTH`

* SmtLeaf single case: check leaf index

* Multiple case: check consistency with leaf index

* use `pub(super)` instead of `pub(crate)`

* use `pub(super)`

* `SmtLeaf`: add `num_entries()` accessor

* Fix `SmtLeaf` serialization

* improve leaf serialization tests
2024-02-14 11:52:40 -08:00
Bobbin Threadbare
61a0764a61 fix: peak index calculation in MmrProof 2024-02-14 11:52:40 -08:00
Philippe Laferrière
3d71a9b59b Smt: remove inner nodes when removing value (#269) 2024-02-14 11:52:40 -08:00
Philippe Laferrière
da12fd258a Add missing methods to Smt (#268) 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
5fcf98669d feat: add PartialMmr::from_parts() constructor 2024-02-14 11:52:40 -08:00
Philippe Laferrière
1cdd3dbbfa Add methods to Smt necessary for VM tests (#264)
* Smt::inner_nodes

* Add conversion Smt -> MerkleStore

* add docstring to `Smt`

* add to docstring

* fmt

* add `leaves()` method to `Smt`

* add `kv_pairs` functions

* rewrite `into_elements()` in terms of `into_kv_pairs()`

* change docstring
2024-02-14 11:52:40 -08:00
Bobbin Threadbare
d59ffe274a feat: add Debug and Clone derives for Falcon signature 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
727ed8fb3e docs: minor padding comment update 2024-02-14 11:52:40 -08:00
Al-Kindi-0
0acceaa526 fix: always pad bytes with 10*0 (#267) 2024-02-14 11:52:40 -08:00
Michael Birch
3882e0f719 fix(dsa): fix deserialization logic (#266) 2024-02-14 11:52:40 -08:00
Augusto F. Hack
70e39e7b39 partialmmr: Method add with support for a single peak and tracking
fixes: #258
2024-02-14 11:52:40 -08:00
Philippe Laferrière
5596db7868 Implement Smt struct (replacement to TieredSmt) (#254) 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
a933ff2fa0 refactor: remove obsolete traits 2024-02-14 11:52:40 -08:00
Philippe Laferrière
8ea37904e3 Introduce SparseMerkleTree trait (#245) 2024-02-14 11:52:40 -08:00
Augusto F. Hack
1004246bfe ci: verify docs syntax 2024-02-14 11:52:40 -08:00
Augusto F. Hack
dae9de9068 docs: fix warnings 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
7e9d4a4316 feat: add to_hex() to RpoDigest and RpxDigest 2024-02-14 11:52:40 -08:00
Al-Kindi-0
c9ab3beccc New padding rule for RPX (#236)
* feat: new padding rule for RPX
* fix: documentation on security
2024-02-14 11:52:40 -08:00
cristiantroy
260592f8e7 Fix: typos (#249)
* tests: fix typos
* full.rs: fix typo
* CONTRIBUTING: fix typo
2024-02-14 11:52:40 -08:00
Bobbin Threadbare
6b5db8a6db fix: clippy 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
3ebee98b0f feat: add PartialMmr::is_tracked() 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
457c985a92 refactor: remove sve feature flag 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
f894ed9cde chore: update CI.yaml 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
ac7593a13c chore: update CI jobs 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
004a3bc7a8 docs: update changelog and readme 2024-02-14 11:52:40 -08:00
Grzegorz Swirski
479fe5e649 feat: use AVX2 instructions whenever available 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
a0f533241f fix: bugfix in PartialMmr apply delta 2024-02-14 11:52:40 -08:00
Al-Kindi-0
05309b19bb chore: export default Winterfell randomcoin 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
be1d631630 feat: add Clone derive to PartialMmr 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
4d0d8d3058 refactor: return MmrPeaks from PartialMmr::peaks() 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
59d93cb8ba fix: typos 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
9baddfd138 feat: implement inner_nodes() iterator for PartialMmr 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
8f92f44a55 feat: add serialization to RpoRandomCoin 2024-02-14 11:52:40 -08:00
Al-Kindi-0
36d3b8dc46 feat: move RpoRandomCoin and define Rng trait
nits: minor

chore: update log and readme
2024-02-14 11:52:40 -08:00
Augusto F. Hack
7e13346e04 serde: for MerklePath, ValuePath, and RootPath 2024-02-14 11:52:40 -08:00
Philippe Laferrière
9a18ed6749 Implement SimpleSmt::set_subtree (#232)
* recompute_nodes_from_indeX_to_root

* MerkleError variant

* set_subtree

* test_simplesmt_set_subtree

* test_simplesmt_set_subtree_entire_tree

* test

* set_subtree: return root
2024-02-14 11:52:40 -08:00
Augusto F. Hack
df2650eb1f bugfix: TSMT failed to verify empty word for depth 64.
When a prefix is pushed to the depth 64, the entry list includes only
the values different than ZERO. This is required, since each block
represents a 2^192 values.

The bug was in the proof membership code, that failed to handle the case
of a key that was not in the list, because the depth is 64 and the value
was not set.
2024-02-14 11:52:40 -08:00
Philippe Laferrière
18310a89f0 MmrPeaks::hash_peaks() returns Digest (#230) 2024-02-14 11:52:40 -08:00
Philippe Laferrière
d719cc2663 Remove ExactSizeIterator constraint from SimpleSmt::with_leaves() (#228)
* Change InvalidNumEntries error

* max computation

* remove length check

* remove ExactSizeIterator constraint

* fix InvalidNumEntries error condition

* 2_usize
2024-02-14 11:52:40 -08:00
Augusto F. Hack
fa475d1929 simplesmt: simplify duplicate check 2024-02-14 11:52:40 -08:00
Philippe Laferrière
25b8cb64ba Introduce SimpleSmt::with_contiguous_leaves() (#227)
* with_contiguous_leaves

* test
2024-02-14 11:52:40 -08:00
Augusto F. Hack
389fcb03c2 simplesmt: bugfix, index must be validated before modifying the tree 2024-02-14 11:52:40 -08:00
Austin Abell
b7cb346e22 feat: memoize Signature polynomial decoding 2024-02-14 11:52:40 -08:00
Philippe Laferriere
fd480f827a Consuming iterator for RpoDigest 2024-02-14 11:52:40 -08:00
Augusto F. Hack
9f95582654 mmr: add into_parts for the peaks 2024-02-14 11:52:40 -08:00
Augusto F. Hack
1f92d5417a simple_smt: reduce serialized size, use static hashes of the empty word 2024-02-14 11:52:40 -08:00
Augusto F. Hack
9b0ce0810b mmr: support accumulator of older forest versions 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
938250453a chore: update changelog 2024-02-14 11:52:40 -08:00
Al-Kindi-0
9ccac2baf0 chore: bump winterfell release to .7 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
525062d023 docs: update bench readme 2024-02-14 11:52:40 -08:00
Augusto F. Hack
3a5264c428 mmr: support proofs with older forest versions 2024-02-14 11:52:40 -08:00
Augusto F. Hack
a8acc0b39d mmr: support arbitrary from/to delta updates 2024-02-14 11:52:40 -08:00
Augusto F. Hack
5f2d170435 mmr: publicly export MmrDelta 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
9d52958f64 docs: update changelog 2024-02-14 11:52:40 -08:00
Al-Kindi-0
a2a26e2aba docs: added RPX benchmarks 2024-02-14 11:52:40 -08:00
Al-Kindi-0
3125144445 feat: RPX (xHash12) hash function implementation 2024-02-14 11:52:40 -08:00
Augusto F. Hack
f33a982f29 rpo: added conversions for digest 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
41f03fbe91 chore: update main readme 2024-02-14 11:52:40 -08:00
Augusto F. Hack
65495aeb18 config: add .editorconfig 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
0a2d440524 chore: update crate version to v0.8 2024-02-14 11:52:40 -08:00
Bobbin Threadbare
c86bdc6d51 Merge pull request #226 from shuoer86/main
chore: fix typos
2023-11-26 15:18:25 -08:00
shuoer86
650508cbc9 chore: fix typos 2023-11-26 21:19:03 +08:00
Augusto Hack
012ad5ae93 Merge pull request #195 from 0xPolygonMiden/hacka-partial-mmr2
mmr: added partial mmr
2023-10-19 20:30:24 +02:00
Augusto F. Hack
bde20f9752 mmr: added partial mmr 2023-10-19 20:15:49 +02:00
Bobbin Threadbare
7f3d4b8966 fix: RPO Falcon build on Windows 2023-10-10 15:16:51 -07:00
Bobbin Threadbare
1a00c7035f fix: glob dependency 2023-10-06 21:20:48 -07:00
Bobbin Threadbare
7ddcdc5e39 docs: add module descriptions 2023-10-06 21:13:42 -07:00
Bobbin Threadbare
bfd05e3d38 Merge pull request #164 from 0xPolygonMiden/next
Tracking PR for v0.7.0 release
2023-10-06 19:54:04 -07:00
Bobbin Threadbare
9235a78afd chore: add date for v0.7 release 2023-10-06 17:06:06 -07:00
Bobbin Threadbare
78aa714b89 Merge pull request #193 from 0xPolygonMiden/bobbin-v0.7-release-prep
v0.7 release prep
2023-10-06 06:57:45 -07:00
Bobbin Threadbare
aeadc96b05 docs: add signature section to main readme 2023-10-06 06:20:15 -07:00
Bobbin Threadbare
0fb1ef837d Merge pull request #192 from 0xPolygonMiden/bobbin-feature-cleanup
Feature clean up
2023-10-04 10:30:59 -07:00
Bobbin Threadbare
cf91c89845 refactor: clean up features 2023-10-03 23:26:45 -07:00
Bobbin Threadbare
025c25fdd9 Merge pull request #179 from 0xPolygonMiden/al-bindings_second_attempt
Falcon DSA
2023-10-03 12:19:08 -07:00
Al-Kindi-0
8078021aff feat: Falcon 512 signature 2023-10-03 20:45:18 +02:00
Bobbin Threadbare
b1dbcee21d Merge pull request #189 from 0xPolygonMiden/frisitano-vault-delta
modify MerkleStore::non_empty_leaves to support TSMT
2023-10-02 18:04:56 -07:00
Bobbin Threadbare
396418659d Merge pull request #190 from reilabs/armv8-a+sve-next
feat: implement RPO hash using SVE instructions
2023-09-25 13:22:01 -07:00
Grzegorz Swirski
01be4d6b9d refactor: move arch specific code to rpo folder, don't run SVE on CI 2023-09-24 22:29:25 +02:00
Grzegorz Swirski
701a187e7f feat: implement RPO hash using SVE instructionss 2023-09-20 12:11:53 +02:00
frisitano
1fa2895724 refactor: modify MerkleStore::non_empty_leaves to support TSMT 2023-09-19 16:19:17 +08:00
Andrey Khmuro
90dd3acb12 Merge pull request #187 from 0xPolygonMiden/andrew-add-empty-word
Create empty word constant
2023-08-31 21:40:36 +03:00
Andrey Khmuro
2f09410e87 refactor: replace with EMPTY_WORD, ZERO and ONE 2023-08-31 20:36:40 +02:00
Bobbin Threadbare
51d527b568 Merge pull request #185 from 0xPolygonMiden/bobbin-leaf-traversal
Add more leaf traversal methods for `MerkleStore`
2023-08-28 16:54:14 -07:00
Bobbin Threadbare
9f54c82d62 feat: implement additional leaf traversal methods on MerkleStore 2023-08-28 16:50:34 -07:00
Bobbin Threadbare
c7f1535974 Merge pull request #182 from 0xPolygonMiden/andrew-tsmt-benchmark
Benchmark of the Tiered SMT
2023-08-17 16:12:23 -07:00
Andrey Khmuro
c1d0612115 refactor: run all benchmarks at once, leave only size run option 2023-08-17 21:50:01 +02:00
Andrey Khmuro
2214ff2425 chore: TSMT benchmark 2023-08-17 20:09:02 +02:00
Augusto Hack
85034af1df Merge pull request #183 from 0xPolygonMiden/hacka-move-error-to-mod
error: moved to its own module
2023-08-15 16:44:11 +02:00
Augusto F. Hack
f7e6922bff error: moved to its own module 2023-08-15 16:36:46 +02:00
Bobbin Threadbare
7780a50dad fix: remove PQClean submodule 2023-08-12 21:31:31 -07:00
Bobbin Threadbare
6d0c7567f0 chore: minor code organization improvement 2023-08-12 09:59:02 -07:00
Bobbin Threadbare
854ade1bfc Merge pull request #181 from 0xPolygonMiden/tohrnii-blake3-ord
feat: derive ord and partialord for blake3digest
2023-08-11 13:42:09 -07:00
tohrnii
fb649df1e7 feat: derive ord and partialord for blake3digest 2023-08-11 20:09:34 +00:00
Augusto Hack
d9e85230a6 Merge pull request #180 from 0xPolygonMiden/hacka-conditional-support-for-serde
feature: add conditional support for serde
2023-08-11 14:03:43 +02:00
Augusto F. Hack
8cf5e9fd2c feature: add conditional support for serde 2023-08-11 13:59:53 +02:00
Bobbin Threadbare
03f89f0aff Merge pull request #177 from 0xPolygonMiden/bobbin-tsmt-delete-64
Bug fix in TSMT for depth 64 removal
2023-08-07 11:13:35 -07:00
Augusto Hack
2fa1b9768a Merge pull request #178 from 0xPolygonMiden/hacka-export-tsmt-error
tsmt: export smt error
2023-08-07 11:25:55 +02:00
Augusto F. Hack
f71d98970b tsmt: export smt error 2023-08-07 11:13:24 +02:00
Bobbin Threadbare
b3e7578ab2 fix: misspelled variant name in TieredSmtProofError 2023-08-04 22:46:23 -07:00
Bobbin Threadbare
5c6a20cb60 fix: bug in TSMT for depth 64 removal 2023-08-04 22:36:45 -07:00
Augusto Hack
bc364b72c0 Merge pull request #176 from 0xPolygonMiden/hacka-tsmt-error-codes
tsmt: return error code instead of panic
2023-08-03 19:11:35 +02:00
Augusto F. Hack
83b6946432 tsmt: return error code instead of panic 2023-08-03 18:57:19 +02:00
Augusto Hack
3dfcc0810f Merge pull request #175 from 0xPolygonMiden/hacka-tsmt-proof-basic-traits
TSMT proof basic traits
2023-08-03 15:54:29 +02:00
Augusto F. Hack
33ef78f8f5 tsmt: add basic traits and into/from parts methods 2023-08-03 15:49:28 +02:00
Augusto Hack
b6eb1f9134 Merge pull request #174 from 0xPolygonMiden/bobbin-tsmt-proof
Implement ability to generate TSMT proofs
2023-08-03 11:31:18 +02:00
Augusto Hack
92bb3ac462 Merge pull request #173 from 0xPolygonMiden/bobbin-tsmt-refactor
Implement value clearing in TSMT
2023-08-03 11:28:54 +02:00
Bobbin Threadbare
1ac30f8989 feat: implement ability to generate TSMT proofs 2023-08-03 01:34:09 -07:00
Bobbin Threadbare
6810b5e3ab fix: node type check in inner_nodes() iterator of TSMT 2023-08-02 20:51:43 -07:00
Bobbin Threadbare
a03f2b5d5e feat: implement iterator over key-value pairs for TSMT 2023-08-01 11:02:29 -07:00
Bobbin Threadbare
1bb75e85dd feat: implement value removal in TSMT 2023-07-31 21:23:18 -07:00
Bobbin Threadbare
1578a9ee1f refactor: simplify TSTM leaf node hashing 2023-07-27 21:15:45 -07:00
Bobbin Threadbare
e49bccd7b7 Merge pull request #165 from 0xPolygonMiden/andrew-replace-mps
Replace MerklePathSet with PartialMerkleTree
2023-07-27 16:26:34 -07:00
Andrey Khmuro
71b04d0734 refactor: replace MerklePathSet with PartialMerkleTree 2023-07-27 22:03:16 +03:00
Bobbin Threadbare
8c749e473a chore: update blake3 dependency to v1.4 2023-07-26 12:10:01 -07:00
frisitano
809b572a40 Merge pull request #171 from 0xPolygonMiden/frisitano-recording-map-finalizer
Introduce TryApplyDiff and refactor RecordingMap finalizer
2023-07-26 15:38:16 +02:00
frisitano
da2d08714d feat: introduce TryApplyDiff and refactor RecordingMap finalizer 2023-07-26 12:14:39 +02:00
frisitano
aaf1788228 Merge pull request #166 from 0xPolygonMiden/frisitano-tx-result
feat: introduce `Diff` traits and objects
2023-07-20 16:45:45 +02:00
frisitano
44e60e7228 feat: introduce diff traits and objects 2023-07-20 16:41:59 +02:00
Andrey Khmuro
08aec4443c Enhancement of the Partial Merkle Tree (#163)
feat: implement additional functionality for the PartialMerkleTree
2023-07-06 00:19:03 +03:00
Bobbin Threadbare
813fe24b88 chore: update crate version to v0.7.0 2023-06-25 02:14:34 -07:00
Bobbin Threadbare
18302d68e0 Merge pull request #154 from 0xPolygonMiden/next
Tracking PR for v0.6.0 release
2023-06-25 02:06:21 -07:00
Bobbin Threadbare
858f95d4a1 chore: update changelog 2023-06-25 01:54:34 -07:00
Bobbin Threadbare
b2d6866d41 refactor: rename Merkle store Node into StoreNode 2023-06-25 01:42:21 -07:00
Bobbin Threadbare
f52ac29a02 Merge pull request #162 from 0xPolygonMiden/frisitano-tx-executor
Introduce data access recording capabilities
2023-06-23 23:33:58 -07:00
Bobbin Threadbare
f08644e4df refactor: simplify recording MerkleStore structure 2023-06-23 23:19:12 -07:00
frisitano
679a30e02e feat: introduce recorder objects 2023-06-23 14:26:57 +01:00
Bobbin Threadbare
cede2e57da Merge pull request #161 from 0xPolygonMiden/bobbin-smt-empty-value
Add `EMPTY_VALUE` associated constant to SMTs
2023-06-14 09:48:14 -07:00
Bobbin Threadbare
4215e83ae5 feat: add EMPTY_VALUE const to SMTs 2023-06-13 22:53:14 -07:00
Bobbin Threadbare
fe5cac9edc fix: compilation errors 2023-06-13 22:43:08 -07:00
Bobbin Threadbare
53d52b8adc Merge pull request #156 from 0xPolygonMiden/andrew-partial-mt
Partial Merkle tree implementation
2023-06-13 22:10:26 -07:00
Bobbin Threadbare
1be64fc43d Merge pull request #157 from 0xPolygonMiden/tohrnii-digest
refactor: refactor crypto APIs to use RpoDigest instead of Word
2023-06-13 15:06:47 -07:00
Bobbin Threadbare
049ae32cbf chore: clean up test code 2023-06-13 14:40:31 -07:00
Andrey Khmuro
b9def61e28 refactor: improve tests, add error tests 2023-06-13 16:14:07 +03:00
tohrnii
0e0a3fda4f refactor: refactor to clean up and simplify things 2023-06-13 10:53:41 +01:00
tohrnii
fe9aa8c28c refactor: refactor crypto APIs to use RpoDigest instead of Word 2023-06-09 21:27:09 +01:00
Andrey Khmuro
766702e37a refactor: improve tests, small fixes 2023-06-09 13:53:50 +03:00
Andrey Khmuro
218a64b5c7 refactor: small fixes 2023-06-07 17:31:38 +03:00
Andrey Khmuro
2708a23649 refactor: optimize code, fix bugs 2023-06-06 01:36:53 +03:00
Andrey Khmuro
43f1a4cb64 refactor: MerkleStore clippy fix 2023-06-06 01:36:53 +03:00
Andrey Khmuro
55cc71dadf fix: fix add_path func leaf determination 2023-06-06 01:36:53 +03:00
Andrey Khmuro
ebf71c2dc7 refactor: optimize code, remove not momentarily necessary functions 2023-06-06 01:36:53 +03:00
Andrey Khmuro
b4324475b6 feat: change constructor from with_leaves to with_paths 2023-06-06 01:36:53 +03:00
Andrey Khmuro
23f448fb33 feat: partial Merkle tree 2023-06-06 01:36:53 +03:00
Bobbin Threadbare
59f7723221 chore: update crete version to v0.6.0 2023-05-26 14:49:58 -07:00
Bobbin Threadbare
2ed880d976 chore: add TieredSmt to readme 2023-05-26 14:41:22 -07:00
Bobbin Threadbare
daa27f49f2 Merge pull request #140 from 0xPolygonMiden/next
Tracking PR for v0.5 release
2023-05-26 14:36:20 -07:00
Bobbin Threadbare
dcda57f71a chore: update changelog 2023-05-26 14:32:17 -07:00
Bobbin Threadbare
d9e3211418 Merge pull request #153 from 0xPolygonMiden/bobbin-tsmt-iter
Tiered SMT iterators
2023-05-20 22:52:59 -07:00
Bobbin Threadbare
21e7a5c07d feat: implement iterators over contents of TieredSmt 2023-05-20 22:47:07 -07:00
Bobbin Threadbare
02673ff87e Merge pull request #152 from 0xPolygonMiden/bobbin-tsmt
Basic Tiered MST
2023-05-16 15:42:34 -07:00
Bobbin Threadbare
b768eade4d feat: added handling of bottom tier to TieredSmt 2023-05-16 15:38:05 -07:00
Bobbin Threadbare
51ce07cc34 feat: implement basic TieredSmt 2023-05-12 11:33:34 -07:00
Bobbin Threadbare
550738bd94 Merge pull request #151 from 0xPolygonMiden/bobbin-mstore-subset
MerkleStore subset and more
2023-05-11 00:41:01 -07:00
Bobbin Threadbare
629494b601 feat: add leaves() iterator to SimpleSmt 2023-05-11 00:37:16 -07:00
Bobbin Threadbare
13aeda5a27 feat: add subset() to MerkleStore 2023-05-09 18:38:21 -07:00
Bobbin Threadbare
e5aba870a2 Merge pull request #149 from 0xPolygonMiden/bobbin-simple-smt
SimpleSmt updates
2023-05-08 07:35:00 -07:00
Bobbin Threadbare
fcf03478ba refactor: update SimpleSmt interfaces 2023-05-08 00:12:24 -07:00
frisitano
0ddd0db89b Merge pull request #148 from 0xPolygonMiden/frisitano-mmr-accumulator
refactor: Mmr accumulator
2023-05-05 17:56:46 +08:00
frisitano
2100d6c861 refactor(mmr): expose method to join mmr peaks in a vector and pad 2023-05-05 12:03:32 +08:00
Bobbin Threadbare
52409ac039 Merge pull request #146 from 0xPolygonMiden/frisitano-merkle-store-inner-nodes
feat: add .inner_nodes() to [MerkleStore]
2023-05-04 13:49:36 -07:00
frisitano
4555fc918f feat: add .inner_nodes() to [MerkleStore] 2023-05-04 19:15:52 +07:00
Bobbin Threadbare
52db23cd42 chore: update crate version to v0.5.0 2023-04-21 15:48:18 -07:00
Bobbin Threadbare
09025b4014 Merge pull request #129 from 0xPolygonMiden/next
Tracking PR for v0.4 release
2023-04-21 15:38:33 -07:00
Bobbin Threadbare
e983e940b2 chore: update changelog 2023-04-21 14:42:08 -07:00
Bobbin Threadbare
ae4e27b6c7 Merge pull request #139 from 0xPolygonMiden/hacka-support-adding-existing-structures-to-store
store: support adding existing structures
2023-04-21 14:32:52 -07:00
Bobbin Threadbare
130ae3d12a feat: add inner node iterator to MerklePath 2023-04-21 14:27:58 -07:00
Bobbin Threadbare
22c9f382c4 fix: serialization test 2023-04-21 11:39:49 -07:00
Bobbin Threadbare
9be4253f19 feat: remove clone requirement for MerkleStore From constructors 2023-04-21 11:22:36 -07:00
Augusto F. Hack
59595a2e04 feat: added From convertions for the MerkleStore 2023-04-21 14:47:58 +02:00
Augusto F. Hack
eb316f51bc store: remove SimpleSMT/MerkleTree/Mmr add/with methods 2023-04-21 14:47:48 +02:00
Augusto F. Hack
8161477d6a store: support adding existing structures 2023-04-20 13:45:31 +02:00
Augusto Hack
158167356d Merge pull request #138 from 0xPolygonMiden/hacka-merge-support-for-leaves
feat: allow merging of leaves
2023-04-17 12:29:13 +02:00
Augusto F. Hack
3996374a8b feat: allow merging of leaves
Consider the case of a MMR with one entry, and a new entry is being
added. Both of these values are quite unique, they are at the same time
the root and only leaf of their corresponding tree. Currently this
representation is not supported by the [MerkleStore], so the leaves are
not in it. Once the two values are merged, they both become leaves of a
new tree under the new parent, and the existing validation didn't permit
that promotion from happening.

This lifts the validation, and changes the method to clarify that not
only `root` are being merged, by arbitrary nodes of a tree (leafs,
internal, or roots), with arbitrary mixing of each.
2023-04-17 12:21:51 +02:00
Augusto Hack
7fa03c7967 Merge pull request #137 from 0xPolygonMiden/frisitano-reexport-mmr-proof
re-export mmr proof
2023-04-14 14:34:58 +02:00
frisitano
79915cc346 feat: re-export MmrProof 2023-04-14 13:25:19 +01:00
Augusto Hack
45412b5cec Merge pull request #134 from 0xPolygonMiden/add-rustfmt-config
config: add rustfmt config
2023-04-11 17:58:06 +02:00
Augusto F. Hack
bbb1e641a3 config: add rustfmt config 2023-04-11 17:38:39 +02:00
Bobbin Threadbare
e02507d11e chore: update version to v0.4.0 2023-04-08 12:46:53 -07:00
Bobbin Threadbare
b5eb68e46c Merge pull request #120 from 0xPolygonMiden/next
Tracking PR for v0.3 release
2023-04-07 23:55:43 -07:00
Bobbin Threadbare
61db888b2c chore: update crate version to v0.3 2023-04-07 23:44:27 -07:00
Bobbin Threadbare
051167f2e5 Merge pull request #76 from 0xPolygonMiden/bobbin-blake3-opt
BLAKE3 hash_elements() optimization
2023-04-07 23:12:41 -07:00
Victor Lopes
498bc93c15 Merge pull request #125 from 0xPolygonMiden/vlopes11-store-get-leaf-depth
feat: add `MerkleStore::get_leaf_depth`
2023-04-06 23:13:54 +02:00
Victor Lopez
00ffc1568a feat: add MerkleStore::get_leaf_depth
This commit introduces `get_leaf_depth`, a tiered SMT helpers that will
retrieve the depth of a leaf for a given root, capped by `64`.

closes #119
2023-04-06 23:01:38 +02:00
Augusto Hack
cbf51dd3e2 Merge pull request #127 from 0xPolygonMiden/hacka-optimized-peak-hash
mmr: optimized peak hash for Miden VM
2023-04-06 19:38:48 +02:00
Augusto F. Hack
ab903a2229 mmr: optimized peak hash for Miden VM 2023-04-06 18:22:01 +02:00
Bobbin Threadbare
86dba195b4 Merge pull request #124 from 0xPolygonMiden/bobbin-merkle-fixes
Merkle fixes
2023-04-05 12:20:41 -07:00
Bobbin Threadbare
bd557bc68c fix: add validation to NodeIndex constructor and remove BitIterator 2023-04-05 12:08:00 -07:00
Augusto Hack
cf94ac07b7 Merge pull request #121 from 0xPolygonMiden/hacka-simple-smt-parent-node-iterator
feat: add parent node iterator for SimpleSMT
2023-04-05 00:46:32 +02:00
Augusto Hack
d873866f52 Merge pull request #118 from 0xPolygonMiden/hacka-support-mmr-in-the-merkle-store
feat: add support for MMR to the MerkleStore
2023-04-04 23:13:43 +02:00
Augusto F. Hack
9275dd00ad feat: add parent node iterator for SimpleSMT 2023-04-04 22:33:26 +02:00
Augusto F. Hack
429d3bab6f feat: add support for MMR to the MerkleStore 2023-04-04 22:33:01 +02:00
Augusto Hack
f19fe6e739 Merge pull request #117 from 0xPolygonMiden/hacka-simplify-consuming-merkle-tree
feat: add node iterator to MerkleTree
2023-04-04 22:14:38 +02:00
Augusto F. Hack
1df4318399 feat: add node iterator to MerkleTree 2023-04-04 22:11:21 +02:00
Bobbin Threadbare
433b467953 feat: optimized hash_elements for blake3 hasher 2023-04-04 01:06:51 -07:00
Augusto Hack
f46d913b20 Merge pull request #116 from 0xPolygonMiden/hacka-remove-merke-store
Remove SimpleSmt store
2023-03-31 03:12:09 +02:00
Augusto F. Hack
f8a62dae76 chore: remove simple_smt::Store 2023-03-31 03:10:01 +02:00
Victor Lopes
49b9029b46 Merge pull request #115 from 0xPolygonMiden/vlopes11-store-smt-depth
feat: Add `depth` as store SMT argument
2023-03-30 01:19:30 +02:00
Victor Lopez
d37f3f5e84 feat: Add depth as store SMT argument
Prior to this commit, MerkleStore allowed the creation of Sparse Merkle
tree only with the maximum depth of 63. However, this doesn't fit the
Tiered Sparse Merkle tree requirements, as it will contain trees of
depth 16.

This commit adds the `depth` argument to the MerkleStore methods that
will create Sparse Merkle trees.
2023-03-30 01:13:05 +02:00
Bobbin Threadbare
9389f2fb40 Merge pull request #80 from 0xPolygonMiden/next
v0.2 tracking PR
2023-03-25 01:28:40 -07:00
Bobbin Threadbare
703692553d chore: add winterfell dependency update to changelog 2023-03-25 00:45:17 -07:00
Bobbin Threadbare
d68be83bc4 chore: add Mmr to readme and changelog 2023-03-25 00:00:24 -07:00
Bobbin Threadbare
80171af872 Merge pull request #114 from 0xPolygonMiden/v0.2.0-release-prep
Prepare v0.2 release
2023-03-24 23:50:41 -07:00
Augusto Hack
75af3d474b Merge pull request #113 from 0xPolygonMiden/hacka-merkle-store-fix-empty-roots
bugfix: fix internal nodes of for empty leafs of a SMT
2023-03-24 23:26:48 +01:00
Augusto F. Hack
9e6c8ff700 bugfix: fix internal nodes of for empty leafs of a SMT
The path returned by `EmptySubtreeRoots` starts at the root, and goes to
the leaf. The MerkleStore constructor assumed the other direction, so
the parent/child hashes were reversed.

This fixes the bug and adds a test.
2023-03-24 23:22:31 +01:00
Bobbin Threadbare
a58922756a chore: update crate versions, dependencies, and CHANGELOG 2023-03-24 14:58:19 -07:00
Augusto Hack
bf15e1331a Merge pull request #112 from 0xPolygonMiden/hacka-add-serde-to-merklestore
Add serde to merklestore
2023-03-24 21:49:50 +01:00
Augusto F. Hack
7957cc929a feat: added MerkleStore serde 2023-03-24 21:44:36 +01:00
Victor Lopes
854892ba9d Merge pull request #111 from 0xPolygonMiden/vlopes11-increase-empty-subtrees
feat: add empty subtree constants to cover u8::MAX depth
2023-03-23 22:50:37 +01:00
Bobbin Threadbare
ce38ee388d Merge pull request #104 from 0xPolygonMiden/hacka-store-docs
Store docs
2023-03-23 13:11:04 -07:00
Bobbin Threadbare
4d1b3628d3 Merge pull request #110 from 0xPolygonMiden/bobbin-pathset-fixes
Fix MerklePathSet issues
2023-03-23 13:10:21 -07:00
Augusto F. Hack
2d1bc3ba34 store: added user documentation on usage and purpose 2023-03-23 14:19:37 +01:00
Victor Lopez
2ff96f40cb feat: add empty subtree constants to cover u8::MAX depth
Prior to this commit, we limited the constants count to 64 for the empty
subtrees depth computation. This is a hard-assumption that every tree of
Miden will have a depth up to 64 - and will cause undefined behavior if
it doesn't.

With the introduction of `MerkleStore::merge_roots` and the deprecation
of `mtree_cwm` instruction from the VM, this assumption is broken and
the user might end with trees with depth greater than 64. This broken
assumption could lead to attack vectors.

We can easily fix that by extending the pre-computed hashes list to the
maximum of `u8` (i.e. 255). This will have zero impact on functionality,
and will be completely safe to use without hard assumptions.
2023-03-23 12:59:47 +01:00
Bobbin Threadbare
9531d2bd34 fix: to paths reduction of MerklePathSet 2023-03-23 01:12:02 -07:00
Bobbin Threadbare
c79351be99 Merge pull request #107 from 0xPolygonMiden/hacka-store-add-merkle-paths
store: added with_merkle_paths constructor
2023-03-22 16:14:45 -07:00
Bobbin Threadbare
b7678619b0 Merge pull request #103 from 0xPolygonMiden/hacka-format-merkle-tree
Format merkle tree
2023-03-22 15:40:16 -07:00
Augusto F. Hack
0375f31035 feat: added utility to format MerkleTree and MerklePath to hex
Example formatted MerkleTree:

```
880abe452320966617646e7740b014954300f19a28780a0889d62ff33f4b0534
  1ade1369091efa31201e9b60c9c28874d0ddce5362b335135a6bb4c917285983
  3e60a9c843b4bb19f7a0572102e6507195f5240767a396335fd21981b048b807
    0100000000000000000000000000000000000000000000000000000000000000
    0200000000000000000000000000000000000000000000000000000000000000
    0300000000000000000000000000000000000000000000000000000000000000
    0400000000000000000000000000000000000000000000000000000000000000
```

Example formatted MerklePath:

```
[0400000000000000000000000000000000000000000000000000000000000000, 1ade1369091efa31201e9b60c9c28874d0ddce5362b335135a6bb4c917285983]
```
2023-03-22 21:53:05 +01:00
Augusto Hack
c96047af9d Merge pull request #102 from 0xPolygonMiden/hacka-merkle-tree-assert-message
chore: clarified assert message
2023-03-22 17:54:54 +01:00
Augusto F. Hack
b250752883 store: added with_merkle_paths constructor
And unit tests for each constructor type.
2023-03-22 14:17:12 +01:00
Augusto Hack
482dab94c5 Merge pull request #101 from 0xPolygonMiden/hacka-fix-benchmark-code
Fix benchmark code
2023-03-22 13:46:22 +01:00
Augusto F. Hack
d6cbd178e1 chore: clarified assert message 2023-03-22 11:30:19 +01:00
Augusto F. Hack
ef342cec23 bugfix: fix store benchmark 2023-03-22 10:53:12 +01:00
Victor Lopes
7305a72295 Merge pull request #99 from 0xPolygonMiden/vlopes11-merkle-store-containers
feat: add merkle path containers and return them on tree update
2023-03-21 20:54:36 +01:00
Victor Lopez
84086bdb95 feat: add merkle path containers and return them on tree update
Returning tuples is often confusing as they don't convey meaning and it
should be used only when there is no possible ambiguity.

For `MerkleStore`, we had a couple of tuples being returned, and reading
the implementation was required in order to distinguish if they were
leaf values or computed roots.

This commit introduces two containers that will self-document these
returns: `RootPath` and `ValuePath`. It will also update `set_node` to
return both the new root & the new path, so we can prevent duplicated
traversals downstream when updating a node (one to update, the second to
fetch the new path/root).
2023-03-21 20:45:01 +01:00
Bobbin Threadbare
a681952982 Merge pull request #97 from 0xPolygonMiden/hacka-storage-benchmark
Storage benchmark
2023-03-21 11:43:12 -07:00
Augusto F. Hack
78e82f2ee6 feat: add benchmark for storages 2023-03-21 14:29:18 +01:00
Victor Lopes
f07ed69d2f Merge pull request #95 from 0xPolygonMiden/vlopes11-fix-merkle-store-bounds
fix: merkle store panics on bounds
2023-03-21 09:51:48 +01:00
Augusto F. Hack
17eb8d78d3 chore: storage -> store 2023-03-21 09:45:36 +01:00
Victor Lopez
8cb245dc1f bugfix: reverse merkle path to match other structures
The store builds the path from root to leaf, this updates the code to
return a path from leaf to root, as it is done by the other structures.

This also added custom error for missing root.
2023-03-21 09:45:29 +01:00
Victor Lopez
867b772d9a fix: merkle store panics on bounds
Prior to this commit, the MerkleStore panicked under certain bounds. It
will prevent such panics by using checked operations.

ilog2, for instance, will panic when the operand is zero. However, there
is a documentation rule enforcing the merkle tree to be size at least 2.
If this rule is checked, then the panic is impossible.
2023-03-18 02:20:11 +01:00
Bobbin Threadbare
33d37d82e2 Merge pull request #79 from 0xPolygonMiden/hacka-ignore-pre-commit-rev
ignore pre commit rev
2023-03-17 00:11:13 -07:00
Augusto Hack
5703fef226 Merge pull request #96 from 0xPolygonMiden/hacka-check-root-in-storage
bugfix: check if the requested root is in the storage
2023-03-16 23:30:56 +01:00
Augusto F. Hack
669ebb49fb bugfix: check if the requested root is in the storage 2023-03-16 23:26:02 +01:00
Victor Lopes
931bcc3cc3 Merge pull request #94 from 0xPolygonMiden/vlopes11-merkle-store-derive
refactor: add derive proc macros to merkle store
2023-03-16 19:13:02 +01:00
Victor Lopez
91667fd7de refactor: add derive proc macros to merkle store
This commit introduce common derive proc macros to MerkleStore. These
are required downstream as the in-memory storage can be cloned.

It also introduces constructors common to the other types of the crate
that will help to build a merkle store, using a build pattern.
2023-03-16 10:28:45 +01:00
Augusto Hack
e4ddf6ffaf Merge pull request #93 from 0xPolygonMiden/hacka-add-merkle-store
Add merkle store
2023-03-15 18:13:48 +01:00
Augusto F. Hack
88a646031f feat: add merkle store 2023-03-15 17:34:42 +01:00
Bobbin Threadbare
2871e4eb27 Merge pull request #87 from 0xPolygonMiden/vlopes11-36-simple-smt-prepare
feat: refactor simple smt to use empty subtree constants
2023-03-07 16:10:24 -08:00
Victor Lopez
3a6a4fcce6 feat: refactor simple smt to use empty subtree constants
Prior to this commit, there was an internal procedure with the merkle
trees to compute empty sub-tree for arbitrary depths.

However, this isn't ideal as this code can be reused in any merkle
implementation that uses RPO as backend.

This commit introduces a structure that will generate these empty
subtrees values.
2023-03-07 20:44:42 +01:00
Augusto Hack
7ffa0cd97d Merge pull request #67 from 0xPolygonMiden/hacka-merkle-mountain-range-memory-implementation
feat: merkle mountain range
2023-03-02 22:27:13 +01:00
Augusto F. Hack
32d37f1591 feat: merkle mountain range 2023-03-02 13:07:55 +01:00
Augusto F. Hack
bc12fcafe9 chore: ignore pre-commit rev 2023-03-01 18:32:24 +01:00
Augusto Hack
8c08243f7a Merge pull request #78 from 0xPolygonMiden/hacka-pre-commit
Add pre commit
2023-03-01 18:31:08 +01:00
Augusto F. Hack
956e4c6fad chore: initial run pre-commit 2023-03-01 17:45:57 +01:00
Augusto F. Hack
efa39e5ce0 feat: added pre-commit hook config 2023-03-01 17:45:33 +01:00
Bobbin Threadbare
ae3f14e0ff Merge pull request #74 from 0xPolygonMiden/hacka-node-index-docs
docs: mention tree form order of NodeIndex docs
2023-02-22 12:19:45 -08:00
Bobbin Threadbare
962a07292f Merge pull request #75 from 0xPolygonMiden/next
v0.1.4 release
2023-02-22 09:32:44 -08:00
Augusto F. Hack
dfb073f784 docs: mention tree form order of NodeIndex docs 2023-02-22 17:23:03 +01:00
Bobbin Threadbare
41c38b4b5d chore: changed version to v0.1.4 in Cargo.toml 2023-02-22 08:22:25 -08:00
Bobbin Threadbare
c4eb4a6b98 Merge pull request #73 from 0xPolygonMiden/vlopes11-72-add-winter-hasher
feat: re-export winter-crypto Hasher, Digest & ElementHasher
2023-02-22 08:15:58 -08:00
Victor Lopez
35b255b5eb feat: re-export winter-crypto Hasher, Digest & ElementHasher
This commit introduces the re-export of the listed primitives.

They will be used inside Miden to report the security level of the
picked primitive, as well as other functionality.

closes #72
2023-02-22 16:56:14 +01:00
Bobbin Threadbare
e94b0c70a9 Merge pull request #71 from 0xPolygonMiden/bobbin-dep-updates
Dependency updates
2023-02-20 23:55:43 -08:00
Bobbin Threadbare
e6bf497500 chore: update dependencies 2023-02-20 23:46:21 -08:00
86 changed files with 16813 additions and 2242 deletions

20
.editorconfig Normal file
View File

@@ -0,0 +1,20 @@
# Documentation available at editorconfig.org
root=true
[*]
ident_style = space
ident_size = 4
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true
[*.rs]
max_line_length = 100
[*.md]
trim_trailing_whitespace = false
[*.yml]
ident_size = 2

View File

@@ -6,4 +6,4 @@
- Commit messages and codestyle follow [conventions](./CONTRIBUTING.md).
- Relevant issues are linked in the PR description.
- Tests added for new functionality.
- Documentation/comments updated according to changes.
- Documentation/comments updated according to changes.

View File

@@ -4,44 +4,76 @@ on:
branches:
- main
pull_request:
types: [opened, repoened, synchronize]
types: [opened, reopened, synchronize]
jobs:
build:
name: Build ${{matrix.toolchain}} on ${{matrix.os}} with ${{matrix.args}}
rustfmt:
name: rustfmt ${{matrix.toolchain}} on ${{matrix.os}}
runs-on: ${{matrix.os}}-latest
strategy:
fail-fast: false
matrix:
toolchain: [stable, nightly]
toolchain: [nightly]
os: [ubuntu]
target: [wasm32-unknown-unknown]
args: [--no-default-features --target wasm32-unknown-unknown]
steps:
- uses: actions/checkout@main
- name: Install rust
- uses: actions/checkout@v4
- name: Install minimal Rust with rustfmt
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: ${{matrix.toolchain}}
components: rustfmt
override: true
- run: rustup target add ${{matrix.target}}
- name: Test
- name: fmt
uses: actions-rs/cargo@v1
with:
command: build
args: ${{matrix.args}}
command: fmt
args: --all -- --check
clippy:
name: clippy ${{matrix.toolchain}} on ${{matrix.os}}
runs-on: ${{matrix.os}}-latest
strategy:
fail-fast: false
matrix:
toolchain: [nightly]
os: [ubuntu]
steps:
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install minimal Rust with clippy
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: ${{matrix.toolchain}}
components: clippy
override: true
- name: Clippy
uses: actions-rs/cargo@v1
with:
command: clippy
args: --all-targets -- -D clippy::all -D warnings
- name: Clippy all features
uses: actions-rs/cargo@v1
with:
command: clippy
args: --all-targets --all-features -- -D clippy::all -D warnings
test:
name: Test ${{matrix.toolchain}} on ${{matrix.os}} with ${{matrix.features}}
name: test ${{matrix.toolchain}} on ${{matrix.os}} with ${{matrix.features}}
runs-on: ${{matrix.os}}-latest
strategy:
fail-fast: false
matrix:
toolchain: [stable, nightly]
os: [ubuntu]
features: [--all-features, --no-default-features]
features: ["--features default,serde", --no-default-features]
timeout-minutes: 30
steps:
- uses: actions/checkout@main
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install rust
uses: actions-rs/toolchain@v1
with:
@@ -53,43 +85,49 @@ jobs:
command: test
args: ${{matrix.features}}
clippy:
name: Clippy with ${{matrix.features}}
no-std:
name: build ${{matrix.toolchain}} no-std for wasm32-unknown-unknown
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
features: [--all-features, --no-default-features]
toolchain: [stable, nightly]
steps:
- uses: actions/checkout@main
- name: Install minimal nightly with clippy
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install rust
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: nightly
components: clippy
toolchain: ${{matrix.toolchain}}
override: true
- name: Clippy
- run: rustup target add wasm32-unknown-unknown
- name: Build
uses: actions-rs/cargo@v1
with:
command: clippy
args: --all ${{matrix.features}} -- -D clippy::all -D warnings
command: build
args: --no-default-features --target wasm32-unknown-unknown
rustfmt:
name: rustfmt
docs:
name: Verify the docs on ${{matrix.toolchain}}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
toolchain: [stable]
steps:
- uses: actions/checkout@main
- name: Install minimal stable with rustfmt
- uses: actions/checkout@v4
with:
submodules: recursive
- name: Install rust
uses: actions-rs/toolchain@v1
with:
profile: minimal
toolchain: stable
components: rustfmt
toolchain: ${{matrix.toolchain}}
override: true
- name: rustfmt
- name: Check docs
uses: actions-rs/cargo@v1
env:
RUSTDOCFLAGS: -D warnings
with:
command: fmt
args: --all -- --check
command: doc
args: --verbose --all-features --keep-going

6
.gitignore vendored
View File

@@ -8,3 +8,9 @@ Cargo.lock
# These are backup files generated by rustfmt
**/*.rs.bk
# Generated by cmake
cmake-build-*
# VS Code
.vscode/

3
.gitmodules vendored Normal file
View File

@@ -0,0 +1,3 @@
[submodule "PQClean"]
path = PQClean
url = https://github.com/PQClean/PQClean.git

43
.pre-commit-config.yaml Normal file
View File

@@ -0,0 +1,43 @@
# See https://pre-commit.com for more information
# See https://pre-commit.com/hooks.html for more hooks
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v3.2.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- id: check-yaml
- id: check-json
- id: check-toml
- id: pretty-format-json
- id: check-added-large-files
- id: check-case-conflict
- id: check-executables-have-shebangs
- id: check-merge-conflict
- id: detect-private-key
- repo: https://github.com/hackaugusto/pre-commit-cargo
rev: v1.0.0
hooks:
# Allows cargo fmt to modify the source code prior to the commit
- id: cargo
name: Cargo fmt
args: ["+stable", "fmt", "--all"]
stages: [commit]
# Requires code to be properly formatted prior to pushing upstream
- id: cargo
name: Cargo fmt --check
args: ["+stable", "fmt", "--all", "--check"]
stages: [push, manual]
- id: cargo
name: Cargo check --all-targets
args: ["+stable", "check", "--all-targets"]
- id: cargo
name: Cargo check --all-targets --no-default-features
args: ["+stable", "check", "--all-targets", "--no-default-features"]
- id: cargo
name: Cargo check --all-targets --features default,std,serde
args: ["+stable", "check", "--all-targets", "--features", "default,std,serde"]
# Unlike fmt, clippy will not be automatically applied
- id: cargo
name: Cargo clippy
args: ["+nightly", "clippy", "--workspace", "--", "--deny", "clippy::all", "--deny", "warnings"]

View File

@@ -1,3 +1,70 @@
## 0.8.0 (2024-02-14)
* Implemented the `PartialMmr` data structure (#195).
* Implemented RPX hash function (#201).
* Added `FeltRng` and `RpoRandomCoin` (#237).
* Accelerated RPO/RPX hash functions using AVX512 instructions (#234).
* Added `inner_nodes()` method to `PartialMmr` (#238).
* Improved `PartialMmr::apply_delta()` (#242).
* Refactored `SimpleSmt` struct (#245).
* Replaced `TieredSmt` struct with `Smt` struct (#254, #277).
* Updated Winterfell dependency to v0.8 (#275).
## 0.7.1 (2023-10-10)
* Fixed RPO Falcon signature build on Windows.
## 0.7.0 (2023-10-05)
* Replaced `MerklePathSet` with `PartialMerkleTree` (#165).
* Implemented clearing of nodes in `TieredSmt` (#173).
* Added ability to generate inclusion proofs for `TieredSmt` (#174).
* Implemented Falcon DSA (#179).
* Added conditional `serde`` support for various structs (#180).
* Implemented benchmarking for `TieredSmt` (#182).
* Added more leaf traversal methods for `MerkleStore` (#185).
* Added SVE acceleration for RPO hash function (#189).
## 0.6.0 (2023-06-25)
* [BREAKING] Added support for recording capabilities for `MerkleStore` (#162).
* [BREAKING] Refactored Merkle struct APIs to use `RpoDigest` instead of `Word` (#157).
* Added initial implementation of `PartialMerkleTree` (#156).
## 0.5.0 (2023-05-26)
* Implemented `TieredSmt` (#152, #153).
* Implemented ability to extract a subset of a `MerkleStore` (#151).
* Cleaned up `SimpleSmt` interface (#149).
* Decoupled hashing and padding of peaks in `Mmr` (#148).
* Added `inner_nodes()` to `MerkleStore` (#146).
## 0.4.0 (2023-04-21)
- Exported `MmrProof` from the crate (#137).
- Allowed merging of leaves in `MerkleStore` (#138).
- [BREAKING] Refactored how existing data structures are added to `MerkleStore` (#139).
## 0.3.0 (2023-04-08)
- Added `depth` parameter to SMT constructors in `MerkleStore` (#115).
- Optimized MMR peak hashing for Miden VM (#120).
- Added `get_leaf_depth` method to `MerkleStore` (#119).
- Added inner node iterators to `MerkleTree`, `SimpleSmt`, and `Mmr` (#117, #118, #121).
## 0.2.0 (2023-03-24)
- Implemented `Mmr` and related structs (#67).
- Implemented `MerkleStore` (#93, #94, #95, #107 #112).
- Added benchmarks for `MerkleStore` vs. other structs (#97).
- Added Merkle path containers (#99).
- Fixed depth handling in `MerklePathSet` (#110).
- Updated Winterfell dependency to v0.6.
## 0.1.4 (2023-02-22)
- Re-export winter-crypto Hasher, Digest & ElementHasher (#72)
## 0.1.3 (2023-02-20)
- Updated Winterfell dependency to v0.5.1 (#68)

View File

@@ -17,7 +17,7 @@ We are using [Github Flow](https://docs.github.com/en/get-started/quickstart/git
### Branching
- The current active branch is `next`. Every branch with a fix/feature must be forked from `next`.
- The branch name should contain a short issue/feature description separated with hyphens [(kebab-case)](https://en.wikipedia.org/wiki/Letter_case#Kebab_case).
- The branch name should contain a short issue/feature description separated with hyphens [(kebab-case)](https://en.wikipedia.org/wiki/Letter_case#Kebab_case).
For example, if the issue title is `Fix functionality X in component Y` then the branch name will be something like: `fix-x-in-y`.
@@ -72,7 +72,7 @@ For example, a new change to the AIR crate might have the following message: `fe
// ================================================================================
```
- [Rustfmt](https://github.com/rust-lang/rustfmt) and [Clippy](https://github.com/rust-lang/rust-clippy) linting is included in CI pipeline. Anyways it's prefferable to run linting locally before push:
- [Rustfmt](https://github.com/rust-lang/rustfmt) and [Clippy](https://github.com/rust-lang/rust-clippy) linting is included in CI pipeline. Anyways it's preferable to run linting locally before push:
```
cargo fix --allow-staged --allow-dirty --all-targets --all-features; cargo fmt; cargo clippy --workspace --all-targets --all-features -- -D warnings
```

View File

@@ -1,14 +1,23 @@
[package]
name = "miden-crypto"
version = "0.1.3"
description="Miden Cryptographic primitives"
version = "0.8.0"
description = "Miden Cryptographic primitives"
authors = ["miden contributors"]
readme="README.md"
readme = "README.md"
license = "MIT"
repository = "https://github.com/0xPolygonMiden/crypto"
documentation = "https://docs.rs/miden-crypto/0.8.0"
categories = ["cryptography", "no-std"]
keywords = ["miden", "crypto", "hash", "merkle"]
edition = "2021"
rust-version = "1.75"
[[bin]]
name = "miden-crypto"
path = "src/main.rs"
bench = false
doctest = false
required-features = ["executable"]
[[bench]]
name = "hash"
@@ -18,17 +27,39 @@ harness = false
name = "smt"
harness = false
[[bench]]
name = "store"
harness = false
[features]
default = ["blake3/default", "std", "winter_crypto/default", "winter_math/default", "winter_utils/default"]
std = ["blake3/std", "winter_crypto/std", "winter_math/std", "winter_utils/std"]
default = ["std"]
executable = ["dep:clap", "dep:rand_utils", "std"]
serde = ["dep:serde", "serde?/alloc", "winter_math/serde"]
std = [
"blake3/std",
"dep:cc",
"dep:libc",
"winter_crypto/std",
"winter_math/std",
"winter_utils/std",
]
[dependencies]
blake3 = { version = "1.0", default-features = false }
winter_crypto = { version = "0.5.1", package = "winter-crypto", default-features = false }
winter_math = { version = "0.5.1", package = "winter-math", default-features = false }
winter_utils = { version = "0.5.1", package = "winter-utils", default-features = false }
blake3 = { version = "1.5", default-features = false }
clap = { version = "4.5", features = ["derive"], optional = true }
libc = { version = "0.2", default-features = false, optional = true }
rand_utils = { version = "0.8", package = "winter-rand-utils", optional = true }
serde = { version = "1.0", features = ["derive"], default-features = false, optional = true }
winter_crypto = { version = "0.8", package = "winter-crypto", default-features = false }
winter_math = { version = "0.8", package = "winter-math", default-features = false }
winter_utils = { version = "0.8", package = "winter-utils", default-features = false }
[dev-dependencies]
criterion = { version = "0.4", features = ["html_reports"] }
proptest = "1.0.0"
rand_utils = { version = "0.4", package = "winter-rand-utils" }
seq-macro = { version = "0.3" }
criterion = { version = "0.5", features = ["html_reports"] }
proptest = "1.4"
rand_utils = { version = "0.8", package = "winter-rand-utils" }
[build-dependencies]
cc = { version = "1.0", features = ["parallel"], optional = true }
glob = "0.3"

View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2022 Polygon Miden
Copyright (c) 2023 Polygon Miden
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

1
PQClean Submodule

Submodule PQClean added at c3abebf4ab

View File

@@ -6,21 +6,36 @@ This crate contains cryptographic primitives used in Polygon Miden.
* [BLAKE3](https://github.com/BLAKE3-team/BLAKE3) hash function with 256-bit, 192-bit, or 160-bit output. The 192-bit and 160-bit outputs are obtained by truncating the 256-bit output of the standard BLAKE3.
* [RPO](https://eprint.iacr.org/2022/1577) hash function with 256-bit output. This hash function is an algebraic hash function suitable for recursive STARKs.
* [RPX](https://eprint.iacr.org/2023/1045) hash function with 256-bit output. Similar to RPO, this hash function is suitable for recursive STARKs but it is about 2x faster as compared to RPO.
For performance benchmarks of these hash functions and their comparison to other popular hash functions please see [here](./benches/).
## Merkle
[Merkle module](./src/merkle/) provides a set of data structures related to Merkle trees. All these data structures are implemented using the RPO hash function described above. The data structures are:
* `MerkleStore`: a collection of Merkle trees of different heights designed to efficiently store trees with common subtrees. When instantiated with `RecordingMap`, a Merkle store records all accesses to the original data.
* `MerkleTree`: a regular fully-balanced binary Merkle tree. The depth of this tree can be at most 64.
* `SimpleSmt`: a Sparse Merkle Tree, mapping 63-bit keys to 4-element leaf values.
* `MerklePathSet`: a collection of Merkle authentication paths all resolving to the same root. The length of the paths can be at most 64.
* `Mmr`: a Merkle mountain range structure designed to function as an append-only log.
* `PartialMerkleTree`: a partial view of a Merkle tree where some sub-trees may not be known. This is similar to a collection of Merkle paths all resolving to the same root. The length of the paths can be at most 64.
* `PartialMmr`: a partial view of a Merkle mountain range structure.
* `SimpleSmt`: a Sparse Merkle Tree (with no compaction), mapping 64-bit keys to 4-element values.
* `Smt`: a Sparse Merkle tree (with compaction at depth 64), mapping 4-element keys to 4-element values.
The module also contains additional supporting components such as `NodeIndex`, `MerklePath`, and `MerkleError` to assist with tree indexation, opening proofs, and reporting inconsistent arguments/state.
## Extra
[Root module](./src/lib.rs) provides a set of constants, types, aliases, and utils required to use the primitives of this library.
## Signatures
[DSA module](./src/dsa) provides a set of digital signature schemes supported by default in the Miden VM. Currently, these schemes are:
* `RPO Falcon512`: a variant of the [Falcon](https://falcon-sign.info/) signature scheme. This variant differs from the standard in that instead of using SHAKE256 hash function in the *hash-to-point* algorithm we use RPO256. This makes the signature more efficient to verify in Miden VM.
For the above signatures, key generation and signing is available only in the `std` context (see [crate features](#crate-features) below), while signature verification is available in `no_std` context as well.
## Pseudo-Random Element Generator
[Pseudo random element generator module](./src/rand/) provides a set of traits and data structures that facilitate generating pseudo-random elements in the context of Miden VM and Miden rollup. The module currently includes:
* `FeltRng`: a trait for generating random field elements and random 4 field elements.
* `RpoRandomCoin`: a struct implementing `FeltRng` as well as the [`RandomCoin`](https://github.com/facebook/winterfell/blob/main/crypto/src/random/mod.rs) trait.
## Crate features
This crate can be compiled with the following features:
@@ -31,6 +46,18 @@ Both of these features imply the use of [alloc](https://doc.rust-lang.org/alloc/
To compile with `no_std`, disable default features via `--no-default-features` flag.
### AVX2 acceleration
On platforms with [AVX2](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions) support, RPO and RPX hash function can be accelerated by using the vector processing unit. To enable AVX2 acceleration, the code needs to be compiled with the `avx2` target feature enabled. For example:
```shell
RUSTFLAGS="-C target-feature=+avx2" cargo build --release
```
### SVE acceleration
On platforms with [SVE](https://en.wikipedia.org/wiki/AArch64#Scalable_Vector_Extension_(SVE)) support, RPO and RPX hash function can be accelerated by using the vector processing unit. To enable SVE acceleration, the code needs to be compiled with the `sve` target feature enabled. For example:
```shell
RUSTFLAGS="-C target-feature=+sve" cargo build --release
```
## Testing
You can use cargo defaults to test the library:

View File

@@ -0,0 +1,78 @@
#include <stddef.h>
#include <arm_sve.h>
#include "library.h"
#include "rpo_hash.h"
// The STATE_WIDTH of RPO hash is 12x u64 elements.
// The current generation of SVE-enabled processors - Neoverse V1
// (e.g. AWS Graviton3) have 256-bit vector registers (4x u64)
// This allows us to split the state into 3 vectors of 4 elements
// and process all 3 independent of each other.
// We see the biggest performance gains by leveraging both
// vector and scalar operations on parts of the state array.
// Due to high latency of vector operations, the processor is able
// to reorder and pipeline scalar instructions while we wait for
// vector results. This effectively gives us some 'free' scalar
// operations and masks vector latency.
//
// This also means that we can fully saturate all four arithmetic
// units of the processor (2x scalar, 2x SIMD)
//
// THIS ANALYSIS NEEDS TO BE PERFORMED AGAIN ONCE PROCESSORS
// GAIN WIDER REGISTERS. It's quite possible that with 8x u64
// vectors processing 2 partially filled vectors might
// be easier and faster than dealing with scalar operations
// on the remainder of the array.
//
// FOR NOW THIS IS ONLY ENABLED ON 4x u64 VECTORS! It falls back
// to the regular, already highly-optimized scalar version
// if the conditions are not met.
bool add_constants_and_apply_sbox(uint64_t state[STATE_WIDTH], uint64_t constants[STATE_WIDTH]) {
const uint64_t vl = svcntd(); // number of u64 numbers in one SVE vector
if (vl != 4) {
return false;
}
svbool_t ptrue = svptrue_b64();
svuint64_t state1 = svld1(ptrue, state + 0*vl);
svuint64_t state2 = svld1(ptrue, state + 1*vl);
svuint64_t const1 = svld1(ptrue, constants + 0*vl);
svuint64_t const2 = svld1(ptrue, constants + 1*vl);
add_constants(ptrue, &state1, &const1, &state2, &const2, state+8, constants+8);
apply_sbox(ptrue, &state1, &state2, state+8);
svst1(ptrue, state + 0*vl, state1);
svst1(ptrue, state + 1*vl, state2);
return true;
}
bool add_constants_and_apply_inv_sbox(uint64_t state[STATE_WIDTH], uint64_t constants[STATE_WIDTH]) {
const uint64_t vl = svcntd(); // number of u64 numbers in one SVE vector
if (vl != 4) {
return false;
}
svbool_t ptrue = svptrue_b64();
svuint64_t state1 = svld1(ptrue, state + 0 * vl);
svuint64_t state2 = svld1(ptrue, state + 1 * vl);
svuint64_t const1 = svld1(ptrue, constants + 0 * vl);
svuint64_t const2 = svld1(ptrue, constants + 1 * vl);
add_constants(ptrue, &state1, &const1, &state2, &const2, state + 8, constants + 8);
apply_inv_sbox(ptrue, &state1, &state2, state + 8);
svst1(ptrue, state + 0 * vl, state1);
svst1(ptrue, state + 1 * vl, state2);
return true;
}

View File

@@ -0,0 +1,12 @@
#ifndef CRYPTO_LIBRARY_H
#define CRYPTO_LIBRARY_H
#include <stdint.h>
#include <stdbool.h>
#define STATE_WIDTH 12
bool add_constants_and_apply_sbox(uint64_t state[STATE_WIDTH], uint64_t constants[STATE_WIDTH]);
bool add_constants_and_apply_inv_sbox(uint64_t state[STATE_WIDTH], uint64_t constants[STATE_WIDTH]);
#endif //CRYPTO_LIBRARY_H

View File

@@ -0,0 +1,221 @@
#ifndef RPO_SVE_RPO_HASH_H
#define RPO_SVE_RPO_HASH_H
#include <arm_sve.h>
#include <stddef.h>
#include <stdint.h>
#include <string.h>
#define COPY(NAME, VIN1, VIN2, SIN3) \
svuint64_t NAME ## _1 = VIN1; \
svuint64_t NAME ## _2 = VIN2; \
uint64_t NAME ## _3[4]; \
memcpy(NAME ## _3, SIN3, 4 * sizeof(uint64_t))
#define MULTIPLY(PRED, DEST, OP) \
mul(PRED, &DEST ## _1, &OP ## _1, &DEST ## _2, &OP ## _2, DEST ## _3, OP ## _3)
#define SQUARE(PRED, NAME) \
sq(PRED, &NAME ## _1, &NAME ## _2, NAME ## _3)
#define SQUARE_DEST(PRED, DEST, SRC) \
COPY(DEST, SRC ## _1, SRC ## _2, SRC ## _3); \
SQUARE(PRED, DEST);
#define POW_ACC(PRED, NAME, CNT, TAIL) \
for (size_t i = 0; i < CNT; i++) { \
SQUARE(PRED, NAME); \
} \
MULTIPLY(PRED, NAME, TAIL);
#define POW_ACC_DEST(PRED, DEST, CNT, HEAD, TAIL) \
COPY(DEST, HEAD ## _1, HEAD ## _2, HEAD ## _3); \
POW_ACC(PRED, DEST, CNT, TAIL)
extern inline void add_constants(
svbool_t pg,
svuint64_t *state1,
svuint64_t *const1,
svuint64_t *state2,
svuint64_t *const2,
uint64_t *state3,
uint64_t *const3
) {
uint64_t Ms = 0xFFFFFFFF00000001ull;
svuint64_t Mv = svindex_u64(Ms, 0);
uint64_t p_1 = Ms - const3[0];
uint64_t p_2 = Ms - const3[1];
uint64_t p_3 = Ms - const3[2];
uint64_t p_4 = Ms - const3[3];
uint64_t x_1, x_2, x_3, x_4;
uint32_t adj_1 = -__builtin_sub_overflow(state3[0], p_1, &x_1);
uint32_t adj_2 = -__builtin_sub_overflow(state3[1], p_2, &x_2);
uint32_t adj_3 = -__builtin_sub_overflow(state3[2], p_3, &x_3);
uint32_t adj_4 = -__builtin_sub_overflow(state3[3], p_4, &x_4);
state3[0] = x_1 - (uint64_t)adj_1;
state3[1] = x_2 - (uint64_t)adj_2;
state3[2] = x_3 - (uint64_t)adj_3;
state3[3] = x_4 - (uint64_t)adj_4;
svuint64_t p1 = svsub_x(pg, Mv, *const1);
svuint64_t p2 = svsub_x(pg, Mv, *const2);
svuint64_t x1 = svsub_x(pg, *state1, p1);
svuint64_t x2 = svsub_x(pg, *state2, p2);
svbool_t pt1 = svcmplt_u64(pg, *state1, p1);
svbool_t pt2 = svcmplt_u64(pg, *state2, p2);
*state1 = svsub_m(pt1, x1, (uint32_t)-1);
*state2 = svsub_m(pt2, x2, (uint32_t)-1);
}
extern inline void mul(
svbool_t pg,
svuint64_t *r1,
const svuint64_t *op1,
svuint64_t *r2,
const svuint64_t *op2,
uint64_t *r3,
const uint64_t *op3
) {
__uint128_t x_1 = r3[0];
__uint128_t x_2 = r3[1];
__uint128_t x_3 = r3[2];
__uint128_t x_4 = r3[3];
x_1 *= (__uint128_t) op3[0];
x_2 *= (__uint128_t) op3[1];
x_3 *= (__uint128_t) op3[2];
x_4 *= (__uint128_t) op3[3];
uint64_t x0_1 = x_1;
uint64_t x0_2 = x_2;
uint64_t x0_3 = x_3;
uint64_t x0_4 = x_4;
svuint64_t l1 = svmul_x(pg, *r1, *op1);
svuint64_t l2 = svmul_x(pg, *r2, *op2);
uint64_t x1_1 = (x_1 >> 64);
uint64_t x1_2 = (x_2 >> 64);
uint64_t x1_3 = (x_3 >> 64);
uint64_t x1_4 = (x_4 >> 64);
uint64_t a_1, a_2, a_3, a_4;
uint64_t e_1 = __builtin_add_overflow(x0_1, (x0_1 << 32), &a_1);
uint64_t e_2 = __builtin_add_overflow(x0_2, (x0_2 << 32), &a_2);
uint64_t e_3 = __builtin_add_overflow(x0_3, (x0_3 << 32), &a_3);
uint64_t e_4 = __builtin_add_overflow(x0_4, (x0_4 << 32), &a_4);
svuint64_t ls1 = svlsl_x(pg, l1, 32);
svuint64_t ls2 = svlsl_x(pg, l2, 32);
svuint64_t a1 = svadd_x(pg, l1, ls1);
svuint64_t a2 = svadd_x(pg, l2, ls2);
svbool_t e1 = svcmplt(pg, a1, l1);
svbool_t e2 = svcmplt(pg, a2, l2);
svuint64_t as1 = svlsr_x(pg, a1, 32);
svuint64_t as2 = svlsr_x(pg, a2, 32);
svuint64_t b1 = svsub_x(pg, a1, as1);
svuint64_t b2 = svsub_x(pg, a2, as2);
b1 = svsub_m(e1, b1, 1);
b2 = svsub_m(e2, b2, 1);
uint64_t b_1 = a_1 - (a_1 >> 32) - e_1;
uint64_t b_2 = a_2 - (a_2 >> 32) - e_2;
uint64_t b_3 = a_3 - (a_3 >> 32) - e_3;
uint64_t b_4 = a_4 - (a_4 >> 32) - e_4;
uint64_t r_1, r_2, r_3, r_4;
uint32_t c_1 = __builtin_sub_overflow(x1_1, b_1, &r_1);
uint32_t c_2 = __builtin_sub_overflow(x1_2, b_2, &r_2);
uint32_t c_3 = __builtin_sub_overflow(x1_3, b_3, &r_3);
uint32_t c_4 = __builtin_sub_overflow(x1_4, b_4, &r_4);
svuint64_t h1 = svmulh_x(pg, *r1, *op1);
svuint64_t h2 = svmulh_x(pg, *r2, *op2);
svuint64_t tr1 = svsub_x(pg, h1, b1);
svuint64_t tr2 = svsub_x(pg, h2, b2);
svbool_t c1 = svcmplt_u64(pg, h1, b1);
svbool_t c2 = svcmplt_u64(pg, h2, b2);
*r1 = svsub_m(c1, tr1, (uint32_t) -1);
*r2 = svsub_m(c2, tr2, (uint32_t) -1);
uint32_t minus1_1 = 0 - c_1;
uint32_t minus1_2 = 0 - c_2;
uint32_t minus1_3 = 0 - c_3;
uint32_t minus1_4 = 0 - c_4;
r3[0] = r_1 - (uint64_t)minus1_1;
r3[1] = r_2 - (uint64_t)minus1_2;
r3[2] = r_3 - (uint64_t)minus1_3;
r3[3] = r_4 - (uint64_t)minus1_4;
}
extern inline void sq(svbool_t pg, svuint64_t *a, svuint64_t *b, uint64_t *c) {
mul(pg, a, a, b, b, c, c);
}
extern inline void apply_sbox(
svbool_t pg,
svuint64_t *state1,
svuint64_t *state2,
uint64_t *state3
) {
COPY(x, *state1, *state2, state3); // copy input to x
SQUARE(pg, x); // x contains input^2
mul(pg, state1, &x_1, state2, &x_2, state3, x_3); // state contains input^3
SQUARE(pg, x); // x contains input^4
mul(pg, state1, &x_1, state2, &x_2, state3, x_3); // state contains input^7
}
extern inline void apply_inv_sbox(
svbool_t pg,
svuint64_t *state_1,
svuint64_t *state_2,
uint64_t *state_3
) {
// base^10
COPY(t1, *state_1, *state_2, state_3);
SQUARE(pg, t1);
// base^100
SQUARE_DEST(pg, t2, t1);
// base^100100
POW_ACC_DEST(pg, t3, 3, t2, t2);
// base^100100100100
POW_ACC_DEST(pg, t4, 6, t3, t3);
// compute base^100100100100100100100100
POW_ACC_DEST(pg, t5, 12, t4, t4);
// compute base^100100100100100100100100100100
POW_ACC_DEST(pg, t6, 6, t5, t3);
// compute base^1001001001001001001001001001000100100100100100100100100100100
POW_ACC_DEST(pg, t7, 31, t6, t6);
// compute base^1001001001001001001001001001000110110110110110110110110110110111
SQUARE(pg, t7);
MULTIPLY(pg, t7, t6);
SQUARE(pg, t7);
SQUARE(pg, t7);
MULTIPLY(pg, t7, t1);
MULTIPLY(pg, t7, t2);
mul(pg, state_1, &t7_1, state_2, &t7_2, state_3, t7_3);
}
#endif //RPO_SVE_RPO_HASH_H

View File

@@ -1,4 +1,4 @@
# Miden VM Hash Functions
# Miden VM Hash Functions
In the Miden VM, we make use of different hash functions. Some of these are "traditional" hash functions, like `BLAKE3`, which are optimized for out-of-STARK performance, while others are algebraic hash functions, like `Rescue Prime`, and are more optimized for a better performance inside the STARK. In what follows, we benchmark several such hash functions and compare against other constructions that are used by other proving systems. More precisely, we benchmark:
* **BLAKE3** as specified [here](https://github.com/BLAKE3-team/BLAKE3-specs/blob/master/blake3.pdf) and implemented [here](https://github.com/BLAKE3-team/BLAKE3) (with a wrapper exposed via this crate).
@@ -6,6 +6,7 @@ In the Miden VM, we make use of different hash functions. Some of these are "tra
* **Poseidon** as specified [here](https://eprint.iacr.org/2019/458.pdf) and implemented [here](https://github.com/mir-protocol/plonky2/blob/806b88d7d6e69a30dc0b4775f7ba275c45e8b63b/plonky2/src/hash/poseidon_goldilocks.rs) (but in pure Rust, without vectorized instructions).
* **Rescue Prime (RP)** as specified [here](https://eprint.iacr.org/2020/1143) and implemented [here](https://github.com/novifinancial/winterfell/blob/46dce1adf0/crypto/src/hash/rescue/rp64_256/mod.rs).
* **Rescue Prime Optimized (RPO)** as specified [here](https://eprint.iacr.org/2022/1577) and implemented in this crate.
* **Rescue Prime Extended (RPX)** a variant of the [xHash](https://eprint.iacr.org/2023/1045) hash function as implemented in this crate.
## Comparison and Instructions
@@ -13,27 +14,33 @@ In the Miden VM, we make use of different hash functions. Some of these are "tra
We benchmark the above hash functions using two scenarios. The first is a 2-to-1 $(a,b)\mapsto h(a,b)$ hashing where both $a$, $b$ and $h(a,b)$ are the digests corresponding to each of the hash functions.
The second scenario is that of sequential hashing where we take a sequence of length $100$ field elements and hash these to produce a single digest. The digests are $4$ field elements in a prime field with modulus $2^{64} - 2^{32} + 1$ (i.e., 32 bytes) for Poseidon, Rescue Prime and RPO, and an array `[u8; 32]` for SHA3 and BLAKE3.
#### Scenario 1: 2-to-1 hashing `h(a,b)`
#### Scenario 1: 2-to-1 hashing `h(a,b)`
| Function | BLAKE3 | SHA3 | Poseidon | Rp64_256 | RPO_256 |
| ------------------- | ------ | --------| --------- | --------- | ------- |
| Apple M1 Pro | 80 ns | 245 ns | 1.5 us | 9.1 us | 5.4 us |
| Apple M2 | 76 ns | 233 ns | 1.3 us | 7.9 us | 5.0 us |
| Amazon Graviton 3 | 116 ns | | | | 8.8 us |
| AMD Ryzen 9 5950X | 64 ns | 273 ns | 1.2 us | 9.1 us | 5.5 us |
| Intel Core i5-8279U | 80 ns | | | | 8.7 us |
| Intel Xeon 8375C | 67 ns | | | | 8.2 us |
| Function | BLAKE3 | SHA3 | Poseidon | Rp64_256 | RPO_256 | RPX_256 |
| ------------------- | ------ | ------- | --------- | --------- | ------- | ------- |
| Apple M1 Pro | 76 ns | 245 ns | 1.5 µs | 9.1 µs | 5.2 µs | 2.7 µs |
| Apple M2 Max | 71 ns | 233 ns | 1.3 µs | 7.9 µs | 4.6 µs | 2.4 µs |
| Amazon Graviton 3 | 108 ns | | | | 5.3 µs | 3.1 µs |
| AMD Ryzen 9 5950X | 64 ns | 273 ns | 1.2 µs | 9.1 µs | 5.5 µs | |
| AMD EPYC 9R14 | 83 ns | | | | 4.3 µs | 2.4 µs |
| Intel Core i5-8279U | 68 ns | 536 ns | 2.0 µs | 13.6 µs | 8.5 µs | 4.4 µs |
| Intel Xeon 8375C | 67 ns | | | | 8.2 µs | |
#### Scenario 2: Sequential hashing of 100 elements `h([a_0,...,a_99])`
| Function | BLAKE3 | SHA3 | Poseidon | Rp64_256 | RPO_256 |
| ------------------- | -------| ------- | --------- | --------- | ------- |
| Apple M1 Pro | 1.1 us | 1.5 us | 19.4 us | 118 us | 70 us |
| Apple M2 | 1.0 us | 1.5 us | 17.4 us | 103 us | 65 us |
| Amazon Graviton 3 | 1.4 us | | | | 114 us |
| AMD Ryzen 9 5950X | 0.8 us | 1.7 us | 15.7 us | 120 us | 72 us |
| Intel Core i5-8279U | 1.0 us | | | | 116 us |
| Intel Xeon 8375C | 0.8 ns | | | | 110 us |
| Function | BLAKE3 | SHA3 | Poseidon | Rp64_256 | RPO_256 | RPX_256 |
| ------------------- | -------| ------- | --------- | --------- | ------- | ------- |
| Apple M1 Pro | 1.0 µs | 1.5 µs | 19.4 µs | 118 µs | 69 µs | 35 µs |
| Apple M2 Max | 0.9 µs | 1.5 µs | 17.4 µs | 103 µs | 60 µs | 31 µs |
| Amazon Graviton 3 | 1.4 µs | | | | 69 µs | 41 µs |
| AMD Ryzen 9 5950X | 0.8 µs | 1.7 µs | 15.7 µs | 120 µs | 72 µs | |
| AMD EPYC 9R14 | 0.9 µs | | | | 56 µs | 32 µs |
| Intel Core i5-8279U | 0.9 µs | | | | 107 µs | 56 µs |
| Intel Xeon 8375C | 0.8 µs | | | | 110 µs | |
Notes:
- On Graviton 3, RPO256 and RPX256 are run with SVE acceleration enabled.
- On AMD EPYC 9R14, RPO256 and RPX256 are run with AVX2 acceleration enabled.
### Instructions
Before you can run the benchmarks, you'll need to make sure you have Rust [installed](https://www.rust-lang.org/tools/install). After that, to run the benchmarks for RPO and BLAKE3, clone the current repository, and from the root directory of the repo run the following:
@@ -46,4 +53,4 @@ To run the benchmarks for Rescue Prime, Poseidon and SHA3, clone the following [
```
cargo bench hash
```
```

View File

@@ -3,6 +3,7 @@ use miden_crypto::{
hash::{
blake::Blake3_256,
rpo::{Rpo256, RpoDigest},
rpx::{Rpx256, RpxDigest},
},
Felt,
};
@@ -31,7 +32,6 @@ fn rpo256_2to1(c: &mut Criterion) {
fn rpo256_sequential(c: &mut Criterion) {
let v: [Felt; 100] = (0..100)
.into_iter()
.map(Felt::new)
.collect::<Vec<Felt>>()
.try_into()
@@ -44,7 +44,6 @@ fn rpo256_sequential(c: &mut Criterion) {
bench.iter_batched(
|| {
let v: [Felt; 100] = (0..100)
.into_iter()
.map(|_| Felt::new(rand_value()))
.collect::<Vec<Felt>>()
.try_into()
@@ -57,6 +56,52 @@ fn rpo256_sequential(c: &mut Criterion) {
});
}
fn rpx256_2to1(c: &mut Criterion) {
let v: [RpxDigest; 2] = [Rpx256::hash(&[1_u8]), Rpx256::hash(&[2_u8])];
c.bench_function("RPX256 2-to-1 hashing (cached)", |bench| {
bench.iter(|| Rpx256::merge(black_box(&v)))
});
c.bench_function("RPX256 2-to-1 hashing (random)", |bench| {
bench.iter_batched(
|| {
[
Rpx256::hash(&rand_value::<u64>().to_le_bytes()),
Rpx256::hash(&rand_value::<u64>().to_le_bytes()),
]
},
|state| Rpx256::merge(&state),
BatchSize::SmallInput,
)
});
}
fn rpx256_sequential(c: &mut Criterion) {
let v: [Felt; 100] = (0..100)
.map(Felt::new)
.collect::<Vec<Felt>>()
.try_into()
.expect("should not fail");
c.bench_function("RPX256 sequential hashing (cached)", |bench| {
bench.iter(|| Rpx256::hash_elements(black_box(&v)))
});
c.bench_function("RPX256 sequential hashing (random)", |bench| {
bench.iter_batched(
|| {
let v: [Felt; 100] = (0..100)
.map(|_| Felt::new(rand_value()))
.collect::<Vec<Felt>>()
.try_into()
.expect("should not fail");
v
},
|state| Rpx256::hash_elements(&state),
BatchSize::SmallInput,
)
});
}
fn blake3_2to1(c: &mut Criterion) {
let v: [<Blake3_256 as Hasher>::Digest; 2] =
[Blake3_256::hash(&[1_u8]), Blake3_256::hash(&[2_u8])];
@@ -80,7 +125,6 @@ fn blake3_2to1(c: &mut Criterion) {
fn blake3_sequential(c: &mut Criterion) {
let v: [Felt; 100] = (0..100)
.into_iter()
.map(Felt::new)
.collect::<Vec<Felt>>()
.try_into()
@@ -93,7 +137,6 @@ fn blake3_sequential(c: &mut Criterion) {
bench.iter_batched(
|| {
let v: [Felt; 100] = (0..100)
.into_iter()
.map(|_| Felt::new(rand_value()))
.collect::<Vec<Felt>>()
.try_into()
@@ -108,6 +151,8 @@ fn blake3_sequential(c: &mut Criterion) {
criterion_group!(
hash_group,
rpx256_2to1,
rpx256_sequential,
rpo256_2to1,
rpo256_sequential,
blake3_2to1,

View File

@@ -1,16 +1,20 @@
use core::mem::swap;
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use miden_crypto::{merkle::SimpleSmt, Felt, Word};
use miden_crypto::{
merkle::{LeafIndex, SimpleSmt},
Felt, Word,
};
use rand_utils::prng_array;
use seq_macro::seq;
fn smt_rpo(c: &mut Criterion) {
// setup trees
let mut seed = [0u8; 32];
let mut trees = vec![];
let leaf = generate_word(&mut seed);
for depth in 14..=20 {
let leaves = ((1 << depth) - 1) as u64;
seq!(DEPTH in 14..=20 {
let leaves = ((1 << DEPTH) - 1) as u64;
for count in [1, leaves / 2, leaves] {
let entries: Vec<_> = (0..count)
.map(|i| {
@@ -18,52 +22,45 @@ fn smt_rpo(c: &mut Criterion) {
(i, word)
})
.collect();
let tree = SimpleSmt::new(entries, depth).unwrap();
trees.push(tree);
let mut tree = SimpleSmt::<DEPTH>::with_leaves(entries).unwrap();
// benchmark 1
let mut insert = c.benchmark_group("smt update_leaf".to_string());
{
let depth = DEPTH;
let key = count >> 2;
insert.bench_with_input(
format!("simple smt(depth:{depth},count:{count})"),
&(key, leaf),
|b, (key, leaf)| {
b.iter(|| {
tree.insert(black_box(LeafIndex::<DEPTH>::new(*key).unwrap()), black_box(*leaf));
});
},
);
}
insert.finish();
// benchmark 2
let mut path = c.benchmark_group("smt get_leaf_path".to_string());
{
let depth = DEPTH;
let key = count >> 2;
path.bench_with_input(
format!("simple smt(depth:{depth},count:{count})"),
&key,
|b, key| {
b.iter(|| {
tree.open(black_box(&LeafIndex::<DEPTH>::new(*key).unwrap()));
});
},
);
}
path.finish();
}
}
let leaf = generate_word(&mut seed);
// benchmarks
let mut insert = c.benchmark_group(format!("smt update_leaf"));
for tree in trees.iter_mut() {
let depth = tree.depth();
let count = tree.leaves_count() as u64;
let key = count >> 2;
insert.bench_with_input(
format!("simple smt(depth:{depth},count:{count})"),
&(key, leaf),
|b, (key, leaf)| {
b.iter(|| {
tree.update_leaf(black_box(*key), black_box(*leaf)).unwrap();
});
},
);
}
insert.finish();
let mut path = c.benchmark_group(format!("smt get_leaf_path"));
for tree in trees.iter_mut() {
let depth = tree.depth();
let count = tree.leaves_count() as u64;
let key = count >> 2;
path.bench_with_input(
format!("simple smt(depth:{depth},count:{count})"),
&key,
|b, key| {
b.iter(|| {
tree.get_leaf_path(black_box(*key)).unwrap();
});
},
);
}
path.finish();
});
}
criterion_group!(smt_group, smt_rpo);
@@ -75,10 +72,5 @@ criterion_main!(smt_group);
fn generate_word(seed: &mut [u8; 32]) -> Word {
swap(seed, &mut prng_array(*seed));
let nums: [u64; 4] = prng_array(*seed);
[
Felt::new(nums[0]),
Felt::new(nums[1]),
Felt::new(nums[2]),
Felt::new(nums[3]),
]
[Felt::new(nums[0]), Felt::new(nums[1]), Felt::new(nums[2]), Felt::new(nums[3])]
}

484
benches/store.rs Normal file
View File

@@ -0,0 +1,484 @@
use criterion::{black_box, criterion_group, criterion_main, BatchSize, BenchmarkId, Criterion};
use miden_crypto::merkle::{
DefaultMerkleStore as MerkleStore, LeafIndex, MerkleTree, NodeIndex, SimpleSmt, SMT_MAX_DEPTH,
};
use miden_crypto::Word;
use miden_crypto::{hash::rpo::RpoDigest, Felt};
use rand_utils::{rand_array, rand_value};
/// Since MerkleTree can only be created when a power-of-two number of elements is used, the sample
/// sizes are limited to that.
static BATCH_SIZES: [usize; 3] = [2usize.pow(4), 2usize.pow(7), 2usize.pow(10)];
/// Generates a random `RpoDigest`.
fn random_rpo_digest() -> RpoDigest {
rand_array::<Felt, 4>().into()
}
/// Generates a random `Word`.
fn random_word() -> Word {
rand_array::<Felt, 4>()
}
/// Generates an index at the specified depth in `0..range`.
fn random_index(range: u64, depth: u8) -> NodeIndex {
let value = rand_value::<u64>() % range;
NodeIndex::new(depth, value).unwrap()
}
/// Benchmarks getting an empty leaf from the SMT and MerkleStore backends.
fn get_empty_leaf_simplesmt(c: &mut Criterion) {
let mut group = c.benchmark_group("get_empty_leaf_simplesmt");
const DEPTH: u8 = SMT_MAX_DEPTH;
let size = u64::MAX;
// both SMT and the store are pre-populated with empty hashes, accessing these values is what is
// being benchmarked here, so no values are inserted into the backends
let smt = SimpleSmt::<DEPTH>::new().unwrap();
let store = MerkleStore::from(&smt);
let root = smt.root();
group.bench_function(BenchmarkId::new("SimpleSmt", DEPTH), |b| {
b.iter_batched(
|| random_index(size, DEPTH),
|index| black_box(smt.get_node(index)),
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore", DEPTH), |b| {
b.iter_batched(
|| random_index(size, DEPTH),
|index| black_box(store.get_node(root, index)),
BatchSize::SmallInput,
)
});
}
/// Benchmarks getting a leaf on Merkle trees and Merkle stores of varying power-of-two sizes.
fn get_leaf_merkletree(c: &mut Criterion) {
let mut group = c.benchmark_group("get_leaf_merkletree");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
let mtree_leaves: Vec<Word> = leaves.iter().map(|v| v.into()).collect();
let mtree = MerkleTree::new(mtree_leaves.clone()).unwrap();
let store = MerkleStore::from(&mtree);
let depth = mtree.depth();
let root = mtree.root();
let size_u64 = size as u64;
group.bench_function(BenchmarkId::new("MerkleTree", size), |b| {
b.iter_batched(
|| random_index(size_u64, depth),
|index| black_box(mtree.get_node(index)),
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore", size), |b| {
b.iter_batched(
|| random_index(size_u64, depth),
|index| black_box(store.get_node(root, index)),
BatchSize::SmallInput,
)
});
}
}
/// Benchmarks getting a leaf on SMT and Merkle stores of varying power-of-two sizes.
fn get_leaf_simplesmt(c: &mut Criterion) {
let mut group = c.benchmark_group("get_leaf_simplesmt");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
let smt_leaves = leaves
.iter()
.enumerate()
.map(|(c, v)| (c.try_into().unwrap(), v.into()))
.collect::<Vec<(u64, Word)>>();
let smt = SimpleSmt::<SMT_MAX_DEPTH>::with_leaves(smt_leaves.clone()).unwrap();
let store = MerkleStore::from(&smt);
let root = smt.root();
let size_u64 = size as u64;
group.bench_function(BenchmarkId::new("SimpleSmt", size), |b| {
b.iter_batched(
|| random_index(size_u64, SMT_MAX_DEPTH),
|index| black_box(smt.get_node(index)),
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore", size), |b| {
b.iter_batched(
|| random_index(size_u64, SMT_MAX_DEPTH),
|index| black_box(store.get_node(root, index)),
BatchSize::SmallInput,
)
});
}
}
/// Benchmarks getting a node at half of the depth of an empty SMT and an empty Merkle store.
fn get_node_of_empty_simplesmt(c: &mut Criterion) {
let mut group = c.benchmark_group("get_node_of_empty_simplesmt");
const DEPTH: u8 = SMT_MAX_DEPTH;
// both SMT and the store are pre-populated with the empty hashes, accessing the internal nodes
// of these values is what is being benchmarked here, so no values are inserted into the
// backends.
let smt = SimpleSmt::<DEPTH>::new().unwrap();
let store = MerkleStore::from(&smt);
let root = smt.root();
let half_depth = DEPTH / 2;
let half_size = 2_u64.pow(half_depth as u32);
group.bench_function(BenchmarkId::new("SimpleSmt", DEPTH), |b| {
b.iter_batched(
|| random_index(half_size, half_depth),
|index| black_box(smt.get_node(index)),
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore", DEPTH), |b| {
b.iter_batched(
|| random_index(half_size, half_depth),
|index| black_box(store.get_node(root, index)),
BatchSize::SmallInput,
)
});
}
/// Benchmarks getting a node at half of the depth of a Merkle tree and Merkle store of varying
/// power-of-two sizes.
fn get_node_merkletree(c: &mut Criterion) {
let mut group = c.benchmark_group("get_node_merkletree");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
let mtree_leaves: Vec<Word> = leaves.iter().map(|v| v.into()).collect();
let mtree = MerkleTree::new(mtree_leaves.clone()).unwrap();
let store = MerkleStore::from(&mtree);
let root = mtree.root();
let half_depth = mtree.depth() / 2;
let half_size = 2_u64.pow(half_depth as u32);
group.bench_function(BenchmarkId::new("MerkleTree", size), |b| {
b.iter_batched(
|| random_index(half_size, half_depth),
|index| black_box(mtree.get_node(index)),
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore", size), |b| {
b.iter_batched(
|| random_index(half_size, half_depth),
|index| black_box(store.get_node(root, index)),
BatchSize::SmallInput,
)
});
}
}
/// Benchmarks getting a node at half the depth on SMT and Merkle stores of varying power-of-two
/// sizes.
fn get_node_simplesmt(c: &mut Criterion) {
let mut group = c.benchmark_group("get_node_simplesmt");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
let smt_leaves = leaves
.iter()
.enumerate()
.map(|(c, v)| (c.try_into().unwrap(), v.into()))
.collect::<Vec<(u64, Word)>>();
let smt = SimpleSmt::<SMT_MAX_DEPTH>::with_leaves(smt_leaves.clone()).unwrap();
let store = MerkleStore::from(&smt);
let root = smt.root();
let half_depth = SMT_MAX_DEPTH / 2;
let half_size = 2_u64.pow(half_depth as u32);
group.bench_function(BenchmarkId::new("SimpleSmt", size), |b| {
b.iter_batched(
|| random_index(half_size, half_depth),
|index| black_box(smt.get_node(index)),
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore", size), |b| {
b.iter_batched(
|| random_index(half_size, half_depth),
|index| black_box(store.get_node(root, index)),
BatchSize::SmallInput,
)
});
}
}
/// Benchmarks getting a path of a leaf on the Merkle tree and Merkle store backends.
fn get_leaf_path_merkletree(c: &mut Criterion) {
let mut group = c.benchmark_group("get_leaf_path_merkletree");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
let mtree_leaves: Vec<Word> = leaves.iter().map(|v| v.into()).collect();
let mtree = MerkleTree::new(mtree_leaves.clone()).unwrap();
let store = MerkleStore::from(&mtree);
let depth = mtree.depth();
let root = mtree.root();
let size_u64 = size as u64;
group.bench_function(BenchmarkId::new("MerkleTree", size), |b| {
b.iter_batched(
|| random_index(size_u64, depth),
|index| black_box(mtree.get_path(index)),
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore", size), |b| {
b.iter_batched(
|| random_index(size_u64, depth),
|index| black_box(store.get_path(root, index)),
BatchSize::SmallInput,
)
});
}
}
/// Benchmarks getting a path of a leaf on the SMT and Merkle store backends.
fn get_leaf_path_simplesmt(c: &mut Criterion) {
let mut group = c.benchmark_group("get_leaf_path_simplesmt");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
let smt_leaves = leaves
.iter()
.enumerate()
.map(|(c, v)| (c.try_into().unwrap(), v.into()))
.collect::<Vec<(u64, Word)>>();
let smt = SimpleSmt::<SMT_MAX_DEPTH>::with_leaves(smt_leaves.clone()).unwrap();
let store = MerkleStore::from(&smt);
let root = smt.root();
let size_u64 = size as u64;
group.bench_function(BenchmarkId::new("SimpleSmt", size), |b| {
b.iter_batched(
|| random_index(size_u64, SMT_MAX_DEPTH),
|index| {
black_box(smt.open(&LeafIndex::<SMT_MAX_DEPTH>::new(index.value()).unwrap()))
},
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore", size), |b| {
b.iter_batched(
|| random_index(size_u64, SMT_MAX_DEPTH),
|index| black_box(store.get_path(root, index)),
BatchSize::SmallInput,
)
});
}
}
/// Benchmarks creation of the different storage backends
fn new(c: &mut Criterion) {
let mut group = c.benchmark_group("new");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
// MerkleTree constructor is optimized to work with vectors. Create a new copy of the data
// and pass it to the benchmark function
group.bench_function(BenchmarkId::new("MerkleTree::new", size), |b| {
b.iter_batched(
|| leaves.iter().map(|v| v.into()).collect::<Vec<Word>>(),
|l| black_box(MerkleTree::new(l)),
BatchSize::SmallInput,
)
});
// This could be done with `bench_with_input`, however to remove variables while comparing
// with MerkleTree it is using `iter_batched`
group.bench_function(BenchmarkId::new("MerkleStore::extend::MerkleTree", size), |b| {
b.iter_batched(
|| leaves.iter().map(|v| v.into()).collect::<Vec<Word>>(),
|l| {
let mtree = MerkleTree::new(l).unwrap();
black_box(MerkleStore::from(&mtree));
},
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("SimpleSmt::new", size), |b| {
b.iter_batched(
|| {
leaves
.iter()
.enumerate()
.map(|(c, v)| (c.try_into().unwrap(), v.into()))
.collect::<Vec<(u64, Word)>>()
},
|l| black_box(SimpleSmt::<SMT_MAX_DEPTH>::with_leaves(l)),
BatchSize::SmallInput,
)
});
group.bench_function(BenchmarkId::new("MerkleStore::extend::SimpleSmt", size), |b| {
b.iter_batched(
|| {
leaves
.iter()
.enumerate()
.map(|(c, v)| (c.try_into().unwrap(), v.into()))
.collect::<Vec<(u64, Word)>>()
},
|l| {
let smt = SimpleSmt::<SMT_MAX_DEPTH>::with_leaves(l).unwrap();
black_box(MerkleStore::from(&smt));
},
BatchSize::SmallInput,
)
});
}
}
/// Benchmarks updating a leaf on MerkleTree and MerkleStore backends.
fn update_leaf_merkletree(c: &mut Criterion) {
let mut group = c.benchmark_group("update_leaf_merkletree");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
let mtree_leaves: Vec<Word> = leaves.iter().map(|v| v.into()).collect();
let mut mtree = MerkleTree::new(mtree_leaves.clone()).unwrap();
let mut store = MerkleStore::from(&mtree);
let depth = mtree.depth();
let root = mtree.root();
let size_u64 = size as u64;
group.bench_function(BenchmarkId::new("MerkleTree", size), |b| {
b.iter_batched(
|| (rand_value::<u64>() % size_u64, random_word()),
|(index, value)| black_box(mtree.update_leaf(index, value)),
BatchSize::SmallInput,
)
});
let mut store_root = root;
group.bench_function(BenchmarkId::new("MerkleStore", size), |b| {
b.iter_batched(
|| (random_index(size_u64, depth), random_word()),
|(index, value)| {
// The MerkleTree automatically updates its internal root, the Store maintains
// the old root and adds the new one. Here we update the root to have a fair
// comparison
store_root = store.set_node(root, index, value.into()).unwrap().root;
black_box(store_root)
},
BatchSize::SmallInput,
)
});
}
}
/// Benchmarks updating a leaf on SMT and MerkleStore backends.
fn update_leaf_simplesmt(c: &mut Criterion) {
let mut group = c.benchmark_group("update_leaf_simplesmt");
let random_data_size = BATCH_SIZES.into_iter().max().unwrap();
let random_data: Vec<RpoDigest> = (0..random_data_size).map(|_| random_rpo_digest()).collect();
for size in BATCH_SIZES {
let leaves = &random_data[..size];
let smt_leaves = leaves
.iter()
.enumerate()
.map(|(c, v)| (c.try_into().unwrap(), v.into()))
.collect::<Vec<(u64, Word)>>();
let mut smt = SimpleSmt::<SMT_MAX_DEPTH>::with_leaves(smt_leaves.clone()).unwrap();
let mut store = MerkleStore::from(&smt);
let root = smt.root();
let size_u64 = size as u64;
group.bench_function(BenchmarkId::new("SimpleSMT", size), |b| {
b.iter_batched(
|| (rand_value::<u64>() % size_u64, random_word()),
|(index, value)| {
black_box(smt.insert(LeafIndex::<SMT_MAX_DEPTH>::new(index).unwrap(), value))
},
BatchSize::SmallInput,
)
});
let mut store_root = root;
group.bench_function(BenchmarkId::new("MerkleStore", size), |b| {
b.iter_batched(
|| (random_index(size_u64, SMT_MAX_DEPTH), random_word()),
|(index, value)| {
// The MerkleTree automatically updates its internal root, the Store maintains
// the old root and adds the new one. Here we update the root to have a fair
// comparison
store_root = store.set_node(root, index, value.into()).unwrap().root;
black_box(store_root)
},
BatchSize::SmallInput,
)
});
}
}
criterion_group!(
store_group,
get_empty_leaf_simplesmt,
get_leaf_merkletree,
get_leaf_path_merkletree,
get_leaf_path_simplesmt,
get_leaf_simplesmt,
get_node_merkletree,
get_node_of_empty_simplesmt,
get_node_simplesmt,
new,
update_leaf_merkletree,
update_leaf_simplesmt,
);
criterion_main!(store_group);

50
build.rs Normal file
View File

@@ -0,0 +1,50 @@
fn main() {
#[cfg(feature = "std")]
compile_rpo_falcon();
#[cfg(target_feature = "sve")]
compile_arch_arm64_sve();
}
#[cfg(feature = "std")]
fn compile_rpo_falcon() {
use std::path::PathBuf;
const RPO_FALCON_PATH: &str = "src/dsa/rpo_falcon512/falcon_c";
println!("cargo:rerun-if-changed={RPO_FALCON_PATH}/falcon.h");
println!("cargo:rerun-if-changed={RPO_FALCON_PATH}/falcon.c");
println!("cargo:rerun-if-changed={RPO_FALCON_PATH}/rpo.h");
println!("cargo:rerun-if-changed={RPO_FALCON_PATH}/rpo.c");
let target_dir: PathBuf = ["PQClean", "crypto_sign", "falcon-512", "clean"].iter().collect();
let common_dir: PathBuf = ["PQClean", "common"].iter().collect();
let scheme_files = glob::glob(target_dir.join("*.c").to_str().unwrap()).unwrap();
let common_files = glob::glob(common_dir.join("*.c").to_str().unwrap()).unwrap();
cc::Build::new()
.include(&common_dir)
.include(target_dir)
.files(scheme_files.into_iter().map(|p| p.unwrap().to_string_lossy().into_owned()))
.files(common_files.into_iter().map(|p| p.unwrap().to_string_lossy().into_owned()))
.file(format!("{RPO_FALCON_PATH}/falcon.c"))
.file(format!("{RPO_FALCON_PATH}/rpo.c"))
.flag("-O3")
.compile("rpo_falcon512");
}
#[cfg(target_feature = "sve")]
fn compile_arch_arm64_sve() {
const RPO_SVE_PATH: &str = "arch/arm64-sve/rpo";
println!("cargo:rerun-if-changed={RPO_SVE_PATH}/library.c");
println!("cargo:rerun-if-changed={RPO_SVE_PATH}/library.h");
println!("cargo:rerun-if-changed={RPO_SVE_PATH}/rpo_hash.h");
cc::Build::new()
.file(format!("{RPO_SVE_PATH}/library.c"))
.flag("-march=armv8-a+sve")
.flag("-O3")
.compile("rpo_sve");
}

21
rustfmt.toml Normal file
View File

@@ -0,0 +1,21 @@
edition = "2021"
array_width = 80
attr_fn_like_width = 80
chain_width = 80
#condense_wildcard_suffixes = true
#enum_discrim_align_threshold = 40
fn_call_width = 80
#fn_single_line = true
#format_code_in_doc_comments = true
#format_macro_matchers = true
#format_strings = true
#group_imports = "StdExternalCrate"
#hex_literal_case = "Lower"
#imports_granularity = "Crate"
newline_style = "Unix"
#normalize_doc_attributes = true
#reorder_impl_items = true
single_line_if_else_max_width = 60
struct_lit_width = 40
use_field_init_shorthand = true
use_try_shorthand = true

3
src/dsa/mod.rs Normal file
View File

@@ -0,0 +1,3 @@
//! Digital signature schemes supported by default in the Miden VM.
pub mod rpo_falcon512;

View File

@@ -0,0 +1,55 @@
use super::{LOG_N, MODULUS, PK_LEN};
use core::fmt;
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum FalconError {
KeyGenerationFailed,
PubKeyDecodingExtraData,
PubKeyDecodingInvalidCoefficient(u32),
PubKeyDecodingInvalidLength(usize),
PubKeyDecodingInvalidTag(u8),
SigDecodingTooBigHighBits(u32),
SigDecodingInvalidRemainder,
SigDecodingNonZeroUnusedBitsLastByte,
SigDecodingMinusZero,
SigDecodingIncorrectEncodingAlgorithm,
SigDecodingNotSupportedDegree(u8),
SigGenerationFailed,
}
impl fmt::Display for FalconError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
use FalconError::*;
match self {
KeyGenerationFailed => write!(f, "Failed to generate a private-public key pair"),
PubKeyDecodingExtraData => {
write!(f, "Failed to decode public key: input not fully consumed")
}
PubKeyDecodingInvalidCoefficient(val) => {
write!(f, "Failed to decode public key: coefficient {val} is greater than or equal to the field modulus {MODULUS}")
}
PubKeyDecodingInvalidLength(len) => {
write!(f, "Failed to decode public key: expected {PK_LEN} bytes but received {len}")
}
PubKeyDecodingInvalidTag(byte) => {
write!(f, "Failed to decode public key: expected the first byte to be {LOG_N} but was {byte}")
}
SigDecodingTooBigHighBits(m) => {
write!(f, "Failed to decode signature: high bits {m} exceed 2048")
}
SigDecodingInvalidRemainder => {
write!(f, "Failed to decode signature: incorrect remaining data")
}
SigDecodingNonZeroUnusedBitsLastByte => {
write!(f, "Failed to decode signature: Non-zero unused bits in the last byte")
}
SigDecodingMinusZero => write!(f, "Failed to decode signature: -0 is forbidden"),
SigDecodingIncorrectEncodingAlgorithm => write!(f, "Failed to decode signature: not supported encoding algorithm"),
SigDecodingNotSupportedDegree(log_n) => write!(f, "Failed to decode signature: only supported irreducible polynomial degree is 512, 2^{log_n} was provided"),
SigGenerationFailed => write!(f, "Failed to generate a signature"),
}
}
}
#[cfg(feature = "std")]
impl std::error::Error for FalconError {}

View File

@@ -0,0 +1,402 @@
/*
* Wrapper for implementing the PQClean API.
*/
#include <string.h>
#include "randombytes.h"
#include "falcon.h"
#include "inner.h"
#include "rpo.h"
#define NONCELEN 40
/*
* Encoding formats (nnnn = log of degree, 9 for Falcon-512, 10 for Falcon-1024)
*
* private key:
* header byte: 0101nnnn
* private f (6 or 5 bits by element, depending on degree)
* private g (6 or 5 bits by element, depending on degree)
* private F (8 bits by element)
*
* public key:
* header byte: 0000nnnn
* public h (14 bits by element)
*
* signature:
* header byte: 0011nnnn
* nonce 40 bytes
* value (12 bits by element)
*
* message + signature:
* signature length (2 bytes, big-endian)
* nonce 40 bytes
* message
* header byte: 0010nnnn
* value (12 bits by element)
* (signature length is 1+len(value), not counting the nonce)
*/
/* see falcon.h */
int PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_from_seed_rpo(
uint8_t *pk,
uint8_t *sk,
unsigned char *seed
) {
union
{
uint8_t b[FALCON_KEYGEN_TEMP_9];
uint64_t dummy_u64;
fpr dummy_fpr;
} tmp;
int8_t f[512], g[512], F[512];
uint16_t h[512];
inner_shake256_context rng;
size_t u, v;
/*
* Generate key pair.
*/
inner_shake256_init(&rng);
inner_shake256_inject(&rng, seed, sizeof seed);
inner_shake256_flip(&rng);
PQCLEAN_FALCON512_CLEAN_keygen(&rng, f, g, F, NULL, h, 9, tmp.b);
inner_shake256_ctx_release(&rng);
/*
* Encode private key.
*/
sk[0] = 0x50 + 9;
u = 1;
v = PQCLEAN_FALCON512_CLEAN_trim_i8_encode(
sk + u, PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES - u,
f, 9, PQCLEAN_FALCON512_CLEAN_max_fg_bits[9]);
if (v == 0)
{
return -1;
}
u += v;
v = PQCLEAN_FALCON512_CLEAN_trim_i8_encode(
sk + u, PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES - u,
g, 9, PQCLEAN_FALCON512_CLEAN_max_fg_bits[9]);
if (v == 0)
{
return -1;
}
u += v;
v = PQCLEAN_FALCON512_CLEAN_trim_i8_encode(
sk + u, PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES - u,
F, 9, PQCLEAN_FALCON512_CLEAN_max_FG_bits[9]);
if (v == 0)
{
return -1;
}
u += v;
if (u != PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES)
{
return -1;
}
/*
* Encode public key.
*/
pk[0] = 0x00 + 9;
v = PQCLEAN_FALCON512_CLEAN_modq_encode(
pk + 1, PQCLEAN_FALCON512_CLEAN_CRYPTO_PUBLICKEYBYTES - 1,
h, 9);
if (v != PQCLEAN_FALCON512_CLEAN_CRYPTO_PUBLICKEYBYTES - 1)
{
return -1;
}
return 0;
}
int PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_rpo(
uint8_t *pk,
uint8_t *sk
) {
unsigned char seed[48];
/*
* Generate a random seed.
*/
randombytes(seed, sizeof seed);
return PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_from_seed_rpo(pk, sk, seed);
}
/*
* Compute the signature. nonce[] receives the nonce and must have length
* NONCELEN bytes. sigbuf[] receives the signature value (without nonce
* or header byte), with *sigbuflen providing the maximum value length and
* receiving the actual value length.
*
* If a signature could be computed but not encoded because it would
* exceed the output buffer size, then a new signature is computed. If
* the provided buffer size is too low, this could loop indefinitely, so
* the caller must provide a size that can accommodate signatures with a
* large enough probability.
*
* Return value: 0 on success, -1 on error.
*/
static int do_sign(
uint8_t *nonce,
uint8_t *sigbuf,
size_t *sigbuflen,
const uint8_t *m,
size_t mlen,
const uint8_t *sk
) {
union
{
uint8_t b[72 * 512];
uint64_t dummy_u64;
fpr dummy_fpr;
} tmp;
int8_t f[512], g[512], F[512], G[512];
struct
{
int16_t sig[512];
uint16_t hm[512];
} r;
unsigned char seed[48];
inner_shake256_context sc;
rpo128_context rc;
size_t u, v;
/*
* Decode the private key.
*/
if (sk[0] != 0x50 + 9)
{
return -1;
}
u = 1;
v = PQCLEAN_FALCON512_CLEAN_trim_i8_decode(
f, 9, PQCLEAN_FALCON512_CLEAN_max_fg_bits[9],
sk + u, PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES - u);
if (v == 0)
{
return -1;
}
u += v;
v = PQCLEAN_FALCON512_CLEAN_trim_i8_decode(
g, 9, PQCLEAN_FALCON512_CLEAN_max_fg_bits[9],
sk + u, PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES - u);
if (v == 0)
{
return -1;
}
u += v;
v = PQCLEAN_FALCON512_CLEAN_trim_i8_decode(
F, 9, PQCLEAN_FALCON512_CLEAN_max_FG_bits[9],
sk + u, PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES - u);
if (v == 0)
{
return -1;
}
u += v;
if (u != PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES)
{
return -1;
}
if (!PQCLEAN_FALCON512_CLEAN_complete_private(G, f, g, F, 9, tmp.b))
{
return -1;
}
/*
* Create a random nonce (40 bytes).
*/
randombytes(nonce, NONCELEN);
/* ==== Start: Deviation from the reference implementation ================================= */
// Transform the nonce into 8 chunks each of size 5 bytes. We do this in order to be sure that
// the conversion to field elements succeeds
uint8_t buffer[64];
memset(buffer, 0, 64);
for (size_t i = 0; i < 8; i++)
{
buffer[8 * i] = nonce[5 * i];
buffer[8 * i + 1] = nonce[5 * i + 1];
buffer[8 * i + 2] = nonce[5 * i + 2];
buffer[8 * i + 3] = nonce[5 * i + 3];
buffer[8 * i + 4] = nonce[5 * i + 4];
}
/*
* Hash message nonce + message into a vector.
*/
rpo128_init(&rc);
rpo128_absorb(&rc, buffer, NONCELEN + 24);
rpo128_absorb(&rc, m, mlen);
rpo128_finalize(&rc);
PQCLEAN_FALCON512_CLEAN_hash_to_point_rpo(&rc, r.hm, 9);
rpo128_release(&rc);
/* ==== End: Deviation from the reference implementation =================================== */
/*
* Initialize a RNG.
*/
randombytes(seed, sizeof seed);
inner_shake256_init(&sc);
inner_shake256_inject(&sc, seed, sizeof seed);
inner_shake256_flip(&sc);
/*
* Compute and return the signature. This loops until a signature
* value is found that fits in the provided buffer.
*/
for (;;)
{
PQCLEAN_FALCON512_CLEAN_sign_dyn(r.sig, &sc, f, g, F, G, r.hm, 9, tmp.b);
v = PQCLEAN_FALCON512_CLEAN_comp_encode(sigbuf, *sigbuflen, r.sig, 9);
if (v != 0)
{
inner_shake256_ctx_release(&sc);
*sigbuflen = v;
return 0;
}
}
}
/*
* Verify a signature. The nonce has size NONCELEN bytes. sigbuf[]
* (of size sigbuflen) contains the signature value, not including the
* header byte or nonce. Return value is 0 on success, -1 on error.
*/
static int do_verify(
const uint8_t *nonce,
const uint8_t *sigbuf,
size_t sigbuflen,
const uint8_t *m,
size_t mlen,
const uint8_t *pk
) {
union
{
uint8_t b[2 * 512];
uint64_t dummy_u64;
fpr dummy_fpr;
} tmp;
uint16_t h[512], hm[512];
int16_t sig[512];
rpo128_context rc;
/*
* Decode public key.
*/
if (pk[0] != 0x00 + 9)
{
return -1;
}
if (PQCLEAN_FALCON512_CLEAN_modq_decode(h, 9,
pk + 1, PQCLEAN_FALCON512_CLEAN_CRYPTO_PUBLICKEYBYTES - 1)
!= PQCLEAN_FALCON512_CLEAN_CRYPTO_PUBLICKEYBYTES - 1)
{
return -1;
}
PQCLEAN_FALCON512_CLEAN_to_ntt_monty(h, 9);
/*
* Decode signature.
*/
if (sigbuflen == 0)
{
return -1;
}
if (PQCLEAN_FALCON512_CLEAN_comp_decode(sig, 9, sigbuf, sigbuflen) != sigbuflen)
{
return -1;
}
/* ==== Start: Deviation from the reference implementation ================================= */
/*
* Hash nonce + message into a vector.
*/
// Transform the nonce into 8 chunks each of size 5 bytes. We do this in order to be sure that
// the conversion to field elements succeeds
uint8_t buffer[64];
memset(buffer, 0, 64);
for (size_t i = 0; i < 8; i++)
{
buffer[8 * i] = nonce[5 * i];
buffer[8 * i + 1] = nonce[5 * i + 1];
buffer[8 * i + 2] = nonce[5 * i + 2];
buffer[8 * i + 3] = nonce[5 * i + 3];
buffer[8 * i + 4] = nonce[5 * i + 4];
}
rpo128_init(&rc);
rpo128_absorb(&rc, buffer, NONCELEN + 24);
rpo128_absorb(&rc, m, mlen);
rpo128_finalize(&rc);
PQCLEAN_FALCON512_CLEAN_hash_to_point_rpo(&rc, hm, 9);
rpo128_release(&rc);
/* === End: Deviation from the reference implementation ==================================== */
/*
* Verify signature.
*/
if (!PQCLEAN_FALCON512_CLEAN_verify_raw(hm, sig, h, 9, tmp.b))
{
return -1;
}
return 0;
}
/* see falcon.h */
int PQCLEAN_FALCON512_CLEAN_crypto_sign_signature_rpo(
uint8_t *sig,
size_t *siglen,
const uint8_t *m,
size_t mlen,
const uint8_t *sk
) {
/*
* The PQCLEAN_FALCON512_CLEAN_CRYPTO_BYTES constant is used for
* the signed message object (as produced by crypto_sign())
* and includes a two-byte length value, so we take care here
* to only generate signatures that are two bytes shorter than
* the maximum. This is done to ensure that crypto_sign()
* and crypto_sign_signature() produce the exact same signature
* value, if used on the same message, with the same private key,
* and using the same output from randombytes() (this is for
* reproducibility of tests).
*/
size_t vlen;
vlen = PQCLEAN_FALCON512_CLEAN_CRYPTO_BYTES - NONCELEN - 3;
if (do_sign(sig + 1, sig + 1 + NONCELEN, &vlen, m, mlen, sk) < 0)
{
return -1;
}
sig[0] = 0x30 + 9;
*siglen = 1 + NONCELEN + vlen;
return 0;
}
/* see falcon.h */
int PQCLEAN_FALCON512_CLEAN_crypto_sign_verify_rpo(
const uint8_t *sig,
size_t siglen,
const uint8_t *m,
size_t mlen,
const uint8_t *pk
) {
if (siglen < 1 + NONCELEN)
{
return -1;
}
if (sig[0] != 0x30 + 9)
{
return -1;
}
return do_verify(sig + 1, sig + 1 + NONCELEN, siglen - 1 - NONCELEN, m, mlen, pk);
}

View File

@@ -0,0 +1,66 @@
#include <stddef.h>
#include <stdint.h>
#define PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES 1281
#define PQCLEAN_FALCON512_CLEAN_CRYPTO_PUBLICKEYBYTES 897
#define PQCLEAN_FALCON512_CLEAN_CRYPTO_BYTES 666
/*
* Generate a new key pair. Public key goes into pk[], private key in sk[].
* Key sizes are exact (in bytes):
* public (pk): PQCLEAN_FALCON512_CLEAN_CRYPTO_PUBLICKEYBYTES
* private (sk): PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES
*
* Return value: 0 on success, -1 on error.
*
* Note: This implementation follows the reference implementation in PQClean
* https://github.com/PQClean/PQClean/tree/master/crypto_sign/falcon-512
* verbatim except for the sections that are marked otherwise.
*/
int PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_rpo(
uint8_t *pk, uint8_t *sk);
/*
* Generate a new key pair from seed. Public key goes into pk[], private key in sk[].
* Key sizes are exact (in bytes):
* public (pk): PQCLEAN_FALCON512_CLEAN_CRYPTO_PUBLICKEYBYTES
* private (sk): PQCLEAN_FALCON512_CLEAN_CRYPTO_SECRETKEYBYTES
*
* Return value: 0 on success, -1 on error.
*/
int PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_from_seed_rpo(
uint8_t *pk, uint8_t *sk, unsigned char *seed);
/*
* Compute a signature on a provided message (m, mlen), with a given
* private key (sk). Signature is written in sig[], with length written
* into *siglen. Signature length is variable; maximum signature length
* (in bytes) is PQCLEAN_FALCON512_CLEAN_CRYPTO_BYTES.
*
* sig[], m[] and sk[] may overlap each other arbitrarily.
*
* Return value: 0 on success, -1 on error.
*
* Note: This implementation follows the reference implementation in PQClean
* https://github.com/PQClean/PQClean/tree/master/crypto_sign/falcon-512
* verbatim except for the sections that are marked otherwise.
*/
int PQCLEAN_FALCON512_CLEAN_crypto_sign_signature_rpo(
uint8_t *sig, size_t *siglen,
const uint8_t *m, size_t mlen, const uint8_t *sk);
/*
* Verify a signature (sig, siglen) on a message (m, mlen) with a given
* public key (pk).
*
* sig[], m[] and pk[] may overlap each other arbitrarily.
*
* Return value: 0 on success, -1 on error.
*
* Note: This implementation follows the reference implementation in PQClean
* https://github.com/PQClean/PQClean/tree/master/crypto_sign/falcon-512
* verbatim except for the sections that are marked otherwise.
*/
int PQCLEAN_FALCON512_CLEAN_crypto_sign_verify_rpo(
const uint8_t *sig, size_t siglen,
const uint8_t *m, size_t mlen, const uint8_t *pk);

View File

@@ -0,0 +1,582 @@
/*
* RPO implementation.
*/
#include <stdint.h>
#include <string.h>
#include <stdlib.h>
/* ================================================================================================
* Modular Arithmetic
*/
#define P 0xFFFFFFFF00000001
#define M 12289
// From https://github.com/ncw/iprime/blob/master/mod_math_noasm.go
static uint64_t add_mod_p(uint64_t a, uint64_t b)
{
a = P - a;
uint64_t res = b - a;
if (b < a)
res += P;
return res;
}
static uint64_t sub_mod_p(uint64_t a, uint64_t b)
{
uint64_t r = a - b;
if (a < b)
r += P;
return r;
}
static uint64_t reduce_mod_p(uint64_t b, uint64_t a)
{
uint32_t d = b >> 32,
c = b;
if (a >= P)
a -= P;
a = sub_mod_p(a, c);
a = sub_mod_p(a, d);
a = add_mod_p(a, ((uint64_t)c) << 32);
return a;
}
static uint64_t mult_mod_p(uint64_t x, uint64_t y)
{
uint32_t a = x,
b = x >> 32,
c = y,
d = y >> 32;
/* first synthesize the product using 32*32 -> 64 bit multiplies */
x = b * (uint64_t)c; /* b*c */
y = a * (uint64_t)d; /* a*d */
uint64_t e = a * (uint64_t)c, /* a*c */
f = b * (uint64_t)d, /* b*d */
t;
x += y; /* b*c + a*d */
/* carry? */
if (x < y)
f += 1LL << 32; /* carry into upper 32 bits - can't overflow */
t = x << 32;
e += t; /* a*c + LSW(b*c + a*d) */
/* carry? */
if (e < t)
f += 1; /* carry into upper 64 bits - can't overflow*/
t = x >> 32;
f += t; /* b*d + MSW(b*c + a*d) */
/* can't overflow */
/* now reduce: (b*d + MSW(b*c + a*d), a*c + LSW(b*c + a*d)) */
return reduce_mod_p(f, e);
}
/* ================================================================================================
* RPO128 Permutation
*/
#define STATE_WIDTH 12
#define NUM_ROUNDS 7
/*
* MDS matrix
*/
static const uint64_t MDS[12][12] = {
{ 7, 23, 8, 26, 13, 10, 9, 7, 6, 22, 21, 8 },
{ 8, 7, 23, 8, 26, 13, 10, 9, 7, 6, 22, 21 },
{ 21, 8, 7, 23, 8, 26, 13, 10, 9, 7, 6, 22 },
{ 22, 21, 8, 7, 23, 8, 26, 13, 10, 9, 7, 6 },
{ 6, 22, 21, 8, 7, 23, 8, 26, 13, 10, 9, 7 },
{ 7, 6, 22, 21, 8, 7, 23, 8, 26, 13, 10, 9 },
{ 9, 7, 6, 22, 21, 8, 7, 23, 8, 26, 13, 10 },
{ 10, 9, 7, 6, 22, 21, 8, 7, 23, 8, 26, 13 },
{ 13, 10, 9, 7, 6, 22, 21, 8, 7, 23, 8, 26 },
{ 26, 13, 10, 9, 7, 6, 22, 21, 8, 7, 23, 8 },
{ 8, 26, 13, 10, 9, 7, 6, 22, 21, 8, 7, 23 },
{ 23, 8, 26, 13, 10, 9, 7, 6, 22, 21, 8, 7 },
};
/*
* Round constants.
*/
static const uint64_t ARK1[7][12] = {
{
5789762306288267392ULL,
6522564764413701783ULL,
17809893479458208203ULL,
107145243989736508ULL,
6388978042437517382ULL,
15844067734406016715ULL,
9975000513555218239ULL,
3344984123768313364ULL,
9959189626657347191ULL,
12960773468763563665ULL,
9602914297752488475ULL,
16657542370200465908ULL,
},
{
12987190162843096997ULL,
653957632802705281ULL,
4441654670647621225ULL,
4038207883745915761ULL,
5613464648874830118ULL,
13222989726778338773ULL,
3037761201230264149ULL,
16683759727265180203ULL,
8337364536491240715ULL,
3227397518293416448ULL,
8110510111539674682ULL,
2872078294163232137ULL,
},
{
18072785500942327487ULL,
6200974112677013481ULL,
17682092219085884187ULL,
10599526828986756440ULL,
975003873302957338ULL,
8264241093196931281ULL,
10065763900435475170ULL,
2181131744534710197ULL,
6317303992309418647ULL,
1401440938888741532ULL,
8884468225181997494ULL,
13066900325715521532ULL,
},
{
5674685213610121970ULL,
5759084860419474071ULL,
13943282657648897737ULL,
1352748651966375394ULL,
17110913224029905221ULL,
1003883795902368422ULL,
4141870621881018291ULL,
8121410972417424656ULL,
14300518605864919529ULL,
13712227150607670181ULL,
17021852944633065291ULL,
6252096473787587650ULL,
},
{
4887609836208846458ULL,
3027115137917284492ULL,
9595098600469470675ULL,
10528569829048484079ULL,
7864689113198939815ULL,
17533723827845969040ULL,
5781638039037710951ULL,
17024078752430719006ULL,
109659393484013511ULL,
7158933660534805869ULL,
2955076958026921730ULL,
7433723648458773977ULL,
},
{
16308865189192447297ULL,
11977192855656444890ULL,
12532242556065780287ULL,
14594890931430968898ULL,
7291784239689209784ULL,
5514718540551361949ULL,
10025733853830934803ULL,
7293794580341021693ULL,
6728552937464861756ULL,
6332385040983343262ULL,
13277683694236792804ULL,
2600778905124452676ULL,
},
{
7123075680859040534ULL,
1034205548717903090ULL,
7717824418247931797ULL,
3019070937878604058ULL,
11403792746066867460ULL,
10280580802233112374ULL,
337153209462421218ULL,
13333398568519923717ULL,
3596153696935337464ULL,
8104208463525993784ULL,
14345062289456085693ULL,
17036731477169661256ULL,
}};
const uint64_t ARK2[7][12] = {
{
6077062762357204287ULL,
15277620170502011191ULL,
5358738125714196705ULL,
14233283787297595718ULL,
13792579614346651365ULL,
11614812331536767105ULL,
14871063686742261166ULL,
10148237148793043499ULL,
4457428952329675767ULL,
15590786458219172475ULL,
10063319113072092615ULL,
14200078843431360086ULL,
},
{
6202948458916099932ULL,
17690140365333231091ULL,
3595001575307484651ULL,
373995945117666487ULL,
1235734395091296013ULL,
14172757457833931602ULL,
707573103686350224ULL,
15453217512188187135ULL,
219777875004506018ULL,
17876696346199469008ULL,
17731621626449383378ULL,
2897136237748376248ULL,
},
{
8023374565629191455ULL,
15013690343205953430ULL,
4485500052507912973ULL,
12489737547229155153ULL,
9500452585969030576ULL,
2054001340201038870ULL,
12420704059284934186ULL,
355990932618543755ULL,
9071225051243523860ULL,
12766199826003448536ULL,
9045979173463556963ULL,
12934431667190679898ULL,
},
{
18389244934624494276ULL,
16731736864863925227ULL,
4440209734760478192ULL,
17208448209698888938ULL,
8739495587021565984ULL,
17000774922218161967ULL,
13533282547195532087ULL,
525402848358706231ULL,
16987541523062161972ULL,
5466806524462797102ULL,
14512769585918244983ULL,
10973956031244051118ULL,
},
{
6982293561042362913ULL,
14065426295947720331ULL,
16451845770444974180ULL,
7139138592091306727ULL,
9012006439959783127ULL,
14619614108529063361ULL,
1394813199588124371ULL,
4635111139507788575ULL,
16217473952264203365ULL,
10782018226466330683ULL,
6844229992533662050ULL,
7446486531695178711ULL,
},
{
3736792340494631448ULL,
577852220195055341ULL,
6689998335515779805ULL,
13886063479078013492ULL,
14358505101923202168ULL,
7744142531772274164ULL,
16135070735728404443ULL,
12290902521256031137ULL,
12059913662657709804ULL,
16456018495793751911ULL,
4571485474751953524ULL,
17200392109565783176ULL,
},
{
17130398059294018733ULL,
519782857322261988ULL,
9625384390925085478ULL,
1664893052631119222ULL,
7629576092524553570ULL,
3485239601103661425ULL,
9755891797164033838ULL,
15218148195153269027ULL,
16460604813734957368ULL,
9643968136937729763ULL,
3611348709641382851ULL,
18256379591337759196ULL,
},
};
static void apply_sbox(uint64_t *const state)
{
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
uint64_t t2 = mult_mod_p(*(state + i), *(state + i));
uint64_t t4 = mult_mod_p(t2, t2);
*(state + i) = mult_mod_p(*(state + i), mult_mod_p(t2, t4));
}
}
static void apply_mds(uint64_t *state)
{
uint64_t res[STATE_WIDTH];
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
res[i] = 0;
}
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
for (uint64_t j = 0; j < STATE_WIDTH; j++)
{
res[i] = add_mod_p(res[i], mult_mod_p(MDS[i][j], *(state + j)));
}
}
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
*(state + i) = res[i];
}
}
static void apply_constants(uint64_t *const state, const uint64_t *ark)
{
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
*(state + i) = add_mod_p(*(state + i), *(ark + i));
}
}
static void exp_acc(const uint64_t m, const uint64_t *base, const uint64_t *tail, uint64_t *const res)
{
for (uint64_t i = 0; i < m; i++)
{
for (uint64_t j = 0; j < STATE_WIDTH; j++)
{
if (i == 0)
{
*(res + j) = mult_mod_p(*(base + j), *(base + j));
}
else
{
*(res + j) = mult_mod_p(*(res + j), *(res + j));
}
}
}
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
*(res + i) = mult_mod_p(*(res + i), *(tail + i));
}
}
static void apply_inv_sbox(uint64_t *const state)
{
uint64_t t1[STATE_WIDTH];
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
t1[i] = 0;
}
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
t1[i] = mult_mod_p(*(state + i), *(state + i));
}
uint64_t t2[STATE_WIDTH];
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
t2[i] = 0;
}
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
t2[i] = mult_mod_p(t1[i], t1[i]);
}
uint64_t t3[STATE_WIDTH];
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
t3[i] = 0;
}
exp_acc(3, t2, t2, t3);
uint64_t t4[STATE_WIDTH];
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
t4[i] = 0;
}
exp_acc(6, t3, t3, t4);
uint64_t tmp[STATE_WIDTH];
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
tmp[i] = 0;
}
exp_acc(12, t4, t4, tmp);
uint64_t t5[STATE_WIDTH];
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
t5[i] = 0;
}
exp_acc(6, tmp, t3, t5);
uint64_t t6[STATE_WIDTH];
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
t6[i] = 0;
}
exp_acc(31, t5, t5, t6);
for (uint64_t i = 0; i < STATE_WIDTH; i++)
{
uint64_t a = mult_mod_p(mult_mod_p(t6[i], t6[i]), t5[i]);
a = mult_mod_p(a, a);
a = mult_mod_p(a, a);
uint64_t b = mult_mod_p(mult_mod_p(t1[i], t2[i]), *(state + i));
*(state + i) = mult_mod_p(a, b);
}
}
static void apply_round(uint64_t *const state, const uint64_t round)
{
apply_mds(state);
apply_constants(state, ARK1[round]);
apply_sbox(state);
apply_mds(state);
apply_constants(state, ARK2[round]);
apply_inv_sbox(state);
}
static void apply_permutation(uint64_t *state)
{
for (uint64_t i = 0; i < NUM_ROUNDS; i++)
{
apply_round(state, i);
}
}
/* ================================================================================================
* RPO128 implementation. This is supposed to substitute SHAKE256 in the hash-to-point algorithm.
*/
#include "rpo.h"
void rpo128_init(rpo128_context *rc)
{
rc->dptr = 32;
memset(rc->st.A, 0, sizeof rc->st.A);
}
void rpo128_absorb(rpo128_context *rc, const uint8_t *in, size_t len)
{
size_t dptr;
dptr = (size_t)rc->dptr;
while (len > 0)
{
size_t clen, u;
/* 136 * 8 = 1088 bit for the rate portion in the case of SHAKE256
* For RPO, this is 64 * 8 = 512 bits
* The capacity for SHAKE256 is at the end while for RPO128 it is at the beginning
*/
clen = 96 - dptr;
if (clen > len)
{
clen = len;
}
for (u = 0; u < clen; u++)
{
rc->st.dbuf[dptr + u] = in[u];
}
dptr += clen;
in += clen;
len -= clen;
if (dptr == 96)
{
apply_permutation(rc->st.A);
dptr = 32;
}
}
rc->dptr = dptr;
}
void rpo128_finalize(rpo128_context *rc)
{
// Set dptr to the end of the buffer, so that first call to extract will call the permutation.
rc->dptr = 96;
}
void rpo128_squeeze(rpo128_context *rc, uint8_t *out, size_t len)
{
size_t dptr;
dptr = (size_t)rc->dptr;
while (len > 0)
{
size_t clen;
if (dptr == 96)
{
apply_permutation(rc->st.A);
dptr = 32;
}
clen = 96 - dptr;
if (clen > len)
{
clen = len;
}
len -= clen;
memcpy(out, rc->st.dbuf + dptr, clen);
dptr += clen;
out += clen;
}
rc->dptr = dptr;
}
void rpo128_release(rpo128_context *rc)
{
memset(rc->st.A, 0, sizeof rc->st.A);
rc->dptr = 32;
}
/* ================================================================================================
* Hash-to-Point algorithm implementation based on RPO128
*/
void PQCLEAN_FALCON512_CLEAN_hash_to_point_rpo(rpo128_context *rc, uint16_t *x, unsigned logn)
{
/*
* This implementation avoids the rejection sampling step needed in the
* per-the-spec implementation. It uses a remark in https://falcon-sign.info/falcon.pdf
* page 31, which argues that the current variant is secure for the parameters set by NIST.
* Avoiding the rejection-sampling step leads to an implementation that is constant-time.
* TODO: Check that the current implementation is indeed constant-time.
*/
size_t n;
n = (size_t)1 << logn;
while (n > 0)
{
uint8_t buf[8];
uint64_t w;
rpo128_squeeze(rc, (void *)buf, sizeof buf);
w = ((uint64_t)(buf[7]) << 56) |
((uint64_t)(buf[6]) << 48) |
((uint64_t)(buf[5]) << 40) |
((uint64_t)(buf[4]) << 32) |
((uint64_t)(buf[3]) << 24) |
((uint64_t)(buf[2]) << 16) |
((uint64_t)(buf[1]) << 8) |
((uint64_t)(buf[0]));
w %= M;
*x++ = (uint16_t)w;
n--;
}
}

View File

@@ -0,0 +1,83 @@
#include <stdint.h>
#include <string.h>
/* ================================================================================================
* RPO hashing algorithm related structs and methods.
*/
/*
* RPO128 context.
*
* This structure is used by the hashing API. It is composed of an internal state that can be
* viewed as either:
* 1. 12 field elements in the Miden VM.
* 2. 96 bytes.
*
* The first view is used for the internal state in the context of the RPO hashing algorithm. The
* second view is used for the buffer used to absorb the data to be hashed.
*
* The pointer to the buffer is updated as the data is absorbed.
*
* 'rpo128_context' must be initialized with rpo128_init() before first use.
*/
typedef struct
{
union
{
uint64_t A[12];
uint8_t dbuf[96];
} st;
uint64_t dptr;
} rpo128_context;
/*
* Initializes an RPO state
*/
void rpo128_init(rpo128_context *rc);
/*
* Absorbs an array of bytes of length 'len' into the state.
*/
void rpo128_absorb(rpo128_context *rc, const uint8_t *in, size_t len);
/*
* Squeezes an array of bytes of length 'len' from the state.
*/
void rpo128_squeeze(rpo128_context *rc, uint8_t *out, size_t len);
/*
* Finalizes the state in preparation for squeezing.
*
* This function should be called after all the data has been absorbed.
*
* Note that the current implementation does not perform any sort of padding for domain separation
* purposes. The reason being that, for our purposes, we always perform the following sequence:
* 1. Absorb a Nonce (which is always 40 bytes packed as 8 field elements).
* 2. Absorb the message (which is always 4 field elements).
* 3. Call finalize.
* 4. Squeeze the output.
* 5. Call release.
*/
void rpo128_finalize(rpo128_context *rc);
/*
* Releases the state.
*
* This function should be called after the squeeze operation is finished.
*/
void rpo128_release(rpo128_context *rc);
/* ================================================================================================
* Hash-to-Point algorithm for signature generation and signature verification.
*/
/*
* Hash-to-Point algorithm.
*
* This function generates a point in Z_q[x]/(phi) from a given message.
*
* It takes a finalized rpo128_context as input and it generates the coefficients of the polynomial
* representing the point. The coefficients are stored in the array 'x'. The number of coefficients
* is given by 'logn', which must in our case is 512.
*/
void PQCLEAN_FALCON512_CLEAN_hash_to_point_rpo(rpo128_context *rc, uint16_t *x, unsigned logn);

View File

@@ -0,0 +1,189 @@
use libc::c_int;
// C IMPLEMENTATION INTERFACE
// ================================================================================================
#[link(name = "rpo_falcon512", kind = "static")]
extern "C" {
/// Generate a new key pair. Public key goes into pk[], private key in sk[].
/// Key sizes are exact (in bytes):
/// - public (pk): 897
/// - private (sk): 1281
///
/// Return value: 0 on success, -1 on error.
pub fn PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_rpo(pk: *mut u8, sk: *mut u8) -> c_int;
/// Generate a new key pair from seed. Public key goes into pk[], private key in sk[].
/// Key sizes are exact (in bytes):
/// - public (pk): 897
/// - private (sk): 1281
///
/// Return value: 0 on success, -1 on error.
pub fn PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_from_seed_rpo(
pk: *mut u8,
sk: *mut u8,
seed: *const u8,
) -> c_int;
/// Compute a signature on a provided message (m, mlen), with a given private key (sk).
/// Signature is written in sig[], with length written into *siglen. Signature length is
/// variable; maximum signature length (in bytes) is 666.
///
/// sig[], m[] and sk[] may overlap each other arbitrarily.
///
/// Return value: 0 on success, -1 on error.
pub fn PQCLEAN_FALCON512_CLEAN_crypto_sign_signature_rpo(
sig: *mut u8,
siglen: *mut usize,
m: *const u8,
mlen: usize,
sk: *const u8,
) -> c_int;
// TEST HELPERS
// --------------------------------------------------------------------------------------------
/// Verify a signature (sig, siglen) on a message (m, mlen) with a given public key (pk).
///
/// sig[], m[] and pk[] may overlap each other arbitrarily.
///
/// Return value: 0 on success, -1 on error.
#[cfg(test)]
pub fn PQCLEAN_FALCON512_CLEAN_crypto_sign_verify_rpo(
sig: *const u8,
siglen: usize,
m: *const u8,
mlen: usize,
pk: *const u8,
) -> c_int;
/// Hash-to-Point algorithm.
///
/// This function generates a point in Z_q[x]/(phi) from a given message.
///
/// It takes a finalized rpo128_context as input and it generates the coefficients of the polynomial
/// representing the point. The coefficients are stored in the array 'x'. The number of coefficients
/// is given by 'logn', which must in our case is 512.
#[cfg(test)]
pub fn PQCLEAN_FALCON512_CLEAN_hash_to_point_rpo(
rc: *mut Rpo128Context,
x: *mut u16,
logn: usize,
);
#[cfg(test)]
pub fn rpo128_init(sc: *mut Rpo128Context);
#[cfg(test)]
pub fn rpo128_absorb(
sc: *mut Rpo128Context,
data: *const ::std::os::raw::c_void,
len: libc::size_t,
);
#[cfg(test)]
pub fn rpo128_finalize(sc: *mut Rpo128Context);
}
#[repr(C)]
#[cfg(test)]
pub struct Rpo128Context {
pub content: [u64; 13usize],
}
// TESTS
// ================================================================================================
#[cfg(all(test, feature = "std"))]
mod tests {
use super::*;
use crate::dsa::rpo_falcon512::{NONCE_LEN, PK_LEN, SIG_LEN, SK_LEN};
use rand_utils::{rand_array, rand_value, rand_vector};
#[test]
fn falcon_ffi() {
unsafe {
//let mut rng = rand::thread_rng();
// --- generate a key pair from a seed ----------------------------
let mut pk = [0u8; PK_LEN];
let mut sk = [0u8; SK_LEN];
let seed: [u8; NONCE_LEN] = rand_array();
assert_eq!(
0,
PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_from_seed_rpo(
pk.as_mut_ptr(),
sk.as_mut_ptr(),
seed.as_ptr()
)
);
// --- sign a message and make sure it verifies -------------------
let mlen: usize = rand_value::<u16>() as usize;
let msg: Vec<u8> = rand_vector(mlen);
let mut detached_sig = [0u8; NONCE_LEN + SIG_LEN];
let mut siglen = 0;
assert_eq!(
0,
PQCLEAN_FALCON512_CLEAN_crypto_sign_signature_rpo(
detached_sig.as_mut_ptr(),
&mut siglen as *mut usize,
msg.as_ptr(),
msg.len(),
sk.as_ptr()
)
);
assert_eq!(
0,
PQCLEAN_FALCON512_CLEAN_crypto_sign_verify_rpo(
detached_sig.as_ptr(),
siglen,
msg.as_ptr(),
msg.len(),
pk.as_ptr()
)
);
// --- check verification of different signature ------------------
assert_eq!(
-1,
PQCLEAN_FALCON512_CLEAN_crypto_sign_verify_rpo(
detached_sig.as_ptr(),
siglen,
msg.as_ptr(),
msg.len() - 1,
pk.as_ptr()
)
);
// --- check verification against a different pub key -------------
let mut pk_alt = [0u8; PK_LEN];
let mut sk_alt = [0u8; SK_LEN];
assert_eq!(
0,
PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_rpo(
pk_alt.as_mut_ptr(),
sk_alt.as_mut_ptr()
)
);
assert_eq!(
-1,
PQCLEAN_FALCON512_CLEAN_crypto_sign_verify_rpo(
detached_sig.as_ptr(),
siglen,
msg.as_ptr(),
msg.len(),
pk_alt.as_ptr()
)
);
}
}
}

View File

@@ -0,0 +1,232 @@
use super::{
ByteReader, ByteWriter, Deserializable, DeserializationError, FalconError, Polynomial,
PublicKeyBytes, Rpo256, SecretKeyBytes, Serializable, Signature, Word,
};
#[cfg(feature = "std")]
use super::{ffi, NonceBytes, NONCE_LEN, PK_LEN, SIG_LEN, SK_LEN};
// PUBLIC KEY
// ================================================================================================
/// A public key for verifying signatures.
///
/// The public key is a [Word] (i.e., 4 field elements) that is the hash of the coefficients of
/// the polynomial representing the raw bytes of the expanded public key.
///
/// For Falcon-512, the first byte of the expanded public key is always equal to log2(512) i.e., 9.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct PublicKey(Word);
impl PublicKey {
/// Returns a new [PublicKey] which is a commitment to the provided expanded public key.
///
/// # Errors
/// Returns an error if the decoding of the public key fails.
pub fn new(pk: PublicKeyBytes) -> Result<Self, FalconError> {
let h = Polynomial::from_pub_key(&pk)?;
let pk_felts = h.to_elements();
let pk_digest = Rpo256::hash_elements(&pk_felts).into();
Ok(Self(pk_digest))
}
/// Verifies the provided signature against provided message and this public key.
pub fn verify(&self, message: Word, signature: &Signature) -> bool {
signature.verify(message, self.0)
}
}
impl From<PublicKey> for Word {
fn from(key: PublicKey) -> Self {
key.0
}
}
// KEY PAIR
// ================================================================================================
/// A key pair (public and secret keys) for signing messages.
///
/// The secret key is a byte array of length [PK_LEN].
/// The public key is a byte array of length [SK_LEN].
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct KeyPair {
public_key: PublicKeyBytes,
secret_key: SecretKeyBytes,
}
#[allow(clippy::new_without_default)]
impl KeyPair {
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Generates a (public_key, secret_key) key pair from OS-provided randomness.
///
/// # Errors
/// Returns an error if key generation fails.
#[cfg(feature = "std")]
pub fn new() -> Result<Self, FalconError> {
let mut public_key = [0u8; PK_LEN];
let mut secret_key = [0u8; SK_LEN];
let res = unsafe {
ffi::PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_rpo(
public_key.as_mut_ptr(),
secret_key.as_mut_ptr(),
)
};
if res == 0 {
Ok(Self { public_key, secret_key })
} else {
Err(FalconError::KeyGenerationFailed)
}
}
/// Generates a (public_key, secret_key) key pair from the provided seed.
///
/// # Errors
/// Returns an error if key generation fails.
#[cfg(feature = "std")]
pub fn from_seed(seed: &NonceBytes) -> Result<Self, FalconError> {
let mut public_key = [0u8; PK_LEN];
let mut secret_key = [0u8; SK_LEN];
let res = unsafe {
ffi::PQCLEAN_FALCON512_CLEAN_crypto_sign_keypair_from_seed_rpo(
public_key.as_mut_ptr(),
secret_key.as_mut_ptr(),
seed.as_ptr(),
)
};
if res == 0 {
Ok(Self { public_key, secret_key })
} else {
Err(FalconError::KeyGenerationFailed)
}
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the public key corresponding to this key pair.
pub fn public_key(&self) -> PublicKey {
// TODO: memoize public key commitment as computing it requires quite a bit of hashing.
// expect() is fine here because we assume that the key pair was constructed correctly.
PublicKey::new(self.public_key).expect("invalid key pair")
}
/// Returns the expanded public key corresponding to this key pair.
pub fn expanded_public_key(&self) -> PublicKeyBytes {
self.public_key
}
// SIGNATURE GENERATION
// --------------------------------------------------------------------------------------------
/// Signs a message with a secret key and a seed.
///
/// # Errors
/// Returns an error of signature generation fails.
#[cfg(feature = "std")]
pub fn sign(&self, message: Word) -> Result<Signature, FalconError> {
let msg = message.iter().flat_map(|e| e.as_int().to_le_bytes()).collect::<Vec<_>>();
let msg_len = msg.len();
let mut sig = [0_u8; SIG_LEN + NONCE_LEN];
let mut sig_len: usize = 0;
let res = unsafe {
ffi::PQCLEAN_FALCON512_CLEAN_crypto_sign_signature_rpo(
sig.as_mut_ptr(),
&mut sig_len as *mut usize,
msg.as_ptr(),
msg_len,
self.secret_key.as_ptr(),
)
};
if res == 0 {
Ok(Signature {
sig,
pk: self.public_key,
pk_polynomial: Default::default(),
sig_polynomial: Default::default(),
})
} else {
Err(FalconError::SigGenerationFailed)
}
}
}
// SERIALIZATION / DESERIALIZATION
// ================================================================================================
impl Serializable for KeyPair {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
target.write_bytes(&self.public_key);
target.write_bytes(&self.secret_key);
}
}
impl Deserializable for KeyPair {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let public_key: PublicKeyBytes = source.read_array()?;
let secret_key: SecretKeyBytes = source.read_array()?;
Ok(Self { public_key, secret_key })
}
}
// TESTS
// ================================================================================================
#[cfg(all(test, feature = "std"))]
mod tests {
use super::{super::Felt, KeyPair, NonceBytes, Word};
use rand_utils::{rand_array, rand_vector};
#[test]
fn test_falcon_verification() {
// generate random keys
let keys = KeyPair::new().unwrap();
let pk = keys.public_key();
// sign a random message
let message: Word = rand_vector::<Felt>(4).try_into().expect("Should not fail.");
let signature = keys.sign(message);
// make sure the signature verifies correctly
assert!(pk.verify(message, signature.as_ref().unwrap()));
// a signature should not verify against a wrong message
let message2: Word = rand_vector::<Felt>(4).try_into().expect("Should not fail.");
assert!(!pk.verify(message2, signature.as_ref().unwrap()));
// a signature should not verify against a wrong public key
let keys2 = KeyPair::new().unwrap();
assert!(!keys2.public_key().verify(message, signature.as_ref().unwrap()))
}
#[test]
fn test_falcon_verification_from_seed() {
// generate keys from a random seed
let seed: NonceBytes = rand_array();
let keys = KeyPair::from_seed(&seed).unwrap();
let pk = keys.public_key();
// sign a random message
let message: Word = rand_vector::<Felt>(4).try_into().expect("Should not fail.");
let signature = keys.sign(message);
// make sure the signature verifies correctly
assert!(pk.verify(message, signature.as_ref().unwrap()));
// a signature should not verify against a wrong message
let message2: Word = rand_vector::<Felt>(4).try_into().expect("Should not fail.");
assert!(!pk.verify(message2, signature.as_ref().unwrap()));
// a signature should not verify against a wrong public key
let keys2 = KeyPair::new().unwrap();
assert!(!keys2.public_key().verify(message, signature.as_ref().unwrap()))
}
}

View File

@@ -0,0 +1,60 @@
use crate::{
hash::rpo::Rpo256,
utils::{
collections::Vec, ByteReader, ByteWriter, Deserializable, DeserializationError,
Serializable,
},
Felt, Word, ZERO,
};
#[cfg(feature = "std")]
mod ffi;
mod error;
mod keys;
mod polynomial;
mod signature;
pub use error::FalconError;
pub use keys::{KeyPair, PublicKey};
pub use polynomial::Polynomial;
pub use signature::Signature;
// CONSTANTS
// ================================================================================================
// The Falcon modulus.
const MODULUS: u16 = 12289;
const MODULUS_MINUS_1_OVER_TWO: u16 = 6144;
// The Falcon parameters for Falcon-512. This is the degree of the polynomial `phi := x^N + 1`
// defining the ring Z_p[x]/(phi).
const N: usize = 512;
const LOG_N: usize = 9;
/// Length of nonce used for key-pair generation.
const NONCE_LEN: usize = 40;
/// Number of filed elements used to encode a nonce.
const NONCE_ELEMENTS: usize = 8;
/// Public key length as a u8 vector.
pub const PK_LEN: usize = 897;
/// Secret key length as a u8 vector.
pub const SK_LEN: usize = 1281;
/// Signature length as a u8 vector.
const SIG_LEN: usize = 626;
/// Bound on the squared-norm of the signature.
const SIG_L2_BOUND: u64 = 34034726;
// TYPE ALIASES
// ================================================================================================
type SignatureBytes = [u8; NONCE_LEN + SIG_LEN];
type PublicKeyBytes = [u8; PK_LEN];
type SecretKeyBytes = [u8; SK_LEN];
type NonceBytes = [u8; NONCE_LEN];
type NonceElements = [Felt; NONCE_ELEMENTS];

View File

@@ -0,0 +1,277 @@
use super::{FalconError, Felt, Vec, LOG_N, MODULUS, MODULUS_MINUS_1_OVER_TWO, N, PK_LEN};
use core::ops::{Add, Mul, Sub};
// FALCON POLYNOMIAL
// ================================================================================================
/// A polynomial over Z_p\[x\]/(phi) where phi := x^512 + 1
#[derive(Debug, Copy, Clone, PartialEq)]
pub struct Polynomial([u16; N]);
impl Polynomial {
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Constructs a new polynomial from a list of coefficients.
///
/// # Safety
/// This constructor validates that the coefficients are in the valid range only in debug mode.
pub unsafe fn new(data: [u16; N]) -> Self {
for value in data {
debug_assert!(value < MODULUS);
}
Self(data)
}
/// Decodes raw bytes representing a public key into a polynomial in Z_p\[x\]/(phi).
///
/// # Errors
/// Returns an error if:
/// - The provided input is not exactly 897 bytes long.
/// - The first byte of the input is not equal to log2(512) i.e., 9.
/// - Any of the coefficients encoded in the provided input is greater than or equal to the
/// Falcon field modulus.
pub fn from_pub_key(input: &[u8]) -> Result<Self, FalconError> {
if input.len() != PK_LEN {
return Err(FalconError::PubKeyDecodingInvalidLength(input.len()));
}
if input[0] != LOG_N as u8 {
return Err(FalconError::PubKeyDecodingInvalidTag(input[0]));
}
let mut acc = 0_u32;
let mut acc_len = 0;
let mut output = [0_u16; N];
let mut output_idx = 0;
for &byte in input.iter().skip(1) {
acc = (acc << 8) | (byte as u32);
acc_len += 8;
if acc_len >= 14 {
acc_len -= 14;
let w = (acc >> acc_len) & 0x3FFF;
if w >= MODULUS as u32 {
return Err(FalconError::PubKeyDecodingInvalidCoefficient(w));
}
output[output_idx] = w as u16;
output_idx += 1;
}
}
if (acc & ((1u32 << acc_len) - 1)) == 0 {
Ok(Self(output))
} else {
Err(FalconError::PubKeyDecodingExtraData)
}
}
/// Decodes the signature into the coefficients of a polynomial in Z_p\[x\]/(phi). It assumes
/// that the signature has been encoded using the uncompressed format.
///
/// # Errors
/// Returns an error if:
/// - The signature has been encoded using a different algorithm than the reference compressed
/// encoding algorithm.
/// - The encoded signature polynomial is in Z_p\[x\]/(phi') where phi' = x^N' + 1 and N' != 512.
/// - While decoding the high bits of a coefficient, the current accumulated value of its
/// high bits is larger than 2048.
/// - The decoded coefficient is -0.
/// - The remaining unused bits in the last byte of `input` are non-zero.
pub fn from_signature(input: &[u8]) -> Result<Self, FalconError> {
let (encoding, log_n) = (input[0] >> 4, input[0] & 0b00001111);
if encoding != 0b0011 {
return Err(FalconError::SigDecodingIncorrectEncodingAlgorithm);
}
if log_n != 0b1001 {
return Err(FalconError::SigDecodingNotSupportedDegree(log_n));
}
let input = &input[41..];
let mut input_idx = 0;
let mut acc = 0u32;
let mut acc_len = 0;
let mut output = [0_u16; N];
for e in output.iter_mut() {
acc = (acc << 8) | (input[input_idx] as u32);
input_idx += 1;
let b = acc >> acc_len;
let s = b & 128;
let mut m = b & 127;
loop {
if acc_len == 0 {
acc = (acc << 8) | (input[input_idx] as u32);
input_idx += 1;
acc_len = 8;
}
acc_len -= 1;
if ((acc >> acc_len) & 1) != 0 {
break;
}
m += 128;
if m >= 2048 {
return Err(FalconError::SigDecodingTooBigHighBits(m));
}
}
if s != 0 && m == 0 {
return Err(FalconError::SigDecodingMinusZero);
}
*e = if s != 0 { (MODULUS as u32 - m) as u16 } else { m as u16 };
}
if (acc & ((1 << acc_len) - 1)) != 0 {
return Err(FalconError::SigDecodingNonZeroUnusedBitsLastByte);
}
Ok(Self(output))
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the coefficients of this polynomial as integers.
pub fn inner(&self) -> [u16; N] {
self.0
}
/// Returns the coefficients of this polynomial as field elements.
pub fn to_elements(&self) -> Vec<Felt> {
self.0.iter().map(|&a| Felt::from(a)).collect()
}
// POLYNOMIAL OPERATIONS
// --------------------------------------------------------------------------------------------
/// Multiplies two polynomials over Z_p\[x\] without reducing modulo p. Given that the degrees
/// of the input polynomials are less than 512 and their coefficients are less than the modulus
/// q equal to 12289, the resulting product polynomial is guaranteed to have coefficients less
/// than the Miden prime.
///
/// Note that this multiplication is not over Z_p\[x\]/(phi).
pub fn mul_modulo_p(a: &Self, b: &Self) -> [u64; 1024] {
let mut c = [0; 2 * N];
for i in 0..N {
for j in 0..N {
c[i + j] += a.0[i] as u64 * b.0[j] as u64;
}
}
c
}
/// Reduces a polynomial, that is the product of two polynomials over Z_p\[x\], modulo
/// the irreducible polynomial phi. This results in an element in Z_p\[x\]/(phi).
pub fn reduce_negacyclic(a: &[u64; 1024]) -> Self {
let mut c = [0; N];
for i in 0..N {
let ai = a[N + i] % MODULUS as u64;
let neg_ai = (MODULUS - ai as u16) % MODULUS;
let bi = (a[i] % MODULUS as u64) as u16;
c[i] = (neg_ai + bi) % MODULUS;
}
Self(c)
}
/// Computes the norm squared of a polynomial in Z_p\[x\]/(phi) after normalizing its
/// coefficients to be in the interval (-p/2, p/2].
pub fn sq_norm(&self) -> u64 {
let mut res = 0;
for e in self.0 {
if e > MODULUS_MINUS_1_OVER_TWO {
res += (MODULUS - e) as u64 * (MODULUS - e) as u64
} else {
res += e as u64 * e as u64
}
}
res
}
}
// Returns a polynomial representing the zero polynomial i.e. default element.
impl Default for Polynomial {
fn default() -> Self {
Self([0_u16; N])
}
}
/// Multiplication over Z_p\[x\]/(phi)
impl Mul for Polynomial {
type Output = Self;
fn mul(self, other: Self) -> <Self as Mul<Self>>::Output {
let mut result = [0_u16; N];
for j in 0..N {
for k in 0..N {
let i = (j + k) % N;
let a = self.0[j] as usize;
let b = other.0[k] as usize;
let q = MODULUS as usize;
let mut prod = a * b % q;
if (N - 1) < (j + k) {
prod = (q - prod) % q;
}
result[i] = ((result[i] as usize + prod) % q) as u16;
}
}
Polynomial(result)
}
}
/// Addition over Z_p\[x\]/(phi)
impl Add for Polynomial {
type Output = Self;
fn add(self, other: Self) -> <Self as Add<Self>>::Output {
let mut res = self;
res.0.iter_mut().zip(other.0.iter()).for_each(|(x, y)| *x = (*x + *y) % MODULUS);
res
}
}
/// Subtraction over Z_p\[x\]/(phi)
impl Sub for Polynomial {
type Output = Self;
fn sub(self, other: Self) -> <Self as Add<Self>>::Output {
let mut res = self;
res.0
.iter_mut()
.zip(other.0.iter())
.for_each(|(x, y)| *x = (*x + MODULUS - *y) % MODULUS);
res
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::{Polynomial, N};
#[test]
fn test_negacyclic_reduction() {
let coef1: [u16; N] = rand_utils::rand_array();
let coef2: [u16; N] = rand_utils::rand_array();
let poly1 = Polynomial(coef1);
let poly2 = Polynomial(coef2);
assert_eq!(
poly1 * poly2,
Polynomial::reduce_negacyclic(&Polynomial::mul_modulo_p(&poly1, &poly2))
);
}
}

View File

@@ -0,0 +1,284 @@
use super::{
ByteReader, ByteWriter, Deserializable, DeserializationError, Felt, NonceBytes, NonceElements,
Polynomial, PublicKeyBytes, Rpo256, Serializable, SignatureBytes, Word, MODULUS, N,
SIG_L2_BOUND, ZERO,
};
use crate::utils::string::ToString;
use core::cell::OnceCell;
// FALCON SIGNATURE
// ================================================================================================
/// An RPO Falcon512 signature over a message.
///
/// The signature is a pair of polynomials (s1, s2) in (Z_p\[x\]/(phi))^2, where:
/// - p := 12289
/// - phi := x^512 + 1
/// - s1 = c - s2 * h
/// - h is a polynomial representing the public key and c is a polynomial that is the hash-to-point
/// of the message being signed.
///
/// The signature verifies if and only if:
/// 1. s1 = c - s2 * h
/// 2. |s1|^2 + |s2|^2 <= SIG_L2_BOUND
///
/// where |.| is the norm.
///
/// [Signature] also includes the extended public key which is serialized as:
/// 1. 1 byte representing the log2(512) i.e., 9.
/// 2. 896 bytes for the public key. This is decoded into the `h` polynomial above.
///
/// The actual signature is serialized as:
/// 1. A header byte specifying the algorithm used to encode the coefficients of the `s2` polynomial
/// together with the degree of the irreducible polynomial phi.
/// The general format of this byte is 0b0cc1nnnn where:
/// a. cc is either 01 when the compressed encoding algorithm is used and 10 when the
/// uncompressed algorithm is used.
/// b. nnnn is log2(N) where N is the degree of the irreducible polynomial phi.
/// The current implementation works always with cc equal to 0b01 and nnnn equal to 0b1001 and
/// thus the header byte is always equal to 0b00111001.
/// 2. 40 bytes for the nonce.
/// 3. 625 bytes encoding the `s2` polynomial above.
///
/// The total size of the signature (including the extended public key) is 1563 bytes.
#[derive(Debug, Clone)]
pub struct Signature {
pub(super) pk: PublicKeyBytes,
pub(super) sig: SignatureBytes,
// Cached polynomial decoding for public key and signatures
pub(super) pk_polynomial: OnceCell<Polynomial>,
pub(super) sig_polynomial: OnceCell<Polynomial>,
}
impl Signature {
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the public key polynomial h.
pub fn pub_key_poly(&self) -> Polynomial {
*self.pk_polynomial.get_or_init(|| {
// we assume that the signature was constructed with a valid public key, and thus
// expect() is OK here.
Polynomial::from_pub_key(&self.pk).expect("invalid public key")
})
}
/// Returns the nonce component of the signature represented as field elements.
///
/// Nonce bytes are converted to field elements by taking consecutive 5 byte chunks
/// of the nonce and interpreting them as field elements.
pub fn nonce(&self) -> NonceElements {
// we assume that the signature was constructed with a valid signature, and thus
// expect() is OK here.
let nonce = self.sig[1..41].try_into().expect("invalid signature");
decode_nonce(nonce)
}
// Returns the polynomial representation of the signature in Z_p[x]/(phi).
pub fn sig_poly(&self) -> Polynomial {
*self.sig_polynomial.get_or_init(|| {
// we assume that the signature was constructed with a valid signature, and thus
// expect() is OK here.
Polynomial::from_signature(&self.sig).expect("invalid signature")
})
}
// HASH-TO-POINT
// --------------------------------------------------------------------------------------------
/// Returns a polynomial in Z_p\[x\]/(phi) representing the hash of the provided message.
pub fn hash_to_point(&self, message: Word) -> Polynomial {
hash_to_point(message, &self.nonce())
}
// SIGNATURE VERIFICATION
// --------------------------------------------------------------------------------------------
/// Returns true if this signature is a valid signature for the specified message generated
/// against key pair matching the specified public key commitment.
pub fn verify(&self, message: Word, pubkey_com: Word) -> bool {
// Make sure the expanded public key matches the provided public key commitment
let h = self.pub_key_poly();
let h_digest: Word = Rpo256::hash_elements(&h.to_elements()).into();
if h_digest != pubkey_com {
return false;
}
// Make sure the signature is valid
let s2 = self.sig_poly();
let c = self.hash_to_point(message);
let s1 = c - s2 * h;
let sq_norm = s1.sq_norm() + s2.sq_norm();
sq_norm <= SIG_L2_BOUND
}
}
// SERIALIZATION / DESERIALIZATION
// ================================================================================================
impl Serializable for Signature {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
target.write_bytes(&self.pk);
target.write_bytes(&self.sig);
}
}
impl Deserializable for Signature {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let pk: PublicKeyBytes = source.read_array()?;
let sig: SignatureBytes = source.read_array()?;
// make sure public key and signature can be decoded correctly
let pk_polynomial = Polynomial::from_pub_key(&pk)
.map_err(|err| DeserializationError::InvalidValue(err.to_string()))?
.into();
let sig_polynomial = Polynomial::from_signature(&sig)
.map_err(|err| DeserializationError::InvalidValue(err.to_string()))?
.into();
Ok(Self { pk, sig, pk_polynomial, sig_polynomial })
}
}
// HELPER FUNCTIONS
// ================================================================================================
/// Returns a polynomial in Z_p[x]/(phi) representing the hash of the provided message and
/// nonce.
fn hash_to_point(message: Word, nonce: &NonceElements) -> Polynomial {
let mut state = [ZERO; Rpo256::STATE_WIDTH];
// absorb the nonce into the state
for (&n, s) in nonce.iter().zip(state[Rpo256::RATE_RANGE].iter_mut()) {
*s = n;
}
Rpo256::apply_permutation(&mut state);
// absorb message into the state
for (&m, s) in message.iter().zip(state[Rpo256::RATE_RANGE].iter_mut()) {
*s = m;
}
// squeeze the coefficients of the polynomial
let mut i = 0;
let mut res = [0_u16; N];
for _ in 0..64 {
Rpo256::apply_permutation(&mut state);
for a in &state[Rpo256::RATE_RANGE] {
res[i] = (a.as_int() % MODULUS as u64) as u16;
i += 1;
}
}
// using the raw constructor is OK here because we reduce all coefficients by the modulus above
unsafe { Polynomial::new(res) }
}
/// Converts byte representation of the nonce into field element representation.
fn decode_nonce(nonce: &NonceBytes) -> NonceElements {
let mut buffer = [0_u8; 8];
let mut result = [ZERO; 8];
for (i, bytes) in nonce.chunks(5).enumerate() {
buffer[..5].copy_from_slice(bytes);
// we can safely (without overflow) create a new Felt from u64 value here since this value
// contains at most 5 bytes
result[i] = Felt::new(u64::from_le_bytes(buffer));
}
result
}
// TESTS
// ================================================================================================
#[cfg(all(test, feature = "std"))]
mod tests {
use super::{
super::{ffi::*, Felt, KeyPair},
*,
};
use libc::c_void;
use rand_utils::rand_vector;
// Wrappers for unsafe functions
impl Rpo128Context {
/// Initializes the RPO state.
pub fn init() -> Self {
let mut ctx = Rpo128Context { content: [0u64; 13] };
unsafe {
rpo128_init(&mut ctx as *mut Rpo128Context);
}
ctx
}
/// Absorbs data into the RPO state.
pub fn absorb(&mut self, data: &[u8]) {
unsafe {
rpo128_absorb(
self as *mut Rpo128Context,
data.as_ptr() as *const c_void,
data.len(),
)
}
}
/// Finalizes the RPO state to prepare for squeezing.
pub fn finalize(&mut self) {
unsafe { rpo128_finalize(self as *mut Rpo128Context) }
}
}
#[test]
fn test_hash_to_point() {
// Create a random message and transform it into a u8 vector
let msg_felts: Word = rand_vector::<Felt>(4).try_into().unwrap();
let msg_bytes = msg_felts.iter().flat_map(|e| e.as_int().to_le_bytes()).collect::<Vec<_>>();
// Create a nonce i.e. a [u8; 40] array and pack into a [Felt; 8] array.
let nonce: [u8; 40] = rand_vector::<u8>(40).try_into().unwrap();
let mut buffer = [0_u8; 64];
for i in 0..8 {
buffer[8 * i] = nonce[5 * i];
buffer[8 * i + 1] = nonce[5 * i + 1];
buffer[8 * i + 2] = nonce[5 * i + 2];
buffer[8 * i + 3] = nonce[5 * i + 3];
buffer[8 * i + 4] = nonce[5 * i + 4];
}
// Initialize the RPO state
let mut rng = Rpo128Context::init();
// Absorb the nonce and message into the RPO state
rng.absorb(&buffer);
rng.absorb(&msg_bytes);
rng.finalize();
// Generate the coefficients of the hash-to-point polynomial.
let mut res: [u16; N] = [0; N];
unsafe {
PQCLEAN_FALCON512_CLEAN_hash_to_point_rpo(
&mut rng as *mut Rpo128Context,
res.as_mut_ptr(),
9,
);
}
// Check that the coefficients are correct
let nonce = decode_nonce(&nonce);
assert_eq!(res, hash_to_point(msg_felts, &nonce).inner());
}
#[test]
fn test_serialization_round_trip() {
let key = KeyPair::new().unwrap();
let signature = key.sign(Word::default()).unwrap();
let serialized = signature.to_bytes();
let deserialized = Signature::read_from_bytes(&serialized).unwrap();
assert_eq!(signature.sig_poly(), deserialized.sig_poly());
assert_eq!(signature.pub_key_poly(), deserialized.pub_key_poly());
}
}

View File

@@ -1,6 +1,7 @@
use super::{Digest, ElementHasher, Felt, FieldElement, Hasher, StarkField};
use super::{Digest, ElementHasher, Felt, FieldElement, Hasher};
use crate::utils::{
uninit_vector, ByteReader, ByteWriter, Deserializable, DeserializationError, Serializable,
bytes_to_hex_string, hex_to_bytes, string::String, ByteReader, ByteWriter, Deserializable,
DeserializationError, HexParseError, Serializable,
};
use core::{
mem::{size_of, transmute, transmute_copy},
@@ -25,7 +26,9 @@ const DIGEST20_BYTES: usize = 20;
///
/// Note: `N` can't be greater than `32` because [`Digest::as_bytes`] currently supports only 32
/// bytes.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
#[derive(Debug, Copy, Clone, Eq, PartialEq, Ord, PartialOrd)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
#[cfg_attr(feature = "serde", serde(into = "String", try_from = "&str"))]
pub struct Blake3Digest<const N: usize>([u8; N]);
impl<const N: usize> Default for Blake3Digest<N> {
@@ -54,6 +57,20 @@ impl<const N: usize> From<[u8; N]> for Blake3Digest<N> {
}
}
impl<const N: usize> From<Blake3Digest<N>> for String {
fn from(value: Blake3Digest<N>) -> Self {
bytes_to_hex_string(value.as_bytes())
}
}
impl<const N: usize> TryFrom<&str> for Blake3Digest<N> {
type Error = HexParseError;
fn try_from(value: &str) -> Result<Self, Self::Error> {
hex_to_bytes(value).map(|v| v.into())
}
}
impl<const N: usize> Serializable for Blake3Digest<N> {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
target.write_bytes(&self.0);
@@ -78,6 +95,7 @@ impl<const N: usize> Digest for Blake3Digest<N> {
// ================================================================================================
/// 256-bit output blake3 hasher.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct Blake3_256;
impl Hasher for Blake3_256 {
@@ -141,6 +159,7 @@ impl Blake3_256 {
// ================================================================================================
/// 192-bit output blake3 hasher.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct Blake3_192;
impl Hasher for Blake3_192 {
@@ -204,6 +223,7 @@ impl Blake3_192 {
// ================================================================================================
/// 160-bit output blake3 hasher.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct Blake3_160;
impl Hasher for Blake3_160 {
@@ -269,10 +289,7 @@ impl Blake3_160 {
/// Zero-copy ref shrink to array.
fn shrink_bytes<const M: usize, const N: usize>(bytes: &[u8; M]) -> &[u8; N] {
// compile-time assertion
assert!(
M >= N,
"N should fit in M so it can be safely transmuted into a smaller slice!"
);
assert!(M >= N, "N should fit in M so it can be safely transmuted into a smaller slice!");
// safety: bytes len is asserted
unsafe { transmute(bytes) }
}
@@ -287,15 +304,25 @@ where
let digest = if Felt::IS_CANONICAL {
blake3::hash(E::elements_as_bytes(elements))
} else {
let base_elements = E::as_base_elements(elements);
let blen = base_elements.len() << 3;
let mut hasher = blake3::Hasher::new();
let mut bytes = unsafe { uninit_vector(blen) };
for (idx, element) in base_elements.iter().enumerate() {
bytes[idx * 8..(idx + 1) * 8].copy_from_slice(&element.as_int().to_le_bytes());
// BLAKE3 state is 64 bytes - so, we can absorb 64 bytes into the state in a single
// permutation. we move the elements into the hasher via the buffer to give the CPU
// a chance to process multiple element-to-byte conversions in parallel
let mut buf = [0_u8; 64];
let mut chunk_iter = E::slice_as_base_elements(elements).chunks_exact(8);
for chunk in chunk_iter.by_ref() {
for i in 0..8 {
buf[i * 8..(i + 1) * 8].copy_from_slice(&chunk[i].as_int().to_le_bytes());
}
hasher.update(&buf);
}
blake3::hash(&bytes)
for element in chunk_iter.remainder() {
hasher.update(&element.as_int().to_le_bytes());
}
hasher.finalize()
};
*shrink_bytes(&digest.into())
}

View File

@@ -1,6 +1,22 @@
use super::*;
use crate::utils::collections::Vec;
use proptest::prelude::*;
use rand_utils::rand_vector;
#[test]
fn blake3_hash_elements() {
// test multiple of 8
let elements = rand_vector::<Felt>(16);
let expected = compute_expected_element_hash(&elements);
let actual: [u8; 32] = hash_elements(&elements);
assert_eq!(&expected, &actual);
// test not multiple of 8
let elements = rand_vector::<Felt>(17);
let expected = compute_expected_element_hash(&elements);
let actual: [u8; 32] = hash_elements(&elements);
assert_eq!(&expected, &actual);
}
proptest! {
#[test]
@@ -18,3 +34,14 @@ proptest! {
Blake3_256::hash(vec);
}
}
// HELPER FUNCTIONS
// ================================================================================================
fn compute_expected_element_hash(elements: &[Felt]) -> blake3::Hash {
let mut bytes = Vec::new();
for element in elements.iter() {
bytes.extend_from_slice(&element.as_int().to_le_bytes());
}
blake3::hash(&bytes)
}

View File

@@ -1,5 +1,19 @@
use super::{Felt, FieldElement, StarkField, ONE, ZERO};
use winter_crypto::{Digest, ElementHasher, Hasher};
//! Cryptographic hash functions used by the Miden VM and the Miden rollup.
use super::{CubeExtension, Felt, FieldElement, StarkField, ONE, ZERO};
pub mod blake;
pub mod rpo;
mod rescue;
pub mod rpo {
pub use super::rescue::{Rpo256, RpoDigest};
}
pub mod rpx {
pub use super::rescue::{Rpx256, RpxDigest};
}
// RE-EXPORTS
// ================================================================================================
pub use winter_crypto::{Digest, ElementHasher, Hasher};

101
src/hash/rescue/arch/mod.rs Normal file
View File

@@ -0,0 +1,101 @@
#[cfg(target_feature = "sve")]
pub mod optimized {
use crate::hash::rescue::STATE_WIDTH;
use crate::Felt;
mod ffi {
#[link(name = "rpo_sve", kind = "static")]
extern "C" {
pub fn add_constants_and_apply_sbox(
state: *mut std::ffi::c_ulong,
constants: *const std::ffi::c_ulong,
) -> bool;
pub fn add_constants_and_apply_inv_sbox(
state: *mut std::ffi::c_ulong,
constants: *const std::ffi::c_ulong,
) -> bool;
}
}
#[inline(always)]
pub fn add_constants_and_apply_sbox(
state: &mut [Felt; STATE_WIDTH],
ark: &[Felt; STATE_WIDTH],
) -> bool {
unsafe {
ffi::add_constants_and_apply_sbox(
state.as_mut_ptr() as *mut u64,
ark.as_ptr() as *const u64,
)
}
}
#[inline(always)]
pub fn add_constants_and_apply_inv_sbox(
state: &mut [Felt; STATE_WIDTH],
ark: &[Felt; STATE_WIDTH],
) -> bool {
unsafe {
ffi::add_constants_and_apply_inv_sbox(
state.as_mut_ptr() as *mut u64,
ark.as_ptr() as *const u64,
)
}
}
}
#[cfg(target_feature = "avx2")]
mod x86_64_avx2;
#[cfg(target_feature = "avx2")]
pub mod optimized {
use super::x86_64_avx2::{apply_inv_sbox, apply_sbox};
use crate::hash::rescue::{add_constants, STATE_WIDTH};
use crate::Felt;
#[inline(always)]
pub fn add_constants_and_apply_sbox(
state: &mut [Felt; STATE_WIDTH],
ark: &[Felt; STATE_WIDTH],
) -> bool {
add_constants(state, ark);
unsafe {
apply_sbox(std::mem::transmute(state));
}
true
}
#[inline(always)]
pub fn add_constants_and_apply_inv_sbox(
state: &mut [Felt; STATE_WIDTH],
ark: &[Felt; STATE_WIDTH],
) -> bool {
add_constants(state, ark);
unsafe {
apply_inv_sbox(std::mem::transmute(state));
}
true
}
}
#[cfg(not(any(target_feature = "avx2", target_feature = "sve")))]
pub mod optimized {
use crate::hash::rescue::STATE_WIDTH;
use crate::Felt;
#[inline(always)]
pub fn add_constants_and_apply_sbox(
_state: &mut [Felt; STATE_WIDTH],
_ark: &[Felt; STATE_WIDTH],
) -> bool {
false
}
#[inline(always)]
pub fn add_constants_and_apply_inv_sbox(
_state: &mut [Felt; STATE_WIDTH],
_ark: &[Felt; STATE_WIDTH],
) -> bool {
false
}
}

View File

@@ -0,0 +1,325 @@
use core::arch::x86_64::*;
// The following AVX2 implementation has been copied from plonky2:
// https://github.com/0xPolygonZero/plonky2/blob/main/plonky2/src/hash/arch/x86_64/poseidon_goldilocks_avx2_bmi2.rs
// Preliminary notes:
// 1. AVX does not support addition with carry but 128-bit (2-word) addition can be easily
// emulated. The method recognizes that for a + b overflowed iff (a + b) < a:
// i. res_lo = a_lo + b_lo
// ii. carry_mask = res_lo < a_lo
// iii. res_hi = a_hi + b_hi - carry_mask
// Notice that carry_mask is subtracted, not added. This is because AVX comparison instructions
// return -1 (all bits 1) for true and 0 for false.
//
// 2. AVX does not have unsigned 64-bit comparisons. Those can be emulated with signed comparisons
// by recognizing that a <u b iff a + (1 << 63) <s b + (1 << 63), where the addition wraps around
// and the comparisons are unsigned and signed respectively. The shift function adds/subtracts
// 1 << 63 to enable this trick.
// Example: addition with carry.
// i. a_lo_s = shift(a_lo)
// ii. res_lo_s = a_lo_s + b_lo
// iii. carry_mask = res_lo_s <s a_lo_s
// iv. res_lo = shift(res_lo_s)
// v. res_hi = a_hi + b_hi - carry_mask
// The suffix _s denotes a value that has been shifted by 1 << 63. The result of addition is
// shifted if exactly one of the operands is shifted, as is the case on line ii. Line iii.
// performs a signed comparison res_lo_s <s a_lo_s on shifted values to emulate unsigned
// comparison res_lo <u a_lo on unshifted values. Finally, line iv. reverses the shift so the
// result can be returned.
// When performing a chain of calculations, we can often save instructions by letting the shift
// propagate through and only undoing it when necessary. For example, to compute the addition of
// three two-word (128-bit) numbers we can do:
// i. a_lo_s = shift(a_lo)
// ii. tmp_lo_s = a_lo_s + b_lo
// iii. tmp_carry_mask = tmp_lo_s <s a_lo_s
// iv. tmp_hi = a_hi + b_hi - tmp_carry_mask
// v. res_lo_s = tmp_lo_s + c_lo
// vi. res_carry_mask = res_lo_s <s tmp_lo_s
// vii. res_lo = shift(res_lo_s)
// viii. res_hi = tmp_hi + c_hi - res_carry_mask
// Notice that the above 3-value addition still only requires two calls to shift, just like our
// 2-value addition.
#[inline(always)]
pub fn branch_hint() {
// NOTE: These are the currently supported assembly architectures. See the
// [nightly reference](https://doc.rust-lang.org/nightly/reference/inline-assembly.html) for
// the most up-to-date list.
#[cfg(any(
target_arch = "aarch64",
target_arch = "arm",
target_arch = "riscv32",
target_arch = "riscv64",
target_arch = "x86",
target_arch = "x86_64",
))]
unsafe {
core::arch::asm!("", options(nomem, nostack, preserves_flags));
}
}
macro_rules! map3 {
($f:ident::<$l:literal>, $v:ident) => {
($f::<$l>($v.0), $f::<$l>($v.1), $f::<$l>($v.2))
};
($f:ident::<$l:literal>, $v1:ident, $v2:ident) => {
($f::<$l>($v1.0, $v2.0), $f::<$l>($v1.1, $v2.1), $f::<$l>($v1.2, $v2.2))
};
($f:ident, $v:ident) => {
($f($v.0), $f($v.1), $f($v.2))
};
($f:ident, $v0:ident, $v1:ident) => {
($f($v0.0, $v1.0), $f($v0.1, $v1.1), $f($v0.2, $v1.2))
};
($f:ident, rep $v0:ident, $v1:ident) => {
($f($v0, $v1.0), $f($v0, $v1.1), $f($v0, $v1.2))
};
($f:ident, $v0:ident, rep $v1:ident) => {
($f($v0.0, $v1), $f($v0.1, $v1), $f($v0.2, $v1))
};
}
#[inline(always)]
unsafe fn square3(
x: (__m256i, __m256i, __m256i),
) -> ((__m256i, __m256i, __m256i), (__m256i, __m256i, __m256i)) {
let x_hi = {
// Move high bits to low position. The high bits of x_hi are ignored. Swizzle is faster than
// bitshift. This instruction only has a floating-point flavor, so we cast to/from float.
// This is safe and free.
let x_ps = map3!(_mm256_castsi256_ps, x);
let x_hi_ps = map3!(_mm256_movehdup_ps, x_ps);
map3!(_mm256_castps_si256, x_hi_ps)
};
// All pairwise multiplications.
let mul_ll = map3!(_mm256_mul_epu32, x, x);
let mul_lh = map3!(_mm256_mul_epu32, x, x_hi);
let mul_hh = map3!(_mm256_mul_epu32, x_hi, x_hi);
// Bignum addition, but mul_lh is shifted by 33 bits (not 32).
let mul_ll_hi = map3!(_mm256_srli_epi64::<33>, mul_ll);
let t0 = map3!(_mm256_add_epi64, mul_lh, mul_ll_hi);
let t0_hi = map3!(_mm256_srli_epi64::<31>, t0);
let res_hi = map3!(_mm256_add_epi64, mul_hh, t0_hi);
// Form low result by adding the mul_ll and the low 31 bits of mul_lh (shifted to the high
// position).
let mul_lh_lo = map3!(_mm256_slli_epi64::<33>, mul_lh);
let res_lo = map3!(_mm256_add_epi64, mul_ll, mul_lh_lo);
(res_lo, res_hi)
}
#[inline(always)]
unsafe fn mul3(
x: (__m256i, __m256i, __m256i),
y: (__m256i, __m256i, __m256i),
) -> ((__m256i, __m256i, __m256i), (__m256i, __m256i, __m256i)) {
let epsilon = _mm256_set1_epi64x(0xffffffff);
let x_hi = {
// Move high bits to low position. The high bits of x_hi are ignored. Swizzle is faster than
// bitshift. This instruction only has a floating-point flavor, so we cast to/from float.
// This is safe and free.
let x_ps = map3!(_mm256_castsi256_ps, x);
let x_hi_ps = map3!(_mm256_movehdup_ps, x_ps);
map3!(_mm256_castps_si256, x_hi_ps)
};
let y_hi = {
let y_ps = map3!(_mm256_castsi256_ps, y);
let y_hi_ps = map3!(_mm256_movehdup_ps, y_ps);
map3!(_mm256_castps_si256, y_hi_ps)
};
// All four pairwise multiplications
let mul_ll = map3!(_mm256_mul_epu32, x, y);
let mul_lh = map3!(_mm256_mul_epu32, x, y_hi);
let mul_hl = map3!(_mm256_mul_epu32, x_hi, y);
let mul_hh = map3!(_mm256_mul_epu32, x_hi, y_hi);
// Bignum addition
// Extract high 32 bits of mul_ll and add to mul_hl. This cannot overflow.
let mul_ll_hi = map3!(_mm256_srli_epi64::<32>, mul_ll);
let t0 = map3!(_mm256_add_epi64, mul_hl, mul_ll_hi);
// Extract low 32 bits of t0 and add to mul_lh. Again, this cannot overflow.
// Also, extract high 32 bits of t0 and add to mul_hh.
let t0_lo = map3!(_mm256_and_si256, t0, rep epsilon);
let t0_hi = map3!(_mm256_srli_epi64::<32>, t0);
let t1 = map3!(_mm256_add_epi64, mul_lh, t0_lo);
let t2 = map3!(_mm256_add_epi64, mul_hh, t0_hi);
// Lastly, extract the high 32 bits of t1 and add to t2.
let t1_hi = map3!(_mm256_srli_epi64::<32>, t1);
let res_hi = map3!(_mm256_add_epi64, t2, t1_hi);
// Form res_lo by combining the low half of mul_ll with the low half of t1 (shifted into high
// position).
let t1_lo = {
let t1_ps = map3!(_mm256_castsi256_ps, t1);
let t1_lo_ps = map3!(_mm256_moveldup_ps, t1_ps);
map3!(_mm256_castps_si256, t1_lo_ps)
};
let res_lo = map3!(_mm256_blend_epi32::<0xaa>, mul_ll, t1_lo);
(res_lo, res_hi)
}
/// Addition, where the second operand is `0 <= y < 0xffffffff00000001`.
#[inline(always)]
unsafe fn add_small(
x_s: (__m256i, __m256i, __m256i),
y: (__m256i, __m256i, __m256i),
) -> (__m256i, __m256i, __m256i) {
let res_wrapped_s = map3!(_mm256_add_epi64, x_s, y);
let mask = map3!(_mm256_cmpgt_epi32, x_s, res_wrapped_s);
let wrapback_amt = map3!(_mm256_srli_epi64::<32>, mask); // EPSILON if overflowed else 0.
let res_s = map3!(_mm256_add_epi64, res_wrapped_s, wrapback_amt);
res_s
}
#[inline(always)]
unsafe fn maybe_adj_sub(res_wrapped_s: __m256i, mask: __m256i) -> __m256i {
// The subtraction is very unlikely to overflow so we're best off branching.
// The even u32s in `mask` are meaningless, so we want to ignore them. `_mm256_testz_pd`
// branches depending on the sign bit of double-precision (64-bit) floats. Bit cast `mask` to
// floating-point (this is free).
let mask_pd = _mm256_castsi256_pd(mask);
// `_mm256_testz_pd(mask_pd, mask_pd) == 1` iff all sign bits are 0, meaning that underflow
// did not occur for any of the vector elements.
if _mm256_testz_pd(mask_pd, mask_pd) == 1 {
res_wrapped_s
} else {
branch_hint();
// Highly unlikely: underflow did occur. Find adjustment per element and apply it.
let adj_amount = _mm256_srli_epi64::<32>(mask); // EPSILON if underflow.
_mm256_sub_epi64(res_wrapped_s, adj_amount)
}
}
/// Addition, where the second operand is much smaller than `0xffffffff00000001`.
#[inline(always)]
unsafe fn sub_tiny(
x_s: (__m256i, __m256i, __m256i),
y: (__m256i, __m256i, __m256i),
) -> (__m256i, __m256i, __m256i) {
let res_wrapped_s = map3!(_mm256_sub_epi64, x_s, y);
let mask = map3!(_mm256_cmpgt_epi32, res_wrapped_s, x_s);
let res_s = map3!(maybe_adj_sub, res_wrapped_s, mask);
res_s
}
#[inline(always)]
unsafe fn reduce3(
(lo0, hi0): ((__m256i, __m256i, __m256i), (__m256i, __m256i, __m256i)),
) -> (__m256i, __m256i, __m256i) {
let sign_bit = _mm256_set1_epi64x(i64::MIN);
let epsilon = _mm256_set1_epi64x(0xffffffff);
let lo0_s = map3!(_mm256_xor_si256, lo0, rep sign_bit);
let hi_hi0 = map3!(_mm256_srli_epi64::<32>, hi0);
let lo1_s = sub_tiny(lo0_s, hi_hi0);
let t1 = map3!(_mm256_mul_epu32, hi0, rep epsilon);
let lo2_s = add_small(lo1_s, t1);
let lo2 = map3!(_mm256_xor_si256, lo2_s, rep sign_bit);
lo2
}
#[inline(always)]
unsafe fn mul_reduce(
a: (__m256i, __m256i, __m256i),
b: (__m256i, __m256i, __m256i),
) -> (__m256i, __m256i, __m256i) {
reduce3(mul3(a, b))
}
#[inline(always)]
unsafe fn square_reduce(state: (__m256i, __m256i, __m256i)) -> (__m256i, __m256i, __m256i) {
reduce3(square3(state))
}
#[inline(always)]
unsafe fn exp_acc(
high: (__m256i, __m256i, __m256i),
low: (__m256i, __m256i, __m256i),
exp: usize,
) -> (__m256i, __m256i, __m256i) {
let mut result = high;
for _ in 0..exp {
result = square_reduce(result);
}
mul_reduce(result, low)
}
#[inline(always)]
unsafe fn do_apply_sbox(state: (__m256i, __m256i, __m256i)) -> (__m256i, __m256i, __m256i) {
let state2 = square_reduce(state);
let state4_unreduced = square3(state2);
let state3_unreduced = mul3(state2, state);
let state4 = reduce3(state4_unreduced);
let state3 = reduce3(state3_unreduced);
let state7_unreduced = mul3(state3, state4);
let state7 = reduce3(state7_unreduced);
state7
}
#[inline(always)]
unsafe fn do_apply_inv_sbox(state: (__m256i, __m256i, __m256i)) -> (__m256i, __m256i, __m256i) {
// compute base^10540996611094048183 using 72 multiplications per array element
// 10540996611094048183 = b1001001001001001001001001001000110110110110110110110110110110111
// compute base^10
let t1 = square_reduce(state);
// compute base^100
let t2 = square_reduce(t1);
// compute base^100100
let t3 = exp_acc(t2, t2, 3);
// compute base^100100100100
let t4 = exp_acc(t3, t3, 6);
// compute base^100100100100100100100100
let t5 = exp_acc(t4, t4, 12);
// compute base^100100100100100100100100100100
let t6 = exp_acc(t5, t3, 6);
// compute base^1001001001001001001001001001000100100100100100100100100100100
let t7 = exp_acc(t6, t6, 31);
// compute base^1001001001001001001001001001000110110110110110110110110110110111
let a = square_reduce(square_reduce(mul_reduce(square_reduce(t7), t6)));
let b = mul_reduce(t1, mul_reduce(t2, state));
mul_reduce(a, b)
}
#[inline(always)]
unsafe fn avx2_load(state: &[u64; 12]) -> (__m256i, __m256i, __m256i) {
(
_mm256_loadu_si256((&state[0..4]).as_ptr().cast::<__m256i>()),
_mm256_loadu_si256((&state[4..8]).as_ptr().cast::<__m256i>()),
_mm256_loadu_si256((&state[8..12]).as_ptr().cast::<__m256i>()),
)
}
#[inline(always)]
unsafe fn avx2_store(buf: &mut [u64; 12], state: (__m256i, __m256i, __m256i)) {
_mm256_storeu_si256((&mut buf[0..4]).as_mut_ptr().cast::<__m256i>(), state.0);
_mm256_storeu_si256((&mut buf[4..8]).as_mut_ptr().cast::<__m256i>(), state.1);
_mm256_storeu_si256((&mut buf[8..12]).as_mut_ptr().cast::<__m256i>(), state.2);
}
#[inline(always)]
pub unsafe fn apply_sbox(buffer: &mut [u64; 12]) {
let mut state = avx2_load(&buffer);
state = do_apply_sbox(state);
avx2_store(buffer, state);
}
#[inline(always)]
pub unsafe fn apply_inv_sbox(buffer: &mut [u64; 12]) {
let mut state = avx2_load(&buffer);
state = do_apply_inv_sbox(state);
avx2_store(buffer, state);
}

View File

@@ -11,7 +11,8 @@
/// divisions by 2 and repeated modular reductions. This is because of our explicit choice of
/// an MDS matrix that has small powers of 2 entries in frequency domain.
/// The following implementation has benefited greatly from the discussions and insights of
/// Hamish Ivey-Law and Jacqueline Nabaglo of Polygon Zero.
/// Hamish Ivey-Law and Jacqueline Nabaglo of Polygon Zero and is base on Nabaglo's Plonky2
/// implementation.
// Rescue MDS matrix in frequency domain.
// More precisely, this is the output of the three 4-point (real) FFTs of the first column of
@@ -26,7 +27,7 @@ const MDS_FREQ_BLOCK_THREE: [i64; 3] = [-8, 1, 1];
// We use split 3 x 4 FFT transform in order to transform our vectors into the frequency domain.
#[inline(always)]
pub(crate) const fn mds_multiply_freq(state: [u64; 12]) -> [u64; 12] {
pub const fn mds_multiply_freq(state: [u64; 12]) -> [u64; 12] {
let [s0, s1, s2, s3, s4, s5, s6, s7, s8, s9, s10, s11] = state;
let (u0, u1, u2) = fft4_real([s0, s3, s6, s9]);
@@ -156,14 +157,14 @@ const fn block3(x: [i64; 3], y: [i64; 3]) -> [i64; 3] {
#[cfg(test)]
mod tests {
use super::super::{Felt, FieldElement, Rpo256, MDS};
use super::super::{apply_mds, Felt, MDS, ZERO};
use proptest::prelude::*;
const STATE_WIDTH: usize = 12;
#[inline(always)]
fn apply_mds_naive(state: &mut [Felt; STATE_WIDTH]) {
let mut result = [Felt::ZERO; STATE_WIDTH];
let mut result = [ZERO; STATE_WIDTH];
result.iter_mut().zip(MDS).for_each(|(r, mds_row)| {
state.iter().zip(mds_row).for_each(|(&s, m)| {
*r += m * s;
@@ -174,9 +175,9 @@ mod tests {
proptest! {
#[test]
fn mds_freq_proptest(a in any::<[u64;STATE_WIDTH]>()) {
fn mds_freq_proptest(a in any::<[u64; STATE_WIDTH]>()) {
let mut v1 = [Felt::ZERO;STATE_WIDTH];
let mut v1 = [ZERO; STATE_WIDTH];
let mut v2;
for i in 0..STATE_WIDTH {
@@ -185,7 +186,7 @@ mod tests {
v2 = v1;
apply_mds_naive(&mut v1);
Rpo256::apply_mds(&mut v2);
apply_mds(&mut v2);
prop_assert_eq!(v1, v2);
}

214
src/hash/rescue/mds/mod.rs Normal file
View File

@@ -0,0 +1,214 @@
use super::{Felt, STATE_WIDTH, ZERO};
mod freq;
pub use freq::mds_multiply_freq;
// MDS MULTIPLICATION
// ================================================================================================
#[inline(always)]
pub fn apply_mds(state: &mut [Felt; STATE_WIDTH]) {
let mut result = [ZERO; STATE_WIDTH];
// Using the linearity of the operations we can split the state into a low||high decomposition
// and operate on each with no overflow and then combine/reduce the result to a field element.
// The no overflow is guaranteed by the fact that the MDS matrix is a small powers of two in
// frequency domain.
let mut state_l = [0u64; STATE_WIDTH];
let mut state_h = [0u64; STATE_WIDTH];
for r in 0..STATE_WIDTH {
let s = state[r].inner();
state_h[r] = s >> 32;
state_l[r] = (s as u32) as u64;
}
let state_h = mds_multiply_freq(state_h);
let state_l = mds_multiply_freq(state_l);
for r in 0..STATE_WIDTH {
let s = state_l[r] as u128 + ((state_h[r] as u128) << 32);
let s_hi = (s >> 64) as u64;
let s_lo = s as u64;
let z = (s_hi << 32) - s_hi;
let (res, over) = s_lo.overflowing_add(z);
result[r] = Felt::from_mont(res.wrapping_add(0u32.wrapping_sub(over as u32) as u64));
}
*state = result;
}
// MDS MATRIX
// ================================================================================================
/// RPO MDS matrix
pub const MDS: [[Felt; STATE_WIDTH]; STATE_WIDTH] = [
[
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
],
[
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
],
[
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
],
[
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
],
[
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
],
[
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
],
[
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
],
[
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
],
[
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
],
[
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
],
[
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
],
[
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
],
];

348
src/hash/rescue/mod.rs Normal file
View File

@@ -0,0 +1,348 @@
use super::{
CubeExtension, Digest, ElementHasher, Felt, FieldElement, Hasher, StarkField, ONE, ZERO,
};
use core::ops::Range;
mod arch;
pub use arch::optimized::{add_constants_and_apply_inv_sbox, add_constants_and_apply_sbox};
mod mds;
use mds::{apply_mds, MDS};
mod rpo;
pub use rpo::{Rpo256, RpoDigest};
mod rpx;
pub use rpx::{Rpx256, RpxDigest};
#[cfg(test)]
mod tests;
// CONSTANTS
// ================================================================================================
/// The number of rounds is set to 7. For the RPO hash functions all rounds are uniform. For the
/// RPX hash function, there are 3 different types of rounds.
const NUM_ROUNDS: usize = 7;
/// Sponge state is set to 12 field elements or 96 bytes; 8 elements are reserved for rate and
/// the remaining 4 elements are reserved for capacity.
const STATE_WIDTH: usize = 12;
/// The rate portion of the state is located in elements 4 through 11.
const RATE_RANGE: Range<usize> = 4..12;
const RATE_WIDTH: usize = RATE_RANGE.end - RATE_RANGE.start;
const INPUT1_RANGE: Range<usize> = 4..8;
const INPUT2_RANGE: Range<usize> = 8..12;
/// The capacity portion of the state is located in elements 0, 1, 2, and 3.
const CAPACITY_RANGE: Range<usize> = 0..4;
/// The output of the hash function is a digest which consists of 4 field elements or 32 bytes.
///
/// The digest is returned from state elements 4, 5, 6, and 7 (the first four elements of the
/// rate portion).
const DIGEST_RANGE: Range<usize> = 4..8;
const DIGEST_SIZE: usize = DIGEST_RANGE.end - DIGEST_RANGE.start;
/// The number of bytes needed to encoded a digest
const DIGEST_BYTES: usize = 32;
/// The number of byte chunks defining a field element when hashing a sequence of bytes
const BINARY_CHUNK_SIZE: usize = 7;
/// S-Box and Inverse S-Box powers;
///
/// The constants are defined for tests only because the exponentiations in the code are unrolled
/// for efficiency reasons.
#[cfg(test)]
const ALPHA: u64 = 7;
#[cfg(test)]
const INV_ALPHA: u64 = 10540996611094048183;
// SBOX FUNCTION
// ================================================================================================
#[inline(always)]
fn apply_sbox(state: &mut [Felt; STATE_WIDTH]) {
state[0] = state[0].exp7();
state[1] = state[1].exp7();
state[2] = state[2].exp7();
state[3] = state[3].exp7();
state[4] = state[4].exp7();
state[5] = state[5].exp7();
state[6] = state[6].exp7();
state[7] = state[7].exp7();
state[8] = state[8].exp7();
state[9] = state[9].exp7();
state[10] = state[10].exp7();
state[11] = state[11].exp7();
}
// INVERSE SBOX FUNCTION
// ================================================================================================
#[inline(always)]
fn apply_inv_sbox(state: &mut [Felt; STATE_WIDTH]) {
// compute base^10540996611094048183 using 72 multiplications per array element
// 10540996611094048183 = b1001001001001001001001001001000110110110110110110110110110110111
// compute base^10
let mut t1 = *state;
t1.iter_mut().for_each(|t| *t = t.square());
// compute base^100
let mut t2 = t1;
t2.iter_mut().for_each(|t| *t = t.square());
// compute base^100100
let t3 = exp_acc::<Felt, STATE_WIDTH, 3>(t2, t2);
// compute base^100100100100
let t4 = exp_acc::<Felt, STATE_WIDTH, 6>(t3, t3);
// compute base^100100100100100100100100
let t5 = exp_acc::<Felt, STATE_WIDTH, 12>(t4, t4);
// compute base^100100100100100100100100100100
let t6 = exp_acc::<Felt, STATE_WIDTH, 6>(t5, t3);
// compute base^1001001001001001001001001001000100100100100100100100100100100
let t7 = exp_acc::<Felt, STATE_WIDTH, 31>(t6, t6);
// compute base^1001001001001001001001001001000110110110110110110110110110110111
for (i, s) in state.iter_mut().enumerate() {
let a = (t7[i].square() * t6[i]).square().square();
let b = t1[i] * t2[i] * *s;
*s = a * b;
}
#[inline(always)]
fn exp_acc<B: StarkField, const N: usize, const M: usize>(
base: [B; N],
tail: [B; N],
) -> [B; N] {
let mut result = base;
for _ in 0..M {
result.iter_mut().for_each(|r| *r = r.square());
}
result.iter_mut().zip(tail).for_each(|(r, t)| *r *= t);
result
}
}
#[inline(always)]
fn add_constants(state: &mut [Felt; STATE_WIDTH], ark: &[Felt; STATE_WIDTH]) {
state.iter_mut().zip(ark).for_each(|(s, &k)| *s += k);
}
// ROUND CONSTANTS
// ================================================================================================
/// Rescue round constants;
/// computed as in [specifications](https://github.com/ASDiscreteMathematics/rpo)
///
/// The constants are broken up into two arrays ARK1 and ARK2; ARK1 contains the constants for the
/// first half of RPO round, and ARK2 contains constants for the second half of RPO round.
const ARK1: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = [
[
Felt::new(5789762306288267392),
Felt::new(6522564764413701783),
Felt::new(17809893479458208203),
Felt::new(107145243989736508),
Felt::new(6388978042437517382),
Felt::new(15844067734406016715),
Felt::new(9975000513555218239),
Felt::new(3344984123768313364),
Felt::new(9959189626657347191),
Felt::new(12960773468763563665),
Felt::new(9602914297752488475),
Felt::new(16657542370200465908),
],
[
Felt::new(12987190162843096997),
Felt::new(653957632802705281),
Felt::new(4441654670647621225),
Felt::new(4038207883745915761),
Felt::new(5613464648874830118),
Felt::new(13222989726778338773),
Felt::new(3037761201230264149),
Felt::new(16683759727265180203),
Felt::new(8337364536491240715),
Felt::new(3227397518293416448),
Felt::new(8110510111539674682),
Felt::new(2872078294163232137),
],
[
Felt::new(18072785500942327487),
Felt::new(6200974112677013481),
Felt::new(17682092219085884187),
Felt::new(10599526828986756440),
Felt::new(975003873302957338),
Felt::new(8264241093196931281),
Felt::new(10065763900435475170),
Felt::new(2181131744534710197),
Felt::new(6317303992309418647),
Felt::new(1401440938888741532),
Felt::new(8884468225181997494),
Felt::new(13066900325715521532),
],
[
Felt::new(5674685213610121970),
Felt::new(5759084860419474071),
Felt::new(13943282657648897737),
Felt::new(1352748651966375394),
Felt::new(17110913224029905221),
Felt::new(1003883795902368422),
Felt::new(4141870621881018291),
Felt::new(8121410972417424656),
Felt::new(14300518605864919529),
Felt::new(13712227150607670181),
Felt::new(17021852944633065291),
Felt::new(6252096473787587650),
],
[
Felt::new(4887609836208846458),
Felt::new(3027115137917284492),
Felt::new(9595098600469470675),
Felt::new(10528569829048484079),
Felt::new(7864689113198939815),
Felt::new(17533723827845969040),
Felt::new(5781638039037710951),
Felt::new(17024078752430719006),
Felt::new(109659393484013511),
Felt::new(7158933660534805869),
Felt::new(2955076958026921730),
Felt::new(7433723648458773977),
],
[
Felt::new(16308865189192447297),
Felt::new(11977192855656444890),
Felt::new(12532242556065780287),
Felt::new(14594890931430968898),
Felt::new(7291784239689209784),
Felt::new(5514718540551361949),
Felt::new(10025733853830934803),
Felt::new(7293794580341021693),
Felt::new(6728552937464861756),
Felt::new(6332385040983343262),
Felt::new(13277683694236792804),
Felt::new(2600778905124452676),
],
[
Felt::new(7123075680859040534),
Felt::new(1034205548717903090),
Felt::new(7717824418247931797),
Felt::new(3019070937878604058),
Felt::new(11403792746066867460),
Felt::new(10280580802233112374),
Felt::new(337153209462421218),
Felt::new(13333398568519923717),
Felt::new(3596153696935337464),
Felt::new(8104208463525993784),
Felt::new(14345062289456085693),
Felt::new(17036731477169661256),
],
];
const ARK2: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = [
[
Felt::new(6077062762357204287),
Felt::new(15277620170502011191),
Felt::new(5358738125714196705),
Felt::new(14233283787297595718),
Felt::new(13792579614346651365),
Felt::new(11614812331536767105),
Felt::new(14871063686742261166),
Felt::new(10148237148793043499),
Felt::new(4457428952329675767),
Felt::new(15590786458219172475),
Felt::new(10063319113072092615),
Felt::new(14200078843431360086),
],
[
Felt::new(6202948458916099932),
Felt::new(17690140365333231091),
Felt::new(3595001575307484651),
Felt::new(373995945117666487),
Felt::new(1235734395091296013),
Felt::new(14172757457833931602),
Felt::new(707573103686350224),
Felt::new(15453217512188187135),
Felt::new(219777875004506018),
Felt::new(17876696346199469008),
Felt::new(17731621626449383378),
Felt::new(2897136237748376248),
],
[
Felt::new(8023374565629191455),
Felt::new(15013690343205953430),
Felt::new(4485500052507912973),
Felt::new(12489737547229155153),
Felt::new(9500452585969030576),
Felt::new(2054001340201038870),
Felt::new(12420704059284934186),
Felt::new(355990932618543755),
Felt::new(9071225051243523860),
Felt::new(12766199826003448536),
Felt::new(9045979173463556963),
Felt::new(12934431667190679898),
],
[
Felt::new(18389244934624494276),
Felt::new(16731736864863925227),
Felt::new(4440209734760478192),
Felt::new(17208448209698888938),
Felt::new(8739495587021565984),
Felt::new(17000774922218161967),
Felt::new(13533282547195532087),
Felt::new(525402848358706231),
Felt::new(16987541523062161972),
Felt::new(5466806524462797102),
Felt::new(14512769585918244983),
Felt::new(10973956031244051118),
],
[
Felt::new(6982293561042362913),
Felt::new(14065426295947720331),
Felt::new(16451845770444974180),
Felt::new(7139138592091306727),
Felt::new(9012006439959783127),
Felt::new(14619614108529063361),
Felt::new(1394813199588124371),
Felt::new(4635111139507788575),
Felt::new(16217473952264203365),
Felt::new(10782018226466330683),
Felt::new(6844229992533662050),
Felt::new(7446486531695178711),
],
[
Felt::new(3736792340494631448),
Felt::new(577852220195055341),
Felt::new(6689998335515779805),
Felt::new(13886063479078013492),
Felt::new(14358505101923202168),
Felt::new(7744142531772274164),
Felt::new(16135070735728404443),
Felt::new(12290902521256031137),
Felt::new(12059913662657709804),
Felt::new(16456018495793751911),
Felt::new(4571485474751953524),
Felt::new(17200392109565783176),
],
[
Felt::new(17130398059294018733),
Felt::new(519782857322261988),
Felt::new(9625384390925085478),
Felt::new(1664893052631119222),
Felt::new(7629576092524553570),
Felt::new(3485239601103661425),
Felt::new(9755891797164033838),
Felt::new(15218148195153269027),
Felt::new(16460604813734957368),
Felt::new(9643968136937729763),
Felt::new(3611348709641382851),
Felt::new(18256379591337759196),
],
];

View File

@@ -0,0 +1,410 @@
use super::{Digest, Felt, StarkField, DIGEST_BYTES, DIGEST_SIZE, ZERO};
use crate::utils::{
bytes_to_hex_string, hex_to_bytes, string::String, ByteReader, ByteWriter, Deserializable,
DeserializationError, HexParseError, Serializable,
};
use core::{cmp::Ordering, fmt::Display, ops::Deref};
use winter_utils::Randomizable;
// DIGEST TRAIT IMPLEMENTATIONS
// ================================================================================================
#[derive(Debug, Default, Copy, Clone, Eq, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
#[cfg_attr(feature = "serde", serde(into = "String", try_from = "&str"))]
pub struct RpoDigest([Felt; DIGEST_SIZE]);
impl RpoDigest {
pub const fn new(value: [Felt; DIGEST_SIZE]) -> Self {
Self(value)
}
pub fn as_elements(&self) -> &[Felt] {
self.as_ref()
}
pub fn as_bytes(&self) -> [u8; DIGEST_BYTES] {
<Self as Digest>::as_bytes(self)
}
pub fn digests_as_elements<'a, I>(digests: I) -> impl Iterator<Item = &'a Felt>
where
I: Iterator<Item = &'a Self>,
{
digests.flat_map(|d| d.0.iter())
}
/// Returns hexadecimal representation of this digest prefixed with `0x`.
pub fn to_hex(&self) -> String {
bytes_to_hex_string(self.as_bytes())
}
}
impl Digest for RpoDigest {
fn as_bytes(&self) -> [u8; DIGEST_BYTES] {
let mut result = [0; DIGEST_BYTES];
result[..8].copy_from_slice(&self.0[0].as_int().to_le_bytes());
result[8..16].copy_from_slice(&self.0[1].as_int().to_le_bytes());
result[16..24].copy_from_slice(&self.0[2].as_int().to_le_bytes());
result[24..].copy_from_slice(&self.0[3].as_int().to_le_bytes());
result
}
}
impl Deref for RpoDigest {
type Target = [Felt; DIGEST_SIZE];
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl Ord for RpoDigest {
fn cmp(&self, other: &Self) -> Ordering {
// compare the inner u64 of both elements.
//
// it will iterate the elements and will return the first computation different than
// `Equal`. Otherwise, the ordering is equal.
//
// the endianness is irrelevant here because since, this being a cryptographically secure
// hash computation, the digest shouldn't have any ordered property of its input.
//
// finally, we use `Felt::inner` instead of `Felt::as_int` so we avoid performing a
// montgomery reduction for every limb. that is safe because every inner element of the
// digest is guaranteed to be in its canonical form (that is, `x in [0,p)`).
self.0.iter().map(Felt::inner).zip(other.0.iter().map(Felt::inner)).fold(
Ordering::Equal,
|ord, (a, b)| match ord {
Ordering::Equal => a.cmp(&b),
_ => ord,
},
)
}
}
impl PartialOrd for RpoDigest {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl Display for RpoDigest {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
let encoded: String = self.into();
write!(f, "{}", encoded)?;
Ok(())
}
}
impl Randomizable for RpoDigest {
const VALUE_SIZE: usize = DIGEST_BYTES;
fn from_random_bytes(bytes: &[u8]) -> Option<Self> {
let bytes_array: Option<[u8; 32]> = bytes.try_into().ok();
if let Some(bytes_array) = bytes_array {
Self::try_from(bytes_array).ok()
} else {
None
}
}
}
// CONVERSIONS: FROM RPO DIGEST
// ================================================================================================
impl From<&RpoDigest> for [Felt; DIGEST_SIZE] {
fn from(value: &RpoDigest) -> Self {
value.0
}
}
impl From<RpoDigest> for [Felt; DIGEST_SIZE] {
fn from(value: RpoDigest) -> Self {
value.0
}
}
impl From<&RpoDigest> for [u64; DIGEST_SIZE] {
fn from(value: &RpoDigest) -> Self {
[
value.0[0].as_int(),
value.0[1].as_int(),
value.0[2].as_int(),
value.0[3].as_int(),
]
}
}
impl From<RpoDigest> for [u64; DIGEST_SIZE] {
fn from(value: RpoDigest) -> Self {
[
value.0[0].as_int(),
value.0[1].as_int(),
value.0[2].as_int(),
value.0[3].as_int(),
]
}
}
impl From<&RpoDigest> for [u8; DIGEST_BYTES] {
fn from(value: &RpoDigest) -> Self {
value.as_bytes()
}
}
impl From<RpoDigest> for [u8; DIGEST_BYTES] {
fn from(value: RpoDigest) -> Self {
value.as_bytes()
}
}
impl From<RpoDigest> for String {
/// The returned string starts with `0x`.
fn from(value: RpoDigest) -> Self {
value.to_hex()
}
}
impl From<&RpoDigest> for String {
/// The returned string starts with `0x`.
fn from(value: &RpoDigest) -> Self {
(*value).into()
}
}
// CONVERSIONS: TO RPO DIGEST
// ================================================================================================
#[derive(Copy, Clone, Debug)]
pub enum RpoDigestError {
/// The provided u64 integer does not fit in the field's moduli.
InvalidInteger,
}
impl From<&[Felt; DIGEST_SIZE]> for RpoDigest {
fn from(value: &[Felt; DIGEST_SIZE]) -> Self {
Self(*value)
}
}
impl From<[Felt; DIGEST_SIZE]> for RpoDigest {
fn from(value: [Felt; DIGEST_SIZE]) -> Self {
Self(value)
}
}
impl TryFrom<[u8; DIGEST_BYTES]> for RpoDigest {
type Error = HexParseError;
fn try_from(value: [u8; DIGEST_BYTES]) -> Result<Self, Self::Error> {
// Note: the input length is known, the conversion from slice to array must succeed so the
// `unwrap`s below are safe
let a = u64::from_le_bytes(value[0..8].try_into().unwrap());
let b = u64::from_le_bytes(value[8..16].try_into().unwrap());
let c = u64::from_le_bytes(value[16..24].try_into().unwrap());
let d = u64::from_le_bytes(value[24..32].try_into().unwrap());
if [a, b, c, d].iter().any(|v| *v >= Felt::MODULUS) {
return Err(HexParseError::OutOfRange);
}
Ok(RpoDigest([Felt::new(a), Felt::new(b), Felt::new(c), Felt::new(d)]))
}
}
impl TryFrom<&[u8; DIGEST_BYTES]> for RpoDigest {
type Error = HexParseError;
fn try_from(value: &[u8; DIGEST_BYTES]) -> Result<Self, Self::Error> {
(*value).try_into()
}
}
impl TryFrom<&[u8]> for RpoDigest {
type Error = HexParseError;
fn try_from(value: &[u8]) -> Result<Self, Self::Error> {
(*value).try_into()
}
}
impl TryFrom<[u64; DIGEST_SIZE]> for RpoDigest {
type Error = RpoDigestError;
fn try_from(value: [u64; DIGEST_SIZE]) -> Result<Self, RpoDigestError> {
Ok(Self([
value[0].try_into().map_err(|_| RpoDigestError::InvalidInteger)?,
value[1].try_into().map_err(|_| RpoDigestError::InvalidInteger)?,
value[2].try_into().map_err(|_| RpoDigestError::InvalidInteger)?,
value[3].try_into().map_err(|_| RpoDigestError::InvalidInteger)?,
]))
}
}
impl TryFrom<&[u64; DIGEST_SIZE]> for RpoDigest {
type Error = RpoDigestError;
fn try_from(value: &[u64; DIGEST_SIZE]) -> Result<Self, RpoDigestError> {
(*value).try_into()
}
}
impl TryFrom<&str> for RpoDigest {
type Error = HexParseError;
/// Expects the string to start with `0x`.
fn try_from(value: &str) -> Result<Self, Self::Error> {
hex_to_bytes(value).and_then(|v| v.try_into())
}
}
impl TryFrom<String> for RpoDigest {
type Error = HexParseError;
/// Expects the string to start with `0x`.
fn try_from(value: String) -> Result<Self, Self::Error> {
value.as_str().try_into()
}
}
impl TryFrom<&String> for RpoDigest {
type Error = HexParseError;
/// Expects the string to start with `0x`.
fn try_from(value: &String) -> Result<Self, Self::Error> {
value.as_str().try_into()
}
}
// SERIALIZATION / DESERIALIZATION
// ================================================================================================
impl Serializable for RpoDigest {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
target.write_bytes(&self.as_bytes());
}
}
impl Deserializable for RpoDigest {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let mut inner: [Felt; DIGEST_SIZE] = [ZERO; DIGEST_SIZE];
for inner in inner.iter_mut() {
let e = source.read_u64()?;
if e >= Felt::MODULUS {
return Err(DeserializationError::InvalidValue(String::from(
"Value not in the appropriate range",
)));
}
*inner = Felt::new(e);
}
Ok(Self(inner))
}
}
// ITERATORS
// ================================================================================================
impl IntoIterator for RpoDigest {
type Item = Felt;
type IntoIter = <[Felt; 4] as IntoIterator>::IntoIter;
fn into_iter(self) -> Self::IntoIter {
self.0.into_iter()
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::{Deserializable, Felt, RpoDigest, Serializable, DIGEST_BYTES, DIGEST_SIZE};
use crate::utils::{string::String, SliceReader};
use rand_utils::rand_value;
#[test]
fn digest_serialization() {
let e1 = Felt::new(rand_value());
let e2 = Felt::new(rand_value());
let e3 = Felt::new(rand_value());
let e4 = Felt::new(rand_value());
let d1 = RpoDigest([e1, e2, e3, e4]);
let mut bytes = vec![];
d1.write_into(&mut bytes);
assert_eq!(DIGEST_BYTES, bytes.len());
let mut reader = SliceReader::new(&bytes);
let d2 = RpoDigest::read_from(&mut reader).unwrap();
assert_eq!(d1, d2);
}
#[test]
fn digest_encoding() {
let digest = RpoDigest([
Felt::new(rand_value()),
Felt::new(rand_value()),
Felt::new(rand_value()),
Felt::new(rand_value()),
]);
let string: String = digest.into();
let round_trip: RpoDigest = string.try_into().expect("decoding failed");
assert_eq!(digest, round_trip);
}
#[test]
fn test_conversions() {
let digest = RpoDigest([
Felt::new(rand_value()),
Felt::new(rand_value()),
Felt::new(rand_value()),
Felt::new(rand_value()),
]);
let v: [Felt; DIGEST_SIZE] = digest.into();
let v2: RpoDigest = v.into();
assert_eq!(digest, v2);
let v: [Felt; DIGEST_SIZE] = (&digest).into();
let v2: RpoDigest = v.into();
assert_eq!(digest, v2);
let v: [u64; DIGEST_SIZE] = digest.into();
let v2: RpoDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: [u64; DIGEST_SIZE] = (&digest).into();
let v2: RpoDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: [u8; DIGEST_BYTES] = digest.into();
let v2: RpoDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: [u8; DIGEST_BYTES] = (&digest).into();
let v2: RpoDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: String = digest.into();
let v2: RpoDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: String = (&digest).into();
let v2: RpoDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: [u8; DIGEST_BYTES] = digest.into();
let v2: RpoDigest = (&v).try_into().unwrap();
assert_eq!(digest, v2);
let v: [u8; DIGEST_BYTES] = (&digest).into();
let v2: RpoDigest = (&v).try_into().unwrap();
assert_eq!(digest, v2);
}
}

317
src/hash/rescue/rpo/mod.rs Normal file
View File

@@ -0,0 +1,317 @@
use super::{
add_constants, add_constants_and_apply_inv_sbox, add_constants_and_apply_sbox, apply_inv_sbox,
apply_mds, apply_sbox, Digest, ElementHasher, Felt, FieldElement, Hasher, StarkField, ARK1,
ARK2, BINARY_CHUNK_SIZE, CAPACITY_RANGE, DIGEST_BYTES, DIGEST_RANGE, DIGEST_SIZE, INPUT1_RANGE,
INPUT2_RANGE, MDS, NUM_ROUNDS, ONE, RATE_RANGE, RATE_WIDTH, STATE_WIDTH, ZERO,
};
use core::{convert::TryInto, ops::Range};
mod digest;
pub use digest::RpoDigest;
#[cfg(test)]
mod tests;
// HASHER IMPLEMENTATION
// ================================================================================================
/// Implementation of the Rescue Prime Optimized hash function with 256-bit output.
///
/// The hash function is implemented according to the Rescue Prime Optimized
/// [specifications](https://eprint.iacr.org/2022/1577)
///
/// The parameters used to instantiate the function are:
/// * Field: 64-bit prime field with modulus 2^64 - 2^32 + 1.
/// * State width: 12 field elements.
/// * Capacity size: 4 field elements.
/// * Number of founds: 7.
/// * S-Box degree: 7.
///
/// The above parameters target a 128-bit security level. The digest consists of four field elements
/// and it can be serialized into 32 bytes (256 bits).
///
/// ## Hash output consistency
/// Functions [hash_elements()](Rpo256::hash_elements), [merge()](Rpo256::merge), and
/// [merge_with_int()](Rpo256::merge_with_int) are internally consistent. That is, computing
/// a hash for the same set of elements using these functions will always produce the same
/// result. For example, merging two digests using [merge()](Rpo256::merge) will produce the
/// same result as hashing 8 elements which make up these digests using
/// [hash_elements()](Rpo256::hash_elements) function.
///
/// However, [hash()](Rpo256::hash) function is not consistent with functions mentioned above.
/// For example, if we take two field elements, serialize them to bytes and hash them using
/// [hash()](Rpo256::hash), the result will differ from the result obtained by hashing these
/// elements directly using [hash_elements()](Rpo256::hash_elements) function. The reason for
/// this difference is that [hash()](Rpo256::hash) function needs to be able to handle
/// arbitrary binary strings, which may or may not encode valid field elements - and thus,
/// deserialization procedure used by this function is different from the procedure used to
/// deserialize valid field elements.
///
/// Thus, if the underlying data consists of valid field elements, it might make more sense
/// to deserialize them into field elements and then hash them using
/// [hash_elements()](Rpo256::hash_elements) function rather then hashing the serialized bytes
/// using [hash()](Rpo256::hash) function.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct Rpo256();
impl Hasher for Rpo256 {
/// Rpo256 collision resistance is 128-bits.
const COLLISION_RESISTANCE: u32 = 128;
type Digest = RpoDigest;
fn hash(bytes: &[u8]) -> Self::Digest {
// initialize the state with zeroes
let mut state = [ZERO; STATE_WIDTH];
// set the capacity (first element) to a flag on whether or not the input length is evenly
// divided by the rate. this will prevent collisions between padded and non-padded inputs,
// and will rule out the need to perform an extra permutation in case of evenly divided
// inputs.
let is_rate_multiple = bytes.len() % RATE_WIDTH == 0;
if !is_rate_multiple {
state[CAPACITY_RANGE.start] = ONE;
}
// initialize a buffer to receive the little-endian elements.
let mut buf = [0_u8; 8];
// iterate the chunks of bytes, creating a field element from each chunk and copying it
// into the state.
//
// every time the rate range is filled, a permutation is performed. if the final value of
// `i` is not zero, then the chunks count wasn't enough to fill the state range, and an
// additional permutation must be performed.
let i = bytes.chunks(BINARY_CHUNK_SIZE).fold(0, |i, chunk| {
// the last element of the iteration may or may not be a full chunk. if it's not, then
// we need to pad the remainder bytes of the chunk with zeroes, separated by a `1`.
// this will avoid collisions.
if chunk.len() == BINARY_CHUNK_SIZE {
buf[..BINARY_CHUNK_SIZE].copy_from_slice(chunk);
} else {
buf.fill(0);
buf[..chunk.len()].copy_from_slice(chunk);
buf[chunk.len()] = 1;
}
// set the current rate element to the input. since we take at most 7 bytes, we are
// guaranteed that the inputs data will fit into a single field element.
state[RATE_RANGE.start + i] = Felt::new(u64::from_le_bytes(buf));
// proceed filling the range. if it's full, then we apply a permutation and reset the
// counter to the beginning of the range.
if i == RATE_WIDTH - 1 {
Self::apply_permutation(&mut state);
0
} else {
i + 1
}
});
// if we absorbed some elements but didn't apply a permutation to them (would happen when
// the number of elements is not a multiple of RATE_WIDTH), apply the RPO permutation. we
// don't need to apply any extra padding because the first capacity element contains a
// flag indicating whether the input is evenly divisible by the rate.
if i != 0 {
state[RATE_RANGE.start + i..RATE_RANGE.end].fill(ZERO);
state[RATE_RANGE.start + i] = ONE;
Self::apply_permutation(&mut state);
}
// return the first 4 elements of the rate as hash result.
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
fn merge(values: &[Self::Digest; 2]) -> Self::Digest {
// initialize the state by copying the digest elements into the rate portion of the state
// (8 total elements), and set the capacity elements to 0.
let mut state = [ZERO; STATE_WIDTH];
let it = Self::Digest::digests_as_elements(values.iter());
for (i, v) in it.enumerate() {
state[RATE_RANGE.start + i] = *v;
}
// apply the RPO permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
fn merge_with_int(seed: Self::Digest, value: u64) -> Self::Digest {
// initialize the state as follows:
// - seed is copied into the first 4 elements of the rate portion of the state.
// - if the value fits into a single field element, copy it into the fifth rate element
// and set the sixth rate element to 1.
// - if the value doesn't fit into a single field element, split it into two field
// elements, copy them into rate elements 5 and 6, and set the seventh rate element
// to 1.
// - set the first capacity element to 1
let mut state = [ZERO; STATE_WIDTH];
state[INPUT1_RANGE].copy_from_slice(seed.as_elements());
state[INPUT2_RANGE.start] = Felt::new(value);
if value < Felt::MODULUS {
state[INPUT2_RANGE.start + 1] = ONE;
} else {
state[INPUT2_RANGE.start + 1] = Felt::new(value / Felt::MODULUS);
state[INPUT2_RANGE.start + 2] = ONE;
}
// common padding for both cases
state[CAPACITY_RANGE.start] = ONE;
// apply the RPO permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
}
impl ElementHasher for Rpo256 {
type BaseField = Felt;
fn hash_elements<E: FieldElement<BaseField = Self::BaseField>>(elements: &[E]) -> Self::Digest {
// convert the elements into a list of base field elements
let elements = E::slice_as_base_elements(elements);
// initialize state to all zeros, except for the first element of the capacity part, which
// is set to 1 if the number of elements is not a multiple of RATE_WIDTH.
let mut state = [ZERO; STATE_WIDTH];
if elements.len() % RATE_WIDTH != 0 {
state[CAPACITY_RANGE.start] = ONE;
}
// absorb elements into the state one by one until the rate portion of the state is filled
// up; then apply the Rescue permutation and start absorbing again; repeat until all
// elements have been absorbed
let mut i = 0;
for &element in elements.iter() {
state[RATE_RANGE.start + i] = element;
i += 1;
if i % RATE_WIDTH == 0 {
Self::apply_permutation(&mut state);
i = 0;
}
}
// if we absorbed some elements but didn't apply a permutation to them (would happen when
// the number of elements is not a multiple of RATE_WIDTH), apply the RPO permutation after
// padding by appending a 1 followed by as many 0 as necessary to make the input length a
// multiple of the RATE_WIDTH.
if i > 0 {
state[RATE_RANGE.start + i] = ONE;
i += 1;
while i != RATE_WIDTH {
state[RATE_RANGE.start + i] = ZERO;
i += 1;
}
Self::apply_permutation(&mut state);
}
// return the first 4 elements of the state as hash result
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
}
// HASH FUNCTION IMPLEMENTATION
// ================================================================================================
impl Rpo256 {
// CONSTANTS
// --------------------------------------------------------------------------------------------
/// The number of rounds is set to 7 to target 128-bit security level.
pub const NUM_ROUNDS: usize = NUM_ROUNDS;
/// Sponge state is set to 12 field elements or 768 bytes; 8 elements are reserved for rate and
/// the remaining 4 elements are reserved for capacity.
pub const STATE_WIDTH: usize = STATE_WIDTH;
/// The rate portion of the state is located in elements 4 through 11 (inclusive).
pub const RATE_RANGE: Range<usize> = RATE_RANGE;
/// The capacity portion of the state is located in elements 0, 1, 2, and 3.
pub const CAPACITY_RANGE: Range<usize> = CAPACITY_RANGE;
/// The output of the hash function can be read from state elements 4, 5, 6, and 7.
pub const DIGEST_RANGE: Range<usize> = DIGEST_RANGE;
/// MDS matrix used for computing the linear layer in a RPO round.
pub const MDS: [[Felt; STATE_WIDTH]; STATE_WIDTH] = MDS;
/// Round constants added to the hasher state in the first half of the RPO round.
pub const ARK1: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = ARK1;
/// Round constants added to the hasher state in the second half of the RPO round.
pub const ARK2: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = ARK2;
// TRAIT PASS-THROUGH FUNCTIONS
// --------------------------------------------------------------------------------------------
/// Returns a hash of the provided sequence of bytes.
#[inline(always)]
pub fn hash(bytes: &[u8]) -> RpoDigest {
<Self as Hasher>::hash(bytes)
}
/// Returns a hash of two digests. This method is intended for use in construction of
/// Merkle trees and verification of Merkle paths.
#[inline(always)]
pub fn merge(values: &[RpoDigest; 2]) -> RpoDigest {
<Self as Hasher>::merge(values)
}
/// Returns a hash of the provided field elements.
#[inline(always)]
pub fn hash_elements<E: FieldElement<BaseField = Felt>>(elements: &[E]) -> RpoDigest {
<Self as ElementHasher>::hash_elements(elements)
}
// DOMAIN IDENTIFIER
// --------------------------------------------------------------------------------------------
/// Returns a hash of two digests and a domain identifier.
pub fn merge_in_domain(values: &[RpoDigest; 2], domain: Felt) -> RpoDigest {
// initialize the state by copying the digest elements into the rate portion of the state
// (8 total elements), and set the capacity elements to 0.
let mut state = [ZERO; STATE_WIDTH];
let it = RpoDigest::digests_as_elements(values.iter());
for (i, v) in it.enumerate() {
state[RATE_RANGE.start + i] = *v;
}
// set the second capacity element to the domain value. The first capacity element is used
// for padding purposes.
state[CAPACITY_RANGE.start + 1] = domain;
// apply the RPO permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
// RESCUE PERMUTATION
// --------------------------------------------------------------------------------------------
/// Applies RPO permutation to the provided state.
#[inline(always)]
pub fn apply_permutation(state: &mut [Felt; STATE_WIDTH]) {
for i in 0..NUM_ROUNDS {
Self::apply_round(state, i);
}
}
/// RPO round function.
#[inline(always)]
pub fn apply_round(state: &mut [Felt; STATE_WIDTH], round: usize) {
// apply first half of RPO round
apply_mds(state);
if !add_constants_and_apply_sbox(state, &ARK1[round]) {
add_constants(state, &ARK1[round]);
apply_sbox(state);
}
// apply second half of RPO round
apply_mds(state);
if !add_constants_and_apply_inv_sbox(state, &ARK2[round]) {
add_constants(state, &ARK2[round]);
apply_inv_sbox(state);
}
}
}

View File

@@ -1,19 +1,15 @@
use super::{
Felt, FieldElement, Hasher, Rpo256, RpoDigest, StarkField, ALPHA, INV_ALPHA, ONE, STATE_WIDTH,
ZERO,
super::{apply_inv_sbox, apply_sbox, ALPHA, INV_ALPHA},
Felt, FieldElement, Hasher, Rpo256, RpoDigest, StarkField, ONE, STATE_WIDTH, ZERO,
};
use crate::{
utils::collections::{BTreeSet, Vec},
Word,
};
use crate::utils::collections::{BTreeSet, Vec};
use core::convert::TryInto;
use proptest::prelude::*;
use rand_utils::rand_value;
#[test]
fn test_alphas() {
let e: Felt = Felt::new(rand_value());
let e_exp = e.exp(ALPHA);
assert_eq!(e, e_exp.exp(INV_ALPHA));
}
#[test]
fn test_sbox() {
let state = [Felt::new(rand_value()); STATE_WIDTH];
@@ -22,7 +18,7 @@ fn test_sbox() {
expected.iter_mut().for_each(|v| *v = v.exp(ALPHA));
let mut actual = state;
Rpo256::apply_sbox(&mut actual);
apply_sbox(&mut actual);
assert_eq!(expected, actual);
}
@@ -35,7 +31,7 @@ fn test_inv_sbox() {
expected.iter_mut().for_each(|v| *v = v.exp(INV_ALPHA));
let mut actual = state;
Rpo256::apply_inv_sbox(&mut actual);
apply_inv_sbox(&mut actual);
assert_eq!(expected, actual);
}
@@ -102,7 +98,7 @@ fn hash_elements_vs_merge_with_int() {
let mut elements = seed.as_elements().to_vec();
elements.push(Felt::new(val));
elements.push(Felt::new(1));
elements.push(ONE);
let h_result = Rpo256::hash_elements(&elements);
assert_eq!(m_result, h_result);
@@ -144,8 +140,8 @@ fn hash_elements_padding() {
#[test]
fn hash_elements() {
let elements = [
Felt::new(0),
Felt::new(1),
ZERO,
ONE,
Felt::new(2),
Felt::new(3),
Felt::new(4),
@@ -167,8 +163,8 @@ fn hash_elements() {
#[test]
fn hash_test_vectors() {
let elements = [
Felt::new(0),
Felt::new(1),
ZERO,
ONE,
Felt::new(2),
Felt::new(3),
Felt::new(4),
@@ -203,7 +199,7 @@ fn sponge_bytes_with_remainder_length_wont_panic() {
// size.
//
// this is a preliminary test to the fuzzy-stress of proptest.
Rpo256::hash(&vec![0; 113]);
Rpo256::hash(&[0; 113]);
}
#[test]
@@ -227,12 +223,12 @@ fn sponge_zeroes_collision() {
proptest! {
#[test]
fn rpo256_wont_panic_with_arbitrary_input(ref vec in any::<Vec<u8>>()) {
Rpo256::hash(&vec);
fn rpo256_wont_panic_with_arbitrary_input(ref bytes in any::<Vec<u8>>()) {
Rpo256::hash(bytes);
}
}
const EXPECTED: [[Felt; 4]; 19] = [
const EXPECTED: [Word; 19] = [
[
Felt::new(1502364727743950833),
Felt::new(5880949717274681448),

View File

@@ -0,0 +1,400 @@
use super::{Digest, Felt, StarkField, DIGEST_BYTES, DIGEST_SIZE, ZERO};
use crate::utils::{
bytes_to_hex_string, hex_to_bytes, string::String, ByteReader, ByteWriter, Deserializable,
DeserializationError, HexParseError, Serializable,
};
use core::{cmp::Ordering, fmt::Display, ops::Deref};
use winter_utils::Randomizable;
// DIGEST TRAIT IMPLEMENTATIONS
// ================================================================================================
#[derive(Debug, Default, Copy, Clone, Eq, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
#[cfg_attr(feature = "serde", serde(into = "String", try_from = "&str"))]
pub struct RpxDigest([Felt; DIGEST_SIZE]);
impl RpxDigest {
pub const fn new(value: [Felt; DIGEST_SIZE]) -> Self {
Self(value)
}
pub fn as_elements(&self) -> &[Felt] {
self.as_ref()
}
pub fn as_bytes(&self) -> [u8; DIGEST_BYTES] {
<Self as Digest>::as_bytes(self)
}
pub fn digests_as_elements<'a, I>(digests: I) -> impl Iterator<Item = &'a Felt>
where
I: Iterator<Item = &'a Self>,
{
digests.flat_map(|d| d.0.iter())
}
/// Returns hexadecimal representation of this digest prefixed with `0x`.
pub fn to_hex(&self) -> String {
bytes_to_hex_string(self.as_bytes())
}
}
impl Digest for RpxDigest {
fn as_bytes(&self) -> [u8; DIGEST_BYTES] {
let mut result = [0; DIGEST_BYTES];
result[..8].copy_from_slice(&self.0[0].as_int().to_le_bytes());
result[8..16].copy_from_slice(&self.0[1].as_int().to_le_bytes());
result[16..24].copy_from_slice(&self.0[2].as_int().to_le_bytes());
result[24..].copy_from_slice(&self.0[3].as_int().to_le_bytes());
result
}
}
impl Deref for RpxDigest {
type Target = [Felt; DIGEST_SIZE];
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl Ord for RpxDigest {
fn cmp(&self, other: &Self) -> Ordering {
// compare the inner u64 of both elements.
//
// it will iterate the elements and will return the first computation different than
// `Equal`. Otherwise, the ordering is equal.
//
// the endianness is irrelevant here because since, this being a cryptographically secure
// hash computation, the digest shouldn't have any ordered property of its input.
//
// finally, we use `Felt::inner` instead of `Felt::as_int` so we avoid performing a
// montgomery reduction for every limb. that is safe because every inner element of the
// digest is guaranteed to be in its canonical form (that is, `x in [0,p)`).
self.0.iter().map(Felt::inner).zip(other.0.iter().map(Felt::inner)).fold(
Ordering::Equal,
|ord, (a, b)| match ord {
Ordering::Equal => a.cmp(&b),
_ => ord,
},
)
}
}
impl PartialOrd for RpxDigest {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
impl Display for RpxDigest {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
let encoded: String = self.into();
write!(f, "{}", encoded)?;
Ok(())
}
}
impl Randomizable for RpxDigest {
const VALUE_SIZE: usize = DIGEST_BYTES;
fn from_random_bytes(bytes: &[u8]) -> Option<Self> {
let bytes_array: Option<[u8; 32]> = bytes.try_into().ok();
if let Some(bytes_array) = bytes_array {
Self::try_from(bytes_array).ok()
} else {
None
}
}
}
// CONVERSIONS: FROM RPX DIGEST
// ================================================================================================
impl From<&RpxDigest> for [Felt; DIGEST_SIZE] {
fn from(value: &RpxDigest) -> Self {
value.0
}
}
impl From<RpxDigest> for [Felt; DIGEST_SIZE] {
fn from(value: RpxDigest) -> Self {
value.0
}
}
impl From<&RpxDigest> for [u64; DIGEST_SIZE] {
fn from(value: &RpxDigest) -> Self {
[
value.0[0].as_int(),
value.0[1].as_int(),
value.0[2].as_int(),
value.0[3].as_int(),
]
}
}
impl From<RpxDigest> for [u64; DIGEST_SIZE] {
fn from(value: RpxDigest) -> Self {
[
value.0[0].as_int(),
value.0[1].as_int(),
value.0[2].as_int(),
value.0[3].as_int(),
]
}
}
impl From<&RpxDigest> for [u8; DIGEST_BYTES] {
fn from(value: &RpxDigest) -> Self {
value.as_bytes()
}
}
impl From<RpxDigest> for [u8; DIGEST_BYTES] {
fn from(value: RpxDigest) -> Self {
value.as_bytes()
}
}
impl From<RpxDigest> for String {
/// The returned string starts with `0x`.
fn from(value: RpxDigest) -> Self {
value.to_hex()
}
}
impl From<&RpxDigest> for String {
/// The returned string starts with `0x`.
fn from(value: &RpxDigest) -> Self {
(*value).into()
}
}
// CONVERSIONS: TO RPX DIGEST
// ================================================================================================
#[derive(Copy, Clone, Debug)]
pub enum RpxDigestError {
/// The provided u64 integer does not fit in the field's moduli.
InvalidInteger,
}
impl From<&[Felt; DIGEST_SIZE]> for RpxDigest {
fn from(value: &[Felt; DIGEST_SIZE]) -> Self {
Self(*value)
}
}
impl From<[Felt; DIGEST_SIZE]> for RpxDigest {
fn from(value: [Felt; DIGEST_SIZE]) -> Self {
Self(value)
}
}
impl TryFrom<[u8; DIGEST_BYTES]> for RpxDigest {
type Error = HexParseError;
fn try_from(value: [u8; DIGEST_BYTES]) -> Result<Self, Self::Error> {
// Note: the input length is known, the conversion from slice to array must succeed so the
// `unwrap`s below are safe
let a = u64::from_le_bytes(value[0..8].try_into().unwrap());
let b = u64::from_le_bytes(value[8..16].try_into().unwrap());
let c = u64::from_le_bytes(value[16..24].try_into().unwrap());
let d = u64::from_le_bytes(value[24..32].try_into().unwrap());
if [a, b, c, d].iter().any(|v| *v >= Felt::MODULUS) {
return Err(HexParseError::OutOfRange);
}
Ok(RpxDigest([Felt::new(a), Felt::new(b), Felt::new(c), Felt::new(d)]))
}
}
impl TryFrom<&[u8; DIGEST_BYTES]> for RpxDigest {
type Error = HexParseError;
fn try_from(value: &[u8; DIGEST_BYTES]) -> Result<Self, Self::Error> {
(*value).try_into()
}
}
impl TryFrom<&[u8]> for RpxDigest {
type Error = HexParseError;
fn try_from(value: &[u8]) -> Result<Self, Self::Error> {
(*value).try_into()
}
}
impl TryFrom<[u64; DIGEST_SIZE]> for RpxDigest {
type Error = RpxDigestError;
fn try_from(value: [u64; DIGEST_SIZE]) -> Result<Self, RpxDigestError> {
Ok(Self([
value[0].try_into().map_err(|_| RpxDigestError::InvalidInteger)?,
value[1].try_into().map_err(|_| RpxDigestError::InvalidInteger)?,
value[2].try_into().map_err(|_| RpxDigestError::InvalidInteger)?,
value[3].try_into().map_err(|_| RpxDigestError::InvalidInteger)?,
]))
}
}
impl TryFrom<&[u64; DIGEST_SIZE]> for RpxDigest {
type Error = RpxDigestError;
fn try_from(value: &[u64; DIGEST_SIZE]) -> Result<Self, RpxDigestError> {
(*value).try_into()
}
}
impl TryFrom<&str> for RpxDigest {
type Error = HexParseError;
/// Expects the string to start with `0x`.
fn try_from(value: &str) -> Result<Self, Self::Error> {
hex_to_bytes(value).and_then(|v| v.try_into())
}
}
impl TryFrom<String> for RpxDigest {
type Error = HexParseError;
/// Expects the string to start with `0x`.
fn try_from(value: String) -> Result<Self, Self::Error> {
value.as_str().try_into()
}
}
impl TryFrom<&String> for RpxDigest {
type Error = HexParseError;
/// Expects the string to start with `0x`.
fn try_from(value: &String) -> Result<Self, Self::Error> {
value.as_str().try_into()
}
}
// SERIALIZATION / DESERIALIZATION
// ================================================================================================
impl Serializable for RpxDigest {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
target.write_bytes(&self.as_bytes());
}
}
impl Deserializable for RpxDigest {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let mut inner: [Felt; DIGEST_SIZE] = [ZERO; DIGEST_SIZE];
for inner in inner.iter_mut() {
let e = source.read_u64()?;
if e >= Felt::MODULUS {
return Err(DeserializationError::InvalidValue(String::from(
"Value not in the appropriate range",
)));
}
*inner = Felt::new(e);
}
Ok(Self(inner))
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::{Deserializable, Felt, RpxDigest, Serializable, DIGEST_BYTES, DIGEST_SIZE};
use crate::utils::{string::String, SliceReader};
use rand_utils::rand_value;
#[test]
fn digest_serialization() {
let e1 = Felt::new(rand_value());
let e2 = Felt::new(rand_value());
let e3 = Felt::new(rand_value());
let e4 = Felt::new(rand_value());
let d1 = RpxDigest([e1, e2, e3, e4]);
let mut bytes = vec![];
d1.write_into(&mut bytes);
assert_eq!(DIGEST_BYTES, bytes.len());
let mut reader = SliceReader::new(&bytes);
let d2 = RpxDigest::read_from(&mut reader).unwrap();
assert_eq!(d1, d2);
}
#[cfg(feature = "std")]
#[test]
fn digest_encoding() {
let digest = RpxDigest([
Felt::new(rand_value()),
Felt::new(rand_value()),
Felt::new(rand_value()),
Felt::new(rand_value()),
]);
let string: String = digest.into();
let round_trip: RpxDigest = string.try_into().expect("decoding failed");
assert_eq!(digest, round_trip);
}
#[test]
fn test_conversions() {
let digest = RpxDigest([
Felt::new(rand_value()),
Felt::new(rand_value()),
Felt::new(rand_value()),
Felt::new(rand_value()),
]);
let v: [Felt; DIGEST_SIZE] = digest.into();
let v2: RpxDigest = v.into();
assert_eq!(digest, v2);
let v: [Felt; DIGEST_SIZE] = (&digest).into();
let v2: RpxDigest = v.into();
assert_eq!(digest, v2);
let v: [u64; DIGEST_SIZE] = digest.into();
let v2: RpxDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: [u64; DIGEST_SIZE] = (&digest).into();
let v2: RpxDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: [u8; DIGEST_BYTES] = digest.into();
let v2: RpxDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: [u8; DIGEST_BYTES] = (&digest).into();
let v2: RpxDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: String = digest.into();
let v2: RpxDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: String = (&digest).into();
let v2: RpxDigest = v.try_into().unwrap();
assert_eq!(digest, v2);
let v: [u8; DIGEST_BYTES] = digest.into();
let v2: RpxDigest = (&v).try_into().unwrap();
assert_eq!(digest, v2);
let v: [u8; DIGEST_BYTES] = (&digest).into();
let v2: RpxDigest = (&v).try_into().unwrap();
assert_eq!(digest, v2);
}
}

352
src/hash/rescue/rpx/mod.rs Normal file
View File

@@ -0,0 +1,352 @@
use super::{
add_constants, add_constants_and_apply_inv_sbox, add_constants_and_apply_sbox, apply_inv_sbox,
apply_mds, apply_sbox, CubeExtension, Digest, ElementHasher, Felt, FieldElement, Hasher,
StarkField, ARK1, ARK2, BINARY_CHUNK_SIZE, CAPACITY_RANGE, DIGEST_BYTES, DIGEST_RANGE,
DIGEST_SIZE, INPUT1_RANGE, INPUT2_RANGE, MDS, NUM_ROUNDS, RATE_RANGE, RATE_WIDTH, STATE_WIDTH,
ZERO,
};
use core::{convert::TryInto, ops::Range};
mod digest;
pub use digest::RpxDigest;
pub type CubicExtElement = CubeExtension<Felt>;
// HASHER IMPLEMENTATION
// ================================================================================================
/// Implementation of the Rescue Prime eXtension hash function with 256-bit output.
///
/// The hash function is based on the XHash12 construction in [specifications](https://eprint.iacr.org/2023/1045)
///
/// The parameters used to instantiate the function are:
/// * Field: 64-bit prime field with modulus 2^64 - 2^32 + 1.
/// * State width: 12 field elements.
/// * Capacity size: 4 field elements.
/// * S-Box degree: 7.
/// * Rounds: There are 3 different types of rounds:
/// - (FB): `apply_mds` → `add_constants` → `apply_sbox` → `apply_mds` → `add_constants` → `apply_inv_sbox`.
/// - (E): `add_constants` → `ext_sbox` (which is raising to power 7 in the degree 3 extension field).
/// - (M): `apply_mds` → `add_constants`.
/// * Permutation: (FB) (E) (FB) (E) (FB) (E) (M).
///
/// The above parameters target a 128-bit security level. The digest consists of four field elements
/// and it can be serialized into 32 bytes (256 bits).
///
/// ## Hash output consistency
/// Functions [hash_elements()](Rpx256::hash_elements), [merge()](Rpx256::merge), and
/// [merge_with_int()](Rpx256::merge_with_int) are internally consistent. That is, computing
/// a hash for the same set of elements using these functions will always produce the same
/// result. For example, merging two digests using [merge()](Rpx256::merge) will produce the
/// same result as hashing 8 elements which make up these digests using
/// [hash_elements()](Rpx256::hash_elements) function.
///
/// However, [hash()](Rpx256::hash) function is not consistent with functions mentioned above.
/// For example, if we take two field elements, serialize them to bytes and hash them using
/// [hash()](Rpx256::hash), the result will differ from the result obtained by hashing these
/// elements directly using [hash_elements()](Rpx256::hash_elements) function. The reason for
/// this difference is that [hash()](Rpx256::hash) function needs to be able to handle
/// arbitrary binary strings, which may or may not encode valid field elements - and thus,
/// deserialization procedure used by this function is different from the procedure used to
/// deserialize valid field elements.
///
/// Thus, if the underlying data consists of valid field elements, it might make more sense
/// to deserialize them into field elements and then hash them using
/// [hash_elements()](Rpx256::hash_elements) function rather then hashing the serialized bytes
/// using [hash()](Rpx256::hash) function.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub struct Rpx256();
impl Hasher for Rpx256 {
/// Rpx256 collision resistance is 128-bits.
const COLLISION_RESISTANCE: u32 = 128;
type Digest = RpxDigest;
fn hash(bytes: &[u8]) -> Self::Digest {
// initialize the state with zeroes
let mut state = [ZERO; STATE_WIDTH];
// determine the number of field elements needed to encode `bytes` when each field element
// represents at most 7 bytes.
let num_field_elem = bytes.len().div_ceil(BINARY_CHUNK_SIZE);
// set the first capacity element to `RATE_WIDTH + (num_field_elem % RATE_WIDTH)`. We do
// this to achieve:
// 1. Domain separating hashing of `[u8]` from hashing of `[Felt]`.
// 2. Avoiding collisions at the `[Felt]` representation of the encoded bytes.
state[CAPACITY_RANGE.start] =
Felt::from((RATE_WIDTH + (num_field_elem % RATE_WIDTH)) as u8);
// initialize a buffer to receive the little-endian elements.
let mut buf = [0_u8; 8];
// iterate the chunks of bytes, creating a field element from each chunk and copying it
// into the state.
//
// every time the rate range is filled, a permutation is performed. if the final value of
// `i` is not zero, then the chunks count wasn't enough to fill the state range, and an
// additional permutation must be performed.
let i = bytes.chunks(BINARY_CHUNK_SIZE).fold(0, |i, chunk| {
// copy the chunk into the buffer
if i != num_field_elem - 1 {
buf[..BINARY_CHUNK_SIZE].copy_from_slice(chunk);
} else {
// on the last iteration, we pad `buf` with a 1 followed by as many 0's as are
// needed to fill it
buf.fill(0);
buf[..chunk.len()].copy_from_slice(chunk);
buf[chunk.len()] = 1;
}
// set the current rate element to the input. since we take at most 7 bytes, we are
// guaranteed that the inputs data will fit into a single field element.
state[RATE_RANGE.start + i] = Felt::new(u64::from_le_bytes(buf));
// proceed filling the range. if it's full, then we apply a permutation and reset the
// counter to the beginning of the range.
if i == RATE_WIDTH - 1 {
Self::apply_permutation(&mut state);
0
} else {
i + 1
}
});
// if we absorbed some elements but didn't apply a permutation to them (would happen when
// the number of elements is not a multiple of RATE_WIDTH), apply the RPX permutation. we
// don't need to apply any extra padding because the first capacity element contains a
// flag indicating the number of field elements constituting the last block when the latter
// is not divisible by `RATE_WIDTH`.
if i != 0 {
state[RATE_RANGE.start + i..RATE_RANGE.end].fill(ZERO);
Self::apply_permutation(&mut state);
}
// return the first 4 elements of the rate as hash result.
RpxDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
fn merge(values: &[Self::Digest; 2]) -> Self::Digest {
// initialize the state by copying the digest elements into the rate portion of the state
// (8 total elements), and set the capacity elements to 0.
let mut state = [ZERO; STATE_WIDTH];
let it = Self::Digest::digests_as_elements(values.iter());
for (i, v) in it.enumerate() {
state[RATE_RANGE.start + i] = *v;
}
// apply the RPX permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpxDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
fn merge_with_int(seed: Self::Digest, value: u64) -> Self::Digest {
// initialize the state as follows:
// - seed is copied into the first 4 elements of the rate portion of the state.
// - if the value fits into a single field element, copy it into the fifth rate element and
// set the first capacity element to 5.
// - if the value doesn't fit into a single field element, split it into two field
// elements, copy them into rate elements 5 and 6 and set the first capacity element to 6.
let mut state = [ZERO; STATE_WIDTH];
state[INPUT1_RANGE].copy_from_slice(seed.as_elements());
state[INPUT2_RANGE.start] = Felt::new(value);
if value < Felt::MODULUS {
state[CAPACITY_RANGE.start] = Felt::from(5_u8);
} else {
state[INPUT2_RANGE.start + 1] = Felt::new(value / Felt::MODULUS);
state[CAPACITY_RANGE.start] = Felt::from(6_u8);
}
// apply the RPX permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpxDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
}
impl ElementHasher for Rpx256 {
type BaseField = Felt;
fn hash_elements<E: FieldElement<BaseField = Self::BaseField>>(elements: &[E]) -> Self::Digest {
// convert the elements into a list of base field elements
let elements = E::slice_as_base_elements(elements);
// initialize state to all zeros, except for the first element of the capacity part, which
// is set to `elements.len() % RATE_WIDTH`.
let mut state = [ZERO; STATE_WIDTH];
state[CAPACITY_RANGE.start] = Self::BaseField::from((elements.len() % RATE_WIDTH) as u8);
// absorb elements into the state one by one until the rate portion of the state is filled
// up; then apply the Rescue permutation and start absorbing again; repeat until all
// elements have been absorbed
let mut i = 0;
for &element in elements.iter() {
state[RATE_RANGE.start + i] = element;
i += 1;
if i % RATE_WIDTH == 0 {
Self::apply_permutation(&mut state);
i = 0;
}
}
// if we absorbed some elements but didn't apply a permutation to them (would happen when
// the number of elements is not a multiple of RATE_WIDTH), apply the RPX permutation after
// padding by as many 0 as necessary to make the input length a multiple of the RATE_WIDTH.
if i > 0 {
while i != RATE_WIDTH {
state[RATE_RANGE.start + i] = ZERO;
i += 1;
}
Self::apply_permutation(&mut state);
}
// return the first 4 elements of the state as hash result
RpxDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
}
// HASH FUNCTION IMPLEMENTATION
// ================================================================================================
impl Rpx256 {
// CONSTANTS
// --------------------------------------------------------------------------------------------
/// Sponge state is set to 12 field elements or 768 bytes; 8 elements are reserved for rate and
/// the remaining 4 elements are reserved for capacity.
pub const STATE_WIDTH: usize = STATE_WIDTH;
/// The rate portion of the state is located in elements 4 through 11 (inclusive).
pub const RATE_RANGE: Range<usize> = RATE_RANGE;
/// The capacity portion of the state is located in elements 0, 1, 2, and 3.
pub const CAPACITY_RANGE: Range<usize> = CAPACITY_RANGE;
/// The output of the hash function can be read from state elements 4, 5, 6, and 7.
pub const DIGEST_RANGE: Range<usize> = DIGEST_RANGE;
/// MDS matrix used for computing the linear layer in the (FB) and (E) rounds.
pub const MDS: [[Felt; STATE_WIDTH]; STATE_WIDTH] = MDS;
/// Round constants added to the hasher state in the first half of the round.
pub const ARK1: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = ARK1;
/// Round constants added to the hasher state in the second half of the round.
pub const ARK2: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = ARK2;
// TRAIT PASS-THROUGH FUNCTIONS
// --------------------------------------------------------------------------------------------
/// Returns a hash of the provided sequence of bytes.
#[inline(always)]
pub fn hash(bytes: &[u8]) -> RpxDigest {
<Self as Hasher>::hash(bytes)
}
/// Returns a hash of two digests. This method is intended for use in construction of
/// Merkle trees and verification of Merkle paths.
#[inline(always)]
pub fn merge(values: &[RpxDigest; 2]) -> RpxDigest {
<Self as Hasher>::merge(values)
}
/// Returns a hash of the provided field elements.
#[inline(always)]
pub fn hash_elements<E: FieldElement<BaseField = Felt>>(elements: &[E]) -> RpxDigest {
<Self as ElementHasher>::hash_elements(elements)
}
// DOMAIN IDENTIFIER
// --------------------------------------------------------------------------------------------
/// Returns a hash of two digests and a domain identifier.
pub fn merge_in_domain(values: &[RpxDigest; 2], domain: Felt) -> RpxDigest {
// initialize the state by copying the digest elements into the rate portion of the state
// (8 total elements), and set the capacity elements to 0.
let mut state = [ZERO; STATE_WIDTH];
let it = RpxDigest::digests_as_elements(values.iter());
for (i, v) in it.enumerate() {
state[RATE_RANGE.start + i] = *v;
}
// set the second capacity element to the domain value. The first capacity element is used
// for padding purposes.
state[CAPACITY_RANGE.start + 1] = domain;
// apply the RPX permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpxDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
// RPX PERMUTATION
// --------------------------------------------------------------------------------------------
/// Applies RPX permutation to the provided state.
#[inline(always)]
pub fn apply_permutation(state: &mut [Felt; STATE_WIDTH]) {
Self::apply_fb_round(state, 0);
Self::apply_ext_round(state, 1);
Self::apply_fb_round(state, 2);
Self::apply_ext_round(state, 3);
Self::apply_fb_round(state, 4);
Self::apply_ext_round(state, 5);
Self::apply_final_round(state, 6);
}
// RPX PERMUTATION ROUND FUNCTIONS
// --------------------------------------------------------------------------------------------
/// (FB) round function.
#[inline(always)]
pub fn apply_fb_round(state: &mut [Felt; STATE_WIDTH], round: usize) {
apply_mds(state);
if !add_constants_and_apply_sbox(state, &ARK1[round]) {
add_constants(state, &ARK1[round]);
apply_sbox(state);
}
apply_mds(state);
if !add_constants_and_apply_inv_sbox(state, &ARK2[round]) {
add_constants(state, &ARK2[round]);
apply_inv_sbox(state);
}
}
/// (E) round function.
#[inline(always)]
pub fn apply_ext_round(state: &mut [Felt; STATE_WIDTH], round: usize) {
// add constants
add_constants(state, &ARK1[round]);
// decompose the state into 4 elements in the cubic extension field and apply the power 7
// map to each of the elements
let [s0, s1, s2, s3, s4, s5, s6, s7, s8, s9, s10, s11] = *state;
let ext0 = Self::exp7(CubicExtElement::new(s0, s1, s2));
let ext1 = Self::exp7(CubicExtElement::new(s3, s4, s5));
let ext2 = Self::exp7(CubicExtElement::new(s6, s7, s8));
let ext3 = Self::exp7(CubicExtElement::new(s9, s10, s11));
// decompose the state back into 12 base field elements
let arr_ext = [ext0, ext1, ext2, ext3];
*state = CubicExtElement::slice_as_base_elements(&arr_ext)
.try_into()
.expect("shouldn't fail");
}
/// (M) round function.
#[inline(always)]
pub fn apply_final_round(state: &mut [Felt; STATE_WIDTH], round: usize) {
apply_mds(state);
add_constants(state, &ARK1[round]);
}
/// Computes an exponentiation to the power 7 in cubic extension field.
#[inline(always)]
pub fn exp7(x: CubeExtension<Felt>) -> CubeExtension<Felt> {
let x2 = x.square();
let x4 = x2.square();
let x3 = x2 * x;
x3 * x4
}
}

9
src/hash/rescue/tests.rs Normal file
View File

@@ -0,0 +1,9 @@
use super::{Felt, FieldElement, ALPHA, INV_ALPHA};
use rand_utils::rand_value;
#[test]
fn test_alphas() {
let e: Felt = Felt::new(rand_value());
let e_exp = e.exp(ALPHA);
assert_eq!(e, e_exp.exp(INV_ALPHA));
}

View File

@@ -1,154 +0,0 @@
use super::{Digest, Felt, StarkField, DIGEST_SIZE, ZERO};
use crate::utils::{
string::String, ByteReader, ByteWriter, Deserializable, DeserializationError, Serializable,
};
use core::{cmp::Ordering, ops::Deref};
// DIGEST TRAIT IMPLEMENTATIONS
// ================================================================================================
#[derive(Debug, Default, Copy, Clone, Eq, PartialEq)]
pub struct RpoDigest([Felt; DIGEST_SIZE]);
impl RpoDigest {
pub fn new(value: [Felt; DIGEST_SIZE]) -> Self {
Self(value)
}
pub fn as_elements(&self) -> &[Felt] {
self.as_ref()
}
pub fn as_bytes(&self) -> [u8; 32] {
<Self as Digest>::as_bytes(self)
}
pub fn digests_as_elements<'a, I>(digests: I) -> impl Iterator<Item = &'a Felt>
where
I: Iterator<Item = &'a Self>,
{
digests.flat_map(|d| d.0.iter())
}
}
impl Digest for RpoDigest {
fn as_bytes(&self) -> [u8; 32] {
let mut result = [0; 32];
result[..8].copy_from_slice(&self.0[0].as_int().to_le_bytes());
result[8..16].copy_from_slice(&self.0[1].as_int().to_le_bytes());
result[16..24].copy_from_slice(&self.0[2].as_int().to_le_bytes());
result[24..].copy_from_slice(&self.0[3].as_int().to_le_bytes());
result
}
}
impl Serializable for RpoDigest {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
target.write_bytes(&self.as_bytes());
}
}
impl Deserializable for RpoDigest {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let mut inner: [Felt; DIGEST_SIZE] = [ZERO; DIGEST_SIZE];
for inner in inner.iter_mut() {
let e = source.read_u64()?;
if e >= Felt::MODULUS {
return Err(DeserializationError::InvalidValue(String::from(
"Value not in the appropriate range",
)));
}
*inner = Felt::new(e);
}
Ok(Self(inner))
}
}
impl From<[Felt; DIGEST_SIZE]> for RpoDigest {
fn from(value: [Felt; DIGEST_SIZE]) -> Self {
Self(value)
}
}
impl From<RpoDigest> for [Felt; DIGEST_SIZE] {
fn from(value: RpoDigest) -> Self {
value.0
}
}
impl From<RpoDigest> for [u8; 32] {
fn from(value: RpoDigest) -> Self {
value.as_bytes()
}
}
impl Deref for RpoDigest {
type Target = [Felt; DIGEST_SIZE];
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl Ord for RpoDigest {
fn cmp(&self, other: &Self) -> Ordering {
// compare the inner u64 of both elements.
//
// it will iterate the elements and will return the first computation different than
// `Equal`. Otherwise, the ordering is equal.
//
// the endianness is irrelevant here because since, this being a cryptographically secure
// hash computation, the digest shouldn't have any ordered property of its input.
//
// finally, we use `Felt::inner` instead of `Felt::as_int` so we avoid performing a
// montgomery reduction for every limb. that is safe because every inner element of the
// digest is guaranteed to be in its canonical form (that is, `x in [0,p)`).
self.0
.iter()
.map(Felt::inner)
.zip(other.0.iter().map(Felt::inner))
.fold(Ordering::Equal, |ord, (a, b)| match ord {
Ordering::Equal => a.cmp(&b),
_ => ord,
})
}
}
impl PartialOrd for RpoDigest {
fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
Some(self.cmp(other))
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::{Deserializable, Felt, RpoDigest, Serializable};
use crate::utils::SliceReader;
use rand_utils::rand_value;
#[test]
fn digest_serialization() {
let e1 = Felt::new(rand_value());
let e2 = Felt::new(rand_value());
let e3 = Felt::new(rand_value());
let e4 = Felt::new(rand_value());
let d1 = RpoDigest([e1, e2, e3, e4]);
let mut bytes = vec![];
d1.write_into(&mut bytes);
assert_eq!(32, bytes.len());
let mut reader = SliceReader::new(&bytes);
let d2 = RpoDigest::read_from(&mut reader).unwrap();
assert_eq!(d1, d2);
}
}

View File

@@ -1,844 +0,0 @@
use super::{Digest, ElementHasher, Felt, FieldElement, Hasher, StarkField, ONE, ZERO};
use core::{convert::TryInto, ops::Range};
mod digest;
pub use digest::RpoDigest;
mod mds_freq;
use mds_freq::mds_multiply_freq;
#[cfg(test)]
mod tests;
// CONSTANTS
// ================================================================================================
/// Sponge state is set to 12 field elements or 96 bytes; 8 elements are reserved for rate and
/// the remaining 4 elements are reserved for capacity.
const STATE_WIDTH: usize = 12;
/// The rate portion of the state is located in elements 4 through 11.
const RATE_RANGE: Range<usize> = 4..12;
const RATE_WIDTH: usize = RATE_RANGE.end - RATE_RANGE.start;
const INPUT1_RANGE: Range<usize> = 4..8;
const INPUT2_RANGE: Range<usize> = 8..12;
/// The capacity portion of the state is located in elements 0, 1, 2, and 3.
const CAPACITY_RANGE: Range<usize> = 0..4;
/// The output of the hash function is a digest which consists of 4 field elements or 32 bytes.
///
/// The digest is returned from state elements 4, 5, 6, and 7 (the first four elements of the
/// rate portion).
const DIGEST_RANGE: Range<usize> = 4..8;
const DIGEST_SIZE: usize = DIGEST_RANGE.end - DIGEST_RANGE.start;
/// The number of rounds is set to 7 to target 128-bit security level
const NUM_ROUNDS: usize = 7;
/// The number of byte chunks defining a field element when hashing a sequence of bytes
const BINARY_CHUNK_SIZE: usize = 7;
/// S-Box and Inverse S-Box powers;
///
/// The constants are defined for tests only because the exponentiations in the code are unrolled
/// for efficiency reasons.
#[cfg(test)]
const ALPHA: u64 = 7;
#[cfg(test)]
const INV_ALPHA: u64 = 10540996611094048183;
// HASHER IMPLEMENTATION
// ================================================================================================
/// Implementation of the Rescue Prime Optimized hash function with 256-bit output.
///
/// The hash function is implemented according to the Rescue Prime Optimized
/// [specifications](https://eprint.iacr.org/2022/1577)
///
/// The parameters used to instantiate the function are:
/// * Field: 64-bit prime field with modulus 2^64 - 2^32 + 1.
/// * State width: 12 field elements.
/// * Capacity size: 4 field elements.
/// * Number of founds: 7.
/// * S-Box degree: 7.
///
/// The above parameters target 128-bit security level. The digest consists of four field elements
/// and it can be serialized into 32 bytes (256 bits).
///
/// ## Hash output consistency
/// Functions [hash_elements()](Rpo256::hash_elements), [merge()](Rpo256::merge), and
/// [merge_with_int()](Rpo256::merge_with_int) are internally consistent. That is, computing
/// a hash for the same set of elements using these functions will always produce the same
/// result. For example, merging two digests using [merge()](Rpo256::merge) will produce the
/// same result as hashing 8 elements which make up these digests using
/// [hash_elements()](Rpo256::hash_elements) function.
///
/// However, [hash()](Rpo256::hash) function is not consistent with functions mentioned above.
/// For example, if we take two field elements, serialize them to bytes and hash them using
/// [hash()](Rpo256::hash), the result will differ from the result obtained by hashing these
/// elements directly using [hash_elements()](Rpo256::hash_elements) function. The reason for
/// this difference is that [hash()](Rpo256::hash) function needs to be able to handle
/// arbitrary binary strings, which may or may not encode valid field elements - and thus,
/// deserialization procedure used by this function is different from the procedure used to
/// deserialize valid field elements.
///
/// Thus, if the underlying data consists of valid field elements, it might make more sense
/// to deserialize them into field elements and then hash them using
/// [hash_elements()](Rpo256::hash_elements) function rather then hashing the serialized bytes
/// using [hash()](Rpo256::hash) function.
pub struct Rpo256();
impl Hasher for Rpo256 {
/// Rpo256 collision resistance is the same as the security level, that is 128-bits.
///
/// #### Collision resistance
///
/// However, our setup of the capacity registers might drop it to 126.
///
/// Related issue: [#69](https://github.com/0xPolygonMiden/crypto/issues/69)
const COLLISION_RESISTANCE: u32 = 128;
type Digest = RpoDigest;
fn hash(bytes: &[u8]) -> Self::Digest {
// initialize the state with zeroes
let mut state = [ZERO; STATE_WIDTH];
// set the capacity (first element) to a flag on whether or not the input length is evenly
// divided by the rate. this will prevent collisions between padded and non-padded inputs,
// and will rule out the need to perform an extra permutation in case of evenly divided
// inputs.
let is_rate_multiple = bytes.len() % RATE_WIDTH == 0;
if !is_rate_multiple {
state[CAPACITY_RANGE.start] = ONE;
}
// initialize a buffer to receive the little-endian elements.
let mut buf = [0_u8; 8];
// iterate the chunks of bytes, creating a field element from each chunk and copying it
// into the state.
//
// every time the rate range is filled, a permutation is performed. if the final value of
// `i` is not zero, then the chunks count wasn't enough to fill the state range, and an
// additional permutation must be performed.
let i = bytes.chunks(BINARY_CHUNK_SIZE).fold(0, |i, chunk| {
// the last element of the iteration may or may not be a full chunk. if it's not, then
// we need to pad the remainder bytes of the chunk with zeroes, separated by a `1`.
// this will avoid collisions.
if chunk.len() == BINARY_CHUNK_SIZE {
buf[..BINARY_CHUNK_SIZE].copy_from_slice(chunk);
} else {
buf.fill(0);
buf[..chunk.len()].copy_from_slice(chunk);
buf[chunk.len()] = 1;
}
// set the current rate element to the input. since we take at most 7 bytes, we are
// guaranteed that the inputs data will fit into a single field element.
state[RATE_RANGE.start + i] = Felt::new(u64::from_le_bytes(buf));
// proceed filling the range. if it's full, then we apply a permutation and reset the
// counter to the beginning of the range.
if i == RATE_WIDTH - 1 {
Self::apply_permutation(&mut state);
0
} else {
i + 1
}
});
// if we absorbed some elements but didn't apply a permutation to them (would happen when
// the number of elements is not a multiple of RATE_WIDTH), apply the RPO permutation. we
// don't need to apply any extra padding because the first capacity element containts a
// flag indicating whether the input is evenly divisible by the rate.
if i != 0 {
state[RATE_RANGE.start + i..RATE_RANGE.end].fill(ZERO);
state[RATE_RANGE.start + i] = ONE;
Self::apply_permutation(&mut state);
}
// return the first 4 elements of the rate as hash result.
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
fn merge(values: &[Self::Digest; 2]) -> Self::Digest {
// initialize the state by copying the digest elements into the rate portion of the state
// (8 total elements), and set the capacity elements to 0.
let mut state = [ZERO; STATE_WIDTH];
let it = Self::Digest::digests_as_elements(values.iter());
for (i, v) in it.enumerate() {
state[RATE_RANGE.start + i] = *v;
}
// apply the RPO permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
fn merge_with_int(seed: Self::Digest, value: u64) -> Self::Digest {
// initialize the state as follows:
// - seed is copied into the first 4 elements of the rate portion of the state.
// - if the value fits into a single field element, copy it into the fifth rate element
// and set the sixth rate element to 1.
// - if the value doesn't fit into a single field element, split it into two field
// elements, copy them into rate elements 5 and 6, and set the seventh rate element
// to 1.
// - set the first capacity element to 1
let mut state = [ZERO; STATE_WIDTH];
state[INPUT1_RANGE].copy_from_slice(seed.as_elements());
state[INPUT2_RANGE.start] = Felt::new(value);
if value < Felt::MODULUS {
state[INPUT2_RANGE.start + 1] = ONE;
} else {
state[INPUT2_RANGE.start + 1] = Felt::new(value / Felt::MODULUS);
state[INPUT2_RANGE.start + 2] = ONE;
}
// common padding for both cases
state[CAPACITY_RANGE.start] = ONE;
// apply the RPO permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
}
impl ElementHasher for Rpo256 {
type BaseField = Felt;
fn hash_elements<E: FieldElement<BaseField = Self::BaseField>>(elements: &[E]) -> Self::Digest {
// convert the elements into a list of base field elements
let elements = E::as_base_elements(elements);
// initialize state to all zeros, except for the first element of the capacity part, which
// is set to 1 if the number of elements is not a multiple of RATE_WIDTH.
let mut state = [ZERO; STATE_WIDTH];
if elements.len() % RATE_WIDTH != 0 {
state[CAPACITY_RANGE.start] = ONE;
}
// absorb elements into the state one by one until the rate portion of the state is filled
// up; then apply the Rescue permutation and start absorbing again; repeat until all
// elements have been absorbed
let mut i = 0;
for &element in elements.iter() {
state[RATE_RANGE.start + i] = element;
i += 1;
if i % RATE_WIDTH == 0 {
Self::apply_permutation(&mut state);
i = 0;
}
}
// if we absorbed some elements but didn't apply a permutation to them (would happen when
// the number of elements is not a multiple of RATE_WIDTH), apply the RPO permutation after
// padding by appending a 1 followed by as many 0 as necessary to make the input length a
// multiple of the RATE_WIDTH.
if i > 0 {
state[RATE_RANGE.start + i] = ONE;
i += 1;
while i != RATE_WIDTH {
state[RATE_RANGE.start + i] = ZERO;
i += 1;
}
Self::apply_permutation(&mut state);
}
// return the first 4 elements of the state as hash result
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
}
// HASH FUNCTION IMPLEMENTATION
// ================================================================================================
impl Rpo256 {
// CONSTANTS
// --------------------------------------------------------------------------------------------
/// The number of rounds is set to 7 to target 128-bit security level.
pub const NUM_ROUNDS: usize = NUM_ROUNDS;
/// Sponge state is set to 12 field elements or 768 bytes; 8 elements are reserved for rate and
/// the remaining 4 elements are reserved for capacity.
pub const STATE_WIDTH: usize = STATE_WIDTH;
/// The rate portion of the state is located in elements 4 through 11 (inclusive).
pub const RATE_RANGE: Range<usize> = RATE_RANGE;
/// The capacity portion of the state is located in elements 0, 1, 2, and 3.
pub const CAPACITY_RANGE: Range<usize> = CAPACITY_RANGE;
/// The output of the hash function can be read from state elements 4, 5, 6, and 7.
pub const DIGEST_RANGE: Range<usize> = DIGEST_RANGE;
/// MDS matrix used for computing the linear layer in a RPO round.
pub const MDS: [[Felt; STATE_WIDTH]; STATE_WIDTH] = MDS;
/// Round constants added to the hasher state in the first half of the RPO round.
pub const ARK1: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = ARK1;
/// Round constants added to the hasher state in the second half of the RPO round.
pub const ARK2: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = ARK2;
// TRAIT PASS-THROUGH FUNCTIONS
// --------------------------------------------------------------------------------------------
/// Returns a hash of the provided sequence of bytes.
#[inline(always)]
pub fn hash(bytes: &[u8]) -> RpoDigest {
<Self as Hasher>::hash(bytes)
}
/// Returns a hash of two digests. This method is intended for use in construction of
/// Merkle trees and verification of Merkle paths.
#[inline(always)]
pub fn merge(values: &[RpoDigest; 2]) -> RpoDigest {
<Self as Hasher>::merge(values)
}
/// Returns a hash of the provided field elements.
#[inline(always)]
pub fn hash_elements<E: FieldElement<BaseField = Felt>>(elements: &[E]) -> RpoDigest {
<Self as ElementHasher>::hash_elements(elements)
}
// DOMAIN IDENTIFIER
// --------------------------------------------------------------------------------------------
/// Returns a hash of two digests and a domain identifier.
pub fn merge_in_domain(values: &[RpoDigest; 2], domain: Felt) -> RpoDigest {
// initialize the state by copying the digest elements into the rate portion of the state
// (8 total elements), and set the capacity elements to 0.
let mut state = [ZERO; STATE_WIDTH];
let it = RpoDigest::digests_as_elements(values.iter());
for (i, v) in it.enumerate() {
state[RATE_RANGE.start + i] = *v;
}
// set the second capacity element to the domain value. The first capacity element is used
// for padding purposes.
state[CAPACITY_RANGE.start + 1] = domain;
// apply the RPO permutation and return the first four elements of the state
Self::apply_permutation(&mut state);
RpoDigest::new(state[DIGEST_RANGE].try_into().unwrap())
}
// RESCUE PERMUTATION
// --------------------------------------------------------------------------------------------
/// Applies RPO permutation to the provided state.
#[inline(always)]
pub fn apply_permutation(state: &mut [Felt; STATE_WIDTH]) {
for i in 0..NUM_ROUNDS {
Self::apply_round(state, i);
}
}
/// RPO round function.
#[inline(always)]
pub fn apply_round(state: &mut [Felt; STATE_WIDTH], round: usize) {
// apply first half of RPO round
Self::apply_mds(state);
Self::add_constants(state, &ARK1[round]);
Self::apply_sbox(state);
// apply second half of RPO round
Self::apply_mds(state);
Self::add_constants(state, &ARK2[round]);
Self::apply_inv_sbox(state);
}
// HELPER FUNCTIONS
// --------------------------------------------------------------------------------------------
#[inline(always)]
fn apply_mds(state: &mut [Felt; STATE_WIDTH]) {
let mut result = [ZERO; STATE_WIDTH];
// Using the linearity of the operations we can split the state into a low||high decomposition
// and operate on each with no overflow and then combine/reduce the result to a field element.
// The no overflow is guaranteed by the fact that the MDS matrix is a small powers of two in
// frequency domain.
let mut state_l = [0u64; STATE_WIDTH];
let mut state_h = [0u64; STATE_WIDTH];
for r in 0..STATE_WIDTH {
let s = state[r].inner();
state_h[r] = s >> 32;
state_l[r] = (s as u32) as u64;
}
let state_h = mds_multiply_freq(state_h);
let state_l = mds_multiply_freq(state_l);
for r in 0..STATE_WIDTH {
let s = state_l[r] as u128 + ((state_h[r] as u128) << 32);
let s_hi = (s >> 64) as u64;
let s_lo = s as u64;
let z = (s_hi << 32) - s_hi;
let (res, over) = s_lo.overflowing_add(z);
result[r] = Felt::from_mont(res.wrapping_add(0u32.wrapping_sub(over as u32) as u64));
}
*state = result;
}
#[inline(always)]
fn add_constants(state: &mut [Felt; STATE_WIDTH], ark: &[Felt; STATE_WIDTH]) {
state.iter_mut().zip(ark).for_each(|(s, &k)| *s += k);
}
#[inline(always)]
fn apply_sbox(state: &mut [Felt; STATE_WIDTH]) {
state[0] = state[0].exp7();
state[1] = state[1].exp7();
state[2] = state[2].exp7();
state[3] = state[3].exp7();
state[4] = state[4].exp7();
state[5] = state[5].exp7();
state[6] = state[6].exp7();
state[7] = state[7].exp7();
state[8] = state[8].exp7();
state[9] = state[9].exp7();
state[10] = state[10].exp7();
state[11] = state[11].exp7();
}
#[inline(always)]
fn apply_inv_sbox(state: &mut [Felt; STATE_WIDTH]) {
// compute base^10540996611094048183 using 72 multiplications per array element
// 10540996611094048183 = b1001001001001001001001001001000110110110110110110110110110110111
// compute base^10
let mut t1 = *state;
t1.iter_mut().for_each(|t| *t = t.square());
// compute base^100
let mut t2 = t1;
t2.iter_mut().for_each(|t| *t = t.square());
// compute base^100100
let t3 = Self::exp_acc::<Felt, STATE_WIDTH, 3>(t2, t2);
// compute base^100100100100
let t4 = Self::exp_acc::<Felt, STATE_WIDTH, 6>(t3, t3);
// compute base^100100100100100100100100
let t5 = Self::exp_acc::<Felt, STATE_WIDTH, 12>(t4, t4);
// compute base^100100100100100100100100100100
let t6 = Self::exp_acc::<Felt, STATE_WIDTH, 6>(t5, t3);
// compute base^1001001001001001001001001001000100100100100100100100100100100
let t7 = Self::exp_acc::<Felt, STATE_WIDTH, 31>(t6, t6);
// compute base^1001001001001001001001001001000110110110110110110110110110110111
for (i, s) in state.iter_mut().enumerate() {
let a = (t7[i].square() * t6[i]).square().square();
let b = t1[i] * t2[i] * *s;
*s = a * b;
}
}
#[inline(always)]
fn exp_acc<B: StarkField, const N: usize, const M: usize>(
base: [B; N],
tail: [B; N],
) -> [B; N] {
let mut result = base;
for _ in 0..M {
result.iter_mut().for_each(|r| *r = r.square());
}
result.iter_mut().zip(tail).for_each(|(r, t)| *r *= t);
result
}
}
// MDS
// ================================================================================================
/// RPO MDS matrix
const MDS: [[Felt; STATE_WIDTH]; STATE_WIDTH] = [
[
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
],
[
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
],
[
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
],
[
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
],
[
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
],
[
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
],
[
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
],
[
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
],
[
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
Felt::new(26),
],
[
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
Felt::new(8),
],
[
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
Felt::new(23),
],
[
Felt::new(23),
Felt::new(8),
Felt::new(26),
Felt::new(13),
Felt::new(10),
Felt::new(9),
Felt::new(7),
Felt::new(6),
Felt::new(22),
Felt::new(21),
Felt::new(8),
Felt::new(7),
],
];
// ROUND CONSTANTS
// ================================================================================================
/// Rescue round constants;
/// computed as in [specifications](https://github.com/ASDiscreteMathematics/rpo)
///
/// The constants are broken up into two arrays ARK1 and ARK2; ARK1 contains the constants for the
/// first half of RPO round, and ARK2 contains constants for the second half of RPO round.
const ARK1: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = [
[
Felt::new(5789762306288267392),
Felt::new(6522564764413701783),
Felt::new(17809893479458208203),
Felt::new(107145243989736508),
Felt::new(6388978042437517382),
Felt::new(15844067734406016715),
Felt::new(9975000513555218239),
Felt::new(3344984123768313364),
Felt::new(9959189626657347191),
Felt::new(12960773468763563665),
Felt::new(9602914297752488475),
Felt::new(16657542370200465908),
],
[
Felt::new(12987190162843096997),
Felt::new(653957632802705281),
Felt::new(4441654670647621225),
Felt::new(4038207883745915761),
Felt::new(5613464648874830118),
Felt::new(13222989726778338773),
Felt::new(3037761201230264149),
Felt::new(16683759727265180203),
Felt::new(8337364536491240715),
Felt::new(3227397518293416448),
Felt::new(8110510111539674682),
Felt::new(2872078294163232137),
],
[
Felt::new(18072785500942327487),
Felt::new(6200974112677013481),
Felt::new(17682092219085884187),
Felt::new(10599526828986756440),
Felt::new(975003873302957338),
Felt::new(8264241093196931281),
Felt::new(10065763900435475170),
Felt::new(2181131744534710197),
Felt::new(6317303992309418647),
Felt::new(1401440938888741532),
Felt::new(8884468225181997494),
Felt::new(13066900325715521532),
],
[
Felt::new(5674685213610121970),
Felt::new(5759084860419474071),
Felt::new(13943282657648897737),
Felt::new(1352748651966375394),
Felt::new(17110913224029905221),
Felt::new(1003883795902368422),
Felt::new(4141870621881018291),
Felt::new(8121410972417424656),
Felt::new(14300518605864919529),
Felt::new(13712227150607670181),
Felt::new(17021852944633065291),
Felt::new(6252096473787587650),
],
[
Felt::new(4887609836208846458),
Felt::new(3027115137917284492),
Felt::new(9595098600469470675),
Felt::new(10528569829048484079),
Felt::new(7864689113198939815),
Felt::new(17533723827845969040),
Felt::new(5781638039037710951),
Felt::new(17024078752430719006),
Felt::new(109659393484013511),
Felt::new(7158933660534805869),
Felt::new(2955076958026921730),
Felt::new(7433723648458773977),
],
[
Felt::new(16308865189192447297),
Felt::new(11977192855656444890),
Felt::new(12532242556065780287),
Felt::new(14594890931430968898),
Felt::new(7291784239689209784),
Felt::new(5514718540551361949),
Felt::new(10025733853830934803),
Felt::new(7293794580341021693),
Felt::new(6728552937464861756),
Felt::new(6332385040983343262),
Felt::new(13277683694236792804),
Felt::new(2600778905124452676),
],
[
Felt::new(7123075680859040534),
Felt::new(1034205548717903090),
Felt::new(7717824418247931797),
Felt::new(3019070937878604058),
Felt::new(11403792746066867460),
Felt::new(10280580802233112374),
Felt::new(337153209462421218),
Felt::new(13333398568519923717),
Felt::new(3596153696935337464),
Felt::new(8104208463525993784),
Felt::new(14345062289456085693),
Felt::new(17036731477169661256),
],
];
const ARK2: [[Felt; STATE_WIDTH]; NUM_ROUNDS] = [
[
Felt::new(6077062762357204287),
Felt::new(15277620170502011191),
Felt::new(5358738125714196705),
Felt::new(14233283787297595718),
Felt::new(13792579614346651365),
Felt::new(11614812331536767105),
Felt::new(14871063686742261166),
Felt::new(10148237148793043499),
Felt::new(4457428952329675767),
Felt::new(15590786458219172475),
Felt::new(10063319113072092615),
Felt::new(14200078843431360086),
],
[
Felt::new(6202948458916099932),
Felt::new(17690140365333231091),
Felt::new(3595001575307484651),
Felt::new(373995945117666487),
Felt::new(1235734395091296013),
Felt::new(14172757457833931602),
Felt::new(707573103686350224),
Felt::new(15453217512188187135),
Felt::new(219777875004506018),
Felt::new(17876696346199469008),
Felt::new(17731621626449383378),
Felt::new(2897136237748376248),
],
[
Felt::new(8023374565629191455),
Felt::new(15013690343205953430),
Felt::new(4485500052507912973),
Felt::new(12489737547229155153),
Felt::new(9500452585969030576),
Felt::new(2054001340201038870),
Felt::new(12420704059284934186),
Felt::new(355990932618543755),
Felt::new(9071225051243523860),
Felt::new(12766199826003448536),
Felt::new(9045979173463556963),
Felt::new(12934431667190679898),
],
[
Felt::new(18389244934624494276),
Felt::new(16731736864863925227),
Felt::new(4440209734760478192),
Felt::new(17208448209698888938),
Felt::new(8739495587021565984),
Felt::new(17000774922218161967),
Felt::new(13533282547195532087),
Felt::new(525402848358706231),
Felt::new(16987541523062161972),
Felt::new(5466806524462797102),
Felt::new(14512769585918244983),
Felt::new(10973956031244051118),
],
[
Felt::new(6982293561042362913),
Felt::new(14065426295947720331),
Felt::new(16451845770444974180),
Felt::new(7139138592091306727),
Felt::new(9012006439959783127),
Felt::new(14619614108529063361),
Felt::new(1394813199588124371),
Felt::new(4635111139507788575),
Felt::new(16217473952264203365),
Felt::new(10782018226466330683),
Felt::new(6844229992533662050),
Felt::new(7446486531695178711),
],
[
Felt::new(3736792340494631448),
Felt::new(577852220195055341),
Felt::new(6689998335515779805),
Felt::new(13886063479078013492),
Felt::new(14358505101923202168),
Felt::new(7744142531772274164),
Felt::new(16135070735728404443),
Felt::new(12290902521256031137),
Felt::new(12059913662657709804),
Felt::new(16456018495793751911),
Felt::new(4571485474751953524),
Felt::new(17200392109565783176),
],
[
Felt::new(17130398059294018733),
Felt::new(519782857322261988),
Felt::new(9625384390925085478),
Felt::new(1664893052631119222),
Felt::new(7629576092524553570),
Felt::new(3485239601103661425),
Felt::new(9755891797164033838),
Felt::new(15218148195153269027),
Felt::new(16460604813734957368),
Felt::new(9643968136937729763),
Felt::new(3611348709641382851),
Felt::new(18256379591337759196),
],
];

View File

@@ -4,22 +4,19 @@
#[cfg_attr(test, macro_use)]
extern crate alloc;
pub mod dsa;
pub mod hash;
pub mod merkle;
pub mod rand;
pub mod utils;
// RE-EXPORTS
// ================================================================================================
pub use winter_crypto::{RandomCoin, RandomCoinError};
pub use winter_math::{fields::f64::BaseElement as Felt, FieldElement, StarkField};
pub mod utils {
pub use winter_utils::{
collections, string, uninit_vector, ByteReader, ByteWriter, Deserializable,
DeserializationError, Serializable, SliceReader,
};
}
pub use winter_math::{
fields::{f64::BaseElement as Felt, CubeExtension, QuadExtension},
FieldElement, StarkField,
};
// TYPE ALIASES
// ================================================================================================
@@ -39,6 +36,9 @@ pub const ZERO: Felt = Felt::ZERO;
/// Field element representing ONE in the Miden base filed.
pub const ONE: Felt = Felt::ONE;
/// Array of field elements representing word of ZEROs in the Miden base field.
pub const EMPTY_WORD: [Felt; 4] = [ZERO; WORD_SIZE];
// TESTS
// ================================================================================================

110
src/main.rs Normal file
View File

@@ -0,0 +1,110 @@
use clap::Parser;
use miden_crypto::{
hash::rpo::{Rpo256, RpoDigest},
merkle::{MerkleError, Smt},
Felt, Word, ONE,
};
use rand_utils::rand_value;
use std::time::Instant;
#[derive(Parser, Debug)]
#[clap(name = "Benchmark", about = "SMT benchmark", version, rename_all = "kebab-case")]
pub struct BenchmarkCmd {
/// Size of the tree
#[clap(short = 's', long = "size")]
size: u64,
}
fn main() {
benchmark_smt();
}
/// Run a benchmark for [`Smt`].
pub fn benchmark_smt() {
let args = BenchmarkCmd::parse();
let tree_size = args.size;
// prepare the `leaves` vector for tree creation
let mut entries = Vec::new();
for i in 0..tree_size {
let key = rand_value::<RpoDigest>();
let value = [ONE, ONE, ONE, Felt::new(i)];
entries.push((key, value));
}
let mut tree = construction(entries, tree_size).unwrap();
insertion(&mut tree, tree_size).unwrap();
proof_generation(&mut tree, tree_size).unwrap();
}
/// Runs the construction benchmark for [`Smt`], returning the constructed tree.
pub fn construction(entries: Vec<(RpoDigest, Word)>, size: u64) -> Result<Smt, MerkleError> {
println!("Running a construction benchmark:");
let now = Instant::now();
let tree = Smt::with_entries(entries)?;
let elapsed = now.elapsed();
println!(
"Constructed a SMT with {} key-value pairs in {:.3} seconds",
size,
elapsed.as_secs_f32(),
);
println!("Number of leaf nodes: {}\n", tree.leaves().count());
Ok(tree)
}
/// Runs the insertion benchmark for the [`Smt`].
pub fn insertion(tree: &mut Smt, size: u64) -> Result<(), MerkleError> {
println!("Running an insertion benchmark:");
let mut insertion_times = Vec::new();
for i in 0..20 {
let test_key = Rpo256::hash(&rand_value::<u64>().to_be_bytes());
let test_value = [ONE, ONE, ONE, Felt::new(size + i)];
let now = Instant::now();
tree.insert(test_key, test_value);
let elapsed = now.elapsed();
insertion_times.push(elapsed.as_secs_f32());
}
println!(
"An average insertion time measured by 20 inserts into a SMT with {} key-value pairs is {:.3} milliseconds\n",
size,
// calculate the average by dividing by 20 and convert to milliseconds by multiplying by
// 1000. As a result, we can only multiply by 50
insertion_times.iter().sum::<f32>() * 50f32,
);
Ok(())
}
/// Runs the proof generation benchmark for the [`Smt`].
pub fn proof_generation(tree: &mut Smt, size: u64) -> Result<(), MerkleError> {
println!("Running a proof generation benchmark:");
let mut insertion_times = Vec::new();
for i in 0..20 {
let test_key = Rpo256::hash(&rand_value::<u64>().to_be_bytes());
let test_value = [ONE, ONE, ONE, Felt::new(size + i)];
tree.insert(test_key, test_value);
let now = Instant::now();
let _proof = tree.open(&test_key);
let elapsed = now.elapsed();
insertion_times.push(elapsed.as_secs_f32());
}
println!(
"An average proving time measured by 20 value proofs in a SMT with {} key-value pairs in {:.3} microseconds",
size,
// calculate the average by dividing by 20 and convert to microseconds by multiplying by
// 1000000. As a result, we can only multiply by 50000
insertion_times.iter().sum::<f32>() * 50000f32,
);
Ok(())
}

1605
src/merkle/empty_roots.rs Normal file

File diff suppressed because it is too large Load Diff

68
src/merkle/error.rs Normal file
View File

@@ -0,0 +1,68 @@
use crate::{
merkle::{MerklePath, NodeIndex, RpoDigest},
utils::collections::Vec,
};
use core::fmt;
use super::smt::SmtLeafError;
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum MerkleError {
ConflictingRoots(Vec<RpoDigest>),
DepthTooSmall(u8),
DepthTooBig(u64),
DuplicateValuesForIndex(u64),
DuplicateValuesForKey(RpoDigest),
InvalidIndex { depth: u8, value: u64 },
InvalidDepth { expected: u8, provided: u8 },
InvalidSubtreeDepth { subtree_depth: u8, tree_depth: u8 },
InvalidPath(MerklePath),
InvalidNumEntries(usize),
NodeNotInSet(NodeIndex),
NodeNotInStore(RpoDigest, NodeIndex),
NumLeavesNotPowerOfTwo(usize),
RootNotInStore(RpoDigest),
SmtLeaf(SmtLeafError),
}
impl fmt::Display for MerkleError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
use MerkleError::*;
match self {
ConflictingRoots(roots) => write!(f, "the merkle paths roots do not match {roots:?}"),
DepthTooSmall(depth) => write!(f, "the provided depth {depth} is too small"),
DepthTooBig(depth) => write!(f, "the provided depth {depth} is too big"),
DuplicateValuesForIndex(key) => write!(f, "multiple values provided for key {key}"),
DuplicateValuesForKey(key) => write!(f, "multiple values provided for key {key}"),
InvalidIndex { depth, value } => {
write!(f, "the index value {value} is not valid for the depth {depth}")
}
InvalidDepth { expected, provided } => {
write!(f, "the provided depth {provided} is not valid for {expected}")
}
InvalidSubtreeDepth { subtree_depth, tree_depth } => {
write!(f, "tried inserting a subtree of depth {subtree_depth} into a tree of depth {tree_depth}")
}
InvalidPath(_path) => write!(f, "the provided path is not valid"),
InvalidNumEntries(max) => write!(f, "number of entries exceeded the maximum: {max}"),
NodeNotInSet(index) => write!(f, "the node with index ({index}) is not in the set"),
NodeNotInStore(hash, index) => {
write!(f, "the node {hash:?} with index ({index}) is not in the store")
}
NumLeavesNotPowerOfTwo(leaves) => {
write!(f, "the leaves count {leaves} is not a power of 2")
}
RootNotInStore(root) => write!(f, "the root {:?} is not in the store", root),
SmtLeaf(smt_leaf_error) => write!(f, "smt leaf error: {smt_leaf_error}"),
}
}
}
#[cfg(feature = "std")]
impl std::error::Error for MerkleError {}
impl From<SmtLeafError> for MerkleError {
fn from(value: SmtLeafError) -> Self {
Self::SmtLeaf(value)
}
}

View File

@@ -1,10 +1,27 @@
use super::{Felt, MerkleError, RpoDigest, StarkField};
use super::{Felt, MerkleError, RpoDigest};
use crate::utils::{ByteReader, ByteWriter, Deserializable, DeserializationError, Serializable};
use core::fmt::Display;
// NODE INDEX
// ================================================================================================
/// A Merkle tree address to an arbitrary node.
/// Address to an arbitrary node in a binary tree using level order form.
///
/// The position is represented by the pair `(depth, pos)`, where for a given depth `d` elements
/// are numbered from $0..(2^d)-1$. Example:
///
/// ```ignore
/// depth
/// 0 0
/// 1 0 1
/// 2 0 1 2 3
/// 3 0 1 2 3 4 5 6 7
/// ```
///
/// The root is represented by the pair $(0, 0)$, its left child is $(1, 0)$ and its right child
/// $(1, 1)$.
#[derive(Debug, Default, Copy, Clone, Eq, PartialEq, PartialOrd, Ord, Hash)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct NodeIndex {
depth: u8,
value: u64,
@@ -15,20 +32,43 @@ impl NodeIndex {
// --------------------------------------------------------------------------------------------
/// Creates a new node index.
pub const fn new(depth: u8, value: u64) -> Self {
///
/// # Errors
/// Returns an error if the `value` is greater than or equal to 2^{depth}.
pub const fn new(depth: u8, value: u64) -> Result<Self, MerkleError> {
if (64 - value.leading_zeros()) > depth as u32 {
Err(MerkleError::InvalidIndex { depth, value })
} else {
Ok(Self { depth, value })
}
}
/// Creates a new node index without checking its validity.
pub const fn new_unchecked(depth: u8, value: u64) -> Self {
debug_assert!((64 - value.leading_zeros()) <= depth as u32);
Self { depth, value }
}
/// Creates a new node index for testing purposes.
///
/// # Panics
/// Panics if the `value` is greater than or equal to 2^{depth}.
#[cfg(test)]
pub fn make(depth: u8, value: u64) -> Self {
Self::new(depth, value).unwrap()
}
/// Creates a node index from a pair of field elements representing the depth and value.
///
/// # Errors
///
/// Will error if the `u64` representation of the depth doesn't fit a `u8`.
/// Returns an error if:
/// - `depth` doesn't fit in a `u8`.
/// - `value` is greater than or equal to 2^{depth}.
pub fn from_elements(depth: &Felt, value: &Felt) -> Result<Self, MerkleError> {
let depth = depth.as_int();
let depth = u8::try_from(depth).map_err(|_| MerkleError::DepthTooBig(depth))?;
let value = value.as_int();
Ok(Self::new(depth, value))
Self::new(depth, value)
}
/// Creates a new node index pointing to the root of the tree.
@@ -36,15 +76,23 @@ impl NodeIndex {
Self { depth: 0, value: 0 }
}
/// Mutates the instance and returns it, replacing the depth.
pub const fn with_depth(mut self, depth: u8) -> Self {
self.depth = depth;
/// Computes sibling index of the current node.
pub const fn sibling(mut self) -> Self {
self.value ^= 1;
self
}
/// Computes the value of the sibling of the current node.
pub fn sibling(mut self) -> Self {
self.value ^= 1;
/// Returns left child index of the current node.
pub const fn left_child(mut self) -> Self {
self.depth += 1;
self.value <<= 1;
self
}
/// Returns right child index of the current node.
pub const fn right_child(mut self) -> Self {
self.depth += 1;
self.value = (self.value << 1) + 1;
self
}
@@ -74,16 +122,11 @@ impl NodeIndex {
self.depth
}
/// Returns the value of the current depth.
/// Returns the value of this index.
pub const fn value(&self) -> u64 {
self.value
}
/// Returns true if the current value fits the current depth for a binary tree.
pub const fn is_valid(&self) -> bool {
self.value < (1 << self.depth as u64)
}
/// Returns true if the current instance points to a right sibling node.
pub const fn is_value_odd(&self) -> bool {
(self.value & 1) == 1
@@ -97,11 +140,42 @@ impl NodeIndex {
// STATE MUTATORS
// --------------------------------------------------------------------------------------------
/// Traverse one level towards the root, decrementing the depth by `1`.
pub fn move_up(&mut self) -> &mut Self {
/// Traverses one level towards the root, decrementing the depth by `1`.
pub fn move_up(&mut self) {
self.depth = self.depth.saturating_sub(1);
self.value >>= 1;
self
}
/// Traverses towards the root until the specified depth is reached.
///
/// Assumes that the specified depth is smaller than the current depth.
pub fn move_up_to(&mut self, depth: u8) {
debug_assert!(depth < self.depth);
let delta = self.depth.saturating_sub(depth);
self.depth = self.depth.saturating_sub(delta);
self.value >>= delta as u32;
}
}
impl Display for NodeIndex {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
write!(f, "depth={}, value={}", self.depth, self.value)
}
}
impl Serializable for NodeIndex {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
target.write_u8(self.depth);
target.write_u64(self.value);
}
}
impl Deserializable for NodeIndex {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let depth = source.read_u8()?;
let value = source.read_u64()?;
NodeIndex::new(depth, value)
.map_err(|_| DeserializationError::InvalidValue("Invalid index".into()))
}
}
@@ -110,14 +184,47 @@ mod tests {
use super::*;
use proptest::prelude::*;
#[test]
fn test_node_index_value_too_high() {
assert_eq!(NodeIndex::new(0, 0).unwrap(), NodeIndex { depth: 0, value: 0 });
let err = NodeIndex::new(0, 1).unwrap_err();
assert_eq!(err, MerkleError::InvalidIndex { depth: 0, value: 1 });
assert_eq!(NodeIndex::new(1, 1).unwrap(), NodeIndex { depth: 1, value: 1 });
let err = NodeIndex::new(1, 2).unwrap_err();
assert_eq!(err, MerkleError::InvalidIndex { depth: 1, value: 2 });
assert_eq!(NodeIndex::new(2, 3).unwrap(), NodeIndex { depth: 2, value: 3 });
let err = NodeIndex::new(2, 4).unwrap_err();
assert_eq!(err, MerkleError::InvalidIndex { depth: 2, value: 4 });
assert_eq!(NodeIndex::new(3, 7).unwrap(), NodeIndex { depth: 3, value: 7 });
let err = NodeIndex::new(3, 8).unwrap_err();
assert_eq!(err, MerkleError::InvalidIndex { depth: 3, value: 8 });
}
#[test]
fn test_node_index_can_represent_depth_64() {
assert!(NodeIndex::new(64, u64::MAX).is_ok());
}
prop_compose! {
fn node_index()(value in 0..2u64.pow(u64::BITS - 1)) -> NodeIndex {
// unwrap never panics because the range of depth is 0..u64::BITS
let mut depth = value.ilog2() as u8;
if value > (1 << depth) { // round up
depth += 1;
}
NodeIndex::new(depth, value).unwrap()
}
}
proptest! {
#[test]
fn arbitrary_index_wont_panic_on_move_up(
depth in prop::num::u8::ANY,
value in prop::num::u64::ANY,
mut index in node_index(),
count in prop::num::u8::ANY,
) {
let mut index = NodeIndex::new(depth, value);
for _ in 0..count {
index.move_up();
}

View File

@@ -1,6 +1,6 @@
use super::{Felt, MerkleError, MerklePath, NodeIndex, Rpo256, RpoDigest, Vec, Word};
use crate::{utils::uninit_vector, FieldElement};
use core::slice;
use super::{InnerNodeInfo, MerkleError, MerklePath, NodeIndex, Rpo256, RpoDigest, Vec, Word};
use crate::utils::{string::String, uninit_vector, word_to_hex};
use core::{fmt, ops::Deref, slice};
use winter_math::log2;
// MERKLE TREE
@@ -8,8 +8,9 @@ use winter_math::log2;
/// A fully-balanced binary Merkle tree (i.e., a tree where the number of leaves is a power of two).
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct MerkleTree {
nodes: Vec<Word>,
nodes: Vec<RpoDigest>,
}
impl MerkleTree {
@@ -19,7 +20,11 @@ impl MerkleTree {
///
/// # Errors
/// Returns an error if the number of leaves is smaller than two or is not a power of two.
pub fn new(leaves: Vec<Word>) -> Result<Self, MerkleError> {
pub fn new<T>(leaves: T) -> Result<Self, MerkleError>
where
T: AsRef<[Word]>,
{
let leaves = leaves.as_ref();
let n = leaves.len();
if n <= 1 {
return Err(MerkleError::DepthTooSmall(n as u8));
@@ -29,10 +34,12 @@ impl MerkleTree {
// create un-initialized vector to hold all tree nodes
let mut nodes = unsafe { uninit_vector(2 * n) };
nodes[0] = [Felt::ZERO; 4];
nodes[0] = RpoDigest::default();
// copy leaves into the second part of the nodes vector
nodes[n..].copy_from_slice(&leaves);
nodes[n..].iter_mut().zip(leaves).for_each(|(node, leaf)| {
*node = RpoDigest::from(*leaf);
});
// re-interpret nodes as an array of two nodes fused together
// Safety: `nodes` will never move here as it is not bound to an external lifetime (i.e.
@@ -42,7 +49,7 @@ impl MerkleTree {
// calculate all internal tree nodes
for i in (1..n).rev() {
nodes[i] = Rpo256::merge(&pairs[i]).into();
nodes[i] = Rpo256::merge(&pairs[i]);
}
Ok(Self { nodes })
@@ -52,7 +59,7 @@ impl MerkleTree {
// --------------------------------------------------------------------------------------------
/// Returns the root of this Merkle tree.
pub fn root(&self) -> Word {
pub fn root(&self) -> RpoDigest {
self.nodes[1]
}
@@ -69,13 +76,11 @@ impl MerkleTree {
/// Returns an error if:
/// * The specified depth is greater than the depth of the tree.
/// * The specified index is not valid for the specified depth.
pub fn get_node(&self, index: NodeIndex) -> Result<Word, MerkleError> {
pub fn get_node(&self, index: NodeIndex) -> Result<RpoDigest, MerkleError> {
if index.is_root() {
return Err(MerkleError::DepthTooSmall(index.depth()));
} else if index.depth() > self.depth() {
return Err(MerkleError::DepthTooBig(index.depth() as u64));
} else if !index.is_valid() {
return Err(MerkleError::InvalidIndex(index));
}
let pos = index.to_scalar_index() as usize;
@@ -94,8 +99,6 @@ impl MerkleTree {
return Err(MerkleError::DepthTooSmall(index.depth()));
} else if index.depth() > self.depth() {
return Err(MerkleError::DepthTooBig(index.depth() as u64));
} else if !index.is_valid() {
return Err(MerkleError::InvalidIndex(index));
}
// TODO should we create a helper in `NodeIndex` that will encapsulate traversal to root so
@@ -108,19 +111,43 @@ impl MerkleTree {
index.move_up();
}
debug_assert!(index.is_root(), "the path walk must go all the way to the root");
Ok(path.into())
}
// ITERATORS
// --------------------------------------------------------------------------------------------
/// Returns an iterator over the leaves of this [MerkleTree].
pub fn leaves(&self) -> impl Iterator<Item = (u64, &Word)> {
let leaves_start = self.nodes.len() / 2;
self.nodes
.iter()
.skip(leaves_start)
.enumerate()
.map(|(i, v)| (i as u64, v.deref()))
}
/// Returns n iterator over every inner node of this [MerkleTree].
///
/// The iterator order is unspecified.
pub fn inner_nodes(&self) -> InnerNodeIterator {
InnerNodeIterator {
nodes: &self.nodes,
index: 1, // index 0 is just padding, start at 1
}
}
// STATE MUTATORS
// --------------------------------------------------------------------------------------------
/// Replaces the leaf at the specified index with the provided value.
///
/// # Errors
/// Returns an error if the specified index value is not a valid leaf value for this tree.
pub fn update_leaf<'a>(&'a mut self, index_value: u64, value: Word) -> Result<(), MerkleError> {
let depth = self.depth();
let mut index = NodeIndex::new(depth, index_value);
if !index.is_valid() {
return Err(MerkleError::InvalidIndex(index));
}
let mut index = NodeIndex::new(self.depth(), index_value)?;
// we don't need to copy the pairs into a new address as we are logically guaranteed to not
// overlap write instructions. however, it's important to bind the lifetime of pairs to
@@ -138,13 +165,13 @@ impl MerkleTree {
// update the current node
let pos = index.to_scalar_index() as usize;
self.nodes[pos] = value;
self.nodes[pos] = value.into();
// traverse to the root, updating each node with the merged values of its parents
for _ in 0..index.depth() {
index.move_up();
let pos = index.to_scalar_index() as usize;
let value = Rpo256::merge(&pairs[pos]).into();
let value = Rpo256::merge(&pairs[pos]);
self.nodes[pos] = value;
}
@@ -152,24 +179,122 @@ impl MerkleTree {
}
}
// CONVERSIONS
// ================================================================================================
impl TryFrom<&[Word]> for MerkleTree {
type Error = MerkleError;
fn try_from(value: &[Word]) -> Result<Self, Self::Error> {
MerkleTree::new(value)
}
}
impl TryFrom<&[RpoDigest]> for MerkleTree {
type Error = MerkleError;
fn try_from(value: &[RpoDigest]) -> Result<Self, Self::Error> {
let value: Vec<Word> = value.iter().map(|v| *v.deref()).collect();
MerkleTree::new(value)
}
}
// ITERATORS
// ================================================================================================
/// An iterator over every inner node of the [MerkleTree].
///
/// Use this to extract the data of the tree, there is no guarantee on the order of the elements.
pub struct InnerNodeIterator<'a> {
nodes: &'a Vec<RpoDigest>,
index: usize,
}
impl<'a> Iterator for InnerNodeIterator<'a> {
type Item = InnerNodeInfo;
fn next(&mut self) -> Option<Self::Item> {
if self.index < self.nodes.len() / 2 {
let value = self.index;
let left = self.index * 2;
let right = left + 1;
self.index += 1;
Some(InnerNodeInfo {
value: self.nodes[value],
left: self.nodes[left],
right: self.nodes[right],
})
} else {
None
}
}
}
// UTILITY FUNCTIONS
// ================================================================================================
/// Utility to visualize a [MerkleTree] in text.
pub fn tree_to_text(tree: &MerkleTree) -> Result<String, fmt::Error> {
let indent = " ";
let mut s = String::new();
s.push_str(&word_to_hex(&tree.root())?);
s.push('\n');
for d in 1..=tree.depth() {
let entries = 2u64.pow(d.into());
for i in 0..entries {
let index = NodeIndex::new(d, i).expect("The index must always be valid");
let node = tree.get_node(index).expect("The node must always be found");
for _ in 0..d {
s.push_str(indent);
}
s.push_str(&word_to_hex(&node)?);
s.push('\n');
}
}
Ok(s)
}
/// Utility to visualize a [MerklePath] in text.
pub fn path_to_text(path: &MerklePath) -> Result<String, fmt::Error> {
let mut s = String::new();
s.push('[');
for el in path.iter() {
s.push_str(&word_to_hex(el)?);
s.push_str(", ");
}
// remove the last ", "
if path.len() != 0 {
s.pop();
s.pop();
}
s.push(']');
Ok(s)
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::*;
use crate::merkle::int_to_node;
use crate::{
merkle::{digests_to_words, int_to_leaf, int_to_node, InnerNodeInfo},
Felt, Word, WORD_SIZE,
};
use core::mem::size_of;
use proptest::prelude::*;
const LEAVES4: [Word; 4] = [
int_to_node(1),
int_to_node(2),
int_to_node(3),
int_to_node(4),
];
const LEAVES4: [RpoDigest; WORD_SIZE] =
[int_to_node(1), int_to_node(2), int_to_node(3), int_to_node(4)];
const LEAVES8: [Word; 8] = [
const LEAVES8: [RpoDigest; 8] = [
int_to_node(1),
int_to_node(2),
int_to_node(3),
@@ -182,7 +307,7 @@ mod tests {
#[test]
fn build_merkle_tree() {
let tree = super::MerkleTree::new(LEAVES4.to_vec()).unwrap();
let tree = super::MerkleTree::new(digests_to_words(&LEAVES4)).unwrap();
assert_eq!(8, tree.nodes.len());
// leaves were copied correctly
@@ -201,58 +326,46 @@ mod tests {
#[test]
fn get_leaf() {
let tree = super::MerkleTree::new(LEAVES4.to_vec()).unwrap();
let tree = super::MerkleTree::new(digests_to_words(&LEAVES4)).unwrap();
// check depth 2
assert_eq!(LEAVES4[0], tree.get_node(NodeIndex::new(2, 0)).unwrap());
assert_eq!(LEAVES4[1], tree.get_node(NodeIndex::new(2, 1)).unwrap());
assert_eq!(LEAVES4[2], tree.get_node(NodeIndex::new(2, 2)).unwrap());
assert_eq!(LEAVES4[3], tree.get_node(NodeIndex::new(2, 3)).unwrap());
assert_eq!(LEAVES4[0], tree.get_node(NodeIndex::make(2, 0)).unwrap());
assert_eq!(LEAVES4[1], tree.get_node(NodeIndex::make(2, 1)).unwrap());
assert_eq!(LEAVES4[2], tree.get_node(NodeIndex::make(2, 2)).unwrap());
assert_eq!(LEAVES4[3], tree.get_node(NodeIndex::make(2, 3)).unwrap());
// check depth 1
let (_, node2, node3) = compute_internal_nodes();
assert_eq!(node2, tree.get_node(NodeIndex::new(1, 0)).unwrap());
assert_eq!(node3, tree.get_node(NodeIndex::new(1, 1)).unwrap());
assert_eq!(node2, tree.get_node(NodeIndex::make(1, 0)).unwrap());
assert_eq!(node3, tree.get_node(NodeIndex::make(1, 1)).unwrap());
}
#[test]
fn get_path() {
let tree = super::MerkleTree::new(LEAVES4.to_vec()).unwrap();
let tree = super::MerkleTree::new(digests_to_words(&LEAVES4)).unwrap();
let (_, node2, node3) = compute_internal_nodes();
// check depth 2
assert_eq!(
vec![LEAVES4[1], node3],
*tree.get_path(NodeIndex::new(2, 0)).unwrap()
);
assert_eq!(
vec![LEAVES4[0], node3],
*tree.get_path(NodeIndex::new(2, 1)).unwrap()
);
assert_eq!(
vec![LEAVES4[3], node2],
*tree.get_path(NodeIndex::new(2, 2)).unwrap()
);
assert_eq!(
vec![LEAVES4[2], node2],
*tree.get_path(NodeIndex::new(2, 3)).unwrap()
);
assert_eq!(vec![LEAVES4[1], node3], *tree.get_path(NodeIndex::make(2, 0)).unwrap());
assert_eq!(vec![LEAVES4[0], node3], *tree.get_path(NodeIndex::make(2, 1)).unwrap());
assert_eq!(vec![LEAVES4[3], node2], *tree.get_path(NodeIndex::make(2, 2)).unwrap());
assert_eq!(vec![LEAVES4[2], node2], *tree.get_path(NodeIndex::make(2, 3)).unwrap());
// check depth 1
assert_eq!(vec![node3], *tree.get_path(NodeIndex::new(1, 0)).unwrap());
assert_eq!(vec![node2], *tree.get_path(NodeIndex::new(1, 1)).unwrap());
assert_eq!(vec![node3], *tree.get_path(NodeIndex::make(1, 0)).unwrap());
assert_eq!(vec![node2], *tree.get_path(NodeIndex::make(1, 1)).unwrap());
}
#[test]
fn update_leaf() {
let mut tree = super::MerkleTree::new(LEAVES8.to_vec()).unwrap();
let mut tree = super::MerkleTree::new(digests_to_words(&LEAVES8)).unwrap();
// update one leaf
let value = 3;
let new_node = int_to_node(9);
let mut expected_leaves = LEAVES8.to_vec();
let new_node = int_to_leaf(9);
let mut expected_leaves = digests_to_words(&LEAVES8);
expected_leaves[value as usize] = new_node;
let expected_tree = super::MerkleTree::new(expected_leaves.clone()).unwrap();
@@ -261,7 +374,7 @@ mod tests {
// update another leaf
let value = 6;
let new_node = int_to_node(10);
let new_node = int_to_leaf(10);
expected_leaves[value as usize] = new_node;
let expected_tree = super::MerkleTree::new(expected_leaves.clone()).unwrap();
@@ -269,6 +382,28 @@ mod tests {
assert_eq!(expected_tree.nodes, tree.nodes);
}
#[test]
fn nodes() -> Result<(), MerkleError> {
let tree = super::MerkleTree::new(digests_to_words(&LEAVES4)).unwrap();
let root = tree.root();
let l1n0 = tree.get_node(NodeIndex::make(1, 0))?;
let l1n1 = tree.get_node(NodeIndex::make(1, 1))?;
let l2n0 = tree.get_node(NodeIndex::make(2, 0))?;
let l2n1 = tree.get_node(NodeIndex::make(2, 1))?;
let l2n2 = tree.get_node(NodeIndex::make(2, 2))?;
let l2n3 = tree.get_node(NodeIndex::make(2, 3))?;
let nodes: Vec<InnerNodeInfo> = tree.inner_nodes().collect();
let expected = vec![
InnerNodeInfo { value: root, left: l1n0, right: l1n1 },
InnerNodeInfo { value: l1n0, left: l2n0, right: l2n1 },
InnerNodeInfo { value: l1n1, left: l2n2, right: l2n3 },
];
assert_eq!(nodes, expected);
Ok(())
}
proptest! {
#[test]
fn arbitrary_word_can_be_represented_as_digest(
@@ -286,8 +421,8 @@ mod tests {
let digest = RpoDigest::from(word);
// assert the addresses are different
let word_ptr = (&word).as_ptr() as *const u8;
let digest_ptr = (&digest).as_ptr() as *const u8;
let word_ptr = word.as_ptr() as *const u8;
let digest_ptr = digest.as_ptr() as *const u8;
assert_ne!(word_ptr, digest_ptr);
// compare the bytes representation
@@ -300,11 +435,13 @@ mod tests {
// HELPER FUNCTIONS
// --------------------------------------------------------------------------------------------
fn compute_internal_nodes() -> (Word, Word, Word) {
let node2 = Rpo256::hash_elements(&[LEAVES4[0], LEAVES4[1]].concat());
let node3 = Rpo256::hash_elements(&[LEAVES4[2], LEAVES4[3]].concat());
fn compute_internal_nodes() -> (RpoDigest, RpoDigest, RpoDigest) {
let node2 =
Rpo256::hash_elements(&[Word::from(LEAVES4[0]), Word::from(LEAVES4[1])].concat());
let node3 =
Rpo256::hash_elements(&[Word::from(LEAVES4[2]), Word::from(LEAVES4[3])].concat());
let root = Rpo256::merge(&[node2, node3]);
(root.into(), node2.into(), node3.into())
(root, node2, node3)
}
}

46
src/merkle/mmr/bit.rs Normal file
View File

@@ -0,0 +1,46 @@
/// Iterate over the bits of a `usize` and yields the bit positions for the true bits.
pub struct TrueBitPositionIterator {
value: usize,
}
impl TrueBitPositionIterator {
pub fn new(value: usize) -> TrueBitPositionIterator {
TrueBitPositionIterator { value }
}
}
impl Iterator for TrueBitPositionIterator {
type Item = u32;
fn next(&mut self) -> Option<<Self as Iterator>::Item> {
// trailing_zeros is computed with the intrinsic cttz. [Rust 1.67.0] x86 uses the `bsf`
// instruction. AArch64 uses the `rbit clz` instructions.
let zeros = self.value.trailing_zeros();
if zeros == usize::BITS {
None
} else {
let bit_position = zeros;
let mask = 1 << bit_position;
self.value ^= mask;
Some(bit_position)
}
}
}
impl DoubleEndedIterator for TrueBitPositionIterator {
fn next_back(&mut self) -> Option<<Self as Iterator>::Item> {
// trailing_zeros is computed with the intrinsic ctlz. [Rust 1.67.0] x86 uses the `bsr`
// instruction. AArch64 uses the `clz` instruction.
let zeros = self.value.leading_zeros();
if zeros == usize::BITS {
None
} else {
let bit_position = usize::BITS - zeros - 1;
let mask = 1 << bit_position;
self.value ^= mask;
Some(bit_position)
}
}
}

16
src/merkle/mmr/delta.rs Normal file
View File

@@ -0,0 +1,16 @@
use super::super::{RpoDigest, Vec};
/// Container for the update data of a [super::PartialMmr]
#[derive(Debug)]
pub struct MmrDelta {
/// The new version of the [super::Mmr]
pub forest: usize,
/// Update data.
///
/// The data is packed as follows:
/// 1. All the elements needed to perform authentication path updates. These are the right
/// siblings required to perform tree merges on the [super::PartialMmr].
/// 2. The new peaks.
pub data: Vec<RpoDigest>,
}

35
src/merkle/mmr/error.rs Normal file
View File

@@ -0,0 +1,35 @@
use crate::merkle::MerkleError;
use core::fmt::{Display, Formatter};
#[cfg(feature = "std")]
use std::error::Error;
#[derive(Debug, PartialEq, Eq, Clone)]
pub enum MmrError {
InvalidPosition(usize),
InvalidPeaks,
InvalidPeak,
InvalidUpdate,
UnknownPeak,
MerkleError(MerkleError),
}
impl Display for MmrError {
fn fmt(&self, fmt: &mut Formatter<'_>) -> Result<(), core::fmt::Error> {
match self {
MmrError::InvalidPosition(pos) => write!(fmt, "Mmr does not contain position {pos}"),
MmrError::InvalidPeaks => write!(fmt, "Invalid peaks count"),
MmrError::InvalidPeak => {
write!(fmt, "Peak values does not match merkle path computed root")
}
MmrError::InvalidUpdate => write!(fmt, "Invalid mmr update"),
MmrError::UnknownPeak => {
write!(fmt, "Peak not in Mmr")
}
MmrError::MerkleError(err) => write!(fmt, "{}", err),
}
}
}
#[cfg(feature = "std")]
impl Error for MmrError {}

416
src/merkle/mmr/full.rs Normal file
View File

@@ -0,0 +1,416 @@
//! A fully materialized Merkle mountain range (MMR).
//!
//! A MMR is a forest structure, i.e. it is an ordered set of disjoint rooted trees. The trees are
//! ordered by size, from the most to least number of leaves. Every tree is a perfect binary tree,
//! meaning a tree has all its leaves at the same depth, and every inner node has a branch-factor
//! of 2 with both children set.
//!
//! Additionally the structure only supports adding leaves to the right-most tree, the one with the
//! least number of leaves. The structure preserves the invariant that each tree has different
//! depths, i.e. as part of adding adding a new element to the forest the trees with same depth are
//! merged, creating a new tree with depth d+1, this process is continued until the property is
//! reestablished.
use super::{
super::{InnerNodeInfo, MerklePath, Vec},
bit::TrueBitPositionIterator,
leaf_to_corresponding_tree, nodes_in_forest, MmrDelta, MmrError, MmrPeaks, MmrProof, Rpo256,
RpoDigest,
};
// MMR
// ===============================================================================================
/// A fully materialized Merkle Mountain Range, with every tree in the forest and all their
/// elements.
///
/// Since this is a full representation of the MMR, elements are never removed and the MMR will
/// grow roughly `O(2n)` in number of leaf elements.
#[derive(Debug, Clone)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct Mmr {
/// Refer to the `forest` method documentation for details of the semantics of this value.
pub(super) forest: usize,
/// Contains every element of the forest.
///
/// The trees are in postorder sequential representation. This representation allows for all
/// the elements of every tree in the forest to be stored in the same sequential buffer. It
/// also means new elements can be added to the forest, and merging of trees is very cheap with
/// no need to copy elements.
pub(super) nodes: Vec<RpoDigest>,
}
impl Default for Mmr {
fn default() -> Self {
Self::new()
}
}
impl Mmr {
// CONSTRUCTORS
// ============================================================================================
/// Constructor for an empty `Mmr`.
pub fn new() -> Mmr {
Mmr { forest: 0, nodes: Vec::new() }
}
// ACCESSORS
// ============================================================================================
/// Returns the MMR forest representation.
///
/// The forest value has the following interpretations:
/// - its value is the number of elements in the forest
/// - bit count corresponds to the number of trees in the forest
/// - each true bit position determines the depth of a tree in the forest
pub const fn forest(&self) -> usize {
self.forest
}
// FUNCTIONALITY
// ============================================================================================
/// Given a leaf position, returns the Merkle path to its corresponding peak. If the position
/// is greater-or-equal than the tree size an error is returned.
///
/// Note: The leaf position is the 0-indexed number corresponding to the order the leaves were
/// added, this corresponds to the MMR size _prior_ to adding the element. So the 1st element
/// has position 0, the second position 1, and so on.
pub fn open(&self, pos: usize, target_forest: usize) -> Result<MmrProof, MmrError> {
// find the target tree responsible for the MMR position
let tree_bit =
leaf_to_corresponding_tree(pos, target_forest).ok_or(MmrError::InvalidPosition(pos))?;
// isolate the trees before the target
let forest_before = target_forest & high_bitmask(tree_bit + 1);
let index_offset = nodes_in_forest(forest_before);
// update the value position from global to the target tree
let relative_pos = pos - forest_before;
// collect the path and the final index of the target value
let (_, path) = self.collect_merkle_path_and_value(tree_bit, relative_pos, index_offset);
Ok(MmrProof {
forest: target_forest,
position: pos,
merkle_path: MerklePath::new(path),
})
}
/// Returns the leaf value at position `pos`.
///
/// Note: The leaf position is the 0-indexed number corresponding to the order the leaves were
/// added, this corresponds to the MMR size _prior_ to adding the element. So the 1st element
/// has position 0, the second position 1, and so on.
pub fn get(&self, pos: usize) -> Result<RpoDigest, MmrError> {
// find the target tree responsible for the MMR position
let tree_bit =
leaf_to_corresponding_tree(pos, self.forest).ok_or(MmrError::InvalidPosition(pos))?;
// isolate the trees before the target
let forest_before = self.forest & high_bitmask(tree_bit + 1);
let index_offset = nodes_in_forest(forest_before);
// update the value position from global to the target tree
let relative_pos = pos - forest_before;
// collect the path and the final index of the target value
let (value, _) = self.collect_merkle_path_and_value(tree_bit, relative_pos, index_offset);
Ok(value)
}
/// Adds a new element to the MMR.
pub fn add(&mut self, el: RpoDigest) {
// Note: every node is also a tree of size 1, adding an element to the forest creates a new
// rooted-tree of size 1. This may temporarily break the invariant that every tree in the
// forest has different sizes, the loop below will eagerly merge trees of same size and
// restore the invariant.
self.nodes.push(el);
let mut left_offset = self.nodes.len().saturating_sub(2);
let mut right = el;
let mut left_tree = 1;
while self.forest & left_tree != 0 {
right = Rpo256::merge(&[self.nodes[left_offset], right]);
self.nodes.push(right);
left_offset = left_offset.saturating_sub(nodes_in_forest(left_tree));
left_tree <<= 1;
}
self.forest += 1;
}
/// Returns an peaks of the MMR for the version specified by `forest`.
pub fn peaks(&self, forest: usize) -> Result<MmrPeaks, MmrError> {
if forest > self.forest {
return Err(MmrError::InvalidPeaks);
}
let peaks: Vec<RpoDigest> = TrueBitPositionIterator::new(forest)
.rev()
.map(|bit| nodes_in_forest(1 << bit))
.scan(0, |offset, el| {
*offset += el;
Some(*offset)
})
.map(|offset| self.nodes[offset - 1])
.collect();
// Safety: the invariant is maintained by the [Mmr]
let peaks = MmrPeaks::new(forest, peaks).unwrap();
Ok(peaks)
}
/// Compute the required update to `original_forest`.
///
/// The result is a packed sequence of the authentication elements required to update the trees
/// that have been merged together, followed by the new peaks of the [Mmr].
pub fn get_delta(&self, from_forest: usize, to_forest: usize) -> Result<MmrDelta, MmrError> {
if to_forest > self.forest || from_forest > to_forest {
return Err(MmrError::InvalidPeaks);
}
if from_forest == to_forest {
return Ok(MmrDelta { forest: to_forest, data: Vec::new() });
}
let mut result = Vec::new();
// Find the largest tree in this [Mmr] which is new to `from_forest`.
let candidate_trees = to_forest ^ from_forest;
let mut new_high = 1 << candidate_trees.ilog2();
// Collect authentication nodes used for tree merges
// ----------------------------------------------------------------------------------------
// Find the trees from `from_forest` that have been merged into `new_high`.
let mut merges = from_forest & (new_high - 1);
// Find the peaks that are common to `from_forest` and this [Mmr]
let common_trees = from_forest ^ merges;
if merges != 0 {
// Skip the smallest trees unknown to `from_forest`.
let mut target = 1 << merges.trailing_zeros();
// Collect siblings required to computed the merged tree's peak
while target < new_high {
// Computes the offset to the smallest know peak
// - common_trees: peaks unchanged in the current update, target comes after these.
// - merges: peaks that have not been merged so far, target comes after these.
// - target: tree from which to load the sibling. On the first iteration this is a
// value known by the partial mmr, on subsequent iterations this value is to be
// computed from the known peaks and provided authentication nodes.
let known = nodes_in_forest(common_trees | merges | target);
let sibling = nodes_in_forest(target);
result.push(self.nodes[known + sibling - 1]);
// Update the target and account for tree merges
target <<= 1;
while merges & target != 0 {
target <<= 1;
}
// Remove the merges done so far
merges ^= merges & (target - 1);
}
} else {
// The new high tree may not be the result of any merges, if it is smaller than all the
// trees of `from_forest`.
new_high = 0;
}
// Collect the new [Mmr] peaks
// ----------------------------------------------------------------------------------------
let mut new_peaks = to_forest ^ common_trees ^ new_high;
let old_peaks = to_forest ^ new_peaks;
let mut offset = nodes_in_forest(old_peaks);
while new_peaks != 0 {
let target = 1 << new_peaks.ilog2();
offset += nodes_in_forest(target);
result.push(self.nodes[offset - 1]);
new_peaks ^= target;
}
Ok(MmrDelta { forest: to_forest, data: result })
}
/// An iterator over inner nodes in the MMR. The order of iteration is unspecified.
pub fn inner_nodes(&self) -> MmrNodes {
MmrNodes {
mmr: self,
forest: 0,
last_right: 0,
index: 0,
}
}
// UTILITIES
// ============================================================================================
/// Internal function used to collect the Merkle path of a value.
///
/// The arguments are relative to the target tree. To compute the opening of the second leaf
/// for a tree with depth 2 in the forest `0b110`:
///
/// - `tree_bit`: Depth of the target tree, e.g. 2 for the smallest tree.
/// - `relative_pos`: 0-indexed leaf position in the target tree, e.g. 1 for the second leaf.
/// - `index_offset`: Node count prior to the target tree, e.g. 7 for the tree of depth 3.
fn collect_merkle_path_and_value(
&self,
tree_bit: u32,
relative_pos: usize,
index_offset: usize,
) -> (RpoDigest, Vec<RpoDigest>) {
// see documentation of `leaf_to_corresponding_tree` for details
let tree_depth = (tree_bit + 1) as usize;
let mut path = Vec::with_capacity(tree_depth);
// The tree walk below goes from the root to the leaf, compute the root index to start
let mut forest_target = 1usize << tree_bit;
let mut index = nodes_in_forest(forest_target) - 1;
// Loop until the leaf is reached
while forest_target > 1 {
// Update the depth of the tree to correspond to a subtree
forest_target >>= 1;
// compute the indices of the right and left subtrees based on the post-order
let right_offset = index - 1;
let left_offset = right_offset - nodes_in_forest(forest_target);
let left_or_right = relative_pos & forest_target;
let sibling = if left_or_right != 0 {
// going down the right subtree, the right child becomes the new root
index = right_offset;
// and the left child is the authentication
self.nodes[index_offset + left_offset]
} else {
index = left_offset;
self.nodes[index_offset + right_offset]
};
path.push(sibling);
}
debug_assert!(path.len() == tree_depth - 1);
// the rest of the codebase has the elements going from leaf to root, adjust it here for
// easy of use/consistency sake
path.reverse();
let value = self.nodes[index_offset + index];
(value, path)
}
}
// CONVERSIONS
// ================================================================================================
impl<T> From<T> for Mmr
where
T: IntoIterator<Item = RpoDigest>,
{
fn from(values: T) -> Self {
let mut mmr = Mmr::new();
for v in values {
mmr.add(v)
}
mmr
}
}
// ITERATOR
// ===============================================================================================
/// Yields inner nodes of the [Mmr].
pub struct MmrNodes<'a> {
/// [Mmr] being yielded, when its `forest` value is matched, the iterations is finished.
mmr: &'a Mmr,
/// Keeps track of the left nodes yielded so far waiting for a right pair, this matches the
/// semantics of the [Mmr]'s forest attribute, since that too works as a buffer of left nodes
/// waiting for a pair to be hashed together.
forest: usize,
/// Keeps track of the last right node yielded, after this value is set, the next iteration
/// will be its parent with its corresponding left node that has been yield already.
last_right: usize,
/// The current index in the `nodes` vector.
index: usize,
}
impl<'a> Iterator for MmrNodes<'a> {
type Item = InnerNodeInfo;
fn next(&mut self) -> Option<Self::Item> {
debug_assert!(self.last_right.count_ones() <= 1, "last_right tracks zero or one element");
// only parent nodes are emitted, remove the single node tree from the forest
let target = self.mmr.forest & (usize::MAX << 1);
if self.forest < target {
if self.last_right == 0 {
// yield the left leaf
debug_assert!(self.last_right == 0, "left must be before right");
self.forest |= 1;
self.index += 1;
// yield the right leaf
debug_assert!((self.forest & 1) == 1, "right must be after left");
self.last_right |= 1;
self.index += 1;
};
debug_assert!(
self.forest & self.last_right != 0,
"parent requires both a left and right",
);
// compute the number of nodes in the right tree, this is the offset to the
// previous left parent
let right_nodes = nodes_in_forest(self.last_right);
// the next parent position is one above the position of the pair
let parent = self.last_right << 1;
// the left node has been paired and the current parent yielded, removed it from the forest
self.forest ^= self.last_right;
if self.forest & parent == 0 {
// this iteration yielded the left parent node
debug_assert!(self.forest & 1 == 0, "next iteration yields a left leaf");
self.last_right = 0;
self.forest ^= parent;
} else {
// the left node of the parent level has been yielded already, this iteration
// was the right parent. Next iteration yields their parent.
self.last_right = parent;
}
// yields a parent
let value = self.mmr.nodes[self.index];
let right = self.mmr.nodes[self.index - 1];
let left = self.mmr.nodes[self.index - 1 - right_nodes];
self.index += 1;
let node = InnerNodeInfo { value, left, right };
Some(node)
} else {
None
}
}
}
// UTILITIES
// ===============================================================================================
/// Return a bitmask for the bits including and above the given position.
pub(crate) const fn high_bitmask(bit: u32) -> usize {
if bit > usize::BITS - 1 {
0
} else {
usize::MAX << bit
}
}

164
src/merkle/mmr/inorder.rs Normal file
View File

@@ -0,0 +1,164 @@
//! Index for nodes of a binary tree based on an in-order tree walk.
//!
//! In-order walks have the parent node index split its left and right subtrees. All the left
//! children have indexes lower than the parent, meanwhile all the right subtree higher indexes.
//! This property makes it is easy to compute changes to the index by adding or subtracting the
//! leaves count.
use core::num::NonZeroUsize;
// IN-ORDER INDEX
// ================================================================================================
/// Index of nodes in a perfectly balanced binary tree based on an in-order tree walk.
#[derive(Debug, Copy, Clone, PartialEq, Eq, PartialOrd, Ord)]
pub struct InOrderIndex {
idx: usize,
}
impl InOrderIndex {
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Returns a new [InOrderIndex] instantiated from the provided value.
pub fn new(idx: NonZeroUsize) -> InOrderIndex {
InOrderIndex { idx: idx.get() }
}
/// Return a new [InOrderIndex] instantiated from the specified leaf position.
///
/// # Panics:
/// If `leaf` is higher than or equal to `usize::MAX / 2`.
pub fn from_leaf_pos(leaf: usize) -> InOrderIndex {
// Convert the position from 0-indexed to 1-indexed, since the bit manipulation in this
// implementation only works 1-indexed counting.
let pos = leaf + 1;
InOrderIndex { idx: pos * 2 - 1 }
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// True if the index is pointing at a leaf.
///
/// Every odd number represents a leaf.
pub fn is_leaf(&self) -> bool {
self.idx & 1 == 1
}
/// Returns true if this note is a left child of its parent.
pub fn is_left_child(&self) -> bool {
self.parent().left_child() == *self
}
/// Returns the level of the index.
///
/// Starts at level zero for leaves and increases by one for each parent.
pub fn level(&self) -> u32 {
self.idx.trailing_zeros()
}
/// Returns the index of the left child.
///
/// # Panics:
/// If the index corresponds to a leaf.
pub fn left_child(&self) -> InOrderIndex {
// The left child is itself a parent, with an index that splits its left/right subtrees. To
// go from the parent index to its left child, it is only necessary to subtract the count
// of elements on the child's right subtree + 1.
let els = 1 << (self.level() - 1);
InOrderIndex { idx: self.idx - els }
}
/// Returns the index of the right child.
///
/// # Panics:
/// If the index corresponds to a leaf.
pub fn right_child(&self) -> InOrderIndex {
// To compute the index of the parent of the right subtree it is sufficient to add the size
// of its left subtree + 1.
let els = 1 << (self.level() - 1);
InOrderIndex { idx: self.idx + els }
}
/// Returns the index of the parent node.
pub fn parent(&self) -> InOrderIndex {
// If the current index corresponds to a node in a left tree, to go up a level it is
// required to add the number of nodes of the right sibling, analogously if the node is a
// right child, going up requires subtracting the number of nodes in its left subtree.
//
// Both of the above operations can be performed by bitwise manipulation. Below the mask
// sets the number of trailing zeros to be equal the new level of the index, and the bit
// marks the parent.
let target = self.level() + 1;
let bit = 1 << target;
let mask = bit - 1;
let idx = self.idx ^ (self.idx & mask);
InOrderIndex { idx: idx | bit }
}
/// Returns the index of the sibling node.
pub fn sibling(&self) -> InOrderIndex {
let parent = self.parent();
if *self > parent {
parent.left_child()
} else {
parent.right_child()
}
}
/// Returns the inner value of this [InOrderIndex].
pub fn inner(&self) -> u64 {
self.idx as u64
}
}
// CONVERSIONS FROM IN-ORDER INDEX
// ------------------------------------------------------------------------------------------------
impl From<InOrderIndex> for u64 {
fn from(index: InOrderIndex) -> Self {
index.idx as u64
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod test {
use super::InOrderIndex;
use proptest::prelude::*;
proptest! {
#[test]
fn proptest_inorder_index_random(count in 1..1000usize) {
let left_pos = count * 2;
let right_pos = count * 2 + 1;
let left = InOrderIndex::from_leaf_pos(left_pos);
let right = InOrderIndex::from_leaf_pos(right_pos);
assert!(left.is_leaf());
assert!(right.is_leaf());
assert_eq!(left.parent(), right.parent());
assert_eq!(left.parent().right_child(), right);
assert_eq!(left, right.parent().left_child());
assert_eq!(left.sibling(), right);
assert_eq!(left, right.sibling());
}
}
#[test]
fn test_inorder_index_basic() {
let left = InOrderIndex::from_leaf_pos(0);
let right = InOrderIndex::from_leaf_pos(1);
assert!(left.is_leaf());
assert!(right.is_leaf());
assert_eq!(left.parent(), right.parent());
assert_eq!(left.parent().right_child(), right);
assert_eq!(left, right.parent().left_child());
assert_eq!(left.sibling(), right);
assert_eq!(left, right.sibling());
}
}

67
src/merkle/mmr/mod.rs Normal file
View File

@@ -0,0 +1,67 @@
mod bit;
mod delta;
mod error;
mod full;
mod inorder;
mod partial;
mod peaks;
mod proof;
#[cfg(test)]
mod tests;
use super::{Felt, Rpo256, RpoDigest, Word};
// REEXPORTS
// ================================================================================================
pub use delta::MmrDelta;
pub use error::MmrError;
pub use full::Mmr;
pub use inorder::InOrderIndex;
pub use partial::PartialMmr;
pub use peaks::MmrPeaks;
pub use proof::MmrProof;
// UTILITIES
// ===============================================================================================
/// Given a 0-indexed leaf position and the current forest, return the tree number responsible for
/// the position.
///
/// Note:
/// The result is a tree position `p`, it has the following interpretations. $p+1$ is the depth of
/// the tree. Because the root element is not part of the proof, $p$ is the length of the
/// authentication path. $2^p$ is equal to the number of leaves in this particular tree. and
/// $2^(p+1)-1$ corresponds to size of the tree.
const fn leaf_to_corresponding_tree(pos: usize, forest: usize) -> Option<u32> {
if pos >= forest {
None
} else {
// - each bit in the forest is a unique tree and the bit position its power-of-two size
// - each tree owns a consecutive range of positions equal to its size from left-to-right
// - this means the first tree owns from `0` up to the `2^k_0` first positions, where `k_0`
// is the highest true bit position, the second tree from `2^k_0 + 1` up to `2^k_1` where
// `k_1` is the second highest bit, so on.
// - this means the highest bits work as a category marker, and the position is owned by
// the first tree which doesn't share a high bit with the position
let before = forest & pos;
let after = forest ^ before;
let tree = after.ilog2();
Some(tree)
}
}
/// Return the total number of nodes of a given forest
///
/// Panics:
///
/// This will panic if the forest has size greater than `usize::MAX / 2`
const fn nodes_in_forest(forest: usize) -> usize {
// - the size of a perfect binary tree is $2^{k+1}-1$ or $2*2^k-1$
// - the forest represents the sum of $2^k$ so a single multiplication is necessary
// - the number of `-1` is the same as the number of trees, which is the same as the number
// bits set
let tree_count = forest.count_ones() as usize;
forest * 2 - tree_count
}

911
src/merkle/mmr/partial.rs Normal file
View File

@@ -0,0 +1,911 @@
use super::{MmrDelta, MmrProof, Rpo256, RpoDigest};
use crate::{
merkle::{
mmr::{leaf_to_corresponding_tree, nodes_in_forest},
InOrderIndex, InnerNodeInfo, MerklePath, MmrError, MmrPeaks,
},
utils::{
collections::{BTreeMap, BTreeSet, Vec},
vec,
},
};
// TYPE ALIASES
// ================================================================================================
type NodeMap = BTreeMap<InOrderIndex, RpoDigest>;
// PARTIAL MERKLE MOUNTAIN RANGE
// ================================================================================================
/// Partially materialized Merkle Mountain Range (MMR), used to efficiently store and update the
/// authentication paths for a subset of the elements in a full MMR.
///
/// This structure store only the authentication path for a value, the value itself is stored
/// separately.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct PartialMmr {
/// The version of the MMR.
///
/// This value serves the following purposes:
///
/// - The forest is a counter for the total number of elements in the MMR.
/// - Since the MMR is an append-only structure, every change to it causes a change to the
/// `forest`, so this value has a dual purpose as a version tag.
/// - The bits in the forest also corresponds to the count and size of every perfect binary
/// tree that composes the MMR structure, which server to compute indexes and perform
/// validation.
pub(crate) forest: usize,
/// The MMR peaks.
///
/// The peaks are used for two reasons:
///
/// 1. It authenticates the addition of an element to the [PartialMmr], ensuring only valid
/// elements are tracked.
/// 2. During a MMR update peaks can be merged by hashing the left and right hand sides. The
/// peaks are used as the left hand.
///
/// All the peaks of every tree in the MMR forest. The peaks are always ordered by number of
/// leaves, starting from the peak with most children, to the one with least.
pub(crate) peaks: Vec<RpoDigest>,
/// Authentication nodes used to construct merkle paths for a subset of the MMR's leaves.
///
/// This does not include the MMR's peaks nor the tracked nodes, only the elements required to
/// construct their authentication paths. This property is used to detect when elements can be
/// safely removed, because they are no longer required to authenticate any element in the
/// [PartialMmr].
///
/// The elements in the MMR are referenced using a in-order tree index. This indexing scheme
/// permits for easy computation of the relative nodes (left/right children, sibling, parent),
/// which is useful for traversal. The indexing is also stable, meaning that merges to the
/// trees in the MMR can be represented without rewrites of the indexes.
pub(crate) nodes: NodeMap,
/// Flag indicating if the odd element should be tracked.
///
/// This flag is necessary because the sibling of the odd doesn't exist yet, so it can not be
/// added into `nodes` to signal the value is being tracked.
pub(crate) track_latest: bool,
}
impl PartialMmr {
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Returns a new [PartialMmr] instantiated from the specified peaks.
pub fn from_peaks(peaks: MmrPeaks) -> Self {
let forest = peaks.num_leaves();
let peaks = peaks.into();
let nodes = BTreeMap::new();
let track_latest = false;
Self { forest, peaks, nodes, track_latest }
}
/// Returns a new [PartialMmr] instantiated from the specified components.
///
/// This constructor does not check the consistency between peaks and nodes. If the specified
/// peaks are nodes are inconsistent, the returned partial MMR may exhibit undefined behavior.
pub fn from_parts(peaks: MmrPeaks, nodes: NodeMap, track_latest: bool) -> Self {
let forest = peaks.num_leaves();
let peaks = peaks.into();
Self { forest, peaks, nodes, track_latest }
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the current `forest` of this [PartialMmr].
///
/// This value corresponds to the version of the [PartialMmr] and the number of leaves in the
/// underlying MMR.
pub fn forest(&self) -> usize {
self.forest
}
/// Returns the number of leaves in the underlying MMR for this [PartialMmr].
pub fn num_leaves(&self) -> usize {
self.forest
}
/// Returns the peaks of the MMR for this [PartialMmr].
pub fn peaks(&self) -> MmrPeaks {
// expect() is OK here because the constructor ensures that MMR peaks can be constructed
// correctly
MmrPeaks::new(self.forest, self.peaks.clone()).expect("invalid MMR peaks")
}
/// Returns true if this partial MMR tracks an authentication path for the leaf at the
/// specified position.
pub fn is_tracked(&self, pos: usize) -> bool {
if pos >= self.forest {
return false;
} else if pos == self.forest - 1 && self.forest & 1 != 0 {
// if the number of leaves in the MMR is odd and the position is for the last leaf
// whether the leaf is tracked is defined by the `track_latest` flag
return self.track_latest;
}
let leaf_index = InOrderIndex::from_leaf_pos(pos);
self.is_tracked_node(&leaf_index)
}
/// Given a leaf position, returns the Merkle path to its corresponding peak, or None if this
/// partial MMR does not track an authentication paths for the specified leaf.
///
/// Note: The leaf position is the 0-indexed number corresponding to the order the leaves were
/// added, this corresponds to the MMR size _prior_ to adding the element. So the 1st element
/// has position 0, the second position 1, and so on.
///
/// # Errors
/// Returns an error if the specified position is greater-or-equal than the number of leaves
/// in the underlying MMR.
pub fn open(&self, pos: usize) -> Result<Option<MmrProof>, MmrError> {
let tree_bit =
leaf_to_corresponding_tree(pos, self.forest).ok_or(MmrError::InvalidPosition(pos))?;
let depth = tree_bit as usize;
let mut nodes = Vec::with_capacity(depth);
let mut idx = InOrderIndex::from_leaf_pos(pos);
while let Some(node) = self.nodes.get(&idx.sibling()) {
nodes.push(*node);
idx = idx.parent();
}
// If there are nodes then the path must be complete, otherwise it is a bug
debug_assert!(nodes.is_empty() || nodes.len() == depth);
if nodes.len() != depth {
// The requested `pos` is not being tracked.
Ok(None)
} else {
Ok(Some(MmrProof {
forest: self.forest,
position: pos,
merkle_path: MerklePath::new(nodes),
}))
}
}
// ITERATORS
// --------------------------------------------------------------------------------------------
/// Returns an iterator nodes of all authentication paths of this [PartialMmr].
pub fn nodes(&self) -> impl Iterator<Item = (&InOrderIndex, &RpoDigest)> {
self.nodes.iter()
}
/// Returns an iterator over inner nodes of this [PartialMmr] for the specified leaves.
///
/// The order of iteration is not defined. If a leaf is not presented in this partial MMR it
/// is silently ignored.
pub fn inner_nodes<'a, I: Iterator<Item = (usize, RpoDigest)> + 'a>(
&'a self,
mut leaves: I,
) -> impl Iterator<Item = InnerNodeInfo> + '_ {
let stack = if let Some((pos, leaf)) = leaves.next() {
let idx = InOrderIndex::from_leaf_pos(pos);
vec![(idx, leaf)]
} else {
Vec::new()
};
InnerNodeIterator {
nodes: &self.nodes,
leaves,
stack,
seen_nodes: BTreeSet::new(),
}
}
// STATE MUTATORS
// --------------------------------------------------------------------------------------------
/// Adds a new peak and optionally track it. Returns a vector of the authentication nodes
/// inserted into this [PartialMmr] as a result of this operation.
///
/// When `track` is `true` the new leaf is tracked.
pub fn add(&mut self, leaf: RpoDigest, track: bool) -> Vec<(InOrderIndex, RpoDigest)> {
self.forest += 1;
let merges = self.forest.trailing_zeros() as usize;
let mut new_nodes = Vec::with_capacity(merges);
let peak = if merges == 0 {
self.track_latest = track;
leaf
} else {
let mut track_right = track;
let mut track_left = self.track_latest;
let mut right = leaf;
let mut right_idx = forest_to_rightmost_index(self.forest);
for _ in 0..merges {
let left = self.peaks.pop().expect("Missing peak");
let left_idx = right_idx.sibling();
if track_right {
let old = self.nodes.insert(left_idx, left);
new_nodes.push((left_idx, left));
debug_assert!(
old.is_none(),
"Idx {:?} already contained an element {:?}",
left_idx,
old
);
};
if track_left {
let old = self.nodes.insert(right_idx, right);
new_nodes.push((right_idx, right));
debug_assert!(
old.is_none(),
"Idx {:?} already contained an element {:?}",
right_idx,
old
);
};
// Update state for the next iteration.
// --------------------------------------------------------------------------------
// This layer is merged, go up one layer.
right_idx = right_idx.parent();
// Merge the current layer. The result is either the right element of the next
// merge, or a new peak.
right = Rpo256::merge(&[left, right]);
// This iteration merged the left and right nodes, the new value is always used as
// the next iteration's right node. Therefore the tracking flags of this iteration
// have to be merged into the right side only.
track_right = track_right || track_left;
// On the next iteration, a peak will be merged. If any of its children are tracked,
// then we have to track the left side
track_left = self.is_tracked_node(&right_idx.sibling());
}
right
};
self.peaks.push(peak);
new_nodes
}
/// Adds the authentication path represented by [MerklePath] if it is valid.
///
/// The `leaf_pos` refers to the global position of the leaf in the MMR, these are 0-indexed
/// values assigned in a strictly monotonic fashion as elements are inserted into the MMR,
/// this value corresponds to the values used in the MMR structure.
///
/// The `leaf` corresponds to the value at `leaf_pos`, and `path` is the authentication path for
/// that element up to its corresponding Mmr peak. The `leaf` is only used to compute the root
/// from the authentication path to valid the data, only the authentication data is saved in
/// the structure. If the value is required it should be stored out-of-band.
pub fn track(
&mut self,
leaf_pos: usize,
leaf: RpoDigest,
path: &MerklePath,
) -> Result<(), MmrError> {
// Checks there is a tree with same depth as the authentication path, if not the path is
// invalid.
let tree = 1 << path.depth();
if tree & self.forest == 0 {
return Err(MmrError::UnknownPeak);
};
if leaf_pos + 1 == self.forest
&& path.depth() == 0
&& self.peaks.last().map_or(false, |v| *v == leaf)
{
self.track_latest = true;
return Ok(());
}
// ignore the trees smaller than the target (these elements are position after the current
// target and don't affect the target leaf_pos)
let target_forest = self.forest ^ (self.forest & (tree - 1));
let peak_pos = (target_forest.count_ones() - 1) as usize;
// translate from mmr leaf_pos to merkle path
let path_idx = leaf_pos - (target_forest ^ tree);
// Compute the root of the authentication path, and check it matches the current version of
// the PartialMmr.
let computed = path.compute_root(path_idx as u64, leaf).map_err(MmrError::MerkleError)?;
if self.peaks[peak_pos] != computed {
return Err(MmrError::InvalidPeak);
}
let mut idx = InOrderIndex::from_leaf_pos(leaf_pos);
for leaf in path.nodes() {
self.nodes.insert(idx.sibling(), *leaf);
idx = idx.parent();
}
Ok(())
}
/// Removes a leaf of the [PartialMmr] and the unused nodes from the authentication path.
///
/// Note: `leaf_pos` corresponds to the position in the MMR and not on an individual tree.
pub fn untrack(&mut self, leaf_pos: usize) {
let mut idx = InOrderIndex::from_leaf_pos(leaf_pos);
self.nodes.remove(&idx.sibling());
// `idx` represent the element that can be computed by the authentication path, because
// these elements can be computed they are not saved for the authentication of the current
// target. In other words, if the idx is present it was added for the authentication of
// another element, and no more elements should be removed otherwise it would remove that
// element's authentication data.
while !self.nodes.contains_key(&idx) {
idx = idx.parent();
self.nodes.remove(&idx.sibling());
}
}
/// Applies updates to this [PartialMmr] and returns a vector of new authentication nodes
/// inserted into the partial MMR.
pub fn apply(&mut self, delta: MmrDelta) -> Result<Vec<(InOrderIndex, RpoDigest)>, MmrError> {
if delta.forest < self.forest {
return Err(MmrError::InvalidPeaks);
}
let mut inserted_nodes = Vec::new();
if delta.forest == self.forest {
if !delta.data.is_empty() {
return Err(MmrError::InvalidUpdate);
}
return Ok(inserted_nodes);
}
// find the tree merges
let changes = self.forest ^ delta.forest;
let largest = 1 << changes.ilog2();
let merges = self.forest & (largest - 1);
debug_assert!(
!self.track_latest || (merges & 1) == 1,
"if there is an odd element, a merge is required"
);
// count the number elements needed to produce largest from the current state
let (merge_count, new_peaks) = if merges != 0 {
let depth = largest.trailing_zeros();
let skipped = merges.trailing_zeros();
let computed = merges.count_ones() - 1;
let merge_count = depth - skipped - computed;
let new_peaks = delta.forest & (largest - 1);
(merge_count, new_peaks)
} else {
(0, changes)
};
// verify the delta size
if (delta.data.len() as u32) != merge_count + new_peaks.count_ones() {
return Err(MmrError::InvalidUpdate);
}
// keeps track of how many data elements from the update have been consumed
let mut update_count = 0;
if merges != 0 {
// starts at the smallest peak and follows the merged peaks
let mut peak_idx = forest_to_root_index(self.forest);
// match order of the update data while applying it
self.peaks.reverse();
// set to true when the data is needed for authentication paths updates
let mut track = self.track_latest;
self.track_latest = false;
let mut peak_count = 0;
let mut target = 1 << merges.trailing_zeros();
let mut new = delta.data[0];
update_count += 1;
while target < largest {
// check if either the left or right subtrees have saved for authentication paths.
// If so, turn tracking on to update those paths.
if target != 1 && !track {
track = self.is_tracked_node(&peak_idx);
}
// update data only contains the nodes from the right subtrees, left nodes are
// either previously known peaks or computed values
let (left, right) = if target & merges != 0 {
let peak = self.peaks[peak_count];
let sibling_idx = peak_idx.sibling();
// if the sibling peak is tracked, add this peaks to the set of
// authentication nodes
if self.is_tracked_node(&sibling_idx) {
self.nodes.insert(peak_idx, new);
inserted_nodes.push((peak_idx, new));
}
peak_count += 1;
(peak, new)
} else {
let update = delta.data[update_count];
update_count += 1;
(new, update)
};
if track {
let sibling_idx = peak_idx.sibling();
if peak_idx.is_left_child() {
self.nodes.insert(sibling_idx, right);
inserted_nodes.push((sibling_idx, right));
} else {
self.nodes.insert(sibling_idx, left);
inserted_nodes.push((sibling_idx, left));
}
}
peak_idx = peak_idx.parent();
new = Rpo256::merge(&[left, right]);
target <<= 1;
}
debug_assert!(peak_count == (merges.count_ones() as usize));
// restore the peaks order
self.peaks.reverse();
// remove the merged peaks
self.peaks.truncate(self.peaks.len() - peak_count);
// add the newly computed peak, the result of the merges
self.peaks.push(new);
}
// The rest of the update data is composed of peaks. None of these elements can contain
// tracked elements because the peaks were unknown, and it is not possible to add elements
// for tacking without authenticating it to a peak.
self.peaks.extend_from_slice(&delta.data[update_count..]);
self.forest = delta.forest;
debug_assert!(self.peaks.len() == (self.forest.count_ones() as usize));
Ok(inserted_nodes)
}
// HELPER METHODS
// --------------------------------------------------------------------------------------------
/// Returns true if this [PartialMmr] tracks authentication path for the node at the specified
/// index.
fn is_tracked_node(&self, node_index: &InOrderIndex) -> bool {
if node_index.is_leaf() {
self.nodes.contains_key(&node_index.sibling())
} else {
let left_child = node_index.left_child();
let right_child = node_index.right_child();
self.nodes.contains_key(&left_child) | self.nodes.contains_key(&right_child)
}
}
}
// CONVERSIONS
// ================================================================================================
impl From<MmrPeaks> for PartialMmr {
fn from(peaks: MmrPeaks) -> Self {
Self::from_peaks(peaks)
}
}
impl From<PartialMmr> for MmrPeaks {
fn from(partial_mmr: PartialMmr) -> Self {
// Safety: the [PartialMmr] maintains the constraints the number of true bits in the forest
// matches the number of peaks, as required by the [MmrPeaks]
MmrPeaks::new(partial_mmr.forest, partial_mmr.peaks).unwrap()
}
}
impl From<&MmrPeaks> for PartialMmr {
fn from(peaks: &MmrPeaks) -> Self {
Self::from_peaks(peaks.clone())
}
}
impl From<&PartialMmr> for MmrPeaks {
fn from(partial_mmr: &PartialMmr) -> Self {
// Safety: the [PartialMmr] maintains the constraints the number of true bits in the forest
// matches the number of peaks, as required by the [MmrPeaks]
MmrPeaks::new(partial_mmr.forest, partial_mmr.peaks.clone()).unwrap()
}
}
// ITERATORS
// ================================================================================================
/// An iterator over every inner node of the [PartialMmr].
pub struct InnerNodeIterator<'a, I: Iterator<Item = (usize, RpoDigest)>> {
nodes: &'a NodeMap,
leaves: I,
stack: Vec<(InOrderIndex, RpoDigest)>,
seen_nodes: BTreeSet<InOrderIndex>,
}
impl<'a, I: Iterator<Item = (usize, RpoDigest)>> Iterator for InnerNodeIterator<'a, I> {
type Item = InnerNodeInfo;
fn next(&mut self) -> Option<Self::Item> {
while let Some((idx, node)) = self.stack.pop() {
let parent_idx = idx.parent();
let new_node = self.seen_nodes.insert(parent_idx);
// if we haven't seen this node's parent before, and the node has a sibling, return
// the inner node defined by the parent of this node, and move up the branch
if new_node {
if let Some(sibling) = self.nodes.get(&idx.sibling()) {
let (left, right) = if parent_idx.left_child() == idx {
(node, *sibling)
} else {
(*sibling, node)
};
let parent = Rpo256::merge(&[left, right]);
let inner_node = InnerNodeInfo { value: parent, left, right };
self.stack.push((parent_idx, parent));
return Some(inner_node);
}
}
// the previous leaf has been processed, try to process the next leaf
if let Some((pos, leaf)) = self.leaves.next() {
let idx = InOrderIndex::from_leaf_pos(pos);
self.stack.push((idx, leaf));
}
}
None
}
}
// UTILS
// ================================================================================================
/// Given the description of a `forest`, returns the index of the root element of the smallest tree
/// in it.
fn forest_to_root_index(forest: usize) -> InOrderIndex {
// Count total size of all trees in the forest.
let nodes = nodes_in_forest(forest);
// Add the count for the parent nodes that separate each tree. These are allocated but
// currently empty, and correspond to the nodes that will be used once the trees are merged.
let open_trees = (forest.count_ones() - 1) as usize;
// Remove the count of the right subtree of the target tree, target tree root index comes
// before the subtree for the in-order tree walk.
let right_subtree_count = ((1u32 << forest.trailing_zeros()) - 1) as usize;
let idx = nodes + open_trees - right_subtree_count;
InOrderIndex::new(idx.try_into().unwrap())
}
/// Given the description of a `forest`, returns the index of the right most element.
fn forest_to_rightmost_index(forest: usize) -> InOrderIndex {
// Count total size of all trees in the forest.
let nodes = nodes_in_forest(forest);
// Add the count for the parent nodes that separate each tree. These are allocated but
// currently empty, and correspond to the nodes that will be used once the trees are merged.
let open_trees = (forest.count_ones() - 1) as usize;
let idx = nodes + open_trees;
InOrderIndex::new(idx.try_into().unwrap())
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::{
forest_to_rightmost_index, forest_to_root_index, BTreeSet, InOrderIndex, MmrPeaks,
PartialMmr, RpoDigest, Vec,
};
use crate::merkle::{int_to_node, MerkleStore, Mmr, NodeIndex};
const LEAVES: [RpoDigest; 7] = [
int_to_node(0),
int_to_node(1),
int_to_node(2),
int_to_node(3),
int_to_node(4),
int_to_node(5),
int_to_node(6),
];
#[test]
fn test_forest_to_root_index() {
fn idx(pos: usize) -> InOrderIndex {
InOrderIndex::new(pos.try_into().unwrap())
}
// When there is a single tree in the forest, the index is equivalent to the number of
// leaves in that tree, which is `2^n`.
assert_eq!(forest_to_root_index(0b0001), idx(1));
assert_eq!(forest_to_root_index(0b0010), idx(2));
assert_eq!(forest_to_root_index(0b0100), idx(4));
assert_eq!(forest_to_root_index(0b1000), idx(8));
assert_eq!(forest_to_root_index(0b0011), idx(5));
assert_eq!(forest_to_root_index(0b0101), idx(9));
assert_eq!(forest_to_root_index(0b1001), idx(17));
assert_eq!(forest_to_root_index(0b0111), idx(13));
assert_eq!(forest_to_root_index(0b1011), idx(21));
assert_eq!(forest_to_root_index(0b1111), idx(29));
assert_eq!(forest_to_root_index(0b0110), idx(10));
assert_eq!(forest_to_root_index(0b1010), idx(18));
assert_eq!(forest_to_root_index(0b1100), idx(20));
assert_eq!(forest_to_root_index(0b1110), idx(26));
}
#[test]
fn test_forest_to_rightmost_index() {
fn idx(pos: usize) -> InOrderIndex {
InOrderIndex::new(pos.try_into().unwrap())
}
for forest in 1..256 {
assert!(forest_to_rightmost_index(forest).inner() % 2 == 1, "Leaves are always odd");
}
assert_eq!(forest_to_rightmost_index(0b0001), idx(1));
assert_eq!(forest_to_rightmost_index(0b0010), idx(3));
assert_eq!(forest_to_rightmost_index(0b0011), idx(5));
assert_eq!(forest_to_rightmost_index(0b0100), idx(7));
assert_eq!(forest_to_rightmost_index(0b0101), idx(9));
assert_eq!(forest_to_rightmost_index(0b0110), idx(11));
assert_eq!(forest_to_rightmost_index(0b0111), idx(13));
assert_eq!(forest_to_rightmost_index(0b1000), idx(15));
assert_eq!(forest_to_rightmost_index(0b1001), idx(17));
assert_eq!(forest_to_rightmost_index(0b1010), idx(19));
assert_eq!(forest_to_rightmost_index(0b1011), idx(21));
assert_eq!(forest_to_rightmost_index(0b1100), idx(23));
assert_eq!(forest_to_rightmost_index(0b1101), idx(25));
assert_eq!(forest_to_rightmost_index(0b1110), idx(27));
assert_eq!(forest_to_rightmost_index(0b1111), idx(29));
}
#[test]
fn test_partial_mmr_apply_delta() {
// build an MMR with 10 nodes (2 peaks) and a partial MMR based on it
let mut mmr = Mmr::default();
(0..10).for_each(|i| mmr.add(int_to_node(i)));
let mut partial_mmr: PartialMmr = mmr.peaks(mmr.forest()).unwrap().into();
// add authentication path for position 1 and 8
{
let node = mmr.get(1).unwrap();
let proof = mmr.open(1, mmr.forest()).unwrap();
partial_mmr.track(1, node, &proof.merkle_path).unwrap();
}
{
let node = mmr.get(8).unwrap();
let proof = mmr.open(8, mmr.forest()).unwrap();
partial_mmr.track(8, node, &proof.merkle_path).unwrap();
}
// add 2 more nodes into the MMR and validate apply_delta()
(10..12).for_each(|i| mmr.add(int_to_node(i)));
validate_apply_delta(&mmr, &mut partial_mmr);
// add 1 more node to the MMR, validate apply_delta() and start tracking the node
mmr.add(int_to_node(12));
validate_apply_delta(&mmr, &mut partial_mmr);
{
let node = mmr.get(12).unwrap();
let proof = mmr.open(12, mmr.forest()).unwrap();
partial_mmr.track(12, node, &proof.merkle_path).unwrap();
assert!(partial_mmr.track_latest);
}
// by this point we are tracking authentication paths for positions: 1, 8, and 12
// add 3 more nodes to the MMR (collapses to 1 peak) and validate apply_delta()
(13..16).for_each(|i| mmr.add(int_to_node(i)));
validate_apply_delta(&mmr, &mut partial_mmr);
}
fn validate_apply_delta(mmr: &Mmr, partial: &mut PartialMmr) {
let tracked_leaves = partial
.nodes
.iter()
.filter_map(|(index, _)| if index.is_leaf() { Some(index.sibling()) } else { None })
.collect::<Vec<_>>();
let nodes_before = partial.nodes.clone();
// compute and apply delta
let delta = mmr.get_delta(partial.forest(), mmr.forest()).unwrap();
let nodes_delta = partial.apply(delta).unwrap();
// new peaks were computed correctly
assert_eq!(mmr.peaks(mmr.forest()).unwrap(), partial.peaks());
let mut expected_nodes = nodes_before;
for (key, value) in nodes_delta {
// nodes should not be duplicated
assert!(expected_nodes.insert(key, value).is_none());
}
// new nodes should be a combination of original nodes and delta
assert_eq!(expected_nodes, partial.nodes);
// make sure tracked leaves open to the same proofs as in the underlying MMR
for index in tracked_leaves {
let index_value: u64 = index.into();
let pos = index_value / 2;
let proof1 = partial.open(pos as usize).unwrap().unwrap();
let proof2 = mmr.open(pos as usize, mmr.forest()).unwrap();
assert_eq!(proof1, proof2);
}
}
#[test]
fn test_partial_mmr_inner_nodes_iterator() {
// build the MMR
let mmr: Mmr = LEAVES.into();
let first_peak = mmr.peaks(mmr.forest).unwrap().peaks()[0];
// -- test single tree ----------------------------
// get path and node for position 1
let node1 = mmr.get(1).unwrap();
let proof1 = mmr.open(1, mmr.forest()).unwrap();
// create partial MMR and add authentication path to node at position 1
let mut partial_mmr: PartialMmr = mmr.peaks(mmr.forest()).unwrap().into();
partial_mmr.track(1, node1, &proof1.merkle_path).unwrap();
// empty iterator should have no nodes
assert_eq!(partial_mmr.inner_nodes([].iter().cloned()).next(), None);
// build Merkle store from authentication paths in partial MMR
let mut store: MerkleStore = MerkleStore::new();
store.extend(partial_mmr.inner_nodes([(1, node1)].iter().cloned()));
let index1 = NodeIndex::new(2, 1).unwrap();
let path1 = store.get_path(first_peak, index1).unwrap().path;
assert_eq!(path1, proof1.merkle_path);
// -- test no duplicates --------------------------
// build the partial MMR
let mut partial_mmr: PartialMmr = mmr.peaks(mmr.forest()).unwrap().into();
let node0 = mmr.get(0).unwrap();
let proof0 = mmr.open(0, mmr.forest()).unwrap();
let node2 = mmr.get(2).unwrap();
let proof2 = mmr.open(2, mmr.forest()).unwrap();
partial_mmr.track(0, node0, &proof0.merkle_path).unwrap();
partial_mmr.track(1, node1, &proof1.merkle_path).unwrap();
partial_mmr.track(2, node2, &proof2.merkle_path).unwrap();
// make sure there are no duplicates
let leaves = [(0, node0), (1, node1), (2, node2)];
let mut nodes = BTreeSet::new();
for node in partial_mmr.inner_nodes(leaves.iter().cloned()) {
assert!(nodes.insert(node.value));
}
// and also that the store is still be built correctly
store.extend(partial_mmr.inner_nodes(leaves.iter().cloned()));
let index0 = NodeIndex::new(2, 0).unwrap();
let index1 = NodeIndex::new(2, 1).unwrap();
let index2 = NodeIndex::new(2, 2).unwrap();
let path0 = store.get_path(first_peak, index0).unwrap().path;
let path1 = store.get_path(first_peak, index1).unwrap().path;
let path2 = store.get_path(first_peak, index2).unwrap().path;
assert_eq!(path0, proof0.merkle_path);
assert_eq!(path1, proof1.merkle_path);
assert_eq!(path2, proof2.merkle_path);
// -- test multiple trees -------------------------
// build the partial MMR
let mut partial_mmr: PartialMmr = mmr.peaks(mmr.forest()).unwrap().into();
let node5 = mmr.get(5).unwrap();
let proof5 = mmr.open(5, mmr.forest()).unwrap();
partial_mmr.track(1, node1, &proof1.merkle_path).unwrap();
partial_mmr.track(5, node5, &proof5.merkle_path).unwrap();
// build Merkle store from authentication paths in partial MMR
let mut store: MerkleStore = MerkleStore::new();
store.extend(partial_mmr.inner_nodes([(1, node1), (5, node5)].iter().cloned()));
let index1 = NodeIndex::new(2, 1).unwrap();
let index5 = NodeIndex::new(1, 1).unwrap();
let second_peak = mmr.peaks(mmr.forest).unwrap().peaks()[1];
let path1 = store.get_path(first_peak, index1).unwrap().path;
let path5 = store.get_path(second_peak, index5).unwrap().path;
assert_eq!(path1, proof1.merkle_path);
assert_eq!(path5, proof5.merkle_path);
}
#[test]
fn test_partial_mmr_add_without_track() {
let mut mmr = Mmr::default();
let empty_peaks = MmrPeaks::new(0, vec![]).unwrap();
let mut partial_mmr = PartialMmr::from_peaks(empty_peaks);
for el in (0..256).map(int_to_node) {
mmr.add(el);
partial_mmr.add(el, false);
let mmr_peaks = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(mmr_peaks, partial_mmr.peaks());
assert_eq!(mmr.forest(), partial_mmr.forest());
}
}
#[test]
fn test_partial_mmr_add_with_track() {
let mut mmr = Mmr::default();
let empty_peaks = MmrPeaks::new(0, vec![]).unwrap();
let mut partial_mmr = PartialMmr::from_peaks(empty_peaks);
for i in 0..256 {
let el = int_to_node(i);
mmr.add(el);
partial_mmr.add(el, true);
let mmr_peaks = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(mmr_peaks, partial_mmr.peaks());
assert_eq!(mmr.forest(), partial_mmr.forest());
for pos in 0..i {
let mmr_proof = mmr.open(pos as usize, mmr.forest()).unwrap();
let partialmmr_proof = partial_mmr.open(pos as usize).unwrap().unwrap();
assert_eq!(mmr_proof, partialmmr_proof);
}
}
}
#[test]
fn test_partial_mmr_add_existing_track() {
let mut mmr = Mmr::from((0..7).map(int_to_node));
// derive a partial Mmr from it which tracks authentication path to leaf 5
let mut partial_mmr = PartialMmr::from_peaks(mmr.peaks(mmr.forest()).unwrap());
let path_to_5 = mmr.open(5, mmr.forest()).unwrap().merkle_path;
let leaf_at_5 = mmr.get(5).unwrap();
partial_mmr.track(5, leaf_at_5, &path_to_5).unwrap();
// add a new leaf to both Mmr and partial Mmr
let leaf_at_7 = int_to_node(7);
mmr.add(leaf_at_7);
partial_mmr.add(leaf_at_7, false);
// the openings should be the same
assert_eq!(mmr.open(5, mmr.forest()).unwrap(), partial_mmr.open(5).unwrap().unwrap());
}
}

140
src/merkle/mmr/peaks.rs Normal file
View File

@@ -0,0 +1,140 @@
use super::{
super::{RpoDigest, Vec, ZERO},
Felt, MmrError, MmrProof, Rpo256, Word,
};
// MMR PEAKS
// ================================================================================================
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct MmrPeaks {
/// The number of leaves is used to differentiate MMRs that have the same number of peaks. This
/// happens because the number of peaks goes up-and-down as the structure is used causing
/// existing trees to be merged and new ones to be created. As an example, every time the MMR
/// has a power-of-two number of leaves there is a single peak.
///
/// Every tree in the MMR forest has a distinct power-of-two size, this means only the right-
/// most tree can have an odd number of elements (e.g. `1`). Additionally this means that the
/// bits in `num_leaves` conveniently encode the size of each individual tree.
///
/// Examples:
///
/// - With 5 leaves, the binary `0b101`. The number of set bits is equal the number
/// of peaks, in this case there are 2 peaks. The 0-indexed least-significant position of
/// the bit determines the number of elements of a tree, so the rightmost tree has `2**0`
/// elements and the left most has `2**2`.
/// - With 12 leaves, the binary is `0b1100`, this case also has 2 peaks, the
/// leftmost tree has `2**3=8` elements, and the right most has `2**2=4` elements.
num_leaves: usize,
/// All the peaks of every tree in the MMR forest. The peaks are always ordered by number of
/// leaves, starting from the peak with most children, to the one with least.
///
/// Invariant: The length of `peaks` must be equal to the number of true bits in `num_leaves`.
peaks: Vec<RpoDigest>,
}
impl MmrPeaks {
// CONSTRUCTOR
// --------------------------------------------------------------------------------------------
/// Returns new [MmrPeaks] instantiated from the provided vector of peaks and the number of
/// leaves in the underlying MMR.
///
/// # Errors
/// Returns an error if the number of leaves and the number of peaks are inconsistent.
pub fn new(num_leaves: usize, peaks: Vec<RpoDigest>) -> Result<Self, MmrError> {
if num_leaves.count_ones() as usize != peaks.len() {
return Err(MmrError::InvalidPeaks);
}
Ok(Self { num_leaves, peaks })
}
// ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns a count of leaves in the underlying MMR.
pub fn num_leaves(&self) -> usize {
self.num_leaves
}
/// Returns the number of peaks of the underlying MMR.
pub fn num_peaks(&self) -> usize {
self.peaks.len()
}
/// Returns the list of peaks of the underlying MMR.
pub fn peaks(&self) -> &[RpoDigest] {
&self.peaks
}
/// Converts this [MmrPeaks] into its components: number of leaves and a vector of peaks of
/// the underlying MMR.
pub fn into_parts(self) -> (usize, Vec<RpoDigest>) {
(self.num_leaves, self.peaks)
}
/// Hashes the peaks.
///
/// The procedure will:
/// - Flatten and pad the peaks to a vector of Felts.
/// - Hash the vector of Felts.
pub fn hash_peaks(&self) -> RpoDigest {
Rpo256::hash_elements(&self.flatten_and_pad_peaks())
}
pub fn verify(&self, value: RpoDigest, opening: MmrProof) -> bool {
let root = &self.peaks[opening.peak_index()];
opening.merkle_path.verify(opening.relative_pos() as u64, value, root)
}
/// Flattens and pads the peaks to make hashing inside of the Miden VM easier.
///
/// The procedure will:
/// - Flatten the vector of Words into a vector of Felts.
/// - Pad the peaks with ZERO to an even number of words, this removes the need to handle RPO
/// padding.
/// - Pad the peaks to a minimum length of 16 words, which reduces the constant cost of
/// hashing.
pub fn flatten_and_pad_peaks(&self) -> Vec<Felt> {
let num_peaks = self.peaks.len();
// To achieve the padding rules above we calculate the length of the final vector.
// This is calculated as the number of field elements. Each peak is 4 field elements.
// The length is calculated as follows:
// - If there are less than 16 peaks, the data is padded to 16 peaks and as such requires
// 64 field elements.
// - If there are more than 16 peaks and the number of peaks is odd, the data is padded to
// an even number of peaks and as such requires `(num_peaks + 1) * 4` field elements.
// - If there are more than 16 peaks and the number of peaks is even, the data is not padded
// and as such requires `num_peaks * 4` field elements.
let len = if num_peaks < 16 {
64
} else if num_peaks % 2 == 1 {
(num_peaks + 1) * 4
} else {
num_peaks * 4
};
let mut elements = Vec::with_capacity(len);
elements.extend_from_slice(
&self
.peaks
.as_slice()
.iter()
.map(|digest| digest.into())
.collect::<Vec<Word>>()
.concat(),
);
elements.resize(len, ZERO);
elements
}
}
impl From<MmrPeaks> for Vec<RpoDigest> {
fn from(peaks: MmrPeaks) -> Self {
peaks.peaks
}
}

106
src/merkle/mmr/proof.rs Normal file
View File

@@ -0,0 +1,106 @@
/// The representation of a single Merkle path.
use super::super::MerklePath;
use super::{full::high_bitmask, leaf_to_corresponding_tree};
// MMR PROOF
// ================================================================================================
#[derive(Debug, Clone, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct MmrProof {
/// The state of the MMR when the MmrProof was created.
pub forest: usize,
/// The position of the leaf value on this MmrProof.
pub position: usize,
/// The Merkle opening, starting from the value's sibling up to and excluding the root of the
/// responsible tree.
pub merkle_path: MerklePath,
}
impl MmrProof {
/// Converts the leaf global position into a local position that can be used to verify the
/// merkle_path.
pub fn relative_pos(&self) -> usize {
let tree_bit = leaf_to_corresponding_tree(self.position, self.forest)
.expect("position must be part of the forest");
let forest_before = self.forest & high_bitmask(tree_bit + 1);
self.position - forest_before
}
/// Returns index of the MMR peak against which the Merkle path in this proof can be verified.
pub fn peak_index(&self) -> usize {
let root = leaf_to_corresponding_tree(self.position, self.forest)
.expect("position must be part of the forest");
let smaller_peak_mask = 2_usize.pow(root) as usize - 1;
let num_smaller_peaks = (self.forest & smaller_peak_mask).count_ones();
(self.forest.count_ones() - num_smaller_peaks - 1) as usize
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::{MerklePath, MmrProof};
#[test]
fn test_peak_index() {
// --- single peak forest ---------------------------------------------
let forest = 11;
// the first 4 leaves belong to peak 0
for position in 0..8 {
let proof = make_dummy_proof(forest, position);
assert_eq!(proof.peak_index(), 0);
}
// --- forest with non-consecutive peaks ------------------------------
let forest = 11;
// the first 8 leaves belong to peak 0
for position in 0..8 {
let proof = make_dummy_proof(forest, position);
assert_eq!(proof.peak_index(), 0);
}
// the next 2 leaves belong to peak 1
for position in 8..10 {
let proof = make_dummy_proof(forest, position);
assert_eq!(proof.peak_index(), 1);
}
// the last leaf is the peak 2
let proof = make_dummy_proof(forest, 10);
assert_eq!(proof.peak_index(), 2);
// --- forest with consecutive peaks ----------------------------------
let forest = 7;
// the first 4 leaves belong to peak 0
for position in 0..4 {
let proof = make_dummy_proof(forest, position);
assert_eq!(proof.peak_index(), 0);
}
// the next 2 leaves belong to peak 1
for position in 4..6 {
let proof = make_dummy_proof(forest, position);
assert_eq!(proof.peak_index(), 1);
}
// the last leaf is the peak 2
let proof = make_dummy_proof(forest, 6);
assert_eq!(proof.peak_index(), 2);
}
fn make_dummy_proof(forest: usize, position: usize) -> MmrProof {
MmrProof {
forest,
position,
merkle_path: MerklePath::default(),
}
}
}

878
src/merkle/mmr/tests.rs Normal file
View File

@@ -0,0 +1,878 @@
use super::{
super::{InnerNodeInfo, Rpo256, RpoDigest, Vec},
bit::TrueBitPositionIterator,
full::high_bitmask,
leaf_to_corresponding_tree, nodes_in_forest, Mmr, MmrPeaks, PartialMmr,
};
use crate::{
merkle::{int_to_node, InOrderIndex, MerklePath, MerkleTree, MmrProof, NodeIndex},
Felt, Word,
};
#[test]
fn test_position_equal_or_higher_than_leafs_is_never_contained() {
let empty_forest = 0;
for pos in 1..1024 {
// pos is index, 0 based
// tree is a length counter, 1 based
// so a valid pos is always smaller, not equal, to tree
assert_eq!(leaf_to_corresponding_tree(pos, pos), None);
assert_eq!(leaf_to_corresponding_tree(pos, pos - 1), None);
// and empty forest has no trees, so no position is valid
assert_eq!(leaf_to_corresponding_tree(pos, empty_forest), None);
}
}
#[test]
fn test_position_zero_is_always_contained_by_the_highest_tree() {
for leaves in 1..1024usize {
let tree = leaves.ilog2();
assert_eq!(leaf_to_corresponding_tree(0, leaves), Some(tree));
}
}
#[test]
fn test_leaf_to_corresponding_tree() {
assert_eq!(leaf_to_corresponding_tree(0, 0b0001), Some(0));
assert_eq!(leaf_to_corresponding_tree(0, 0b0010), Some(1));
assert_eq!(leaf_to_corresponding_tree(0, 0b0011), Some(1));
assert_eq!(leaf_to_corresponding_tree(0, 0b1011), Some(3));
// position one is always owned by the left-most tree
assert_eq!(leaf_to_corresponding_tree(1, 0b0010), Some(1));
assert_eq!(leaf_to_corresponding_tree(1, 0b0011), Some(1));
assert_eq!(leaf_to_corresponding_tree(1, 0b1011), Some(3));
// position two starts as its own root, and then it is merged with the left-most tree
assert_eq!(leaf_to_corresponding_tree(2, 0b0011), Some(0));
assert_eq!(leaf_to_corresponding_tree(2, 0b0100), Some(2));
assert_eq!(leaf_to_corresponding_tree(2, 0b1011), Some(3));
// position tree is merged on the left-most tree
assert_eq!(leaf_to_corresponding_tree(3, 0b0011), None);
assert_eq!(leaf_to_corresponding_tree(3, 0b0100), Some(2));
assert_eq!(leaf_to_corresponding_tree(3, 0b1011), Some(3));
assert_eq!(leaf_to_corresponding_tree(4, 0b0101), Some(0));
assert_eq!(leaf_to_corresponding_tree(4, 0b0110), Some(1));
assert_eq!(leaf_to_corresponding_tree(4, 0b0111), Some(1));
assert_eq!(leaf_to_corresponding_tree(4, 0b1000), Some(3));
assert_eq!(leaf_to_corresponding_tree(12, 0b01101), Some(0));
assert_eq!(leaf_to_corresponding_tree(12, 0b01110), Some(1));
assert_eq!(leaf_to_corresponding_tree(12, 0b01111), Some(1));
assert_eq!(leaf_to_corresponding_tree(12, 0b10000), Some(4));
}
#[test]
fn test_high_bitmask() {
assert_eq!(high_bitmask(0), usize::MAX);
assert_eq!(high_bitmask(1), usize::MAX << 1);
assert_eq!(high_bitmask(usize::BITS - 2), 0b11usize.rotate_right(2));
assert_eq!(high_bitmask(usize::BITS - 1), 0b1usize.rotate_right(1));
assert_eq!(high_bitmask(usize::BITS), 0, "overflow should be handled");
}
#[test]
fn test_nodes_in_forest() {
assert_eq!(nodes_in_forest(0b0000), 0);
assert_eq!(nodes_in_forest(0b0001), 1);
assert_eq!(nodes_in_forest(0b0010), 3);
assert_eq!(nodes_in_forest(0b0011), 4);
assert_eq!(nodes_in_forest(0b0100), 7);
assert_eq!(nodes_in_forest(0b0101), 8);
assert_eq!(nodes_in_forest(0b0110), 10);
assert_eq!(nodes_in_forest(0b0111), 11);
assert_eq!(nodes_in_forest(0b1000), 15);
assert_eq!(nodes_in_forest(0b1001), 16);
assert_eq!(nodes_in_forest(0b1010), 18);
assert_eq!(nodes_in_forest(0b1011), 19);
}
#[test]
fn test_nodes_in_forest_single_bit() {
assert_eq!(nodes_in_forest(2usize.pow(0)), 2usize.pow(1) - 1);
assert_eq!(nodes_in_forest(2usize.pow(1)), 2usize.pow(2) - 1);
assert_eq!(nodes_in_forest(2usize.pow(2)), 2usize.pow(3) - 1);
assert_eq!(nodes_in_forest(2usize.pow(3)), 2usize.pow(4) - 1);
for bit in 0..(usize::BITS - 1) {
let size = 2usize.pow(bit + 1) - 1;
assert_eq!(nodes_in_forest(1usize << bit), size);
}
}
const LEAVES: [RpoDigest; 7] = [
int_to_node(0),
int_to_node(1),
int_to_node(2),
int_to_node(3),
int_to_node(4),
int_to_node(5),
int_to_node(6),
];
#[test]
fn test_mmr_simple() {
let mut postorder = vec![
LEAVES[0],
LEAVES[1],
merge(LEAVES[0], LEAVES[1]),
LEAVES[2],
LEAVES[3],
merge(LEAVES[2], LEAVES[3]),
];
postorder.push(merge(postorder[2], postorder[5]));
postorder.push(LEAVES[4]);
postorder.push(LEAVES[5]);
postorder.push(merge(LEAVES[4], LEAVES[5]));
postorder.push(LEAVES[6]);
let mut mmr = Mmr::new();
assert_eq!(mmr.forest(), 0);
assert_eq!(mmr.nodes.len(), 0);
mmr.add(LEAVES[0]);
assert_eq!(mmr.forest(), 1);
assert_eq!(mmr.nodes.len(), 1);
assert_eq!(mmr.nodes.as_slice(), &postorder[0..mmr.nodes.len()]);
let acc = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(acc.num_leaves(), 1);
assert_eq!(acc.peaks(), &[postorder[0]]);
mmr.add(LEAVES[1]);
assert_eq!(mmr.forest(), 2);
assert_eq!(mmr.nodes.len(), 3);
assert_eq!(mmr.nodes.as_slice(), &postorder[0..mmr.nodes.len()]);
let acc = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(acc.num_leaves(), 2);
assert_eq!(acc.peaks(), &[postorder[2]]);
mmr.add(LEAVES[2]);
assert_eq!(mmr.forest(), 3);
assert_eq!(mmr.nodes.len(), 4);
assert_eq!(mmr.nodes.as_slice(), &postorder[0..mmr.nodes.len()]);
let acc = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(acc.num_leaves(), 3);
assert_eq!(acc.peaks(), &[postorder[2], postorder[3]]);
mmr.add(LEAVES[3]);
assert_eq!(mmr.forest(), 4);
assert_eq!(mmr.nodes.len(), 7);
assert_eq!(mmr.nodes.as_slice(), &postorder[0..mmr.nodes.len()]);
let acc = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(acc.num_leaves(), 4);
assert_eq!(acc.peaks(), &[postorder[6]]);
mmr.add(LEAVES[4]);
assert_eq!(mmr.forest(), 5);
assert_eq!(mmr.nodes.len(), 8);
assert_eq!(mmr.nodes.as_slice(), &postorder[0..mmr.nodes.len()]);
let acc = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(acc.num_leaves(), 5);
assert_eq!(acc.peaks(), &[postorder[6], postorder[7]]);
mmr.add(LEAVES[5]);
assert_eq!(mmr.forest(), 6);
assert_eq!(mmr.nodes.len(), 10);
assert_eq!(mmr.nodes.as_slice(), &postorder[0..mmr.nodes.len()]);
let acc = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(acc.num_leaves(), 6);
assert_eq!(acc.peaks(), &[postorder[6], postorder[9]]);
mmr.add(LEAVES[6]);
assert_eq!(mmr.forest(), 7);
assert_eq!(mmr.nodes.len(), 11);
assert_eq!(mmr.nodes.as_slice(), &postorder[0..mmr.nodes.len()]);
let acc = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(acc.num_leaves(), 7);
assert_eq!(acc.peaks(), &[postorder[6], postorder[9], postorder[10]]);
}
#[test]
fn test_mmr_open() {
let mmr: Mmr = LEAVES.into();
let h01 = merge(LEAVES[0], LEAVES[1]);
let h23 = merge(LEAVES[2], LEAVES[3]);
// node at pos 7 is the root
assert!(
mmr.open(7, mmr.forest()).is_err(),
"Element 7 is not in the tree, result should be None"
);
// node at pos 6 is the root
let empty: MerklePath = MerklePath::new(vec![]);
let opening = mmr
.open(6, mmr.forest())
.expect("Element 6 is contained in the tree, expected an opening result.");
assert_eq!(opening.merkle_path, empty);
assert_eq!(opening.forest, mmr.forest);
assert_eq!(opening.position, 6);
assert!(
mmr.peaks(mmr.forest()).unwrap().verify(LEAVES[6], opening),
"MmrProof should be valid for the current accumulator."
);
// nodes 4,5 are depth 1
let root_to_path = MerklePath::new(vec![LEAVES[4]]);
let opening = mmr
.open(5, mmr.forest())
.expect("Element 5 is contained in the tree, expected an opening result.");
assert_eq!(opening.merkle_path, root_to_path);
assert_eq!(opening.forest, mmr.forest);
assert_eq!(opening.position, 5);
assert!(
mmr.peaks(mmr.forest()).unwrap().verify(LEAVES[5], opening),
"MmrProof should be valid for the current accumulator."
);
let root_to_path = MerklePath::new(vec![LEAVES[5]]);
let opening = mmr
.open(4, mmr.forest())
.expect("Element 4 is contained in the tree, expected an opening result.");
assert_eq!(opening.merkle_path, root_to_path);
assert_eq!(opening.forest, mmr.forest);
assert_eq!(opening.position, 4);
assert!(
mmr.peaks(mmr.forest()).unwrap().verify(LEAVES[4], opening),
"MmrProof should be valid for the current accumulator."
);
// nodes 0,1,2,3 are detph 2
let root_to_path = MerklePath::new(vec![LEAVES[2], h01]);
let opening = mmr
.open(3, mmr.forest())
.expect("Element 3 is contained in the tree, expected an opening result.");
assert_eq!(opening.merkle_path, root_to_path);
assert_eq!(opening.forest, mmr.forest);
assert_eq!(opening.position, 3);
assert!(
mmr.peaks(mmr.forest()).unwrap().verify(LEAVES[3], opening),
"MmrProof should be valid for the current accumulator."
);
let root_to_path = MerklePath::new(vec![LEAVES[3], h01]);
let opening = mmr
.open(2, mmr.forest())
.expect("Element 2 is contained in the tree, expected an opening result.");
assert_eq!(opening.merkle_path, root_to_path);
assert_eq!(opening.forest, mmr.forest);
assert_eq!(opening.position, 2);
assert!(
mmr.peaks(mmr.forest()).unwrap().verify(LEAVES[2], opening),
"MmrProof should be valid for the current accumulator."
);
let root_to_path = MerklePath::new(vec![LEAVES[0], h23]);
let opening = mmr
.open(1, mmr.forest())
.expect("Element 1 is contained in the tree, expected an opening result.");
assert_eq!(opening.merkle_path, root_to_path);
assert_eq!(opening.forest, mmr.forest);
assert_eq!(opening.position, 1);
assert!(
mmr.peaks(mmr.forest()).unwrap().verify(LEAVES[1], opening),
"MmrProof should be valid for the current accumulator."
);
let root_to_path = MerklePath::new(vec![LEAVES[1], h23]);
let opening = mmr
.open(0, mmr.forest())
.expect("Element 0 is contained in the tree, expected an opening result.");
assert_eq!(opening.merkle_path, root_to_path);
assert_eq!(opening.forest, mmr.forest);
assert_eq!(opening.position, 0);
assert!(
mmr.peaks(mmr.forest()).unwrap().verify(LEAVES[0], opening),
"MmrProof should be valid for the current accumulator."
);
}
#[test]
fn test_mmr_open_older_version() {
let mmr: Mmr = LEAVES.into();
fn is_even(v: &usize) -> bool {
v & 1 == 0
}
// merkle path of a node is empty if there are no elements to pair with it
for pos in (0..mmr.forest()).filter(is_even) {
let forest = pos + 1;
let proof = mmr.open(pos, forest).unwrap();
assert_eq!(proof.forest, forest);
assert_eq!(proof.merkle_path.nodes(), []);
assert_eq!(proof.position, pos);
}
// openings match that of a merkle tree
let mtree: MerkleTree = LEAVES[..4].try_into().unwrap();
for forest in 4..=LEAVES.len() {
for pos in 0..4 {
let idx = NodeIndex::new(2, pos).unwrap();
let path = mtree.get_path(idx).unwrap();
let proof = mmr.open(pos as usize, forest).unwrap();
assert_eq!(path, proof.merkle_path);
}
}
let mtree: MerkleTree = LEAVES[4..6].try_into().unwrap();
for forest in 6..=LEAVES.len() {
for pos in 0..2 {
let idx = NodeIndex::new(1, pos).unwrap();
let path = mtree.get_path(idx).unwrap();
// account for the bigger tree with 4 elements
let mmr_pos = (pos + 4) as usize;
let proof = mmr.open(mmr_pos, forest).unwrap();
assert_eq!(path, proof.merkle_path);
}
}
}
/// Tests the openings of a simple Mmr with a single tree of depth 8.
#[test]
fn test_mmr_open_eight() {
let leaves = [
int_to_node(0),
int_to_node(1),
int_to_node(2),
int_to_node(3),
int_to_node(4),
int_to_node(5),
int_to_node(6),
int_to_node(7),
];
let mtree: MerkleTree = leaves.as_slice().try_into().unwrap();
let forest = leaves.len();
let mmr: Mmr = leaves.into();
let root = mtree.root();
let position = 0;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path = mtree.get_path(NodeIndex::new(3, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(position as u64, leaves[position]).unwrap(), root);
let position = 1;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path = mtree.get_path(NodeIndex::new(3, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(position as u64, leaves[position]).unwrap(), root);
let position = 2;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path = mtree.get_path(NodeIndex::new(3, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(position as u64, leaves[position]).unwrap(), root);
let position = 3;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path = mtree.get_path(NodeIndex::new(3, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(position as u64, leaves[position]).unwrap(), root);
let position = 4;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path = mtree.get_path(NodeIndex::new(3, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(position as u64, leaves[position]).unwrap(), root);
let position = 5;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path = mtree.get_path(NodeIndex::new(3, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(position as u64, leaves[position]).unwrap(), root);
let position = 6;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path = mtree.get_path(NodeIndex::new(3, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(position as u64, leaves[position]).unwrap(), root);
let position = 7;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path = mtree.get_path(NodeIndex::new(3, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(position as u64, leaves[position]).unwrap(), root);
}
/// Tests the openings of Mmr with a 3 trees of depths 4, 2, and 1.
#[test]
fn test_mmr_open_seven() {
let mtree1: MerkleTree = LEAVES[..4].try_into().unwrap();
let mtree2: MerkleTree = LEAVES[4..6].try_into().unwrap();
let forest = LEAVES.len();
let mmr: Mmr = LEAVES.into();
let position = 0;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path: MerklePath =
mtree1.get_path(NodeIndex::new(2, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(0, LEAVES[0]).unwrap(), mtree1.root());
let position = 1;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path: MerklePath =
mtree1.get_path(NodeIndex::new(2, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(1, LEAVES[1]).unwrap(), mtree1.root());
let position = 2;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path: MerklePath =
mtree1.get_path(NodeIndex::new(2, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(2, LEAVES[2]).unwrap(), mtree1.root());
let position = 3;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path: MerklePath =
mtree1.get_path(NodeIndex::new(2, position as u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(3, LEAVES[3]).unwrap(), mtree1.root());
let position = 4;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path: MerklePath = mtree2.get_path(NodeIndex::new(1, 0u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(0, LEAVES[4]).unwrap(), mtree2.root());
let position = 5;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path: MerklePath = mtree2.get_path(NodeIndex::new(1, 1u64).unwrap()).unwrap();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(1, LEAVES[5]).unwrap(), mtree2.root());
let position = 6;
let proof = mmr.open(position, mmr.forest()).unwrap();
let merkle_path: MerklePath = [].as_ref().into();
assert_eq!(proof, MmrProof { forest, position, merkle_path });
assert_eq!(proof.merkle_path.compute_root(0, LEAVES[6]).unwrap(), LEAVES[6]);
}
#[test]
fn test_mmr_get() {
let mmr: Mmr = LEAVES.into();
assert_eq!(mmr.get(0).unwrap(), LEAVES[0], "value at pos 0 must correspond");
assert_eq!(mmr.get(1).unwrap(), LEAVES[1], "value at pos 1 must correspond");
assert_eq!(mmr.get(2).unwrap(), LEAVES[2], "value at pos 2 must correspond");
assert_eq!(mmr.get(3).unwrap(), LEAVES[3], "value at pos 3 must correspond");
assert_eq!(mmr.get(4).unwrap(), LEAVES[4], "value at pos 4 must correspond");
assert_eq!(mmr.get(5).unwrap(), LEAVES[5], "value at pos 5 must correspond");
assert_eq!(mmr.get(6).unwrap(), LEAVES[6], "value at pos 6 must correspond");
assert!(mmr.get(7).is_err());
}
#[test]
fn test_mmr_invariants() {
let mut mmr = Mmr::new();
for v in 1..=1028 {
mmr.add(int_to_node(v));
let accumulator = mmr.peaks(mmr.forest()).unwrap();
assert_eq!(v as usize, mmr.forest(), "MMR leaf count must increase by one on every add");
assert_eq!(
v as usize,
accumulator.num_leaves(),
"MMR and its accumulator must match leaves count"
);
assert_eq!(
accumulator.num_leaves().count_ones() as usize,
accumulator.peaks().len(),
"bits on leaves must match the number of peaks"
);
let expected_nodes: usize = TrueBitPositionIterator::new(mmr.forest())
.map(|bit_pos| nodes_in_forest(1 << bit_pos))
.sum();
assert_eq!(
expected_nodes,
mmr.nodes.len(),
"the sum of every tree size must be equal to the number of nodes in the MMR (forest: {:b})",
mmr.forest(),
);
}
}
#[test]
fn test_bit_position_iterator() {
assert_eq!(TrueBitPositionIterator::new(0).count(), 0);
assert_eq!(TrueBitPositionIterator::new(0).rev().count(), 0);
assert_eq!(TrueBitPositionIterator::new(1).collect::<Vec<u32>>(), vec![0]);
assert_eq!(TrueBitPositionIterator::new(1).rev().collect::<Vec<u32>>(), vec![0],);
assert_eq!(TrueBitPositionIterator::new(2).collect::<Vec<u32>>(), vec![1]);
assert_eq!(TrueBitPositionIterator::new(2).rev().collect::<Vec<u32>>(), vec![1],);
assert_eq!(TrueBitPositionIterator::new(3).collect::<Vec<u32>>(), vec![0, 1],);
assert_eq!(TrueBitPositionIterator::new(3).rev().collect::<Vec<u32>>(), vec![1, 0],);
assert_eq!(
TrueBitPositionIterator::new(0b11010101).collect::<Vec<u32>>(),
vec![0, 2, 4, 6, 7],
);
assert_eq!(
TrueBitPositionIterator::new(0b11010101).rev().collect::<Vec<u32>>(),
vec![7, 6, 4, 2, 0],
);
}
#[test]
fn test_mmr_inner_nodes() {
let mmr: Mmr = LEAVES.into();
let nodes: Vec<InnerNodeInfo> = mmr.inner_nodes().collect();
let h01 = Rpo256::merge(&[LEAVES[0], LEAVES[1]]);
let h23 = Rpo256::merge(&[LEAVES[2], LEAVES[3]]);
let h0123 = Rpo256::merge(&[h01, h23]);
let h45 = Rpo256::merge(&[LEAVES[4], LEAVES[5]]);
let postorder = vec![
InnerNodeInfo {
value: h01,
left: LEAVES[0],
right: LEAVES[1],
},
InnerNodeInfo {
value: h23,
left: LEAVES[2],
right: LEAVES[3],
},
InnerNodeInfo { value: h0123, left: h01, right: h23 },
InnerNodeInfo {
value: h45,
left: LEAVES[4],
right: LEAVES[5],
},
];
assert_eq!(postorder, nodes);
}
#[test]
fn test_mmr_peaks() {
let mmr: Mmr = LEAVES.into();
let forest = 0b0001;
let acc = mmr.peaks(forest).unwrap();
assert_eq!(acc.num_leaves(), forest);
assert_eq!(acc.peaks(), &[mmr.nodes[0]]);
let forest = 0b0010;
let acc = mmr.peaks(forest).unwrap();
assert_eq!(acc.num_leaves(), forest);
assert_eq!(acc.peaks(), &[mmr.nodes[2]]);
let forest = 0b0011;
let acc = mmr.peaks(forest).unwrap();
assert_eq!(acc.num_leaves(), forest);
assert_eq!(acc.peaks(), &[mmr.nodes[2], mmr.nodes[3]]);
let forest = 0b0100;
let acc = mmr.peaks(forest).unwrap();
assert_eq!(acc.num_leaves(), forest);
assert_eq!(acc.peaks(), &[mmr.nodes[6]]);
let forest = 0b0101;
let acc = mmr.peaks(forest).unwrap();
assert_eq!(acc.num_leaves(), forest);
assert_eq!(acc.peaks(), &[mmr.nodes[6], mmr.nodes[7]]);
let forest = 0b0110;
let acc = mmr.peaks(forest).unwrap();
assert_eq!(acc.num_leaves(), forest);
assert_eq!(acc.peaks(), &[mmr.nodes[6], mmr.nodes[9]]);
let forest = 0b0111;
let acc = mmr.peaks(forest).unwrap();
assert_eq!(acc.num_leaves(), forest);
assert_eq!(acc.peaks(), &[mmr.nodes[6], mmr.nodes[9], mmr.nodes[10]]);
}
#[test]
fn test_mmr_hash_peaks() {
let mmr: Mmr = LEAVES.into();
let peaks = mmr.peaks(mmr.forest()).unwrap();
let first_peak = Rpo256::merge(&[
Rpo256::merge(&[LEAVES[0], LEAVES[1]]),
Rpo256::merge(&[LEAVES[2], LEAVES[3]]),
]);
let second_peak = Rpo256::merge(&[LEAVES[4], LEAVES[5]]);
let third_peak = LEAVES[6];
// minimum length is 16
let mut expected_peaks = [first_peak, second_peak, third_peak].to_vec();
expected_peaks.resize(16, RpoDigest::default());
assert_eq!(peaks.hash_peaks(), Rpo256::hash_elements(&digests_to_elements(&expected_peaks)));
}
#[test]
fn test_mmr_peaks_hash_less_than_16() {
let mut peaks = Vec::new();
for i in 0..16 {
peaks.push(int_to_node(i));
let num_leaves = (1 << peaks.len()) - 1;
let accumulator = MmrPeaks::new(num_leaves, peaks.clone()).unwrap();
// minimum length is 16
let mut expected_peaks = peaks.clone();
expected_peaks.resize(16, RpoDigest::default());
assert_eq!(
accumulator.hash_peaks(),
Rpo256::hash_elements(&digests_to_elements(&expected_peaks))
);
}
}
#[test]
fn test_mmr_peaks_hash_odd() {
let peaks: Vec<_> = (0..=17).map(int_to_node).collect();
let num_leaves = (1 << peaks.len()) - 1;
let accumulator = MmrPeaks::new(num_leaves, peaks.clone()).unwrap();
// odd length bigger than 16 is padded to the next even number
let mut expected_peaks = peaks;
expected_peaks.resize(18, RpoDigest::default());
assert_eq!(
accumulator.hash_peaks(),
Rpo256::hash_elements(&digests_to_elements(&expected_peaks))
);
}
#[test]
fn test_mmr_delta() {
let mmr: Mmr = LEAVES.into();
let acc = mmr.peaks(mmr.forest()).unwrap();
// original_forest can't have more elements
assert!(
mmr.get_delta(LEAVES.len() + 1, mmr.forest()).is_err(),
"Can not provide updates for a newer Mmr"
);
// if the number of elements is the same there is no change
assert!(
mmr.get_delta(LEAVES.len(), mmr.forest()).unwrap().data.is_empty(),
"There are no updates for the same Mmr version"
);
// missing the last element added, which is itself a tree peak
assert_eq!(mmr.get_delta(6, mmr.forest()).unwrap().data, vec![acc.peaks()[2]], "one peak");
// missing the sibling to complete the tree of depth 2, and the last element
assert_eq!(
mmr.get_delta(5, mmr.forest()).unwrap().data,
vec![LEAVES[5], acc.peaks()[2]],
"one sibling, one peak"
);
// missing the whole last two trees, only send the peaks
assert_eq!(
mmr.get_delta(4, mmr.forest()).unwrap().data,
vec![acc.peaks()[1], acc.peaks()[2]],
"two peaks"
);
// missing the sibling to complete the first tree, and the two last trees
assert_eq!(
mmr.get_delta(3, mmr.forest()).unwrap().data,
vec![LEAVES[3], acc.peaks()[1], acc.peaks()[2]],
"one sibling, two peaks"
);
// missing half of the first tree, only send the computed element (not the leaves), and the new
// peaks
assert_eq!(
mmr.get_delta(2, mmr.forest()).unwrap().data,
vec![mmr.nodes[5], acc.peaks()[1], acc.peaks()[2]],
"one sibling, two peaks"
);
assert_eq!(
mmr.get_delta(1, mmr.forest()).unwrap().data,
vec![LEAVES[1], mmr.nodes[5], acc.peaks()[1], acc.peaks()[2]],
"one sibling, two peaks"
);
assert_eq!(&mmr.get_delta(0, mmr.forest()).unwrap().data, acc.peaks(), "all peaks");
}
#[test]
fn test_mmr_delta_old_forest() {
let mmr: Mmr = LEAVES.into();
// from_forest must be smaller-or-equal to to_forest
for version in 1..=mmr.forest() {
assert!(mmr.get_delta(version + 1, version).is_err());
}
// when from_forest and to_forest are equal, there are no updates
for version in 1..=mmr.forest() {
let delta = mmr.get_delta(version, version).unwrap();
assert!(delta.data.is_empty());
assert_eq!(delta.forest, version);
}
// test update which merges the odd peak to the right
for count in 0..(mmr.forest() / 2) {
// *2 because every iteration tests a pair
// +1 because the Mmr is 1-indexed
let from_forest = (count * 2) + 1;
let to_forest = (count * 2) + 2;
let delta = mmr.get_delta(from_forest, to_forest).unwrap();
// *2 because every iteration tests a pair
// +1 because sibling is the odd element
let sibling = (count * 2) + 1;
assert_eq!(delta.data, [LEAVES[sibling]]);
assert_eq!(delta.forest, to_forest);
}
let version = 4;
let delta = mmr.get_delta(1, version).unwrap();
assert_eq!(delta.data, [mmr.nodes[1], mmr.nodes[5]]);
assert_eq!(delta.forest, version);
let version = 5;
let delta = mmr.get_delta(1, version).unwrap();
assert_eq!(delta.data, [mmr.nodes[1], mmr.nodes[5], mmr.nodes[7]]);
assert_eq!(delta.forest, version);
}
#[test]
fn test_partial_mmr_simple() {
let mmr: Mmr = LEAVES.into();
let peaks = mmr.peaks(mmr.forest()).unwrap();
let mut partial: PartialMmr = peaks.clone().into();
// check initial state of the partial mmr
assert_eq!(partial.peaks(), peaks);
assert_eq!(partial.forest(), peaks.num_leaves());
assert_eq!(partial.forest(), LEAVES.len());
assert_eq!(partial.peaks().num_peaks(), 3);
assert_eq!(partial.nodes.len(), 0);
// check state after adding tracking one element
let proof1 = mmr.open(0, mmr.forest()).unwrap();
let el1 = mmr.get(proof1.position).unwrap();
partial.track(proof1.position, el1, &proof1.merkle_path).unwrap();
// check the number of nodes increased by the number of nodes in the proof
assert_eq!(partial.nodes.len(), proof1.merkle_path.len());
// check the values match
let idx = InOrderIndex::from_leaf_pos(proof1.position);
assert_eq!(partial.nodes[&idx.sibling()], proof1.merkle_path[0]);
let idx = idx.parent();
assert_eq!(partial.nodes[&idx.sibling()], proof1.merkle_path[1]);
let proof2 = mmr.open(1, mmr.forest()).unwrap();
let el2 = mmr.get(proof2.position).unwrap();
partial.track(proof2.position, el2, &proof2.merkle_path).unwrap();
// check the number of nodes increased by a single element (the one that is not shared)
assert_eq!(partial.nodes.len(), 3);
// check the values match
let idx = InOrderIndex::from_leaf_pos(proof2.position);
assert_eq!(partial.nodes[&idx.sibling()], proof2.merkle_path[0]);
let idx = idx.parent();
assert_eq!(partial.nodes[&idx.sibling()], proof2.merkle_path[1]);
}
#[test]
fn test_partial_mmr_update_single() {
let mut full = Mmr::new();
let zero = int_to_node(0);
full.add(zero);
let mut partial: PartialMmr = full.peaks(full.forest()).unwrap().into();
let proof = full.open(0, full.forest()).unwrap();
partial.track(proof.position, zero, &proof.merkle_path).unwrap();
for i in 1..100 {
let node = int_to_node(i);
full.add(node);
let delta = full.get_delta(partial.forest(), full.forest()).unwrap();
partial.apply(delta).unwrap();
assert_eq!(partial.forest(), full.forest());
assert_eq!(partial.peaks(), full.peaks(full.forest()).unwrap());
let proof1 = full.open(i as usize, full.forest()).unwrap();
partial.track(proof1.position, node, &proof1.merkle_path).unwrap();
let proof2 = partial.open(proof1.position).unwrap().unwrap();
assert_eq!(proof1.merkle_path, proof2.merkle_path);
}
}
#[test]
fn test_mmr_add_invalid_odd_leaf() {
let mmr: Mmr = LEAVES.into();
let acc = mmr.peaks(mmr.forest()).unwrap();
let mut partial: PartialMmr = acc.clone().into();
let empty = MerklePath::new(Vec::new());
// None of the other leaves should work
for node in LEAVES.iter().cloned().rev().skip(1) {
let result = partial.track(LEAVES.len() - 1, node, &empty);
assert!(result.is_err());
}
let result = partial.track(LEAVES.len() - 1, LEAVES[6], &empty);
assert!(result.is_ok());
}
mod property_tests {
use super::leaf_to_corresponding_tree;
use proptest::prelude::*;
proptest! {
#[test]
fn test_last_position_is_always_contained_in_the_last_tree(leaves in any::<usize>().prop_filter("cant have an empty tree", |v| *v != 0)) {
let last_pos = leaves - 1;
let lowest_bit = leaves.trailing_zeros();
assert_eq!(
leaf_to_corresponding_tree(last_pos, leaves),
Some(lowest_bit),
);
}
}
proptest! {
#[test]
fn test_contained_tree_is_always_power_of_two((leaves, pos) in any::<usize>().prop_flat_map(|v| (Just(v), 0..v))) {
let tree_bit = leaf_to_corresponding_tree(pos, leaves).expect("pos is smaller than leaves, there should always be a corresponding tree");
let mask = 1usize << tree_bit;
assert!(tree_bit < usize::BITS, "the result must be a bit in usize");
assert!(mask & leaves != 0, "the result should be a tree in leaves");
}
}
}
// HELPER FUNCTIONS
// ================================================================================================
fn digests_to_elements(digests: &[RpoDigest]) -> Vec<Felt> {
digests.iter().flat_map(Word::from).collect()
}
// short hand for the rpo hash, used to make test code more concise and easy to read
fn merge(l: RpoDigest, r: RpoDigest) -> RpoDigest {
Rpo256::merge(&[l, r])
}

View File

@@ -1,71 +1,61 @@
//! Data structures related to Merkle trees based on RPO256 hash function.
use super::{
hash::rpo::{Rpo256, RpoDigest},
utils::collections::{vec, BTreeMap, Vec},
Felt, StarkField, Word, ZERO,
utils::collections::{vec, BTreeMap, BTreeSet, KvMap, RecordingMap, Vec},
Felt, Word, EMPTY_WORD, ZERO,
};
use core::fmt;
// REEXPORTS
// ================================================================================================
mod empty_roots;
pub use empty_roots::EmptySubtreeRoots;
mod index;
pub use index::NodeIndex;
mod merkle_tree;
pub use merkle_tree::MerkleTree;
pub use merkle_tree::{path_to_text, tree_to_text, MerkleTree};
mod path;
pub use path::MerklePath;
pub use path::{MerklePath, RootPath, ValuePath};
mod path_set;
pub use path_set::MerklePathSet;
mod smt;
pub use smt::{
LeafIndex, SimpleSmt, Smt, SmtLeaf, SmtLeafError, SmtProof, SmtProofError, SMT_DEPTH,
SMT_MAX_DEPTH, SMT_MIN_DEPTH,
};
mod simple_smt;
pub use simple_smt::SimpleSmt;
mod mmr;
pub use mmr::{InOrderIndex, Mmr, MmrDelta, MmrError, MmrPeaks, MmrProof, PartialMmr};
// ERRORS
// ================================================================================================
mod store;
pub use store::{DefaultMerkleStore, MerkleStore, RecordingMerkleStore, StoreNode};
#[derive(Clone, Debug)]
pub enum MerkleError {
DepthTooSmall(u8),
DepthTooBig(u64),
NumLeavesNotPowerOfTwo(usize),
InvalidIndex(NodeIndex),
InvalidDepth { expected: u8, provided: u8 },
InvalidPath(MerklePath),
InvalidEntriesCount(usize, usize),
NodeNotInSet(u64),
}
mod node;
pub use node::InnerNodeInfo;
impl fmt::Display for MerkleError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
use MerkleError::*;
match self {
DepthTooSmall(depth) => write!(f, "the provided depth {depth} is too small"),
DepthTooBig(depth) => write!(f, "the provided depth {depth} is too big"),
NumLeavesNotPowerOfTwo(leaves) => {
write!(f, "the leaves count {leaves} is not a power of 2")
}
InvalidIndex(index) => write!(
f,
"the index value {} is not valid for the depth {}", index.value(), index.depth()
),
InvalidDepth { expected, provided } => write!(
f,
"the provided depth {provided} is not valid for {expected}"
),
InvalidPath(_path) => write!(f, "the provided path is not valid"),
InvalidEntriesCount(max, provided) => write!(f, "the provided number of entries is {provided}, but the maximum for the given depth is {max}"),
NodeNotInSet(index) => write!(f, "the node indexed by {index} is not in the set"),
}
}
}
mod partial_mt;
pub use partial_mt::PartialMerkleTree;
#[cfg(feature = "std")]
impl std::error::Error for MerkleError {}
mod error;
pub use error::MerkleError;
// HELPER FUNCTIONS
// ================================================================================================
#[cfg(test)]
const fn int_to_node(value: u64) -> Word {
const fn int_to_node(value: u64) -> RpoDigest {
RpoDigest::new([Felt::new(value), ZERO, ZERO, ZERO])
}
#[cfg(test)]
const fn int_to_leaf(value: u64) -> Word {
[Felt::new(value), ZERO, ZERO, ZERO]
}
#[cfg(test)]
fn digests_to_words(digests: &[RpoDigest]) -> Vec<Word> {
digests.iter().map(|d| d.into()).collect()
}

10
src/merkle/node.rs Normal file
View File

@@ -0,0 +1,10 @@
use super::RpoDigest;
/// Representation of a node with two children used for iterating over containers.
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct InnerNodeInfo {
pub value: RpoDigest,
pub left: RpoDigest,
pub right: RpoDigest,
}

View File

@@ -0,0 +1,467 @@
use super::{
BTreeMap, BTreeSet, InnerNodeInfo, MerkleError, MerklePath, NodeIndex, Rpo256, RpoDigest,
ValuePath, Vec, Word, EMPTY_WORD,
};
use crate::utils::{
format, string::String, vec, word_to_hex, ByteReader, ByteWriter, Deserializable,
DeserializationError, Serializable,
};
use core::fmt;
#[cfg(test)]
mod tests;
// CONSTANTS
// ================================================================================================
/// Index of the root node.
const ROOT_INDEX: NodeIndex = NodeIndex::root();
/// An RpoDigest consisting of 4 ZERO elements.
const EMPTY_DIGEST: RpoDigest = RpoDigest::new(EMPTY_WORD);
// PARTIAL MERKLE TREE
// ================================================================================================
/// A partial Merkle tree with NodeIndex keys and 4-element RpoDigest leaf values. Partial Merkle
/// Tree allows to create Merkle Tree by providing Merkle paths of different lengths.
///
/// The root of the tree is recomputed on each new leaf update.
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct PartialMerkleTree {
max_depth: u8,
nodes: BTreeMap<NodeIndex, RpoDigest>,
leaves: BTreeSet<NodeIndex>,
}
impl Default for PartialMerkleTree {
fn default() -> Self {
Self::new()
}
}
impl PartialMerkleTree {
// CONSTANTS
// --------------------------------------------------------------------------------------------
/// Minimum supported depth.
pub const MIN_DEPTH: u8 = 1;
/// Maximum supported depth.
pub const MAX_DEPTH: u8 = 64;
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Returns a new empty [PartialMerkleTree].
pub fn new() -> Self {
PartialMerkleTree {
max_depth: 0,
nodes: BTreeMap::new(),
leaves: BTreeSet::new(),
}
}
/// Appends the provided paths iterator into the set.
///
/// Analogous to [Self::add_path].
pub fn with_paths<I>(paths: I) -> Result<Self, MerkleError>
where
I: IntoIterator<Item = (u64, RpoDigest, MerklePath)>,
{
// create an empty tree
let tree = PartialMerkleTree::new();
paths.into_iter().try_fold(tree, |mut tree, (index, value, path)| {
tree.add_path(index, value, path)?;
Ok(tree)
})
}
/// Returns a new [PartialMerkleTree] instantiated with leaves map as specified by the provided
/// entries.
///
/// # Errors
/// Returns an error if:
/// - If the depth is 0 or is greater than 64.
/// - The number of entries exceeds the maximum tree capacity, that is 2^{depth}.
/// - The provided entries contain an insufficient set of nodes.
pub fn with_leaves<R, I>(entries: R) -> Result<Self, MerkleError>
where
R: IntoIterator<IntoIter = I>,
I: Iterator<Item = (NodeIndex, RpoDigest)> + ExactSizeIterator,
{
let mut layers: BTreeMap<u8, Vec<u64>> = BTreeMap::new();
let mut leaves = BTreeSet::new();
let mut nodes = BTreeMap::new();
// add data to the leaves and nodes maps and also fill layers map, where the key is the
// depth of the node and value is its index.
for (node_index, hash) in entries.into_iter() {
leaves.insert(node_index);
nodes.insert(node_index, hash);
layers
.entry(node_index.depth())
.and_modify(|layer_vec| layer_vec.push(node_index.value()))
.or_insert(vec![node_index.value()]);
}
// check if the number of leaves can be accommodated by the tree's depth; we use a min
// depth of 63 because we consider passing in a vector of size 2^64 infeasible.
let max = 2usize.pow(63);
if layers.len() > max {
return Err(MerkleError::InvalidNumEntries(max));
}
// Get maximum depth
let max_depth = *layers.keys().next_back().unwrap_or(&0);
// fill layers without nodes with empty vector
for depth in 0..max_depth {
layers.entry(depth).or_default();
}
let mut layer_iter = layers.into_values().rev();
let mut parent_layer = layer_iter.next().unwrap();
let mut current_layer;
for depth in (1..max_depth + 1).rev() {
// set current_layer = parent_layer and parent_layer = layer_iter.next()
current_layer = layer_iter.next().unwrap();
core::mem::swap(&mut current_layer, &mut parent_layer);
for index_value in current_layer {
// get the parent node index
let parent_node = NodeIndex::new(depth - 1, index_value / 2)?;
// Check if the parent hash was already calculated. In about half of the cases, we
// don't need to do anything.
if !parent_layer.contains(&parent_node.value()) {
// create current node index
let index = NodeIndex::new(depth, index_value)?;
// get hash of the current node
let node = nodes.get(&index).ok_or(MerkleError::NodeNotInSet(index))?;
// get hash of the sibling node
let sibling = nodes
.get(&index.sibling())
.ok_or(MerkleError::NodeNotInSet(index.sibling()))?;
// get parent hash
let parent = Rpo256::merge(&index.build_node(*node, *sibling));
// add index value of the calculated node to the parents layer
parent_layer.push(parent_node.value());
// add index and hash to the nodes map
nodes.insert(parent_node, parent);
}
}
}
Ok(PartialMerkleTree { max_depth, nodes, leaves })
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the root of this Merkle tree.
pub fn root(&self) -> RpoDigest {
self.nodes.get(&ROOT_INDEX).cloned().unwrap_or(EMPTY_DIGEST)
}
/// Returns the depth of this Merkle tree.
pub fn max_depth(&self) -> u8 {
self.max_depth
}
/// Returns a node at the specified NodeIndex.
///
/// # Errors
/// Returns an error if the specified NodeIndex is not contained in the nodes map.
pub fn get_node(&self, index: NodeIndex) -> Result<RpoDigest, MerkleError> {
self.nodes.get(&index).ok_or(MerkleError::NodeNotInSet(index)).copied()
}
/// Returns true if provided index contains in the leaves set, false otherwise.
pub fn is_leaf(&self, index: NodeIndex) -> bool {
self.leaves.contains(&index)
}
/// Returns a vector of paths from every leaf to the root.
pub fn to_paths(&self) -> Vec<(NodeIndex, ValuePath)> {
let mut paths = Vec::new();
self.leaves.iter().for_each(|&leaf| {
paths.push((
leaf,
ValuePath {
value: self.get_node(leaf).expect("Failed to get leaf node"),
path: self.get_path(leaf).expect("Failed to get path"),
},
));
});
paths
}
/// Returns a Merkle path from the node at the specified index to the root.
///
/// The node itself is not included in the path.
///
/// # Errors
/// Returns an error if:
/// - the specified index has depth set to 0 or the depth is greater than the depth of this
/// Merkle tree.
/// - the specified index is not contained in the nodes map.
pub fn get_path(&self, mut index: NodeIndex) -> Result<MerklePath, MerkleError> {
if index.is_root() {
return Err(MerkleError::DepthTooSmall(index.depth()));
} else if index.depth() > self.max_depth() {
return Err(MerkleError::DepthTooBig(index.depth() as u64));
}
if !self.nodes.contains_key(&index) {
return Err(MerkleError::NodeNotInSet(index));
}
let mut path = Vec::new();
for _ in 0..index.depth() {
let sibling_index = index.sibling();
index.move_up();
let sibling =
self.nodes.get(&sibling_index).cloned().expect("Sibling node not in the map");
path.push(sibling);
}
Ok(MerklePath::new(path))
}
// ITERATORS
// --------------------------------------------------------------------------------------------
/// Returns an iterator over the leaves of this [PartialMerkleTree].
pub fn leaves(&self) -> impl Iterator<Item = (NodeIndex, RpoDigest)> + '_ {
self.leaves.iter().map(|&leaf| {
(
leaf,
self.get_node(leaf)
.unwrap_or_else(|_| panic!("Leaf with {leaf} is not in the nodes map")),
)
})
}
/// Returns an iterator over the inner nodes of this Merkle tree.
pub fn inner_nodes(&self) -> impl Iterator<Item = InnerNodeInfo> + '_ {
let inner_nodes = self.nodes.iter().filter(|(index, _)| !self.leaves.contains(index));
inner_nodes.map(|(index, digest)| {
let left_hash =
self.nodes.get(&index.left_child()).expect("Failed to get left child hash");
let right_hash =
self.nodes.get(&index.right_child()).expect("Failed to get right child hash");
InnerNodeInfo {
value: *digest,
left: *left_hash,
right: *right_hash,
}
})
}
// STATE MUTATORS
// --------------------------------------------------------------------------------------------
/// Adds the nodes of the specified Merkle path to this [PartialMerkleTree]. The `index_value`
/// and `value` parameters specify the leaf node at which the path starts.
///
/// # Errors
/// Returns an error if:
/// - The depth of the specified node_index is greater than 64 or smaller than 1.
/// - The specified path is not consistent with other paths in the set (i.e., resolves to a
/// different root).
pub fn add_path(
&mut self,
index_value: u64,
value: RpoDigest,
path: MerklePath,
) -> Result<(), MerkleError> {
let index_value = NodeIndex::new(path.len() as u8, index_value)?;
Self::check_depth(index_value.depth())?;
self.update_depth(index_value.depth());
// add provided node and its sibling to the leaves set
self.leaves.insert(index_value);
let sibling_node_index = index_value.sibling();
self.leaves.insert(sibling_node_index);
// add provided node and its sibling to the nodes map
self.nodes.insert(index_value, value);
self.nodes.insert(sibling_node_index, path[0]);
// traverse to the root, updating the nodes
let mut index_value = index_value;
let node = Rpo256::merge(&index_value.build_node(value, path[0]));
let root = path.iter().skip(1).copied().fold(node, |node, hash| {
index_value.move_up();
// insert calculated node to the nodes map
self.nodes.insert(index_value, node);
// if the calculated node was a leaf, remove it from leaves set.
self.leaves.remove(&index_value);
let sibling_node = index_value.sibling();
// Insert node from Merkle path to the nodes map. This sibling node becomes a leaf only
// if it is a new node (it wasn't in nodes map).
// Node can be in 3 states: internal node, leaf of the tree and not a tree node at all.
// - Internal node can only stay in this state -- addition of a new path can't make it
// a leaf or remove it from the tree.
// - Leaf node can stay in the same state (remain a leaf) or can become an internal
// node. In the first case we don't need to do anything, and the second case is handled
// by the call of `self.leaves.remove(&index_value);`
// - New node can be a calculated node or a "sibling" node from a Merkle Path:
// --- Calculated node, obviously, never can be a leaf.
// --- Sibling node can be only a leaf, because otherwise it is not a new node.
if self.nodes.insert(sibling_node, hash).is_none() {
self.leaves.insert(sibling_node);
}
Rpo256::merge(&index_value.build_node(node, hash))
});
// if the path set is empty (the root is all ZEROs), set the root to the root of the added
// path; otherwise, the root of the added path must be identical to the current root
if self.root() == EMPTY_DIGEST {
self.nodes.insert(ROOT_INDEX, root);
} else if self.root() != root {
return Err(MerkleError::ConflictingRoots([self.root(), root].to_vec()));
}
Ok(())
}
/// Updates value of the leaf at the specified index returning the old leaf value.
/// By default the specified index is assumed to belong to the deepest layer. If the considered
/// node does not belong to the tree, the first node on the way to the root will be changed.
///
/// By default the specified index is assumed to belong to the deepest layer. If the considered
/// node does not belong to the tree, the first node on the way to the root will be changed.
///
/// This also recomputes all hashes between the leaf and the root, updating the root itself.
///
/// # Errors
/// Returns an error if:
/// - The specified index is greater than the maximum number of nodes on the deepest layer.
pub fn update_leaf(&mut self, index: u64, value: Word) -> Result<RpoDigest, MerkleError> {
let mut node_index = NodeIndex::new(self.max_depth(), index)?;
// proceed to the leaf
for _ in 0..node_index.depth() {
if !self.leaves.contains(&node_index) {
node_index.move_up();
}
}
// add node value to the nodes Map
let old_value = self
.nodes
.insert(node_index, value.into())
.ok_or(MerkleError::NodeNotInSet(node_index))?;
// if the old value and new value are the same, there is nothing to update
if value == *old_value {
return Ok(old_value);
}
let mut value = value.into();
for _ in 0..node_index.depth() {
let sibling = self.nodes.get(&node_index.sibling()).expect("sibling should exist");
value = Rpo256::merge(&node_index.build_node(value, *sibling));
node_index.move_up();
self.nodes.insert(node_index, value);
}
Ok(old_value)
}
// UTILITY FUNCTIONS
// --------------------------------------------------------------------------------------------
/// Utility to visualize a [PartialMerkleTree] in text.
pub fn print(&self) -> Result<String, fmt::Error> {
let indent = " ";
let mut s = String::new();
s.push_str("root: ");
s.push_str(&word_to_hex(&self.root())?);
s.push('\n');
for d in 1..=self.max_depth() {
let entries = 2u64.pow(d.into());
for i in 0..entries {
let index = NodeIndex::new(d, i).expect("The index must always be valid");
let node = self.get_node(index);
let node = match node {
Err(_) => continue,
Ok(node) => node,
};
for _ in 0..d {
s.push_str(indent);
}
s.push_str(&format!("({}, {}): ", index.depth(), index.value()));
s.push_str(&word_to_hex(&node)?);
s.push('\n');
}
}
Ok(s)
}
// HELPER METHODS
// --------------------------------------------------------------------------------------------
/// Updates depth value with the maximum of current and provided depth.
fn update_depth(&mut self, new_depth: u8) {
self.max_depth = new_depth.max(self.max_depth);
}
/// Returns an error if the depth is 0 or is greater than 64.
fn check_depth(depth: u8) -> Result<(), MerkleError> {
// validate the range of the depth.
if depth < Self::MIN_DEPTH {
return Err(MerkleError::DepthTooSmall(depth));
} else if Self::MAX_DEPTH < depth {
return Err(MerkleError::DepthTooBig(depth as u64));
}
Ok(())
}
}
// SERIALIZATION
// ================================================================================================
impl Serializable for PartialMerkleTree {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
// write leaf nodes
target.write_u64(self.leaves.len() as u64);
for leaf_index in self.leaves.iter() {
leaf_index.write_into(target);
self.get_node(*leaf_index).expect("Leaf hash not found").write_into(target);
}
}
}
impl Deserializable for PartialMerkleTree {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let leaves_len = source.read_u64()? as usize;
let mut leaf_nodes = Vec::with_capacity(leaves_len);
// add leaf nodes to the vector
for _ in 0..leaves_len {
let index = NodeIndex::read_from(source)?;
let hash = RpoDigest::read_from(source)?;
leaf_nodes.push((index, hash));
}
let pmt = PartialMerkleTree::with_leaves(leaf_nodes).map_err(|_| {
DeserializationError::InvalidValue("Invalid data for PartialMerkleTree creation".into())
})?;
Ok(pmt)
}
}

View File

@@ -0,0 +1,463 @@
use super::{
super::{
digests_to_words, int_to_node, BTreeMap, DefaultMerkleStore as MerkleStore, MerkleTree,
NodeIndex, PartialMerkleTree,
},
Deserializable, InnerNodeInfo, RpoDigest, Serializable, ValuePath, Vec,
};
// TEST DATA
// ================================================================================================
const NODE10: NodeIndex = NodeIndex::new_unchecked(1, 0);
const NODE11: NodeIndex = NodeIndex::new_unchecked(1, 1);
const NODE20: NodeIndex = NodeIndex::new_unchecked(2, 0);
const NODE21: NodeIndex = NodeIndex::new_unchecked(2, 1);
const NODE22: NodeIndex = NodeIndex::new_unchecked(2, 2);
const NODE23: NodeIndex = NodeIndex::new_unchecked(2, 3);
const NODE30: NodeIndex = NodeIndex::new_unchecked(3, 0);
const NODE31: NodeIndex = NodeIndex::new_unchecked(3, 1);
const NODE32: NodeIndex = NodeIndex::new_unchecked(3, 2);
const NODE33: NodeIndex = NodeIndex::new_unchecked(3, 3);
const VALUES8: [RpoDigest; 8] = [
int_to_node(30),
int_to_node(31),
int_to_node(32),
int_to_node(33),
int_to_node(34),
int_to_node(35),
int_to_node(36),
int_to_node(37),
];
// TESTS
// ================================================================================================
// For the Partial Merkle Tree tests we will use parts of the Merkle Tree which full form is
// illustrated below:
//
// __________ root __________
// / \
// ____ 10 ____ ____ 11 ____
// / \ / \
// 20 21 22 23
// / \ / \ / \ / \
// (30) (31) (32) (33) (34) (35) (36) (37)
//
// Where node number is a concatenation of its depth and index. For example, node with
// NodeIndex(3, 5) will be labeled as `35`. Leaves of the tree are shown as nodes with parenthesis
// (33).
/// Checks that creation of the PMT with `with_leaves()` constructor is working correctly.
#[test]
fn with_leaves() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let leaf_nodes_vec = vec![
(NODE20, mt.get_node(NODE20).unwrap()),
(NODE32, mt.get_node(NODE32).unwrap()),
(NODE33, mt.get_node(NODE33).unwrap()),
(NODE22, mt.get_node(NODE22).unwrap()),
(NODE23, mt.get_node(NODE23).unwrap()),
];
let leaf_nodes: BTreeMap<NodeIndex, RpoDigest> = leaf_nodes_vec.into_iter().collect();
let pmt = PartialMerkleTree::with_leaves(leaf_nodes).unwrap();
assert_eq!(expected_root, pmt.root())
}
/// Checks that `with_leaves()` function returns an error when using incomplete set of nodes.
#[test]
fn err_with_leaves() {
// NODE22 is missing
let leaf_nodes_vec = vec![
(NODE20, int_to_node(20)),
(NODE32, int_to_node(32)),
(NODE33, int_to_node(33)),
(NODE23, int_to_node(23)),
];
let leaf_nodes: BTreeMap<NodeIndex, RpoDigest> = leaf_nodes_vec.into_iter().collect();
assert!(PartialMerkleTree::with_leaves(leaf_nodes).is_err());
}
/// Checks that root returned by `root()` function is equal to the expected one.
#[test]
fn get_root() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let pmt = PartialMerkleTree::with_paths([(3, path33.value, path33.path)]).unwrap();
assert_eq!(expected_root, pmt.root());
}
/// This test checks correctness of the `add_path()` and `get_path()` functions. First it creates a
/// PMT using `add_path()` by adding Merkle Paths from node 33 and node 22 to the empty PMT. Then
/// it checks that paths returned by `get_path()` function are equal to the expected ones.
#[test]
fn add_and_get_paths() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let expected_path33 = ms.get_path(expected_root, NODE33).unwrap();
let expected_path22 = ms.get_path(expected_root, NODE22).unwrap();
let mut pmt = PartialMerkleTree::new();
pmt.add_path(3, expected_path33.value, expected_path33.path.clone()).unwrap();
pmt.add_path(2, expected_path22.value, expected_path22.path.clone()).unwrap();
let path33 = pmt.get_path(NODE33).unwrap();
let path22 = pmt.get_path(NODE22).unwrap();
let actual_root = pmt.root();
assert_eq!(expected_path33.path, path33);
assert_eq!(expected_path22.path, path22);
assert_eq!(expected_root, actual_root);
}
/// Checks that function `get_node` used on nodes 10 and 32 returns expected values.
#[test]
fn get_node() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let pmt = PartialMerkleTree::with_paths([(3, path33.value, path33.path)]).unwrap();
assert_eq!(ms.get_node(expected_root, NODE32).unwrap(), pmt.get_node(NODE32).unwrap());
assert_eq!(ms.get_node(expected_root, NODE10).unwrap(), pmt.get_node(NODE10).unwrap());
}
/// Updates leaves of the PMT using `update_leaf()` function and checks that new root of the tree
/// is equal to the expected one.
#[test]
fn update_leaf() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let root = mt.root();
let mut ms = MerkleStore::from(&mt);
let path33 = ms.get_path(root, NODE33).unwrap();
let mut pmt = PartialMerkleTree::with_paths([(3, path33.value, path33.path)]).unwrap();
let new_value32 = int_to_node(132);
let expected_root = ms.set_node(root, NODE32, new_value32).unwrap().root;
pmt.update_leaf(2, *new_value32).unwrap();
let actual_root = pmt.root();
assert_eq!(expected_root, actual_root);
let new_value20 = int_to_node(120);
let expected_root = ms.set_node(expected_root, NODE20, new_value20).unwrap().root;
pmt.update_leaf(0, *new_value20).unwrap();
let actual_root = pmt.root();
assert_eq!(expected_root, actual_root);
let new_value11 = int_to_node(111);
let expected_root = ms.set_node(expected_root, NODE11, new_value11).unwrap().root;
pmt.update_leaf(6, *new_value11).unwrap();
let actual_root = pmt.root();
assert_eq!(expected_root, actual_root);
}
/// Checks that paths of the PMT returned by `paths()` function are equal to the expected ones.
#[test]
fn get_paths() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let path22 = ms.get_path(expected_root, NODE22).unwrap();
let mut pmt = PartialMerkleTree::new();
pmt.add_path(3, path33.value, path33.path).unwrap();
pmt.add_path(2, path22.value, path22.path).unwrap();
// After PMT creation with path33 (33; 32, 20, 11) and path22 (22; 23, 10) we will have this
// tree:
//
// ______root______
// / \
// ___10___ ___11___
// / \ / \
// (20) 21 (22) (23)
// / \
// (32) (33)
//
// Which have leaf nodes 20, 22, 23, 32 and 33. Hence overall we will have 5 paths -- one path
// for each leaf.
let leaves = [NODE20, NODE22, NODE23, NODE32, NODE33];
let expected_paths: Vec<(NodeIndex, ValuePath)> = leaves
.iter()
.map(|&leaf| {
(
leaf,
ValuePath {
value: mt.get_node(leaf).unwrap(),
path: mt.get_path(leaf).unwrap(),
},
)
})
.collect();
let actual_paths = pmt.to_paths();
assert_eq!(expected_paths, actual_paths);
}
// Checks correctness of leaves determination when using the `leaves()` function.
#[test]
fn leaves() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let path22 = ms.get_path(expected_root, NODE22).unwrap();
let mut pmt = PartialMerkleTree::with_paths([(3, path33.value, path33.path)]).unwrap();
// After PMT creation with path33 (33; 32, 20, 11) we will have this tree:
//
// ______root______
// / \
// ___10___ (11)
// / \
// (20) 21
// / \
// (32) (33)
//
// Which have leaf nodes 11, 20, 32 and 33.
let value11 = mt.get_node(NODE11).unwrap();
let value20 = mt.get_node(NODE20).unwrap();
let value32 = mt.get_node(NODE32).unwrap();
let value33 = mt.get_node(NODE33).unwrap();
let leaves = [(NODE11, value11), (NODE20, value20), (NODE32, value32), (NODE33, value33)];
let expected_leaves = leaves.iter().copied();
assert!(expected_leaves.eq(pmt.leaves()));
pmt.add_path(2, path22.value, path22.path).unwrap();
// After adding the path22 (22; 23, 10) to the existing PMT we will have this tree:
//
// ______root______
// / \
// ___10___ ___11___
// / \ / \
// (20) 21 (22) (23)
// / \
// (32) (33)
//
// Which have leaf nodes 20, 22, 23, 32 and 33.
let value20 = mt.get_node(NODE20).unwrap();
let value22 = mt.get_node(NODE22).unwrap();
let value23 = mt.get_node(NODE23).unwrap();
let value32 = mt.get_node(NODE32).unwrap();
let value33 = mt.get_node(NODE33).unwrap();
let leaves = vec![
(NODE20, value20),
(NODE22, value22),
(NODE23, value23),
(NODE32, value32),
(NODE33, value33),
];
let expected_leaves = leaves.iter().copied();
assert!(expected_leaves.eq(pmt.leaves()));
}
/// Checks that nodes of the PMT returned by `inner_nodes()` function are equal to the expected ones.
#[test]
fn test_inner_node_iterator() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let path22 = ms.get_path(expected_root, NODE22).unwrap();
let mut pmt = PartialMerkleTree::with_paths([(3, path33.value, path33.path)]).unwrap();
// get actual inner nodes
let actual: Vec<InnerNodeInfo> = pmt.inner_nodes().collect();
let expected_n00 = mt.root();
let expected_n10 = mt.get_node(NODE10).unwrap();
let expected_n11 = mt.get_node(NODE11).unwrap();
let expected_n20 = mt.get_node(NODE20).unwrap();
let expected_n21 = mt.get_node(NODE21).unwrap();
let expected_n32 = mt.get_node(NODE32).unwrap();
let expected_n33 = mt.get_node(NODE33).unwrap();
// create vector of the expected inner nodes
let mut expected = vec![
InnerNodeInfo {
value: expected_n00,
left: expected_n10,
right: expected_n11,
},
InnerNodeInfo {
value: expected_n10,
left: expected_n20,
right: expected_n21,
},
InnerNodeInfo {
value: expected_n21,
left: expected_n32,
right: expected_n33,
},
];
assert_eq!(actual, expected);
// add another path to the Partial Merkle Tree
pmt.add_path(2, path22.value, path22.path).unwrap();
// get new actual inner nodes
let actual: Vec<InnerNodeInfo> = pmt.inner_nodes().collect();
let expected_n22 = mt.get_node(NODE22).unwrap();
let expected_n23 = mt.get_node(NODE23).unwrap();
let info_11 = InnerNodeInfo {
value: expected_n11,
left: expected_n22,
right: expected_n23,
};
// add new inner node to the existing vertor
expected.insert(2, info_11);
assert_eq!(actual, expected);
}
/// Checks that serialization and deserialization implementations for the PMT are working
/// correctly.
#[test]
fn serialization() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let path22 = ms.get_path(expected_root, NODE22).unwrap();
let pmt = PartialMerkleTree::with_paths([
(3, path33.value, path33.path),
(2, path22.value, path22.path),
])
.unwrap();
let serialized_pmt = pmt.to_bytes();
let deserialized_pmt = PartialMerkleTree::read_from_bytes(&serialized_pmt).unwrap();
assert_eq!(deserialized_pmt, pmt);
}
/// Checks that deserialization fails with incorrect data.
#[test]
fn err_deserialization() {
let mut tree_bytes: Vec<u8> = vec![5];
tree_bytes.append(&mut NODE20.to_bytes());
tree_bytes.append(&mut int_to_node(20).to_bytes());
tree_bytes.append(&mut NODE21.to_bytes());
tree_bytes.append(&mut int_to_node(21).to_bytes());
// node with depth 1 could have index 0 or 1, but it has 2
tree_bytes.append(&mut vec![1, 2]);
tree_bytes.append(&mut int_to_node(11).to_bytes());
assert!(PartialMerkleTree::read_from_bytes(&tree_bytes).is_err());
}
/// Checks that addition of the path with different root will cause an error.
#[test]
fn err_add_path() {
let path33 = vec![int_to_node(1), int_to_node(2), int_to_node(3)].into();
let path22 = vec![int_to_node(4), int_to_node(5)].into();
let mut pmt = PartialMerkleTree::new();
pmt.add_path(3, int_to_node(6), path33).unwrap();
assert!(pmt.add_path(2, int_to_node(7), path22).is_err());
}
/// Checks that the request of the node which is not in the PMT will cause an error.
#[test]
fn err_get_node() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let pmt = PartialMerkleTree::with_paths([(3, path33.value, path33.path)]).unwrap();
assert!(pmt.get_node(NODE22).is_err());
assert!(pmt.get_node(NODE23).is_err());
assert!(pmt.get_node(NODE30).is_err());
assert!(pmt.get_node(NODE31).is_err());
}
/// Checks that the request of the path from the leaf which is not in the PMT will cause an error.
#[test]
fn err_get_path() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let pmt = PartialMerkleTree::with_paths([(3, path33.value, path33.path)]).unwrap();
assert!(pmt.get_path(NODE22).is_err());
assert!(pmt.get_path(NODE23).is_err());
assert!(pmt.get_path(NODE30).is_err());
assert!(pmt.get_path(NODE31).is_err());
}
#[test]
fn err_update_leaf() {
let mt = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let expected_root = mt.root();
let ms = MerkleStore::from(&mt);
let path33 = ms.get_path(expected_root, NODE33).unwrap();
let mut pmt = PartialMerkleTree::with_paths([(3, path33.value, path33.path)]).unwrap();
assert!(pmt.update_leaf(8, *int_to_node(38)).is_err());
}

View File

@@ -1,13 +1,17 @@
use super::{vec, NodeIndex, Rpo256, Vec, Word};
use crate::Word;
use super::{vec, InnerNodeInfo, MerkleError, NodeIndex, Rpo256, RpoDigest, Vec};
use core::ops::{Deref, DerefMut};
use winter_utils::{ByteReader, Deserializable, DeserializationError, Serializable};
// MERKLE PATH
// ================================================================================================
/// A merkle path container, composed of a sequence of nodes of a Merkle tree.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct MerklePath {
nodes: Vec<Word>,
nodes: Vec<RpoDigest>,
}
impl MerklePath {
@@ -15,47 +19,90 @@ impl MerklePath {
// --------------------------------------------------------------------------------------------
/// Creates a new Merkle path from a list of nodes.
pub fn new(nodes: Vec<Word>) -> Self {
pub fn new(nodes: Vec<RpoDigest>) -> Self {
assert!(nodes.len() <= u8::MAX.into(), "MerklePath may have at most 256 items");
Self { nodes }
}
// PROVIDERS
// --------------------------------------------------------------------------------------------
/// Computes the merkle root for this opening.
pub fn compute_root(&self, index_value: u64, node: Word) -> Word {
let mut index = NodeIndex::new(self.depth(), index_value);
self.nodes.iter().copied().fold(node, |node, sibling| {
// compute the node and move to the next iteration.
let input = index.build_node(node.into(), sibling.into());
index.move_up();
Rpo256::merge(&input).into()
})
}
/// Returns the depth in which this Merkle path proof is valid.
pub fn depth(&self) -> u8 {
self.nodes.len() as u8
}
/// Returns a reference to the [MerklePath]'s nodes.
pub fn nodes(&self) -> &[RpoDigest] {
&self.nodes
}
/// Computes the merkle root for this opening.
pub fn compute_root(&self, index: u64, node: RpoDigest) -> Result<RpoDigest, MerkleError> {
let mut index = NodeIndex::new(self.depth(), index)?;
let root = self.nodes.iter().copied().fold(node, |node, sibling| {
// compute the node and move to the next iteration.
let input = index.build_node(node, sibling);
index.move_up();
Rpo256::merge(&input)
});
Ok(root)
}
/// Verifies the Merkle opening proof towards the provided root.
///
/// Returns `true` if `node` exists at `index` in a Merkle tree with `root`.
pub fn verify(&self, index: u64, node: Word, root: &Word) -> bool {
root == &self.compute_root(index, node)
pub fn verify(&self, index: u64, node: RpoDigest, root: &RpoDigest) -> bool {
match self.compute_root(index, node) {
Ok(computed_root) => root == &computed_root,
Err(_) => false,
}
}
/// Returns an iterator over every inner node of this [MerklePath].
///
/// The iteration order is unspecified.
///
/// # Errors
/// Returns an error if the specified index is not valid for this path.
pub fn inner_nodes(
&self,
index: u64,
node: RpoDigest,
) -> Result<InnerNodeIterator, MerkleError> {
Ok(InnerNodeIterator {
nodes: &self.nodes,
index: NodeIndex::new(self.depth(), index)?,
value: node,
})
}
}
impl From<Vec<Word>> for MerklePath {
fn from(path: Vec<Word>) -> Self {
// CONVERSIONS
// ================================================================================================
impl From<MerklePath> for Vec<RpoDigest> {
fn from(path: MerklePath) -> Self {
path.nodes
}
}
impl From<Vec<RpoDigest>> for MerklePath {
fn from(path: Vec<RpoDigest>) -> Self {
Self::new(path)
}
}
impl From<&[RpoDigest]> for MerklePath {
fn from(path: &[RpoDigest]) -> Self {
Self::new(path.to_vec())
}
}
impl Deref for MerklePath {
// we use `Vec` here instead of slice so we can call vector mutation methods directly from the
// merkle path (example: `Vec::remove`).
type Target = Vec<Word>;
type Target = Vec<RpoDigest>;
fn deref(&self) -> &Self::Target {
&self.nodes
@@ -68,17 +115,157 @@ impl DerefMut for MerklePath {
}
}
impl FromIterator<Word> for MerklePath {
fn from_iter<T: IntoIterator<Item = Word>>(iter: T) -> Self {
// ITERATORS
// ================================================================================================
impl FromIterator<RpoDigest> for MerklePath {
fn from_iter<T: IntoIterator<Item = RpoDigest>>(iter: T) -> Self {
Self::new(iter.into_iter().collect())
}
}
impl IntoIterator for MerklePath {
type Item = Word;
type IntoIter = vec::IntoIter<Word>;
type Item = RpoDigest;
type IntoIter = vec::IntoIter<RpoDigest>;
fn into_iter(self) -> Self::IntoIter {
self.nodes.into_iter()
}
}
/// An iterator over internal nodes of a [MerklePath].
pub struct InnerNodeIterator<'a> {
nodes: &'a Vec<RpoDigest>,
index: NodeIndex,
value: RpoDigest,
}
impl<'a> Iterator for InnerNodeIterator<'a> {
type Item = InnerNodeInfo;
fn next(&mut self) -> Option<Self::Item> {
if !self.index.is_root() {
let sibling_pos = self.nodes.len() - self.index.depth() as usize;
let (left, right) = if self.index.is_value_odd() {
(self.nodes[sibling_pos], self.value)
} else {
(self.value, self.nodes[sibling_pos])
};
self.value = Rpo256::merge(&[left, right]);
self.index.move_up();
Some(InnerNodeInfo { value: self.value, left, right })
} else {
None
}
}
}
// MERKLE PATH CONTAINERS
// ================================================================================================
/// A container for a [crate::Word] value and its [MerklePath] opening.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct ValuePath {
/// The node value opening for `path`.
pub value: RpoDigest,
/// The path from `value` to `root` (exclusive).
pub path: MerklePath,
}
impl ValuePath {
/// Returns a new [ValuePath] instantiated from the specified value and path.
pub fn new(value: RpoDigest, path: MerklePath) -> Self {
Self { value, path }
}
}
impl From<(MerklePath, Word)> for ValuePath {
fn from((path, value): (MerklePath, Word)) -> Self {
ValuePath::new(value.into(), path)
}
}
/// A container for a [MerklePath] and its [crate::Word] root.
///
/// This structure does not provide any guarantees regarding the correctness of the path to the
/// root. For more information, check [MerklePath::verify].
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct RootPath {
/// The node value opening for `path`.
pub root: RpoDigest,
/// The path from `value` to `root` (exclusive).
pub path: MerklePath,
}
// SERIALIZATION
// ================================================================================================
impl Serializable for MerklePath {
fn write_into<W: winter_utils::ByteWriter>(&self, target: &mut W) {
assert!(self.nodes.len() <= u8::MAX.into(), "Length enforced in the constructor");
target.write_u8(self.nodes.len() as u8);
target.write_many(&self.nodes);
}
}
impl Deserializable for MerklePath {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let count = source.read_u8()?.into();
let nodes = source.read_many::<RpoDigest>(count)?;
Ok(Self { nodes })
}
}
impl Serializable for ValuePath {
fn write_into<W: winter_utils::ByteWriter>(&self, target: &mut W) {
self.value.write_into(target);
self.path.write_into(target);
}
}
impl Deserializable for ValuePath {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let value = RpoDigest::read_from(source)?;
let path = MerklePath::read_from(source)?;
Ok(Self { value, path })
}
}
impl Serializable for RootPath {
fn write_into<W: winter_utils::ByteWriter>(&self, target: &mut W) {
self.root.write_into(target);
self.path.write_into(target);
}
}
impl Deserializable for RootPath {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let root = RpoDigest::read_from(source)?;
let path = MerklePath::read_from(source)?;
Ok(Self { root, path })
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use crate::merkle::{int_to_node, MerklePath};
#[test]
fn test_inner_nodes() {
let nodes = vec![int_to_node(1), int_to_node(2), int_to_node(3), int_to_node(4)];
let merkle_path = MerklePath::new(nodes);
let index = 6;
let node = int_to_node(5);
let root = merkle_path.compute_root(index, node).unwrap();
let inner_root = merkle_path.inner_nodes(index, node).unwrap().last().unwrap().value;
assert_eq!(root, inner_root);
}
}

View File

@@ -1,350 +0,0 @@
use super::{BTreeMap, MerkleError, MerklePath, NodeIndex, Rpo256, Vec, Word, ZERO};
// MERKLE PATH SET
// ================================================================================================
/// A set of Merkle paths.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct MerklePathSet {
root: Word,
total_depth: u8,
paths: BTreeMap<u64, MerklePath>,
}
impl MerklePathSet {
// CONSTRUCTOR
// --------------------------------------------------------------------------------------------
/// Returns an empty MerklePathSet.
pub fn new(depth: u8) -> Self {
let root = [ZERO; 4];
let paths = BTreeMap::new();
Self {
root,
total_depth: depth,
paths,
}
}
/// Appends the provided paths iterator into the set.
///
/// Analogous to `[Self::add_path]`.
pub fn with_paths<I>(self, paths: I) -> Result<Self, MerkleError>
where
I: IntoIterator<Item = (u64, Word, MerklePath)>,
{
paths
.into_iter()
.try_fold(self, |mut set, (index, value, path)| {
set.add_path(index, value, path)?;
Ok(set)
})
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the root to which all paths in this set resolve.
pub const fn root(&self) -> Word {
self.root
}
/// Returns the depth of the Merkle tree implied by the paths stored in this set.
///
/// Merkle tree of depth 1 has two leaves, depth 2 has four leaves etc.
pub const fn depth(&self) -> u8 {
self.total_depth
}
/// Returns a node at the specified index.
///
/// # Errors
/// Returns an error if:
/// * The specified index is not valid for the depth of structure.
/// * Requested node does not exist in the set.
pub fn get_node(&self, index: NodeIndex) -> Result<Word, MerkleError> {
if !index.with_depth(self.total_depth).is_valid() {
return Err(MerkleError::InvalidIndex(
index.with_depth(self.total_depth),
));
}
if index.depth() != self.total_depth {
return Err(MerkleError::InvalidDepth {
expected: self.total_depth,
provided: index.depth(),
});
}
let index_value = index.to_scalar_index();
let parity = index_value & 1;
let index_value = index_value / 2;
self.paths
.get(&index_value)
.ok_or(MerkleError::NodeNotInSet(index_value))
.map(|path| path[parity as usize])
}
/// Returns a Merkle path to the node at the specified index. The node itself is
/// not included in the path.
///
/// # Errors
/// Returns an error if:
/// * The specified index is not valid for the depth of structure.
/// * Node of the requested path does not exist in the set.
pub fn get_path(&self, index: NodeIndex) -> Result<MerklePath, MerkleError> {
if !index.with_depth(self.total_depth).is_valid() {
return Err(MerkleError::InvalidIndex(index));
}
if index.depth() != self.total_depth {
return Err(MerkleError::InvalidDepth {
expected: self.total_depth,
provided: index.depth(),
});
}
let index_value = index.to_scalar_index();
let index = index_value / 2;
let parity = index_value & 1;
let mut path = self
.paths
.get(&index)
.cloned()
.ok_or(MerkleError::NodeNotInSet(index))?;
path.remove(parity as usize);
Ok(path)
}
// STATE MUTATORS
// --------------------------------------------------------------------------------------------
/// Adds the specified Merkle path to this [MerklePathSet]. The `index` and `value` parameters
/// specify the leaf node at which the path starts.
///
/// # Errors
/// Returns an error if:
/// - The specified index is is not valid in the context of this Merkle path set (i.e., the
/// index implies a greater depth than is specified for this set).
/// - The specified path is not consistent with other paths in the set (i.e., resolves to a
/// different root).
pub fn add_path(
&mut self,
index_value: u64,
value: Word,
mut path: MerklePath,
) -> Result<(), MerkleError> {
let depth = (path.len() + 1) as u8;
let mut index = NodeIndex::new(depth, index_value);
if index.depth() != self.total_depth {
return Err(MerkleError::InvalidDepth {
expected: self.total_depth,
provided: index.depth(),
});
}
// update the current path
let index_value = index.to_scalar_index();
let upper_index_value = index_value / 2;
let parity = index_value & 1;
path.insert(parity as usize, value);
// traverse to the root, updating the nodes
let root: Word = Rpo256::merge(&[path[0].into(), path[1].into()]).into();
let root = path.iter().skip(2).copied().fold(root, |root, hash| {
index.move_up();
Rpo256::merge(&index.build_node(root.into(), hash.into())).into()
});
// if the path set is empty (the root is all ZEROs), set the root to the root of the added
// path; otherwise, the root of the added path must be identical to the current root
if self.root == [ZERO; 4] {
self.root = root;
} else if self.root != root {
return Err(MerkleError::InvalidPath(path));
}
// finish updating the path
self.paths.insert(upper_index_value, path);
Ok(())
}
/// Replaces the leaf at the specified index with the provided value.
///
/// # Errors
/// Returns an error if:
/// * Requested node does not exist in the set.
pub fn update_leaf(&mut self, base_index_value: u64, value: Word) -> Result<(), MerkleError> {
let depth = self.depth();
let mut index = NodeIndex::new(depth, base_index_value);
if !index.is_valid() {
return Err(MerkleError::InvalidIndex(index));
}
let path = match self
.paths
.get_mut(&index.clone().move_up().to_scalar_index())
{
Some(path) => path,
None => return Err(MerkleError::NodeNotInSet(base_index_value)),
};
// Fill old_hashes vector -----------------------------------------------------------------
let mut current_index = index;
let mut old_hashes = Vec::with_capacity(path.len().saturating_sub(2));
let mut root: Word = Rpo256::merge(&[path[0].into(), path[1].into()]).into();
for hash in path.iter().skip(2).copied() {
old_hashes.push(root);
current_index.move_up();
let input = current_index.build_node(hash.into(), root.into());
root = Rpo256::merge(&input).into();
}
// Fill new_hashes vector -----------------------------------------------------------------
path[index.is_value_odd() as usize] = value;
let mut new_hashes = Vec::with_capacity(path.len().saturating_sub(2));
let mut new_root: Word = Rpo256::merge(&[path[0].into(), path[1].into()]).into();
for path_hash in path.iter().skip(2).copied() {
new_hashes.push(new_root);
index.move_up();
let input = current_index.build_node(path_hash.into(), new_root.into());
new_root = Rpo256::merge(&input).into();
}
self.root = new_root;
// update paths ---------------------------------------------------------------------------
for path in self.paths.values_mut() {
for i in (0..old_hashes.len()).rev() {
if path[i + 2] == old_hashes[i] {
path[i + 2] = new_hashes[i];
break;
}
}
}
Ok(())
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::*;
use crate::merkle::int_to_node;
#[test]
fn get_root() {
let leaf0 = int_to_node(0);
let leaf1 = int_to_node(1);
let leaf2 = int_to_node(2);
let leaf3 = int_to_node(3);
let parent0 = calculate_parent_hash(leaf0, 0, leaf1);
let parent1 = calculate_parent_hash(leaf2, 2, leaf3);
let root_exp = calculate_parent_hash(parent0, 0, parent1);
let set = super::MerklePathSet::new(3)
.with_paths([(0, leaf0, vec![leaf1, parent1].into())])
.unwrap();
assert_eq!(set.root(), root_exp);
}
#[test]
fn add_and_get_path() {
let path_6 = vec![int_to_node(7), int_to_node(45), int_to_node(123)];
let hash_6 = int_to_node(6);
let index = 6_u64;
let depth = 4_u8;
let set = super::MerklePathSet::new(depth)
.with_paths([(index, hash_6, path_6.clone().into())])
.unwrap();
let stored_path_6 = set.get_path(NodeIndex::new(depth, index)).unwrap();
assert_eq!(path_6, *stored_path_6);
assert!(set.get_path(NodeIndex::new(depth, 15_u64)).is_err())
}
#[test]
fn get_node() {
let path_6 = vec![int_to_node(7), int_to_node(45), int_to_node(123)];
let hash_6 = int_to_node(6);
let index = 6_u64;
let depth = 4_u8;
let set = MerklePathSet::new(depth)
.with_paths([(index, hash_6, path_6.into())])
.unwrap();
assert_eq!(
int_to_node(6u64),
set.get_node(NodeIndex::new(depth, index)).unwrap()
);
assert!(set.get_node(NodeIndex::new(depth, 15_u64)).is_err());
}
#[test]
fn update_leaf() {
let hash_4 = int_to_node(4);
let hash_5 = int_to_node(5);
let hash_6 = int_to_node(6);
let hash_7 = int_to_node(7);
let hash_45 = calculate_parent_hash(hash_4, 12u64, hash_5);
let hash_67 = calculate_parent_hash(hash_6, 14u64, hash_7);
let hash_0123 = int_to_node(123);
let path_6 = vec![hash_7, hash_45, hash_0123];
let path_5 = vec![hash_4, hash_67, hash_0123];
let path_4 = vec![hash_5, hash_67, hash_0123];
let index_6 = 6_u64;
let index_5 = 5_u64;
let index_4 = 4_u64;
let depth = 4_u8;
let mut set = MerklePathSet::new(depth)
.with_paths([
(index_6, hash_6, path_6.into()),
(index_5, hash_5, path_5.into()),
(index_4, hash_4, path_4.into()),
])
.unwrap();
let new_hash_6 = int_to_node(100);
let new_hash_5 = int_to_node(55);
set.update_leaf(index_6, new_hash_6).unwrap();
let new_path_4 = set.get_path(NodeIndex::new(depth, index_4)).unwrap();
let new_hash_67 = calculate_parent_hash(new_hash_6, 14_u64, hash_7);
assert_eq!(new_hash_67, new_path_4[1]);
set.update_leaf(index_5, new_hash_5).unwrap();
let new_path_4 = set.get_path(NodeIndex::new(depth, index_4)).unwrap();
let new_path_6 = set.get_path(NodeIndex::new(depth, index_6)).unwrap();
let new_hash_45 = calculate_parent_hash(new_hash_5, 13_u64, hash_4);
assert_eq!(new_hash_45, new_path_6[1]);
assert_eq!(new_hash_5, new_path_4[0]);
}
// HELPER FUNCTIONS
// --------------------------------------------------------------------------------------------
const fn is_even(pos: u64) -> bool {
pos & 1 == 0
}
/// Calculates the hash of the parent node by two sibling ones
/// - node — current node
/// - node_pos — position of the current node
/// - sibling — neighboring vertex in the tree
fn calculate_parent_hash(node: Word, node_pos: u64, sibling: Word) -> Word {
if is_even(node_pos) {
Rpo256::merge(&[node.into(), sibling.into()]).into()
} else {
Rpo256::merge(&[sibling.into(), node.into()]).into()
}
}
}

View File

@@ -1,263 +0,0 @@
use super::{BTreeMap, MerkleError, MerklePath, NodeIndex, Rpo256, RpoDigest, Vec, Word};
#[cfg(test)]
mod tests;
// SPARSE MERKLE TREE
// ================================================================================================
/// A sparse Merkle tree with 63-bit keys and 4-element leaf values, without compaction.
/// Manipulation and retrieval of leaves and internal nodes is provided by its internal `Store`.
/// The root of the tree is recomputed on each new leaf update.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct SimpleSmt {
root: Word,
depth: u8,
store: Store,
}
impl SimpleSmt {
// CONSTANTS
// --------------------------------------------------------------------------------------------
/// Minimum supported depth.
pub const MIN_DEPTH: u8 = 1;
/// Maximum supported depth.
pub const MAX_DEPTH: u8 = 63;
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Creates a new simple SMT.
///
/// The provided entries will be tuples of the leaves and their corresponding keys.
///
/// # Errors
///
/// The function will fail if the provided entries count exceed the maximum tree capacity, that
/// is `2^{depth}`.
pub fn new<R, I>(entries: R, depth: u8) -> Result<Self, MerkleError>
where
R: IntoIterator<IntoIter = I>,
I: Iterator<Item = (u64, Word)> + ExactSizeIterator,
{
let mut entries = entries.into_iter();
// validate the range of the depth.
let max = 1 << depth;
if depth < Self::MIN_DEPTH {
return Err(MerkleError::DepthTooSmall(depth));
} else if Self::MAX_DEPTH < depth {
return Err(MerkleError::DepthTooBig(depth as u64));
} else if entries.len() > max {
return Err(MerkleError::InvalidEntriesCount(max, entries.len()));
}
let (store, root) = Store::new(depth);
let mut tree = Self { root, depth, store };
entries.try_for_each(|(key, leaf)| tree.insert_leaf(key, leaf))?;
Ok(tree)
}
/// Returns the root of this Merkle tree.
pub const fn root(&self) -> Word {
self.root
}
/// Returns the depth of this Merkle tree.
pub const fn depth(&self) -> u8 {
self.depth
}
/// Returns the set count of the keys of the leaves.
pub fn leaves_count(&self) -> usize {
self.store.leaves_count()
}
/// Returns a node at the specified key
///
/// # Errors
/// Returns an error if:
/// * The specified depth is greater than the depth of the tree.
/// * The specified key does not exist
pub fn get_node(&self, index: &NodeIndex) -> Result<Word, MerkleError> {
if index.is_root() {
Err(MerkleError::DepthTooSmall(index.depth()))
} else if index.depth() > self.depth() {
Err(MerkleError::DepthTooBig(index.depth() as u64))
} else if index.depth() == self.depth() {
self.store.get_leaf_node(index.value())
} else {
let branch_node = self.store.get_branch_node(index)?;
Ok(Rpo256::merge(&[branch_node.left, branch_node.right]).into())
}
}
/// Returns a Merkle path from the node at the specified key to the root. The node itself is
/// not included in the path.
///
/// # Errors
/// Returns an error if:
/// * The specified key does not exist as a branch or leaf node
/// * The specified depth is greater than the depth of the tree.
pub fn get_path(&self, mut index: NodeIndex) -> Result<MerklePath, MerkleError> {
if index.is_root() {
return Err(MerkleError::DepthTooSmall(index.depth()));
} else if index.depth() > self.depth() {
return Err(MerkleError::DepthTooBig(index.depth() as u64));
} else if index.depth() == self.depth() && !self.store.check_leaf_node_exists(index.value())
{
return Err(MerkleError::InvalidIndex(index.with_depth(self.depth())));
}
let mut path = Vec::with_capacity(index.depth() as usize);
for _ in 0..index.depth() {
let is_right = index.is_value_odd();
index.move_up();
let BranchNode { left, right } = self.store.get_branch_node(&index)?;
let value = if is_right { left } else { right };
path.push(*value);
}
Ok(path.into())
}
/// Return a Merkle path from the leaf at the specified key to the root. The leaf itself is not
/// included in the path.
///
/// # Errors
/// Returns an error if:
/// * The specified key does not exist as a leaf node.
pub fn get_leaf_path(&self, key: u64) -> Result<MerklePath, MerkleError> {
self.get_path(NodeIndex::new(self.depth(), key))
}
/// Replaces the leaf located at the specified key, and recomputes hashes by walking up the tree
///
/// # Errors
/// Returns an error if the specified key is not a valid leaf index for this tree.
pub fn update_leaf(&mut self, key: u64, value: Word) -> Result<(), MerkleError> {
if !self.store.check_leaf_node_exists(key) {
return Err(MerkleError::InvalidIndex(NodeIndex::new(self.depth(), key)));
}
self.insert_leaf(key, value)?;
Ok(())
}
/// Inserts a leaf located at the specified key, and recomputes hashes by walking up the tree
pub fn insert_leaf(&mut self, key: u64, value: Word) -> Result<(), MerkleError> {
self.store.insert_leaf_node(key, value);
// TODO consider using a map `index |-> word` instead of `index |-> (word, word)`
let mut index = NodeIndex::new(self.depth(), key);
let mut value = RpoDigest::from(value);
for _ in 0..index.depth() {
let is_right = index.is_value_odd();
index.move_up();
let BranchNode { left, right } = self
.store
.get_branch_node(&index)
.unwrap_or_else(|_| self.store.get_empty_node(index.depth() as usize + 1));
let (left, right) = if is_right {
(left, value)
} else {
(value, right)
};
self.store.insert_branch_node(index, left, right);
value = Rpo256::merge(&[left, right]);
}
self.root = value.into();
Ok(())
}
}
// STORE
// ================================================================================================
/// A data store for sparse Merkle tree key-value pairs.
/// Leaves and branch nodes are stored separately in B-tree maps, indexed by key and (key, depth)
/// respectively. Hashes for blank subtrees at each layer are stored in `empty_hashes`, beginning
/// with the root hash of an empty tree, and ending with the zero value of a leaf node.
#[derive(Debug, Clone, PartialEq, Eq)]
struct Store {
branches: BTreeMap<NodeIndex, BranchNode>,
leaves: BTreeMap<u64, Word>,
empty_hashes: Vec<RpoDigest>,
depth: u8,
}
#[derive(Debug, Default, Clone, PartialEq, Eq)]
struct BranchNode {
left: RpoDigest,
right: RpoDigest,
}
impl Store {
fn new(depth: u8) -> (Self, Word) {
let branches = BTreeMap::new();
let leaves = BTreeMap::new();
// Construct empty node digests for each layer of the tree
let empty_hashes: Vec<RpoDigest> = (0..depth + 1)
.scan(Word::default().into(), |state, _| {
let value = *state;
*state = Rpo256::merge(&[value, value]);
Some(value)
})
.collect::<Vec<_>>()
.into_iter()
.rev()
.collect();
let root = empty_hashes[0].into();
let store = Self {
branches,
leaves,
empty_hashes,
depth,
};
(store, root)
}
fn get_empty_node(&self, depth: usize) -> BranchNode {
let digest = self.empty_hashes[depth];
BranchNode {
left: digest,
right: digest,
}
}
fn check_leaf_node_exists(&self, key: u64) -> bool {
self.leaves.contains_key(&key)
}
fn get_leaf_node(&self, key: u64) -> Result<Word, MerkleError> {
self.leaves
.get(&key)
.cloned()
.ok_or(MerkleError::InvalidIndex(NodeIndex::new(self.depth, key)))
}
fn insert_leaf_node(&mut self, key: u64, node: Word) {
self.leaves.insert(key, node);
}
fn get_branch_node(&self, index: &NodeIndex) -> Result<BranchNode, MerkleError> {
self.branches
.get(index)
.cloned()
.ok_or(MerkleError::InvalidIndex(*index))
}
fn insert_branch_node(&mut self, index: NodeIndex, left: RpoDigest, right: RpoDigest) {
let branch = BranchNode { left, right };
self.branches.insert(index, branch);
}
fn leaves_count(&self) -> usize {
self.leaves.len()
}
}

View File

@@ -1,281 +0,0 @@
use super::{
super::{MerkleTree, RpoDigest, SimpleSmt},
NodeIndex, Rpo256, Vec, Word,
};
use crate::{Felt, FieldElement};
use core::iter;
use proptest::prelude::*;
use rand_utils::prng_array;
const KEYS4: [u64; 4] = [0, 1, 2, 3];
const KEYS8: [u64; 8] = [0, 1, 2, 3, 4, 5, 6, 7];
const VALUES4: [Word; 4] = [
int_to_node(1),
int_to_node(2),
int_to_node(3),
int_to_node(4),
];
const VALUES8: [Word; 8] = [
int_to_node(1),
int_to_node(2),
int_to_node(3),
int_to_node(4),
int_to_node(5),
int_to_node(6),
int_to_node(7),
int_to_node(8),
];
const ZERO_VALUES8: [Word; 8] = [int_to_node(0); 8];
#[test]
fn build_empty_tree() {
let smt = SimpleSmt::new(iter::empty(), 3).unwrap();
let mt = MerkleTree::new(ZERO_VALUES8.to_vec()).unwrap();
assert_eq!(mt.root(), smt.root());
}
#[test]
fn empty_digests_are_consistent() {
let depth = 5;
let root = SimpleSmt::new(iter::empty(), depth).unwrap().root();
let computed: [RpoDigest; 2] = (0..depth).fold([Default::default(); 2], |state, _| {
let digest = Rpo256::merge(&state);
[digest; 2]
});
assert_eq!(Word::from(computed[0]), root);
}
#[test]
fn build_sparse_tree() {
let mut smt = SimpleSmt::new(iter::empty(), 3).unwrap();
let mut values = ZERO_VALUES8.to_vec();
// insert single value
let key = 6;
let new_node = int_to_node(7);
values[key as usize] = new_node;
smt.insert_leaf(key, new_node)
.expect("Failed to insert leaf");
let mt2 = MerkleTree::new(values.clone()).unwrap();
assert_eq!(mt2.root(), smt.root());
assert_eq!(
mt2.get_path(NodeIndex::new(3, 6)).unwrap(),
smt.get_path(NodeIndex::new(3, 6)).unwrap()
);
// insert second value at distinct leaf branch
let key = 2;
let new_node = int_to_node(3);
values[key as usize] = new_node;
smt.insert_leaf(key, new_node)
.expect("Failed to insert leaf");
let mt3 = MerkleTree::new(values).unwrap();
assert_eq!(mt3.root(), smt.root());
assert_eq!(
mt3.get_path(NodeIndex::new(3, 2)).unwrap(),
smt.get_path(NodeIndex::new(3, 2)).unwrap()
);
}
#[test]
fn build_full_tree() {
let tree = SimpleSmt::new(KEYS4.into_iter().zip(VALUES4.into_iter()), 2).unwrap();
let (root, node2, node3) = compute_internal_nodes();
assert_eq!(root, tree.root());
assert_eq!(node2, tree.get_node(&NodeIndex::new(1, 0)).unwrap());
assert_eq!(node3, tree.get_node(&NodeIndex::new(1, 1)).unwrap());
}
#[test]
fn get_values() {
let tree = SimpleSmt::new(KEYS4.into_iter().zip(VALUES4.into_iter()), 2).unwrap();
// check depth 2
assert_eq!(VALUES4[0], tree.get_node(&NodeIndex::new(2, 0)).unwrap());
assert_eq!(VALUES4[1], tree.get_node(&NodeIndex::new(2, 1)).unwrap());
assert_eq!(VALUES4[2], tree.get_node(&NodeIndex::new(2, 2)).unwrap());
assert_eq!(VALUES4[3], tree.get_node(&NodeIndex::new(2, 3)).unwrap());
}
#[test]
fn get_path() {
let tree = SimpleSmt::new(KEYS4.into_iter().zip(VALUES4.into_iter()), 2).unwrap();
let (_, node2, node3) = compute_internal_nodes();
// check depth 2
assert_eq!(
vec![VALUES4[1], node3],
*tree.get_path(NodeIndex::new(2, 0)).unwrap()
);
assert_eq!(
vec![VALUES4[0], node3],
*tree.get_path(NodeIndex::new(2, 1)).unwrap()
);
assert_eq!(
vec![VALUES4[3], node2],
*tree.get_path(NodeIndex::new(2, 2)).unwrap()
);
assert_eq!(
vec![VALUES4[2], node2],
*tree.get_path(NodeIndex::new(2, 3)).unwrap()
);
// check depth 1
assert_eq!(vec![node3], *tree.get_path(NodeIndex::new(1, 0)).unwrap());
assert_eq!(vec![node2], *tree.get_path(NodeIndex::new(1, 1)).unwrap());
}
#[test]
fn update_leaf() {
let mut tree = SimpleSmt::new(KEYS8.into_iter().zip(VALUES8.into_iter()), 3).unwrap();
// update one value
let key = 3;
let new_node = int_to_node(9);
let mut expected_values = VALUES8.to_vec();
expected_values[key] = new_node;
let expected_tree = SimpleSmt::new(
KEYS8.into_iter().zip(expected_values.clone().into_iter()),
3,
)
.unwrap();
tree.update_leaf(key as u64, new_node).unwrap();
assert_eq!(expected_tree.root, tree.root);
// update another value
let key = 6;
let new_node = int_to_node(10);
expected_values[key] = new_node;
let expected_tree =
SimpleSmt::new(KEYS8.into_iter().zip(expected_values.into_iter()), 3).unwrap();
tree.update_leaf(key as u64, new_node).unwrap();
assert_eq!(expected_tree.root, tree.root);
}
#[test]
fn small_tree_opening_is_consistent() {
// ____k____
// / \
// _i_ _j_
// / \ / \
// e f g h
// / \ / \ / \ / \
// a b 0 0 c 0 0 d
let z = Word::from(RpoDigest::default());
let a = Word::from(Rpo256::merge(&[z.into(); 2]));
let b = Word::from(Rpo256::merge(&[a.into(); 2]));
let c = Word::from(Rpo256::merge(&[b.into(); 2]));
let d = Word::from(Rpo256::merge(&[c.into(); 2]));
let e = Word::from(Rpo256::merge(&[a.into(), b.into()]));
let f = Word::from(Rpo256::merge(&[z.into(), z.into()]));
let g = Word::from(Rpo256::merge(&[c.into(), z.into()]));
let h = Word::from(Rpo256::merge(&[z.into(), d.into()]));
let i = Word::from(Rpo256::merge(&[e.into(), f.into()]));
let j = Word::from(Rpo256::merge(&[g.into(), h.into()]));
let k = Word::from(Rpo256::merge(&[i.into(), j.into()]));
let depth = 3;
let entries = vec![(0, a), (1, b), (4, c), (7, d)];
let tree = SimpleSmt::new(entries, depth).unwrap();
assert_eq!(tree.root(), Word::from(k));
let cases: Vec<(u8, u64, Vec<Word>)> = vec![
(3, 0, vec![b, f, j]),
(3, 1, vec![a, f, j]),
(3, 4, vec![z, h, i]),
(3, 7, vec![z, g, i]),
(2, 0, vec![f, j]),
(2, 1, vec![e, j]),
(2, 2, vec![h, i]),
(2, 3, vec![g, i]),
(1, 0, vec![j]),
(1, 1, vec![i]),
];
for (depth, key, path) in cases {
let opening = tree.get_path(NodeIndex::new(depth, key)).unwrap();
assert_eq!(path, *opening);
}
}
proptest! {
#[test]
fn arbitrary_openings_single_leaf(
depth in SimpleSmt::MIN_DEPTH..SimpleSmt::MAX_DEPTH,
key in prop::num::u64::ANY,
leaf in prop::num::u64::ANY,
) {
let mut tree = SimpleSmt::new(iter::empty(), depth).unwrap();
let key = key % (1 << depth as u64);
let leaf = int_to_node(leaf);
tree.insert_leaf(key, leaf.into()).unwrap();
tree.get_leaf_path(key).unwrap();
// traverse to root, fetching all paths
for d in 1..depth {
let k = key >> (depth - d);
tree.get_path(NodeIndex::new(d, k)).unwrap();
}
}
#[test]
fn arbitrary_openings_multiple_leaves(
depth in SimpleSmt::MIN_DEPTH..SimpleSmt::MAX_DEPTH,
count in 2u8..10u8,
ref seed in any::<[u8; 32]>()
) {
let mut tree = SimpleSmt::new(iter::empty(), depth).unwrap();
let mut seed = *seed;
let leaves = (1 << depth) - 1;
for _ in 0..count {
seed = prng_array(seed);
let mut key = [0u8; 8];
let mut leaf = [0u8; 8];
key.copy_from_slice(&seed[..8]);
leaf.copy_from_slice(&seed[8..16]);
let key = u64::from_le_bytes(key);
let key = key % leaves;
let leaf = u64::from_le_bytes(leaf);
let leaf = int_to_node(leaf);
tree.insert_leaf(key, leaf).unwrap();
tree.get_leaf_path(key).unwrap();
}
}
}
// HELPER FUNCTIONS
// --------------------------------------------------------------------------------------------
fn compute_internal_nodes() -> (Word, Word, Word) {
let node2 = Rpo256::hash_elements(&[VALUES4[0], VALUES4[1]].concat());
let node3 = Rpo256::hash_elements(&[VALUES4[2], VALUES4[3]].concat());
let root = Rpo256::merge(&[node2, node3]);
(root.into(), node2.into(), node3.into())
}
const fn int_to_node(value: u64) -> Word {
[Felt::new(value), Felt::ZERO, Felt::ZERO, Felt::ZERO]
}

View File

@@ -0,0 +1,86 @@
use core::fmt;
use crate::{
hash::rpo::RpoDigest,
merkle::{LeafIndex, SMT_DEPTH},
utils::collections::Vec,
Word,
};
// SMT LEAF ERROR
// =================================================================================================
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum SmtLeafError {
InconsistentKeys {
entries: Vec<(RpoDigest, Word)>,
key_1: RpoDigest,
key_2: RpoDigest,
},
InvalidNumEntriesForMultiple(usize),
SingleKeyInconsistentWithLeafIndex {
key: RpoDigest,
leaf_index: LeafIndex<SMT_DEPTH>,
},
MultipleKeysInconsistentWithLeafIndex {
leaf_index_from_keys: LeafIndex<SMT_DEPTH>,
leaf_index_supplied: LeafIndex<SMT_DEPTH>,
},
}
#[cfg(feature = "std")]
impl std::error::Error for SmtLeafError {}
impl fmt::Display for SmtLeafError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
use SmtLeafError::*;
match self {
InvalidNumEntriesForMultiple(num_entries) => {
write!(f, "Multiple leaf requires 2 or more entries. Got: {num_entries}")
}
InconsistentKeys { entries, key_1, key_2 } => {
write!(f, "Multiple leaf requires all keys to map to the same leaf index. Offending keys: {key_1} and {key_2}. Entries: {entries:?}.")
}
SingleKeyInconsistentWithLeafIndex { key, leaf_index } => {
write!(
f,
"Single key in leaf inconsistent with leaf index. Key: {key}, leaf index: {}",
leaf_index.value()
)
}
MultipleKeysInconsistentWithLeafIndex {
leaf_index_from_keys,
leaf_index_supplied,
} => {
write!(
f,
"Keys in entries map to leaf index {}, but leaf index {} was supplied",
leaf_index_from_keys.value(),
leaf_index_supplied.value()
)
}
}
}
}
// SMT PROOF ERROR
// =================================================================================================
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum SmtProofError {
InvalidPathLength(usize),
}
#[cfg(feature = "std")]
impl std::error::Error for SmtProofError {}
impl fmt::Display for SmtProofError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
use SmtProofError::*;
match self {
InvalidPathLength(path_length) => {
write!(f, "Invalid Merkle path length. Expected {SMT_DEPTH}, got {path_length}")
}
}
}
}

372
src/merkle/smt/full/leaf.rs Normal file
View File

@@ -0,0 +1,372 @@
use core::cmp::Ordering;
use crate::utils::{collections::Vec, string::ToString, vec};
use winter_utils::{ByteReader, ByteWriter, Deserializable, DeserializationError, Serializable};
use super::{Felt, LeafIndex, Rpo256, RpoDigest, SmtLeafError, Word, EMPTY_WORD, SMT_DEPTH};
#[derive(Clone, Debug, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub enum SmtLeaf {
Empty(LeafIndex<SMT_DEPTH>),
Single((RpoDigest, Word)),
Multiple(Vec<(RpoDigest, Word)>),
}
impl SmtLeaf {
// CONSTRUCTORS
// ---------------------------------------------------------------------------------------------
/// Returns a new leaf with the specified entries
///
/// # Errors
/// - Returns an error if 2 keys in `entries` map to a different leaf index
/// - Returns an error if 1 or more keys in `entries` map to a leaf index
/// different from `leaf_index`
pub fn new(
entries: Vec<(RpoDigest, Word)>,
leaf_index: LeafIndex<SMT_DEPTH>,
) -> Result<Self, SmtLeafError> {
match entries.len() {
0 => Ok(Self::new_empty(leaf_index)),
1 => {
let (key, value) = entries[0];
if LeafIndex::<SMT_DEPTH>::from(key) != leaf_index {
return Err(SmtLeafError::SingleKeyInconsistentWithLeafIndex {
key,
leaf_index,
});
}
Ok(Self::new_single(key, value))
}
_ => {
let leaf = Self::new_multiple(entries)?;
// `new_multiple()` checked that all keys map to the same leaf index. We still need
// to ensure that that leaf index is `leaf_index`.
if leaf.index() != leaf_index {
Err(SmtLeafError::MultipleKeysInconsistentWithLeafIndex {
leaf_index_from_keys: leaf.index(),
leaf_index_supplied: leaf_index,
})
} else {
Ok(leaf)
}
}
}
}
/// Returns a new empty leaf with the specified leaf index
pub fn new_empty(leaf_index: LeafIndex<SMT_DEPTH>) -> Self {
Self::Empty(leaf_index)
}
/// Returns a new single leaf with the specified entry. The leaf index is derived from the
/// entry's key.
pub fn new_single(key: RpoDigest, value: Word) -> Self {
Self::Single((key, value))
}
/// Returns a new single leaf with the specified entry. The leaf index is derived from the
/// entries' keys.
///
/// # Errors
/// - Returns an error if 2 keys in `entries` map to a different leaf index
pub fn new_multiple(entries: Vec<(RpoDigest, Word)>) -> Result<Self, SmtLeafError> {
if entries.len() < 2 {
return Err(SmtLeafError::InvalidNumEntriesForMultiple(entries.len()));
}
// Check that all keys map to the same leaf index
{
let mut keys = entries.iter().map(|(key, _)| key);
let first_key = *keys.next().expect("ensured at least 2 entries");
let first_leaf_index: LeafIndex<SMT_DEPTH> = first_key.into();
for &next_key in keys {
let next_leaf_index: LeafIndex<SMT_DEPTH> = next_key.into();
if next_leaf_index != first_leaf_index {
return Err(SmtLeafError::InconsistentKeys {
entries,
key_1: first_key,
key_2: next_key,
});
}
}
}
Ok(Self::Multiple(entries))
}
// PUBLIC ACCESSORS
// ---------------------------------------------------------------------------------------------
/// Returns true if the leaf is empty
pub fn is_empty(&self) -> bool {
matches!(self, Self::Empty(_))
}
/// Returns the leaf's index in the [`super::Smt`]
pub fn index(&self) -> LeafIndex<SMT_DEPTH> {
match self {
SmtLeaf::Empty(leaf_index) => *leaf_index,
SmtLeaf::Single((key, _)) => key.into(),
SmtLeaf::Multiple(entries) => {
// Note: All keys are guaranteed to have the same leaf index
let (first_key, _) = entries[0];
first_key.into()
}
}
}
/// Returns the number of entries stored in the leaf
pub fn num_entries(&self) -> u64 {
match self {
SmtLeaf::Empty(_) => 0,
SmtLeaf::Single(_) => 1,
SmtLeaf::Multiple(entries) => {
entries.len().try_into().expect("shouldn't have more than 2^64 entries")
}
}
}
/// Computes the hash of the leaf
pub fn hash(&self) -> RpoDigest {
match self {
SmtLeaf::Empty(_) => EMPTY_WORD.into(),
SmtLeaf::Single((key, value)) => Rpo256::merge(&[*key, value.into()]),
SmtLeaf::Multiple(kvs) => {
let elements: Vec<Felt> = kvs.iter().copied().flat_map(kv_to_elements).collect();
Rpo256::hash_elements(&elements)
}
}
}
// ITERATORS
// ---------------------------------------------------------------------------------------------
/// Returns the key-value pairs in the leaf
pub fn entries(&self) -> Vec<&(RpoDigest, Word)> {
match self {
SmtLeaf::Empty(_) => Vec::new(),
SmtLeaf::Single(kv_pair) => vec![kv_pair],
SmtLeaf::Multiple(kv_pairs) => kv_pairs.iter().collect(),
}
}
// CONVERSIONS
// ---------------------------------------------------------------------------------------------
/// Converts a leaf to a list of field elements
pub fn to_elements(&self) -> Vec<Felt> {
self.clone().into_elements()
}
/// Converts a leaf to a list of field elements
pub fn into_elements(self) -> Vec<Felt> {
self.into_entries().into_iter().flat_map(kv_to_elements).collect()
}
/// Converts a leaf the key-value pairs in the leaf
pub fn into_entries(self) -> Vec<(RpoDigest, Word)> {
match self {
SmtLeaf::Empty(_) => Vec::new(),
SmtLeaf::Single(kv_pair) => vec![kv_pair],
SmtLeaf::Multiple(kv_pairs) => kv_pairs,
}
}
// HELPERS
// ---------------------------------------------------------------------------------------------
/// Returns the value associated with `key` in the leaf, or `None` if `key` maps to another leaf.
pub(super) fn get_value(&self, key: &RpoDigest) -> Option<Word> {
// Ensure that `key` maps to this leaf
if self.index() != key.into() {
return None;
}
match self {
SmtLeaf::Empty(_) => Some(EMPTY_WORD),
SmtLeaf::Single((key_in_leaf, value_in_leaf)) => {
if key == key_in_leaf {
Some(*value_in_leaf)
} else {
Some(EMPTY_WORD)
}
}
SmtLeaf::Multiple(kv_pairs) => {
for (key_in_leaf, value_in_leaf) in kv_pairs {
if key == key_in_leaf {
return Some(*value_in_leaf);
}
}
Some(EMPTY_WORD)
}
}
}
/// Inserts key-value pair into the leaf; returns the previous value associated with `key`, if
/// any.
///
/// The caller needs to ensure that `key` has the same leaf index as all other keys in the leaf
pub(super) fn insert(&mut self, key: RpoDigest, value: Word) -> Option<Word> {
match self {
SmtLeaf::Empty(_) => {
*self = SmtLeaf::new_single(key, value);
None
}
SmtLeaf::Single(kv_pair) => {
if kv_pair.0 == key {
// the key is already in this leaf. Update the value and return the previous
// value
let old_value = kv_pair.1;
kv_pair.1 = value;
Some(old_value)
} else {
// Another entry is present in this leaf. Transform the entry into a list
// entry, and make sure the key-value pairs are sorted by key
let mut pairs = vec![*kv_pair, (key, value)];
pairs.sort_by(|(key_1, _), (key_2, _)| cmp_keys(*key_1, *key_2));
*self = SmtLeaf::Multiple(pairs);
None
}
}
SmtLeaf::Multiple(kv_pairs) => {
match kv_pairs.binary_search_by(|kv_pair| cmp_keys(kv_pair.0, key)) {
Ok(pos) => {
let old_value = kv_pairs[pos].1;
kv_pairs[pos].1 = value;
Some(old_value)
}
Err(pos) => {
kv_pairs.insert(pos, (key, value));
None
}
}
}
}
}
/// Removes key-value pair from the leaf stored at key; returns the previous value associated
/// with `key`, if any. Also returns an `is_empty` flag, indicating whether the leaf became
/// empty, and must be removed from the data structure it is contained in.
pub(super) fn remove(&mut self, key: RpoDigest) -> (Option<Word>, bool) {
match self {
SmtLeaf::Empty(_) => (None, false),
SmtLeaf::Single((key_at_leaf, value_at_leaf)) => {
if *key_at_leaf == key {
// our key was indeed stored in the leaf, so we return the value that was stored
// in it, and indicate that the leaf should be removed
let old_value = *value_at_leaf;
// Note: this is not strictly needed, since the caller is expected to drop this
// `SmtLeaf` object.
*self = SmtLeaf::new_empty(key.into());
(Some(old_value), true)
} else {
// another key is stored at leaf; nothing to update
(None, false)
}
}
SmtLeaf::Multiple(kv_pairs) => {
match kv_pairs.binary_search_by(|kv_pair| cmp_keys(kv_pair.0, key)) {
Ok(pos) => {
let old_value = kv_pairs[pos].1;
kv_pairs.remove(pos);
debug_assert!(!kv_pairs.is_empty());
if kv_pairs.len() == 1 {
// convert the leaf into `Single`
*self = SmtLeaf::Single(kv_pairs[0]);
}
(Some(old_value), false)
}
Err(_) => {
// other keys are stored at leaf; nothing to update
(None, false)
}
}
}
}
}
}
impl Serializable for SmtLeaf {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
// Write: num entries
self.num_entries().write_into(target);
// Write: leaf index
let leaf_index: u64 = self.index().value();
leaf_index.write_into(target);
// Write: entries
for (key, value) in self.entries() {
key.write_into(target);
value.write_into(target);
}
}
}
impl Deserializable for SmtLeaf {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
// Read: num entries
let num_entries = source.read_u64()?;
// Read: leaf index
let leaf_index: LeafIndex<SMT_DEPTH> = {
let value = source.read_u64()?;
LeafIndex::new_max_depth(value)
};
// Read: entries
let mut entries: Vec<(RpoDigest, Word)> = Vec::new();
for _ in 0..num_entries {
let key: RpoDigest = source.read()?;
let value: Word = source.read()?;
entries.push((key, value));
}
Self::new(entries, leaf_index)
.map_err(|err| DeserializationError::InvalidValue(err.to_string()))
}
}
// HELPER FUNCTIONS
// ================================================================================================
/// Converts a key-value tuple to an iterator of `Felt`s
fn kv_to_elements((key, value): (RpoDigest, Word)) -> impl Iterator<Item = Felt> {
let key_elements = key.into_iter();
let value_elements = value.into_iter();
key_elements.chain(value_elements)
}
/// Compares two keys, compared element-by-element using their integer representations starting with
/// the most significant element.
fn cmp_keys(key_1: RpoDigest, key_2: RpoDigest) -> Ordering {
for (v1, v2) in key_1.iter().zip(key_2.iter()).rev() {
let v1 = v1.as_int();
let v2 = v2.as_int();
if v1 != v2 {
return v1.cmp(&v2);
}
}
Ordering::Equal
}

299
src/merkle/smt/full/mod.rs Normal file
View File

@@ -0,0 +1,299 @@
use crate::hash::rpo::Rpo256;
use crate::merkle::{EmptySubtreeRoots, InnerNodeInfo};
use crate::utils::collections::{BTreeMap, BTreeSet};
use crate::{Felt, EMPTY_WORD};
use super::{
InnerNode, LeafIndex, MerkleError, MerklePath, NodeIndex, RpoDigest, SparseMerkleTree, Word,
};
mod error;
pub use error::{SmtLeafError, SmtProofError};
mod leaf;
pub use leaf::SmtLeaf;
mod proof;
pub use proof::SmtProof;
#[cfg(test)]
mod tests;
// CONSTANTS
// ================================================================================================
pub const SMT_DEPTH: u8 = 64;
// SMT
// ================================================================================================
/// Sparse Merkle tree mapping 256-bit keys to 256-bit values. Both keys and values are represented
/// by 4 field elements.
///
/// All leaves sit at depth 64. The most significant element of the key is used to identify the leaf to
/// which the key maps.
///
/// A leaf is either empty, or holds one or more key-value pairs. An empty leaf hashes to the empty
/// word. Otherwise, a leaf hashes to the hash of its key-value pairs, ordered by key first, value
/// second.
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct Smt {
root: RpoDigest,
leaves: BTreeMap<u64, SmtLeaf>,
inner_nodes: BTreeMap<NodeIndex, InnerNode>,
}
impl Smt {
// CONSTANTS
// --------------------------------------------------------------------------------------------
/// The default value used to compute the hash of empty leaves
pub const EMPTY_VALUE: Word = <Self as SparseMerkleTree<SMT_DEPTH>>::EMPTY_VALUE;
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Returns a new [Smt].
///
/// All leaves in the returned tree are set to [Self::EMPTY_VALUE].
pub fn new() -> Self {
let root = *EmptySubtreeRoots::entry(SMT_DEPTH, 0);
Self {
root,
leaves: BTreeMap::new(),
inner_nodes: BTreeMap::new(),
}
}
/// Returns a new [Smt] instantiated with leaves set as specified by the provided entries.
///
/// All leaves omitted from the entries list are set to [Self::EMPTY_VALUE].
///
/// # Errors
/// Returns an error if the provided entries contain multiple values for the same key.
pub fn with_entries(
entries: impl IntoIterator<Item = (RpoDigest, Word)>,
) -> Result<Self, MerkleError> {
// create an empty tree
let mut tree = Self::new();
// This being a sparse data structure, the EMPTY_WORD is not assigned to the `BTreeMap`, so
// entries with the empty value need additional tracking.
let mut key_set_to_zero = BTreeSet::new();
for (key, value) in entries {
let old_value = tree.insert(key, value);
if old_value != EMPTY_WORD || key_set_to_zero.contains(&key) {
return Err(MerkleError::DuplicateValuesForIndex(
LeafIndex::<SMT_DEPTH>::from(key).value(),
));
}
if value == EMPTY_WORD {
key_set_to_zero.insert(key);
};
}
Ok(tree)
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the depth of the tree
pub const fn depth(&self) -> u8 {
SMT_DEPTH
}
/// Returns the root of the tree
pub fn root(&self) -> RpoDigest {
<Self as SparseMerkleTree<SMT_DEPTH>>::root(self)
}
/// Returns the leaf to which `key` maps
pub fn get_leaf(&self, key: &RpoDigest) -> SmtLeaf {
<Self as SparseMerkleTree<SMT_DEPTH>>::get_leaf(self, key)
}
/// Returns the value associated with `key`
pub fn get_value(&self, key: &RpoDigest) -> Word {
let leaf_pos = LeafIndex::<SMT_DEPTH>::from(*key).value();
match self.leaves.get(&leaf_pos) {
Some(leaf) => leaf.get_value(key).unwrap_or_default(),
None => EMPTY_WORD,
}
}
/// Returns an opening of the leaf associated with `key`. Conceptually, an opening is a Merkle
/// path to the leaf, as well as the leaf itself.
pub fn open(&self, key: &RpoDigest) -> SmtProof {
<Self as SparseMerkleTree<SMT_DEPTH>>::open(self, key)
}
// ITERATORS
// --------------------------------------------------------------------------------------------
/// Returns an iterator over the leaves of this [Smt].
pub fn leaves(&self) -> impl Iterator<Item = (LeafIndex<SMT_DEPTH>, &SmtLeaf)> {
self.leaves
.iter()
.map(|(leaf_index, leaf)| (LeafIndex::new_max_depth(*leaf_index), leaf))
}
/// Returns an iterator over the key-value pairs of this [Smt].
pub fn entries(&self) -> impl Iterator<Item = &(RpoDigest, Word)> {
self.leaves().flat_map(|(_, leaf)| leaf.entries())
}
/// Returns an iterator over the inner nodes of this [Smt].
pub fn inner_nodes(&self) -> impl Iterator<Item = InnerNodeInfo> + '_ {
self.inner_nodes.values().map(|e| InnerNodeInfo {
value: e.hash(),
left: e.left,
right: e.right,
})
}
// STATE MUTATORS
// --------------------------------------------------------------------------------------------
/// Inserts a value at the specified key, returning the previous value associated with that key.
/// Recall that by definition, any key that hasn't been updated is associated with
/// [`Self::EMPTY_VALUE`].
///
/// This also recomputes all hashes between the leaf (associated with the key) and the root,
/// updating the root itself.
pub fn insert(&mut self, key: RpoDigest, value: Word) -> Word {
<Self as SparseMerkleTree<SMT_DEPTH>>::insert(self, key, value)
}
// HELPERS
// --------------------------------------------------------------------------------------------
/// Inserts `value` at leaf index pointed to by `key`. `value` is guaranteed to not be the empty
/// value, such that this is indeed an insertion.
fn perform_insert(&mut self, key: RpoDigest, value: Word) -> Option<Word> {
debug_assert_ne!(value, Self::EMPTY_VALUE);
let leaf_index: LeafIndex<SMT_DEPTH> = Self::key_to_leaf_index(&key);
match self.leaves.get_mut(&leaf_index.value()) {
Some(leaf) => leaf.insert(key, value),
None => {
self.leaves.insert(leaf_index.value(), SmtLeaf::Single((key, value)));
None
}
}
}
/// Removes key-value pair at leaf index pointed to by `key` if it exists.
fn perform_remove(&mut self, key: RpoDigest) -> Option<Word> {
let leaf_index: LeafIndex<SMT_DEPTH> = Self::key_to_leaf_index(&key);
if let Some(leaf) = self.leaves.get_mut(&leaf_index.value()) {
let (old_value, is_empty) = leaf.remove(key);
if is_empty {
self.leaves.remove(&leaf_index.value());
}
old_value
} else {
// there's nothing stored at the leaf; nothing to update
None
}
}
}
impl SparseMerkleTree<SMT_DEPTH> for Smt {
type Key = RpoDigest;
type Value = Word;
type Leaf = SmtLeaf;
type Opening = SmtProof;
const EMPTY_VALUE: Self::Value = EMPTY_WORD;
fn root(&self) -> RpoDigest {
self.root
}
fn set_root(&mut self, root: RpoDigest) {
self.root = root;
}
fn get_inner_node(&self, index: NodeIndex) -> InnerNode {
self.inner_nodes.get(&index).cloned().unwrap_or_else(|| {
let node = EmptySubtreeRoots::entry(SMT_DEPTH, index.depth() + 1);
InnerNode { left: *node, right: *node }
})
}
fn insert_inner_node(&mut self, index: NodeIndex, inner_node: InnerNode) {
self.inner_nodes.insert(index, inner_node);
}
fn remove_inner_node(&mut self, index: NodeIndex) {
let _ = self.inner_nodes.remove(&index);
}
fn insert_value(&mut self, key: Self::Key, value: Self::Value) -> Option<Self::Value> {
// inserting an `EMPTY_VALUE` is equivalent to removing any value associated with `key`
if value != Self::EMPTY_VALUE {
self.perform_insert(key, value)
} else {
self.perform_remove(key)
}
}
fn get_leaf(&self, key: &RpoDigest) -> Self::Leaf {
let leaf_pos = LeafIndex::<SMT_DEPTH>::from(*key).value();
match self.leaves.get(&leaf_pos) {
Some(leaf) => leaf.clone(),
None => SmtLeaf::new_empty(key.into()),
}
}
fn hash_leaf(leaf: &Self::Leaf) -> RpoDigest {
leaf.hash()
}
fn key_to_leaf_index(key: &RpoDigest) -> LeafIndex<SMT_DEPTH> {
let most_significant_felt = key[3];
LeafIndex::new_max_depth(most_significant_felt.as_int())
}
fn path_and_leaf_to_opening(path: MerklePath, leaf: SmtLeaf) -> SmtProof {
SmtProof::new_unchecked(path, leaf)
}
}
impl Default for Smt {
fn default() -> Self {
Self::new()
}
}
// CONVERSIONS
// ================================================================================================
impl From<Word> for LeafIndex<SMT_DEPTH> {
fn from(value: Word) -> Self {
// We use the most significant `Felt` of a `Word` as the leaf index.
Self::new_max_depth(value[3].as_int())
}
}
impl From<RpoDigest> for LeafIndex<SMT_DEPTH> {
fn from(value: RpoDigest) -> Self {
Word::from(value).into()
}
}
impl From<&RpoDigest> for LeafIndex<SMT_DEPTH> {
fn from(value: &RpoDigest) -> Self {
Word::from(value).into()
}
}

View File

@@ -0,0 +1,114 @@
use crate::utils::string::ToString;
use winter_utils::{ByteReader, ByteWriter, Deserializable, DeserializationError, Serializable};
use super::{MerklePath, RpoDigest, SmtLeaf, SmtProofError, Word, SMT_DEPTH};
/// A proof which can be used to assert membership (or non-membership) of key-value pairs in a
/// [`super::Smt`].
///
/// The proof consists of a Merkle path and leaf which describes the node located at the base of the
/// path.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct SmtProof {
path: MerklePath,
leaf: SmtLeaf,
}
impl SmtProof {
// CONSTRUCTOR
// --------------------------------------------------------------------------------------------
/// Returns a new instance of [`SmtProof`] instantiated from the specified path and leaf.
///
/// # Errors
/// Returns an error if the path length is not [`SMT_DEPTH`].
pub fn new(path: MerklePath, leaf: SmtLeaf) -> Result<Self, SmtProofError> {
if path.len() != SMT_DEPTH.into() {
return Err(SmtProofError::InvalidPathLength(path.len()));
}
Ok(Self { path, leaf })
}
/// Returns a new instance of [`SmtProof`] instantiated from the specified path and leaf.
///
/// The length of the path is not checked. Reserved for internal use.
pub(super) fn new_unchecked(path: MerklePath, leaf: SmtLeaf) -> Self {
Self { path, leaf }
}
// PROOF VERIFIER
// --------------------------------------------------------------------------------------------
/// Returns true if a [`super::Smt`] with the specified root contains the provided
/// key-value pair.
///
/// Note: this method cannot be used to assert non-membership. That is, if false is returned,
/// it does not mean that the provided key-value pair is not in the tree.
pub fn verify_membership(&self, key: &RpoDigest, value: &Word, root: &RpoDigest) -> bool {
let maybe_value_in_leaf = self.leaf.get_value(key);
match maybe_value_in_leaf {
Some(value_in_leaf) => {
// The value must match for the proof to be valid
if value_in_leaf != *value {
return false;
}
// make sure the Merkle path resolves to the correct root
self.compute_root() == *root
}
// If the key maps to a different leaf, the proof cannot verify membership of `value`
None => false,
}
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the value associated with the specific key according to this proof, or None if
/// this proof does not contain a value for the specified key.
///
/// A key-value pair generated by using this method should pass the `verify_membership()` check.
pub fn get(&self, key: &RpoDigest) -> Option<Word> {
self.leaf.get_value(key)
}
/// Computes the root of a [`super::Smt`] to which this proof resolves.
pub fn compute_root(&self) -> RpoDigest {
self.path
.compute_root(self.leaf.index().value(), self.leaf.hash())
.expect("failed to compute Merkle path root")
}
/// Returns the proof's Merkle path.
pub fn path(&self) -> &MerklePath {
&self.path
}
/// Returns the leaf associated with the proof.
pub fn leaf(&self) -> &SmtLeaf {
&self.leaf
}
/// Consume the proof and returns its parts.
pub fn into_parts(self) -> (MerklePath, SmtLeaf) {
(self.path, self.leaf)
}
}
impl Serializable for SmtProof {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
self.path.write_into(target);
self.leaf.write_into(target);
}
}
impl Deserializable for SmtProof {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let path = MerklePath::read_from(source)?;
let leaf = SmtLeaf::read_from(source)?;
Self::new(path, leaf).map_err(|err| DeserializationError::InvalidValue(err.to_string()))
}
}

View File

@@ -0,0 +1,408 @@
use winter_utils::{Deserializable, Serializable};
use super::*;
use crate::{
merkle::{EmptySubtreeRoots, MerkleStore},
utils::collections::Vec,
ONE, WORD_SIZE,
};
// SMT
// --------------------------------------------------------------------------------------------
/// This test checks that inserting twice at the same key functions as expected. The test covers
/// only the case where the key is alone in its leaf
#[test]
fn test_smt_insert_at_same_key() {
let mut smt = Smt::default();
let mut store: MerkleStore = MerkleStore::default();
assert_eq!(smt.root(), *EmptySubtreeRoots::entry(SMT_DEPTH, 0));
let key_1: RpoDigest = {
let raw = 0b_01101001_01101100_00011111_11111111_10010110_10010011_11100000_00000000_u64;
RpoDigest::from([ONE, ONE, ONE, Felt::new(raw)])
};
let key_1_index: NodeIndex = LeafIndex::<SMT_DEPTH>::from(key_1).into();
let value_1 = [ONE; WORD_SIZE];
let value_2 = [ONE + ONE; WORD_SIZE];
// Insert value 1 and ensure root is as expected
{
let leaf_node = build_empty_or_single_leaf_node(key_1, value_1);
let tree_root = store.set_node(smt.root(), key_1_index, leaf_node).unwrap().root;
let old_value_1 = smt.insert(key_1, value_1);
assert_eq!(old_value_1, EMPTY_WORD);
assert_eq!(smt.root(), tree_root);
}
// Insert value 2 and ensure root is as expected
{
let leaf_node = build_empty_or_single_leaf_node(key_1, value_2);
let tree_root = store.set_node(smt.root(), key_1_index, leaf_node).unwrap().root;
let old_value_2 = smt.insert(key_1, value_2);
assert_eq!(old_value_2, value_1);
assert_eq!(smt.root(), tree_root);
}
}
/// This test checks that inserting twice at the same key functions as expected. The test covers
/// only the case where the leaf type is `SmtLeaf::Multiple`
#[test]
fn test_smt_insert_at_same_key_2() {
// The most significant u64 used for both keys (to ensure they map to the same leaf)
let key_msb: u64 = 42;
let key_already_present: RpoDigest =
RpoDigest::from([2_u32.into(), 2_u32.into(), 2_u32.into(), Felt::new(key_msb)]);
let key_already_present_index: NodeIndex =
LeafIndex::<SMT_DEPTH>::from(key_already_present).into();
let value_already_present = [ONE + ONE + ONE; WORD_SIZE];
let mut smt =
Smt::with_entries(core::iter::once((key_already_present, value_already_present))).unwrap();
let mut store: MerkleStore = {
let mut store = MerkleStore::default();
let leaf_node = build_empty_or_single_leaf_node(key_already_present, value_already_present);
store
.set_node(*EmptySubtreeRoots::entry(SMT_DEPTH, 0), key_already_present_index, leaf_node)
.unwrap();
store
};
let key_1: RpoDigest = RpoDigest::from([ONE, ONE, ONE, Felt::new(key_msb)]);
let key_1_index: NodeIndex = LeafIndex::<SMT_DEPTH>::from(key_1).into();
assert_eq!(key_1_index, key_already_present_index);
let value_1 = [ONE; WORD_SIZE];
let value_2 = [ONE + ONE; WORD_SIZE];
// Insert value 1 and ensure root is as expected
{
// Note: key_1 comes first because it is smaller
let leaf_node = build_multiple_leaf_node(&[
(key_1, value_1),
(key_already_present, value_already_present),
]);
let tree_root = store.set_node(smt.root(), key_1_index, leaf_node).unwrap().root;
let old_value_1 = smt.insert(key_1, value_1);
assert_eq!(old_value_1, EMPTY_WORD);
assert_eq!(smt.root(), tree_root);
}
// Insert value 2 and ensure root is as expected
{
let leaf_node = build_multiple_leaf_node(&[
(key_1, value_2),
(key_already_present, value_already_present),
]);
let tree_root = store.set_node(smt.root(), key_1_index, leaf_node).unwrap().root;
let old_value_2 = smt.insert(key_1, value_2);
assert_eq!(old_value_2, value_1);
assert_eq!(smt.root(), tree_root);
}
}
/// This test ensures that the root of the tree is as expected when we add/remove 3 items at 3
/// different keys. This also tests that the merkle paths produced are as expected.
#[test]
fn test_smt_insert_and_remove_multiple_values() {
fn insert_values_and_assert_path(
smt: &mut Smt,
store: &mut MerkleStore,
key_values: &[(RpoDigest, Word)],
) {
for &(key, value) in key_values {
let key_index: NodeIndex = LeafIndex::<SMT_DEPTH>::from(key).into();
let leaf_node = build_empty_or_single_leaf_node(key, value);
let tree_root = store.set_node(smt.root(), key_index, leaf_node).unwrap().root;
let _ = smt.insert(key, value);
assert_eq!(smt.root(), tree_root);
let expected_path = store.get_path(tree_root, key_index).unwrap();
assert_eq!(smt.open(&key).into_parts().0, expected_path.path);
}
}
let mut smt = Smt::default();
let mut store: MerkleStore = MerkleStore::default();
assert_eq!(smt.root(), *EmptySubtreeRoots::entry(SMT_DEPTH, 0));
let key_1: RpoDigest = {
let raw = 0b_01101001_01101100_00011111_11111111_10010110_10010011_11100000_00000000_u64;
RpoDigest::from([ONE, ONE, ONE, Felt::new(raw)])
};
let key_2: RpoDigest = {
let raw = 0b_11111111_11111111_11111111_11111111_11111111_11111111_11111111_11111111_u64;
RpoDigest::from([ONE, ONE, ONE, Felt::new(raw)])
};
let key_3: RpoDigest = {
let raw = 0b_00000000_00000000_00000000_00000000_00000000_00000000_00000000_00000000_u64;
RpoDigest::from([ONE, ONE, ONE, Felt::new(raw)])
};
let value_1 = [ONE; WORD_SIZE];
let value_2 = [ONE + ONE; WORD_SIZE];
let value_3 = [ONE + ONE + ONE; WORD_SIZE];
// Insert values in the tree
let key_values = [(key_1, value_1), (key_2, value_2), (key_3, value_3)];
insert_values_and_assert_path(&mut smt, &mut store, &key_values);
// Remove values from the tree
let key_empty_values = [(key_1, EMPTY_WORD), (key_2, EMPTY_WORD), (key_3, EMPTY_WORD)];
insert_values_and_assert_path(&mut smt, &mut store, &key_empty_values);
let empty_root = *EmptySubtreeRoots::entry(SMT_DEPTH, 0);
assert_eq!(smt.root(), empty_root);
// an empty tree should have no leaves or inner nodes
assert!(smt.leaves.is_empty());
assert!(smt.inner_nodes.is_empty());
}
/// This tests that inserting the empty value does indeed remove the key-value contained at the
/// leaf. We insert & remove 3 values at the same leaf to ensure that all cases are covered (empty,
/// single, multiple).
#[test]
fn test_smt_removal() {
let mut smt = Smt::default();
let raw = 0b_01101001_01101100_00011111_11111111_10010110_10010011_11100000_00000000_u64;
let key_1: RpoDigest = RpoDigest::from([ONE, ONE, ONE, Felt::new(raw)]);
let key_2: RpoDigest =
RpoDigest::from([2_u32.into(), 2_u32.into(), 2_u32.into(), Felt::new(raw)]);
let key_3: RpoDigest =
RpoDigest::from([3_u32.into(), 3_u32.into(), 3_u32.into(), Felt::new(raw)]);
let value_1 = [ONE; WORD_SIZE];
let value_2 = [2_u32.into(); WORD_SIZE];
let value_3: [Felt; 4] = [3_u32.into(); WORD_SIZE];
// insert key-value 1
{
let old_value_1 = smt.insert(key_1, value_1);
assert_eq!(old_value_1, EMPTY_WORD);
assert_eq!(smt.get_leaf(&key_1), SmtLeaf::Single((key_1, value_1)));
}
// insert key-value 2
{
let old_value_2 = smt.insert(key_2, value_2);
assert_eq!(old_value_2, EMPTY_WORD);
assert_eq!(
smt.get_leaf(&key_2),
SmtLeaf::Multiple(vec![(key_1, value_1), (key_2, value_2)])
);
}
// insert key-value 3
{
let old_value_3 = smt.insert(key_3, value_3);
assert_eq!(old_value_3, EMPTY_WORD);
assert_eq!(
smt.get_leaf(&key_3),
SmtLeaf::Multiple(vec![(key_1, value_1), (key_2, value_2), (key_3, value_3)])
);
}
// remove key 3
{
let old_value_3 = smt.insert(key_3, EMPTY_WORD);
assert_eq!(old_value_3, value_3);
assert_eq!(
smt.get_leaf(&key_3),
SmtLeaf::Multiple(vec![(key_1, value_1), (key_2, value_2)])
);
}
// remove key 2
{
let old_value_2 = smt.insert(key_2, EMPTY_WORD);
assert_eq!(old_value_2, value_2);
assert_eq!(smt.get_leaf(&key_2), SmtLeaf::Single((key_1, value_1)));
}
// remove key 1
{
let old_value_1 = smt.insert(key_1, EMPTY_WORD);
assert_eq!(old_value_1, value_1);
assert_eq!(smt.get_leaf(&key_1), SmtLeaf::new_empty(key_1.into()));
}
}
/// Tests that 2 key-value pairs stored in the same leaf have the same path
#[test]
fn test_smt_path_to_keys_in_same_leaf_are_equal() {
let raw = 0b_01101001_01101100_00011111_11111111_10010110_10010011_11100000_00000000_u64;
let key_1: RpoDigest = RpoDigest::from([ONE, ONE, ONE, Felt::new(raw)]);
let key_2: RpoDigest =
RpoDigest::from([2_u32.into(), 2_u32.into(), 2_u32.into(), Felt::new(raw)]);
let value_1 = [ONE; WORD_SIZE];
let value_2 = [2_u32.into(); WORD_SIZE];
let smt = Smt::with_entries([(key_1, value_1), (key_2, value_2)]).unwrap();
assert_eq!(smt.open(&key_1), smt.open(&key_2));
}
/// Tests that an empty leaf hashes to the empty word
#[test]
fn test_empty_leaf_hash() {
let smt = Smt::default();
let leaf = smt.get_leaf(&RpoDigest::default());
assert_eq!(leaf.hash(), EMPTY_WORD.into());
}
/// Tests that `get_value()` works as expected
#[test]
fn test_smt_get_value() {
let key_1: RpoDigest = RpoDigest::from([ONE, ONE, ONE, ONE]);
let key_2: RpoDigest =
RpoDigest::from([2_u32.into(), 2_u32.into(), 2_u32.into(), 2_u32.into()]);
let value_1 = [ONE; WORD_SIZE];
let value_2 = [2_u32.into(); WORD_SIZE];
let smt = Smt::with_entries([(key_1, value_1), (key_2, value_2)]).unwrap();
let returned_value_1 = smt.get_value(&key_1);
let returned_value_2 = smt.get_value(&key_2);
assert_eq!(value_1, returned_value_1);
assert_eq!(value_2, returned_value_2);
// Check that a key with no inserted value returns the empty word
let key_no_value =
RpoDigest::from([42_u32.into(), 42_u32.into(), 42_u32.into(), 42_u32.into()]);
assert_eq!(EMPTY_WORD, smt.get_value(&key_no_value));
}
/// Tests that `entries()` works as expected
#[test]
fn test_smt_entries() {
let key_1: RpoDigest = RpoDigest::from([ONE, ONE, ONE, ONE]);
let key_2: RpoDigest =
RpoDigest::from([2_u32.into(), 2_u32.into(), 2_u32.into(), 2_u32.into()]);
let value_1 = [ONE; WORD_SIZE];
let value_2 = [2_u32.into(); WORD_SIZE];
let smt = Smt::with_entries([(key_1, value_1), (key_2, value_2)]).unwrap();
let mut entries = smt.entries();
// Note: for simplicity, we assume the order `(k1,v1), (k2,v2)`. If a new implementation
// switches the order, it is OK to modify the order here as well.
assert_eq!(&(key_1, value_1), entries.next().unwrap());
assert_eq!(&(key_2, value_2), entries.next().unwrap());
assert!(entries.next().is_none());
}
// SMT LEAF
// --------------------------------------------------------------------------------------------
#[test]
fn test_empty_smt_leaf_serialization() {
let empty_leaf = SmtLeaf::new_empty(LeafIndex::new_max_depth(42));
let mut serialized = empty_leaf.to_bytes();
// extend buffer with random bytes
serialized.extend([1, 2, 3, 4, 5]);
let deserialized = SmtLeaf::read_from_bytes(&serialized).unwrap();
assert_eq!(empty_leaf, deserialized);
}
#[test]
fn test_single_smt_leaf_serialization() {
let single_leaf = SmtLeaf::new_single(
RpoDigest::from([10_u32.into(), 11_u32.into(), 12_u32.into(), 13_u32.into()]),
[1_u32.into(), 2_u32.into(), 3_u32.into(), 4_u32.into()],
);
let mut serialized = single_leaf.to_bytes();
// extend buffer with random bytes
serialized.extend([1, 2, 3, 4, 5]);
let deserialized = SmtLeaf::read_from_bytes(&serialized).unwrap();
assert_eq!(single_leaf, deserialized);
}
#[test]
fn test_multiple_smt_leaf_serialization_success() {
let multiple_leaf = SmtLeaf::new_multiple(vec![
(
RpoDigest::from([10_u32.into(), 11_u32.into(), 12_u32.into(), 13_u32.into()]),
[1_u32.into(), 2_u32.into(), 3_u32.into(), 4_u32.into()],
),
(
RpoDigest::from([100_u32.into(), 101_u32.into(), 102_u32.into(), 13_u32.into()]),
[11_u32.into(), 12_u32.into(), 13_u32.into(), 14_u32.into()],
),
])
.unwrap();
let mut serialized = multiple_leaf.to_bytes();
// extend buffer with random bytes
serialized.extend([1, 2, 3, 4, 5]);
let deserialized = SmtLeaf::read_from_bytes(&serialized).unwrap();
assert_eq!(multiple_leaf, deserialized);
}
// HELPERS
// --------------------------------------------------------------------------------------------
fn build_empty_or_single_leaf_node(key: RpoDigest, value: Word) -> RpoDigest {
if value == EMPTY_WORD {
SmtLeaf::new_empty(key.into()).hash()
} else {
SmtLeaf::Single((key, value)).hash()
}
}
fn build_multiple_leaf_node(kv_pairs: &[(RpoDigest, Word)]) -> RpoDigest {
let elements: Vec<Felt> = kv_pairs
.iter()
.flat_map(|(key, value)| {
let key_elements = key.into_iter();
let value_elements = (*value).into_iter();
key_elements.chain(value_elements)
})
.collect();
Rpo256::hash_elements(&elements)
}

245
src/merkle/smt/mod.rs Normal file
View File

@@ -0,0 +1,245 @@
use crate::{
hash::rpo::{Rpo256, RpoDigest},
Word,
};
use super::{EmptySubtreeRoots, MerkleError, MerklePath, NodeIndex, Vec};
mod full;
pub use full::{Smt, SmtLeaf, SmtLeafError, SmtProof, SmtProofError, SMT_DEPTH};
mod simple;
pub use simple::SimpleSmt;
// CONSTANTS
// ================================================================================================
/// Minimum supported depth.
pub const SMT_MIN_DEPTH: u8 = 1;
/// Maximum supported depth.
pub const SMT_MAX_DEPTH: u8 = 64;
// SPARSE MERKLE TREE
// ================================================================================================
/// An abstract description of a sparse Merkle tree.
///
/// A sparse Merkle tree is a key-value map which also supports proving that a given value is indeed
/// stored at a given key in the tree. It is viewed as always being fully populated. If a leaf's
/// value was not explicitly set, then its value is the default value. Typically, the vast majority
/// of leaves will store the default value (hence it is "sparse"), and therefore the internal
/// representation of the tree will only keep track of the leaves that have a different value from
/// the default.
///
/// All leaves sit at the same depth. The deeper the tree, the more leaves it has; but also the
/// longer its proofs are - of exactly `log(depth)` size. A tree cannot have depth 0, since such a
/// tree is just a single value, and is probably a programming mistake.
///
/// Every key maps to one leaf. If there are as many keys as there are leaves, then
/// [Self::Leaf] should be the same type as [Self::Value], as is the case with
/// [crate::merkle::SimpleSmt]. However, if there are more keys than leaves, then [`Self::Leaf`]
/// must accomodate all keys that map to the same leaf.
///
/// [SparseMerkleTree] currently doesn't support optimizations that compress Merkle proofs.
pub(crate) trait SparseMerkleTree<const DEPTH: u8> {
/// The type for a key
type Key: Clone;
/// The type for a value
type Value: Clone + PartialEq;
/// The type for a leaf
type Leaf;
/// The type for an opening (i.e. a "proof") of a leaf
type Opening;
/// The default value used to compute the hash of empty leaves
const EMPTY_VALUE: Self::Value;
// PROVIDED METHODS
// ---------------------------------------------------------------------------------------------
/// Returns an opening of the leaf associated with `key`. Conceptually, an opening is a Merkle
/// path to the leaf, as well as the leaf itself.
fn open(&self, key: &Self::Key) -> Self::Opening {
let leaf = self.get_leaf(key);
let mut index: NodeIndex = {
let leaf_index: LeafIndex<DEPTH> = Self::key_to_leaf_index(key);
leaf_index.into()
};
let merkle_path = {
let mut path = Vec::with_capacity(index.depth() as usize);
for _ in 0..index.depth() {
let is_right = index.is_value_odd();
index.move_up();
let InnerNode { left, right } = self.get_inner_node(index);
let value = if is_right { left } else { right };
path.push(value);
}
MerklePath::new(path)
};
Self::path_and_leaf_to_opening(merkle_path, leaf)
}
/// Inserts a value at the specified key, returning the previous value associated with that key.
/// Recall that by definition, any key that hasn't been updated is associated with
/// [`Self::EMPTY_VALUE`].
///
/// This also recomputes all hashes between the leaf (associated with the key) and the root,
/// updating the root itself.
fn insert(&mut self, key: Self::Key, value: Self::Value) -> Self::Value {
let old_value = self.insert_value(key.clone(), value.clone()).unwrap_or(Self::EMPTY_VALUE);
// if the old value and new value are the same, there is nothing to update
if value == old_value {
return value;
}
let leaf = self.get_leaf(&key);
let node_index = {
let leaf_index: LeafIndex<DEPTH> = Self::key_to_leaf_index(&key);
leaf_index.into()
};
self.recompute_nodes_from_index_to_root(node_index, Self::hash_leaf(&leaf));
old_value
}
/// Recomputes the branch nodes (including the root) from `index` all the way to the root.
/// `node_hash_at_index` is the hash of the node stored at index.
fn recompute_nodes_from_index_to_root(
&mut self,
mut index: NodeIndex,
node_hash_at_index: RpoDigest,
) {
let mut node_hash = node_hash_at_index;
for node_depth in (0..index.depth()).rev() {
let is_right = index.is_value_odd();
index.move_up();
let InnerNode { left, right } = self.get_inner_node(index);
let (left, right) = if is_right {
(left, node_hash)
} else {
(node_hash, right)
};
node_hash = Rpo256::merge(&[left, right]);
if node_hash == *EmptySubtreeRoots::entry(DEPTH, node_depth) {
// If a subtree is empty, when can remove the inner node, since it's equal to the
// default value
self.remove_inner_node(index)
} else {
self.insert_inner_node(index, InnerNode { left, right });
}
}
self.set_root(node_hash);
}
// REQUIRED METHODS
// ---------------------------------------------------------------------------------------------
/// The root of the tree
fn root(&self) -> RpoDigest;
/// Sets the root of the tree
fn set_root(&mut self, root: RpoDigest);
/// Retrieves an inner node at the given index
fn get_inner_node(&self, index: NodeIndex) -> InnerNode;
/// Inserts an inner node at the given index
fn insert_inner_node(&mut self, index: NodeIndex, inner_node: InnerNode);
/// Removes an inner node at the given index
fn remove_inner_node(&mut self, index: NodeIndex);
/// Inserts a leaf node, and returns the value at the key if already exists
fn insert_value(&mut self, key: Self::Key, value: Self::Value) -> Option<Self::Value>;
/// Returns the leaf at the specified index.
fn get_leaf(&self, key: &Self::Key) -> Self::Leaf;
/// Returns the hash of a leaf
fn hash_leaf(leaf: &Self::Leaf) -> RpoDigest;
/// Maps a key to a leaf index
fn key_to_leaf_index(key: &Self::Key) -> LeafIndex<DEPTH>;
/// Maps a (MerklePath, Self::Leaf) to an opening.
///
/// The length `path` is guaranteed to be equal to `DEPTH`
fn path_and_leaf_to_opening(path: MerklePath, leaf: Self::Leaf) -> Self::Opening;
}
// INNER NODE
// ================================================================================================
#[derive(Debug, Default, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub(crate) struct InnerNode {
pub left: RpoDigest,
pub right: RpoDigest,
}
impl InnerNode {
pub fn hash(&self) -> RpoDigest {
Rpo256::merge(&[self.left, self.right])
}
}
// LEAF INDEX
// ================================================================================================
/// The index of a leaf, at a depth known at compile-time.
#[derive(Debug, Default, Copy, Clone, Eq, PartialEq, PartialOrd, Ord, Hash)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct LeafIndex<const DEPTH: u8> {
index: NodeIndex,
}
impl<const DEPTH: u8> LeafIndex<DEPTH> {
pub fn new(value: u64) -> Result<Self, MerkleError> {
if DEPTH < SMT_MIN_DEPTH {
return Err(MerkleError::DepthTooSmall(DEPTH));
}
Ok(LeafIndex { index: NodeIndex::new(DEPTH, value)? })
}
pub fn value(&self) -> u64 {
self.index.value()
}
}
impl LeafIndex<SMT_MAX_DEPTH> {
pub const fn new_max_depth(value: u64) -> Self {
LeafIndex {
index: NodeIndex::new_unchecked(SMT_MAX_DEPTH, value),
}
}
}
impl<const DEPTH: u8> From<LeafIndex<DEPTH>> for NodeIndex {
fn from(value: LeafIndex<DEPTH>) -> Self {
value.index
}
}
impl<const DEPTH: u8> TryFrom<NodeIndex> for LeafIndex<DEPTH> {
type Error = MerkleError;
fn try_from(node_index: NodeIndex) -> Result<Self, Self::Error> {
if node_index.depth() != DEPTH {
return Err(MerkleError::InvalidDepth {
expected: DEPTH,
provided: node_index.depth(),
});
}
Self::new(node_index.value())
}
}

View File

@@ -0,0 +1,309 @@
use crate::{
merkle::{EmptySubtreeRoots, InnerNodeInfo, MerklePath, ValuePath},
EMPTY_WORD,
};
use super::{
InnerNode, LeafIndex, MerkleError, NodeIndex, RpoDigest, SparseMerkleTree, Word, SMT_MAX_DEPTH,
SMT_MIN_DEPTH,
};
use crate::utils::collections::{BTreeMap, BTreeSet};
#[cfg(test)]
mod tests;
// SPARSE MERKLE TREE
// ================================================================================================
/// A sparse Merkle tree with 64-bit keys and 4-element leaf values, without compaction.
///
/// The root of the tree is recomputed on each new leaf update.
#[derive(Debug, Clone, PartialEq, Eq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct SimpleSmt<const DEPTH: u8> {
root: RpoDigest,
leaves: BTreeMap<u64, Word>,
inner_nodes: BTreeMap<NodeIndex, InnerNode>,
}
impl<const DEPTH: u8> SimpleSmt<DEPTH> {
// CONSTANTS
// --------------------------------------------------------------------------------------------
/// The default value used to compute the hash of empty leaves
pub const EMPTY_VALUE: Word = <Self as SparseMerkleTree<DEPTH>>::EMPTY_VALUE;
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Returns a new [SimpleSmt].
///
/// All leaves in the returned tree are set to [ZERO; 4].
///
/// # Errors
/// Returns an error if DEPTH is 0 or is greater than 64.
pub fn new() -> Result<Self, MerkleError> {
// validate the range of the depth.
if DEPTH < SMT_MIN_DEPTH {
return Err(MerkleError::DepthTooSmall(DEPTH));
} else if SMT_MAX_DEPTH < DEPTH {
return Err(MerkleError::DepthTooBig(DEPTH as u64));
}
let root = *EmptySubtreeRoots::entry(DEPTH, 0);
Ok(Self {
root,
leaves: BTreeMap::new(),
inner_nodes: BTreeMap::new(),
})
}
/// Returns a new [SimpleSmt] instantiated with leaves set as specified by the provided entries.
///
/// All leaves omitted from the entries list are set to [ZERO; 4].
///
/// # Errors
/// Returns an error if:
/// - If the depth is 0 or is greater than 64.
/// - The number of entries exceeds the maximum tree capacity, that is 2^{depth}.
/// - The provided entries contain multiple values for the same key.
pub fn with_leaves(
entries: impl IntoIterator<Item = (u64, Word)>,
) -> Result<Self, MerkleError> {
// create an empty tree
let mut tree = Self::new()?;
// compute the max number of entries. We use an upper bound of depth 63 because we consider
// passing in a vector of size 2^64 infeasible.
let max_num_entries = 2_usize.pow(DEPTH.min(63).into());
// This being a sparse data structure, the EMPTY_WORD is not assigned to the `BTreeMap`, so
// entries with the empty value need additional tracking.
let mut key_set_to_zero = BTreeSet::new();
for (idx, (key, value)) in entries.into_iter().enumerate() {
if idx >= max_num_entries {
return Err(MerkleError::InvalidNumEntries(max_num_entries));
}
let old_value = tree.insert(LeafIndex::<DEPTH>::new(key)?, value);
if old_value != Self::EMPTY_VALUE || key_set_to_zero.contains(&key) {
return Err(MerkleError::DuplicateValuesForIndex(key));
}
if value == Self::EMPTY_VALUE {
key_set_to_zero.insert(key);
};
}
Ok(tree)
}
/// Wrapper around [`SimpleSmt::with_leaves`] which inserts leaves at contiguous indices
/// starting at index 0.
pub fn with_contiguous_leaves(
entries: impl IntoIterator<Item = Word>,
) -> Result<Self, MerkleError> {
Self::with_leaves(
entries
.into_iter()
.enumerate()
.map(|(idx, word)| (idx.try_into().expect("tree max depth is 2^8"), word)),
)
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns the depth of the tree
pub const fn depth(&self) -> u8 {
DEPTH
}
/// Returns the root of the tree
pub fn root(&self) -> RpoDigest {
<Self as SparseMerkleTree<DEPTH>>::root(self)
}
/// Returns the leaf at the specified index.
pub fn get_leaf(&self, key: &LeafIndex<DEPTH>) -> Word {
<Self as SparseMerkleTree<DEPTH>>::get_leaf(self, key)
}
/// Returns a node at the specified index.
///
/// # Errors
/// Returns an error if the specified index has depth set to 0 or the depth is greater than
/// the depth of this Merkle tree.
pub fn get_node(&self, index: NodeIndex) -> Result<RpoDigest, MerkleError> {
if index.is_root() {
Err(MerkleError::DepthTooSmall(index.depth()))
} else if index.depth() > DEPTH {
Err(MerkleError::DepthTooBig(index.depth() as u64))
} else if index.depth() == DEPTH {
let leaf = self.get_leaf(&LeafIndex::<DEPTH>::try_from(index)?);
Ok(leaf.into())
} else {
Ok(self.get_inner_node(index).hash())
}
}
/// Returns an opening of the leaf associated with `key`. Conceptually, an opening is a Merkle
/// path to the leaf, as well as the leaf itself.
pub fn open(&self, key: &LeafIndex<DEPTH>) -> ValuePath {
<Self as SparseMerkleTree<DEPTH>>::open(self, key)
}
// ITERATORS
// --------------------------------------------------------------------------------------------
/// Returns an iterator over the leaves of this [SimpleSmt].
pub fn leaves(&self) -> impl Iterator<Item = (u64, &Word)> {
self.leaves.iter().map(|(i, w)| (*i, w))
}
/// Returns an iterator over the inner nodes of this [SimpleSmt].
pub fn inner_nodes(&self) -> impl Iterator<Item = InnerNodeInfo> + '_ {
self.inner_nodes.values().map(|e| InnerNodeInfo {
value: e.hash(),
left: e.left,
right: e.right,
})
}
// STATE MUTATORS
// --------------------------------------------------------------------------------------------
/// Inserts a value at the specified key, returning the previous value associated with that key.
/// Recall that by definition, any key that hasn't been updated is associated with
/// [`EMPTY_WORD`].
///
/// This also recomputes all hashes between the leaf (associated with the key) and the root,
/// updating the root itself.
pub fn insert(&mut self, key: LeafIndex<DEPTH>, value: Word) -> Word {
<Self as SparseMerkleTree<DEPTH>>::insert(self, key, value)
}
/// Inserts a subtree at the specified index. The depth at which the subtree is inserted is
/// computed as `DEPTH - SUBTREE_DEPTH`.
///
/// Returns the new root.
pub fn set_subtree<const SUBTREE_DEPTH: u8>(
&mut self,
subtree_insertion_index: u64,
subtree: SimpleSmt<SUBTREE_DEPTH>,
) -> Result<RpoDigest, MerkleError> {
if SUBTREE_DEPTH > DEPTH {
return Err(MerkleError::InvalidSubtreeDepth {
subtree_depth: SUBTREE_DEPTH,
tree_depth: DEPTH,
});
}
// Verify that `subtree_insertion_index` is valid.
let subtree_root_insertion_depth = DEPTH - SUBTREE_DEPTH;
let subtree_root_index =
NodeIndex::new(subtree_root_insertion_depth, subtree_insertion_index)?;
// add leaves
// --------------
// The subtree's leaf indices live in their own context - i.e. a subtree of depth `d`. If we
// insert the subtree at `subtree_insertion_index = 0`, then the subtree leaf indices are
// valid as they are. However, consider what happens when we insert at
// `subtree_insertion_index = 1`. The first leaf of our subtree now will have index `2^d`;
// you can see it as there's a full subtree sitting on its left. In general, for
// `subtree_insertion_index = i`, there are `i` subtrees sitting before the subtree we want
// to insert, so we need to adjust all its leaves by `i * 2^d`.
let leaf_index_shift: u64 = subtree_insertion_index * 2_u64.pow(SUBTREE_DEPTH.into());
for (subtree_leaf_idx, leaf_value) in subtree.leaves() {
let new_leaf_idx = leaf_index_shift + subtree_leaf_idx;
debug_assert!(new_leaf_idx < 2_u64.pow(DEPTH.into()));
self.leaves.insert(new_leaf_idx, *leaf_value);
}
// add subtree's branch nodes (which includes the root)
// --------------
for (branch_idx, branch_node) in subtree.inner_nodes {
let new_branch_idx = {
let new_depth = subtree_root_insertion_depth + branch_idx.depth();
let new_value = subtree_insertion_index * 2_u64.pow(branch_idx.depth().into())
+ branch_idx.value();
NodeIndex::new(new_depth, new_value).expect("index guaranteed to be valid")
};
self.inner_nodes.insert(new_branch_idx, branch_node);
}
// recompute nodes starting from subtree root
// --------------
self.recompute_nodes_from_index_to_root(subtree_root_index, subtree.root);
Ok(self.root)
}
}
impl<const DEPTH: u8> SparseMerkleTree<DEPTH> for SimpleSmt<DEPTH> {
type Key = LeafIndex<DEPTH>;
type Value = Word;
type Leaf = Word;
type Opening = ValuePath;
const EMPTY_VALUE: Self::Value = EMPTY_WORD;
fn root(&self) -> RpoDigest {
self.root
}
fn set_root(&mut self, root: RpoDigest) {
self.root = root;
}
fn get_inner_node(&self, index: NodeIndex) -> InnerNode {
self.inner_nodes.get(&index).cloned().unwrap_or_else(|| {
let node = EmptySubtreeRoots::entry(DEPTH, index.depth() + 1);
InnerNode { left: *node, right: *node }
})
}
fn insert_inner_node(&mut self, index: NodeIndex, inner_node: InnerNode) {
self.inner_nodes.insert(index, inner_node);
}
fn remove_inner_node(&mut self, index: NodeIndex) {
let _ = self.inner_nodes.remove(&index);
}
fn insert_value(&mut self, key: LeafIndex<DEPTH>, value: Word) -> Option<Word> {
self.leaves.insert(key.value(), value)
}
fn get_leaf(&self, key: &LeafIndex<DEPTH>) -> Word {
// the lookup in empty_hashes could fail only if empty_hashes were not built correctly
// by the constructor as we check the depth of the lookup above.
let leaf_pos = key.value();
match self.leaves.get(&leaf_pos) {
Some(word) => *word,
None => Word::from(*EmptySubtreeRoots::entry(DEPTH, DEPTH)),
}
}
fn hash_leaf(leaf: &Word) -> RpoDigest {
// `SimpleSmt` takes the leaf value itself as the hash
leaf.into()
}
fn key_to_leaf_index(key: &LeafIndex<DEPTH>) -> LeafIndex<DEPTH> {
*key
}
fn path_and_leaf_to_opening(path: MerklePath, leaf: Word) -> ValuePath {
(path, leaf).into()
}
}

View File

@@ -0,0 +1,437 @@
use super::{
super::{MerkleError, RpoDigest, SimpleSmt},
NodeIndex,
};
use crate::{
hash::rpo::Rpo256,
merkle::{
digests_to_words, int_to_leaf, int_to_node, smt::SparseMerkleTree, EmptySubtreeRoots,
InnerNodeInfo, LeafIndex, MerkleTree,
},
utils::collections::Vec,
Word, EMPTY_WORD,
};
// TEST DATA
// ================================================================================================
const KEYS4: [u64; 4] = [0, 1, 2, 3];
const KEYS8: [u64; 8] = [0, 1, 2, 3, 4, 5, 6, 7];
const VALUES4: [RpoDigest; 4] = [int_to_node(1), int_to_node(2), int_to_node(3), int_to_node(4)];
const VALUES8: [RpoDigest; 8] = [
int_to_node(1),
int_to_node(2),
int_to_node(3),
int_to_node(4),
int_to_node(5),
int_to_node(6),
int_to_node(7),
int_to_node(8),
];
const ZERO_VALUES8: [Word; 8] = [int_to_leaf(0); 8];
// TESTS
// ================================================================================================
#[test]
fn build_empty_tree() {
// tree of depth 3
let smt = SimpleSmt::<3>::new().unwrap();
let mt = MerkleTree::new(ZERO_VALUES8).unwrap();
assert_eq!(mt.root(), smt.root());
}
#[test]
fn build_sparse_tree() {
const DEPTH: u8 = 3;
let mut smt = SimpleSmt::<DEPTH>::new().unwrap();
let mut values = ZERO_VALUES8.to_vec();
// insert single value
let key = 6;
let new_node = int_to_leaf(7);
values[key as usize] = new_node;
let old_value = smt.insert(LeafIndex::<DEPTH>::new(key).unwrap(), new_node);
let mt2 = MerkleTree::new(values.clone()).unwrap();
assert_eq!(mt2.root(), smt.root());
assert_eq!(
mt2.get_path(NodeIndex::make(3, 6)).unwrap(),
smt.open(&LeafIndex::<3>::new(6).unwrap()).path
);
assert_eq!(old_value, EMPTY_WORD);
// insert second value at distinct leaf branch
let key = 2;
let new_node = int_to_leaf(3);
values[key as usize] = new_node;
let old_value = smt.insert(LeafIndex::<DEPTH>::new(key).unwrap(), new_node);
let mt3 = MerkleTree::new(values).unwrap();
assert_eq!(mt3.root(), smt.root());
assert_eq!(
mt3.get_path(NodeIndex::make(3, 2)).unwrap(),
smt.open(&LeafIndex::<3>::new(2).unwrap()).path
);
assert_eq!(old_value, EMPTY_WORD);
}
/// Tests that [`SimpleSmt::with_contiguous_leaves`] works as expected
#[test]
fn build_contiguous_tree() {
let tree_with_leaves =
SimpleSmt::<2>::with_leaves([0, 1, 2, 3].into_iter().zip(digests_to_words(&VALUES4)))
.unwrap();
let tree_with_contiguous_leaves =
SimpleSmt::<2>::with_contiguous_leaves(digests_to_words(&VALUES4)).unwrap();
assert_eq!(tree_with_leaves, tree_with_contiguous_leaves);
}
#[test]
fn test_depth2_tree() {
let tree =
SimpleSmt::<2>::with_leaves(KEYS4.into_iter().zip(digests_to_words(&VALUES4))).unwrap();
// check internal structure
let (root, node2, node3) = compute_internal_nodes();
assert_eq!(root, tree.root());
assert_eq!(node2, tree.get_node(NodeIndex::make(1, 0)).unwrap());
assert_eq!(node3, tree.get_node(NodeIndex::make(1, 1)).unwrap());
// check get_node()
assert_eq!(VALUES4[0], tree.get_node(NodeIndex::make(2, 0)).unwrap());
assert_eq!(VALUES4[1], tree.get_node(NodeIndex::make(2, 1)).unwrap());
assert_eq!(VALUES4[2], tree.get_node(NodeIndex::make(2, 2)).unwrap());
assert_eq!(VALUES4[3], tree.get_node(NodeIndex::make(2, 3)).unwrap());
// check get_path(): depth 2
assert_eq!(vec![VALUES4[1], node3], *tree.open(&LeafIndex::<2>::new(0).unwrap()).path);
assert_eq!(vec![VALUES4[0], node3], *tree.open(&LeafIndex::<2>::new(1).unwrap()).path);
assert_eq!(vec![VALUES4[3], node2], *tree.open(&LeafIndex::<2>::new(2).unwrap()).path);
assert_eq!(vec![VALUES4[2], node2], *tree.open(&LeafIndex::<2>::new(3).unwrap()).path);
}
#[test]
fn test_inner_node_iterator() -> Result<(), MerkleError> {
let tree =
SimpleSmt::<2>::with_leaves(KEYS4.into_iter().zip(digests_to_words(&VALUES4))).unwrap();
// check depth 2
assert_eq!(VALUES4[0], tree.get_node(NodeIndex::make(2, 0)).unwrap());
assert_eq!(VALUES4[1], tree.get_node(NodeIndex::make(2, 1)).unwrap());
assert_eq!(VALUES4[2], tree.get_node(NodeIndex::make(2, 2)).unwrap());
assert_eq!(VALUES4[3], tree.get_node(NodeIndex::make(2, 3)).unwrap());
// get parent nodes
let root = tree.root();
let l1n0 = tree.get_node(NodeIndex::make(1, 0))?;
let l1n1 = tree.get_node(NodeIndex::make(1, 1))?;
let l2n0 = tree.get_node(NodeIndex::make(2, 0))?;
let l2n1 = tree.get_node(NodeIndex::make(2, 1))?;
let l2n2 = tree.get_node(NodeIndex::make(2, 2))?;
let l2n3 = tree.get_node(NodeIndex::make(2, 3))?;
let nodes: Vec<InnerNodeInfo> = tree.inner_nodes().collect();
let expected = vec![
InnerNodeInfo { value: root, left: l1n0, right: l1n1 },
InnerNodeInfo { value: l1n0, left: l2n0, right: l2n1 },
InnerNodeInfo { value: l1n1, left: l2n2, right: l2n3 },
];
assert_eq!(nodes, expected);
Ok(())
}
#[test]
fn update_leaf() {
const DEPTH: u8 = 3;
let mut tree =
SimpleSmt::<DEPTH>::with_leaves(KEYS8.into_iter().zip(digests_to_words(&VALUES8))).unwrap();
// update one value
let key = 3;
let new_node = int_to_leaf(9);
let mut expected_values = digests_to_words(&VALUES8);
expected_values[key] = new_node;
let expected_tree = MerkleTree::new(expected_values.clone()).unwrap();
let old_leaf = tree.insert(LeafIndex::<DEPTH>::new(key as u64).unwrap(), new_node);
assert_eq!(expected_tree.root(), tree.root);
assert_eq!(old_leaf, *VALUES8[key]);
// update another value
let key = 6;
let new_node = int_to_leaf(10);
expected_values[key] = new_node;
let expected_tree = MerkleTree::new(expected_values.clone()).unwrap();
let old_leaf = tree.insert(LeafIndex::<DEPTH>::new(key as u64).unwrap(), new_node);
assert_eq!(expected_tree.root(), tree.root);
assert_eq!(old_leaf, *VALUES8[key]);
}
#[test]
fn small_tree_opening_is_consistent() {
// ____k____
// / \
// _i_ _j_
// / \ / \
// e f g h
// / \ / \ / \ / \
// a b 0 0 c 0 0 d
let z = EMPTY_WORD;
let a = Word::from(Rpo256::merge(&[z.into(); 2]));
let b = Word::from(Rpo256::merge(&[a.into(); 2]));
let c = Word::from(Rpo256::merge(&[b.into(); 2]));
let d = Word::from(Rpo256::merge(&[c.into(); 2]));
let e = Rpo256::merge(&[a.into(), b.into()]);
let f = Rpo256::merge(&[z.into(), z.into()]);
let g = Rpo256::merge(&[c.into(), z.into()]);
let h = Rpo256::merge(&[z.into(), d.into()]);
let i = Rpo256::merge(&[e, f]);
let j = Rpo256::merge(&[g, h]);
let k = Rpo256::merge(&[i, j]);
let entries = vec![(0, a), (1, b), (4, c), (7, d)];
let tree = SimpleSmt::<3>::with_leaves(entries).unwrap();
assert_eq!(tree.root(), k);
let cases: Vec<(u64, Vec<RpoDigest>)> = vec![
(0, vec![b.into(), f, j]),
(1, vec![a.into(), f, j]),
(4, vec![z.into(), h, i]),
(7, vec![z.into(), g, i]),
];
for (key, path) in cases {
let opening = tree.open(&LeafIndex::<3>::new(key).unwrap());
assert_eq!(path, *opening.path);
}
}
#[test]
fn test_simplesmt_fail_on_duplicates() {
let values = [
// same key, same value
(int_to_leaf(1), int_to_leaf(1)),
// same key, different values
(int_to_leaf(1), int_to_leaf(2)),
// same key, set to zero
(EMPTY_WORD, int_to_leaf(1)),
// same key, re-set to zero
(int_to_leaf(1), EMPTY_WORD),
// same key, set to zero twice
(EMPTY_WORD, EMPTY_WORD),
];
for (first, second) in values.iter() {
// consecutive
let entries = [(1, *first), (1, *second)];
let smt = SimpleSmt::<64>::with_leaves(entries);
assert_eq!(smt.unwrap_err(), MerkleError::DuplicateValuesForIndex(1));
// not consecutive
let entries = [(1, *first), (5, int_to_leaf(5)), (1, *second)];
let smt = SimpleSmt::<64>::with_leaves(entries);
assert_eq!(smt.unwrap_err(), MerkleError::DuplicateValuesForIndex(1));
}
}
#[test]
fn with_no_duplicates_empty_node() {
let entries = [(1_u64, int_to_leaf(0)), (5, int_to_leaf(2))];
let smt = SimpleSmt::<64>::with_leaves(entries);
assert!(smt.is_ok());
}
#[test]
fn test_simplesmt_with_leaves_nonexisting_leaf() {
// TESTING WITH EMPTY WORD
// --------------------------------------------------------------------------------------------
// Depth 1 has 2 leaf. Position is 0-indexed, position 2 doesn't exist.
let leaves = [(2, EMPTY_WORD)];
let result = SimpleSmt::<1>::with_leaves(leaves);
assert!(result.is_err());
// Depth 2 has 4 leaves. Position is 0-indexed, position 4 doesn't exist.
let leaves = [(4, EMPTY_WORD)];
let result = SimpleSmt::<2>::with_leaves(leaves);
assert!(result.is_err());
// Depth 3 has 8 leaves. Position is 0-indexed, position 8 doesn't exist.
let leaves = [(8, EMPTY_WORD)];
let result = SimpleSmt::<3>::with_leaves(leaves);
assert!(result.is_err());
// TESTING WITH A VALUE
// --------------------------------------------------------------------------------------------
let value = int_to_node(1);
// Depth 1 has 2 leaves. Position is 0-indexed, position 2 doesn't exist.
let leaves = [(2, *value)];
let result = SimpleSmt::<1>::with_leaves(leaves);
assert!(result.is_err());
// Depth 2 has 4 leaves. Position is 0-indexed, position 4 doesn't exist.
let leaves = [(4, *value)];
let result = SimpleSmt::<2>::with_leaves(leaves);
assert!(result.is_err());
// Depth 3 has 8 leaves. Position is 0-indexed, position 8 doesn't exist.
let leaves = [(8, *value)];
let result = SimpleSmt::<3>::with_leaves(leaves);
assert!(result.is_err());
}
#[test]
fn test_simplesmt_set_subtree() {
// Final Tree:
//
// ____k____
// / \
// _i_ _j_
// / \ / \
// e f g h
// / \ / \ / \ / \
// a b 0 0 c 0 0 d
let z = EMPTY_WORD;
let a = Word::from(Rpo256::merge(&[z.into(); 2]));
let b = Word::from(Rpo256::merge(&[a.into(); 2]));
let c = Word::from(Rpo256::merge(&[b.into(); 2]));
let d = Word::from(Rpo256::merge(&[c.into(); 2]));
let e = Rpo256::merge(&[a.into(), b.into()]);
let f = Rpo256::merge(&[z.into(), z.into()]);
let g = Rpo256::merge(&[c.into(), z.into()]);
let h = Rpo256::merge(&[z.into(), d.into()]);
let i = Rpo256::merge(&[e, f]);
let j = Rpo256::merge(&[g, h]);
let k = Rpo256::merge(&[i, j]);
// subtree:
// g
// / \
// c 0
let subtree = {
let entries = vec![(0, c)];
SimpleSmt::<1>::with_leaves(entries).unwrap()
};
// insert subtree
const TREE_DEPTH: u8 = 3;
let tree = {
let entries = vec![(0, a), (1, b), (7, d)];
let mut tree = SimpleSmt::<TREE_DEPTH>::with_leaves(entries).unwrap();
tree.set_subtree(2, subtree).unwrap();
tree
};
assert_eq!(tree.root(), k);
assert_eq!(tree.get_leaf(&LeafIndex::<TREE_DEPTH>::new(4).unwrap()), c);
assert_eq!(tree.get_inner_node(NodeIndex::new_unchecked(2, 2)).hash(), g);
}
/// Ensures that an invalid input node index into `set_subtree()` incurs no mutation of the tree
#[test]
fn test_simplesmt_set_subtree_unchanged_for_wrong_index() {
// Final Tree:
//
// ____k____
// / \
// _i_ _j_
// / \ / \
// e f g h
// / \ / \ / \ / \
// a b 0 0 c 0 0 d
let z = EMPTY_WORD;
let a = Word::from(Rpo256::merge(&[z.into(); 2]));
let b = Word::from(Rpo256::merge(&[a.into(); 2]));
let c = Word::from(Rpo256::merge(&[b.into(); 2]));
let d = Word::from(Rpo256::merge(&[c.into(); 2]));
// subtree:
// g
// / \
// c 0
let subtree = {
let entries = vec![(0, c)];
SimpleSmt::<1>::with_leaves(entries).unwrap()
};
let mut tree = {
let entries = vec![(0, a), (1, b), (7, d)];
SimpleSmt::<3>::with_leaves(entries).unwrap()
};
let tree_root_before_insertion = tree.root();
// insert subtree
assert!(tree.set_subtree(500, subtree).is_err());
assert_eq!(tree.root(), tree_root_before_insertion);
}
/// We insert an empty subtree that has the same depth as the original tree
#[test]
fn test_simplesmt_set_subtree_entire_tree() {
// Initial Tree:
//
// ____k____
// / \
// _i_ _j_
// / \ / \
// e f g h
// / \ / \ / \ / \
// a b 0 0 c 0 0 d
let z = EMPTY_WORD;
let a = Word::from(Rpo256::merge(&[z.into(); 2]));
let b = Word::from(Rpo256::merge(&[a.into(); 2]));
let c = Word::from(Rpo256::merge(&[b.into(); 2]));
let d = Word::from(Rpo256::merge(&[c.into(); 2]));
// subtree: E3
const DEPTH: u8 = 3;
let subtree = { SimpleSmt::<DEPTH>::with_leaves(Vec::new()).unwrap() };
assert_eq!(subtree.root(), *EmptySubtreeRoots::entry(DEPTH, 0));
// insert subtree
let mut tree = {
let entries = vec![(0, a), (1, b), (4, c), (7, d)];
SimpleSmt::<3>::with_leaves(entries).unwrap()
};
tree.set_subtree(0, subtree).unwrap();
assert_eq!(tree.root(), *EmptySubtreeRoots::entry(DEPTH, 0));
}
// HELPER FUNCTIONS
// --------------------------------------------------------------------------------------------
fn compute_internal_nodes() -> (RpoDigest, RpoDigest, RpoDigest) {
let node2 = Rpo256::merge(&[VALUES4[0], VALUES4[1]]);
let node3 = Rpo256::merge(&[VALUES4[2], VALUES4[3]]);
let root = Rpo256::merge(&[node2, node3]);
(root, node2, node3)
}

617
src/merkle/store/mod.rs Normal file
View File

@@ -0,0 +1,617 @@
use super::{
mmr::Mmr, BTreeMap, EmptySubtreeRoots, InnerNodeInfo, KvMap, MerkleError, MerklePath,
MerkleTree, NodeIndex, PartialMerkleTree, RecordingMap, RootPath, Rpo256, RpoDigest, SimpleSmt,
Smt, ValuePath, Vec,
};
use crate::utils::{ByteReader, ByteWriter, Deserializable, DeserializationError, Serializable};
use core::borrow::Borrow;
#[cfg(test)]
mod tests;
// MERKLE STORE
// ================================================================================================
/// A default [MerkleStore] which uses a simple [BTreeMap] as the backing storage.
pub type DefaultMerkleStore = MerkleStore<BTreeMap<RpoDigest, StoreNode>>;
/// A [MerkleStore] with recording capabilities which uses [RecordingMap] as the backing storage.
pub type RecordingMerkleStore = MerkleStore<RecordingMap<RpoDigest, StoreNode>>;
#[derive(Debug, Default, Copy, Clone, Eq, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct StoreNode {
left: RpoDigest,
right: RpoDigest,
}
/// An in-memory data store for Merkelized data.
///
/// This is a in memory data store for Merkle trees, this store allows all the nodes of multiple
/// trees to live as long as necessary and without duplication, this allows the implementation of
/// space efficient persistent data structures.
///
/// Example usage:
///
/// ```rust
/// # use miden_crypto::{ZERO, Felt, Word};
/// # use miden_crypto::merkle::{NodeIndex, MerkleStore, MerkleTree};
/// # use miden_crypto::hash::rpo::Rpo256;
/// # const fn int_to_node(value: u64) -> Word {
/// # [Felt::new(value), ZERO, ZERO, ZERO]
/// # }
/// # let A = int_to_node(1);
/// # let B = int_to_node(2);
/// # let C = int_to_node(3);
/// # let D = int_to_node(4);
/// # let E = int_to_node(5);
/// # let F = int_to_node(6);
/// # let G = int_to_node(7);
/// # let H0 = int_to_node(8);
/// # let H1 = int_to_node(9);
/// # let T0 = MerkleTree::new([A, B, C, D, E, F, G, H0].to_vec()).expect("even number of leaves provided");
/// # let T1 = MerkleTree::new([A, B, C, D, E, F, G, H1].to_vec()).expect("even number of leaves provided");
/// # let ROOT0 = T0.root();
/// # let ROOT1 = T1.root();
/// let mut store: MerkleStore = MerkleStore::new();
///
/// // the store is initialized with the SMT empty nodes
/// assert_eq!(store.num_internal_nodes(), 255);
///
/// let tree1 = MerkleTree::new(vec![A, B, C, D, E, F, G, H0]).unwrap();
/// let tree2 = MerkleTree::new(vec![A, B, C, D, E, F, G, H1]).unwrap();
///
/// // populates the store with two merkle trees, common nodes are shared
/// store.extend(tree1.inner_nodes());
/// store.extend(tree2.inner_nodes());
///
/// // every leaf except the last are the same
/// for i in 0..7 {
/// let idx0 = NodeIndex::new(3, i).unwrap();
/// let d0 = store.get_node(ROOT0, idx0).unwrap();
/// let idx1 = NodeIndex::new(3, i).unwrap();
/// let d1 = store.get_node(ROOT1, idx1).unwrap();
/// assert_eq!(d0, d1, "Both trees have the same leaf at pos {i}");
/// }
///
/// // The leafs A-B-C-D are the same for both trees, so are their 2 immediate parents
/// for i in 0..4 {
/// let idx0 = NodeIndex::new(3, i).unwrap();
/// let d0 = store.get_path(ROOT0, idx0).unwrap();
/// let idx1 = NodeIndex::new(3, i).unwrap();
/// let d1 = store.get_path(ROOT1, idx1).unwrap();
/// assert_eq!(d0.path[0..2], d1.path[0..2], "Both sub-trees are equal up to two levels");
/// }
///
/// // Common internal nodes are shared, the two added trees have a total of 30, but the store has
/// // only 10 new entries, corresponding to the 10 unique internal nodes of these trees.
/// assert_eq!(store.num_internal_nodes() - 255, 10);
/// ```
#[derive(Debug, Clone, Eq, PartialEq)]
#[cfg_attr(feature = "serde", derive(serde::Deserialize, serde::Serialize))]
pub struct MerkleStore<T: KvMap<RpoDigest, StoreNode> = BTreeMap<RpoDigest, StoreNode>> {
nodes: T,
}
impl<T: KvMap<RpoDigest, StoreNode>> Default for MerkleStore<T> {
fn default() -> Self {
Self::new()
}
}
impl<T: KvMap<RpoDigest, StoreNode>> MerkleStore<T> {
// CONSTRUCTORS
// --------------------------------------------------------------------------------------------
/// Creates an empty `MerkleStore` instance.
pub fn new() -> MerkleStore<T> {
// pre-populate the store with the empty hashes
let nodes = empty_hashes().into_iter().collect();
MerkleStore { nodes }
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Return a count of the non-leaf nodes in the store.
pub fn num_internal_nodes(&self) -> usize {
self.nodes.len()
}
/// Returns the node at `index` rooted on the tree `root`.
///
/// # Errors
/// This method can return the following errors:
/// - `RootNotInStore` if the `root` is not present in the store.
/// - `NodeNotInStore` if a node needed to traverse from `root` to `index` is not present in
/// the store.
pub fn get_node(&self, root: RpoDigest, index: NodeIndex) -> Result<RpoDigest, MerkleError> {
let mut hash = root;
// corner case: check the root is in the store when called with index `NodeIndex::root()`
self.nodes.get(&hash).ok_or(MerkleError::RootNotInStore(hash))?;
for i in (0..index.depth()).rev() {
let node = self.nodes.get(&hash).ok_or(MerkleError::NodeNotInStore(hash, index))?;
let bit = (index.value() >> i) & 1;
hash = if bit == 0 { node.left } else { node.right }
}
Ok(hash)
}
/// Returns the node at the specified `index` and its opening to the `root`.
///
/// The path starts at the sibling of the target leaf.
///
/// # Errors
/// This method can return the following errors:
/// - `RootNotInStore` if the `root` is not present in the store.
/// - `NodeNotInStore` if a node needed to traverse from `root` to `index` is not present in
/// the store.
pub fn get_path(&self, root: RpoDigest, index: NodeIndex) -> Result<ValuePath, MerkleError> {
let mut hash = root;
let mut path = Vec::with_capacity(index.depth().into());
// corner case: check the root is in the store when called with index `NodeIndex::root()`
self.nodes.get(&hash).ok_or(MerkleError::RootNotInStore(hash))?;
for i in (0..index.depth()).rev() {
let node = self.nodes.get(&hash).ok_or(MerkleError::NodeNotInStore(hash, index))?;
let bit = (index.value() >> i) & 1;
hash = if bit == 0 {
path.push(node.right);
node.left
} else {
path.push(node.left);
node.right
}
}
// the path is computed from root to leaf, so it must be reversed
path.reverse();
Ok(ValuePath::new(hash, MerklePath::new(path)))
}
// LEAF TRAVERSAL
// --------------------------------------------------------------------------------------------
/// Returns the depth of the first leaf or an empty node encountered while traversing the tree
/// from the specified root down according to the provided index.
///
/// The `tree_depth` parameter specifies the depth of the tree rooted at `root`. The
/// maximum value the argument accepts is [u64::BITS].
///
/// # Errors
/// Will return an error if:
/// - The provided root is not found.
/// - The provided `tree_depth` is greater than 64.
/// - The provided `index` is not valid for a depth equivalent to `tree_depth`.
/// - No leaf or an empty node was found while traversing the tree down to `tree_depth`.
pub fn get_leaf_depth(
&self,
root: RpoDigest,
tree_depth: u8,
index: u64,
) -> Result<u8, MerkleError> {
// validate depth and index
if tree_depth > 64 {
return Err(MerkleError::DepthTooBig(tree_depth as u64));
}
NodeIndex::new(tree_depth, index)?;
// check if the root exists, providing the proper error report if it doesn't
let empty = EmptySubtreeRoots::empty_hashes(tree_depth);
let mut hash = root;
if !self.nodes.contains_key(&hash) {
return Err(MerkleError::RootNotInStore(hash));
}
// we traverse from root to leaf, so the path is reversed
let mut path = (index << (64 - tree_depth)).reverse_bits();
// iterate every depth and reconstruct the path from root to leaf
for depth in 0..=tree_depth {
// we short-circuit if an empty node has been found
if hash == empty[depth as usize] {
return Ok(depth);
}
// fetch the children pair, mapped by its parent hash
let children = match self.nodes.get(&hash) {
Some(node) => node,
None => return Ok(depth),
};
// traverse down
hash = if path & 1 == 0 { children.left } else { children.right };
path >>= 1;
}
// return an error because we exhausted the index but didn't find either a leaf or an
// empty node
Err(MerkleError::DepthTooBig(tree_depth as u64 + 1))
}
/// Returns index and value of a leaf node which is the only leaf node in a subtree defined by
/// the provided root. If the subtree contains zero or more than one leaf nodes None is
/// returned.
///
/// The `tree_depth` parameter specifies the depth of the parent tree such that `root` is
/// located in this tree at `root_index`. The maximum value the argument accepts is
/// [u64::BITS].
///
/// # Errors
/// Will return an error if:
/// - The provided root is not found.
/// - The provided `tree_depth` is greater than 64.
/// - The provided `root_index` has depth greater than `tree_depth`.
/// - A lone node at depth `tree_depth` is not a leaf node.
pub fn find_lone_leaf(
&self,
root: RpoDigest,
root_index: NodeIndex,
tree_depth: u8,
) -> Result<Option<(NodeIndex, RpoDigest)>, MerkleError> {
// we set max depth at u64::BITS as this is the largest meaningful value for a 64-bit index
const MAX_DEPTH: u8 = u64::BITS as u8;
if tree_depth > MAX_DEPTH {
return Err(MerkleError::DepthTooBig(tree_depth as u64));
}
let empty = EmptySubtreeRoots::empty_hashes(MAX_DEPTH);
let mut node = root;
if !self.nodes.contains_key(&node) {
return Err(MerkleError::RootNotInStore(node));
}
let mut index = root_index;
if index.depth() > tree_depth {
return Err(MerkleError::DepthTooBig(index.depth() as u64));
}
// traverse down following the path of single non-empty nodes; this works because if a
// node has two empty children it cannot contain a lone leaf. similarly if a node has
// two non-empty children it must contain at least two leaves.
for depth in index.depth()..tree_depth {
// if the node is a leaf, return; otherwise, examine the node's children
let children = match self.nodes.get(&node) {
Some(node) => node,
None => return Ok(Some((index, node))),
};
let empty_node = empty[depth as usize + 1];
node = if children.left != empty_node && children.right == empty_node {
index = index.left_child();
children.left
} else if children.left == empty_node && children.right != empty_node {
index = index.right_child();
children.right
} else {
return Ok(None);
};
}
// if we are here, we got to `tree_depth`; thus, either the current node is a leaf node,
// and so we return it, or it is an internal node, and then we return an error
if self.nodes.contains_key(&node) {
Err(MerkleError::DepthTooBig(tree_depth as u64 + 1))
} else {
Ok(Some((index, node)))
}
}
// DATA EXTRACTORS
// --------------------------------------------------------------------------------------------
/// Returns a subset of this Merkle store such that the returned Merkle store contains all
/// nodes which are descendants of the specified roots.
///
/// The roots for which no descendants exist in this Merkle store are ignored.
pub fn subset<I, R>(&self, roots: I) -> MerkleStore<T>
where
I: Iterator<Item = R>,
R: Borrow<RpoDigest>,
{
let mut store = MerkleStore::new();
for root in roots {
let root = *root.borrow();
store.clone_tree_from(root, self);
}
store
}
/// Iterator over the inner nodes of the [MerkleStore].
pub fn inner_nodes(&self) -> impl Iterator<Item = InnerNodeInfo> + '_ {
self.nodes
.iter()
.map(|(r, n)| InnerNodeInfo { value: *r, left: n.left, right: n.right })
}
/// Iterator over the non-empty leaves of the Merkle tree associated with the specified `root`
/// and `max_depth`.
pub fn non_empty_leaves(
&self,
root: RpoDigest,
max_depth: u8,
) -> impl Iterator<Item = (NodeIndex, RpoDigest)> + '_ {
let empty_roots = EmptySubtreeRoots::empty_hashes(max_depth);
let mut stack = Vec::new();
stack.push((NodeIndex::new_unchecked(0, 0), root));
core::iter::from_fn(move || {
while let Some((index, node_hash)) = stack.pop() {
// if we are at the max depth then we have reached a leaf
if index.depth() == max_depth {
return Some((index, node_hash));
}
// fetch the nodes children and push them onto the stack if they are not the roots
// of empty subtrees
if let Some(node) = self.nodes.get(&node_hash) {
if !empty_roots.contains(&node.left) {
stack.push((index.left_child(), node.left));
}
if !empty_roots.contains(&node.right) {
stack.push((index.right_child(), node.right));
}
// if the node is not in the store assume it is a leaf
} else {
return Some((index, node_hash));
}
}
None
})
}
// STATE MUTATORS
// --------------------------------------------------------------------------------------------
/// Adds all the nodes of a Merkle path represented by `path`, opening to `node`. Returns the
/// new root.
///
/// This will compute the sibling elements determined by the Merkle `path` and `node`, and
/// include all the nodes into the store.
pub fn add_merkle_path(
&mut self,
index: u64,
node: RpoDigest,
path: MerklePath,
) -> Result<RpoDigest, MerkleError> {
let root = path.inner_nodes(index, node)?.fold(RpoDigest::default(), |_, node| {
let value: RpoDigest = node.value;
let left: RpoDigest = node.left;
let right: RpoDigest = node.right;
debug_assert_eq!(Rpo256::merge(&[left, right]), value);
self.nodes.insert(value, StoreNode { left, right });
node.value
});
Ok(root)
}
/// Adds all the nodes of multiple Merkle paths into the store.
///
/// This will compute the sibling elements for each Merkle `path` and include all the nodes
/// into the store.
///
/// For further reference, check [MerkleStore::add_merkle_path].
pub fn add_merkle_paths<I>(&mut self, paths: I) -> Result<(), MerkleError>
where
I: IntoIterator<Item = (u64, RpoDigest, MerklePath)>,
{
for (index_value, node, path) in paths.into_iter() {
self.add_merkle_path(index_value, node, path)?;
}
Ok(())
}
/// Sets a node to `value`.
///
/// # Errors
/// This method can return the following errors:
/// - `RootNotInStore` if the `root` is not present in the store.
/// - `NodeNotInStore` if a node needed to traverse from `root` to `index` is not present in
/// the store.
pub fn set_node(
&mut self,
mut root: RpoDigest,
index: NodeIndex,
value: RpoDigest,
) -> Result<RootPath, MerkleError> {
let node = value;
let ValuePath { value, path } = self.get_path(root, index)?;
// performs the update only if the node value differs from the opening
if node != value {
root = self.add_merkle_path(index.value(), node, path.clone())?;
}
Ok(RootPath { root, path })
}
/// Merges two elements and adds the resulting node into the store.
///
/// Merges arbitrary values. They may be leafs, nodes, or a mixture of both.
pub fn merge_roots(
&mut self,
left_root: RpoDigest,
right_root: RpoDigest,
) -> Result<RpoDigest, MerkleError> {
let parent = Rpo256::merge(&[left_root, right_root]);
self.nodes.insert(parent, StoreNode { left: left_root, right: right_root });
Ok(parent)
}
// DESTRUCTURING
// --------------------------------------------------------------------------------------------
/// Returns the inner storage of this MerkleStore while consuming `self`.
pub fn into_inner(self) -> T {
self.nodes
}
// HELPER METHODS
// --------------------------------------------------------------------------------------------
/// Recursively clones a tree with the specified root from the specified source into self.
///
/// If the source store does not contain a tree with the specified root, this is a noop.
fn clone_tree_from(&mut self, root: RpoDigest, source: &Self) {
// process the node only if it is in the source
if let Some(node) = source.nodes.get(&root) {
// if the node has already been inserted, no need to process it further as all of its
// descendants should be already cloned from the source store
if self.nodes.insert(root, *node).is_none() {
self.clone_tree_from(node.left, source);
self.clone_tree_from(node.right, source);
}
}
}
}
// CONVERSIONS
// ================================================================================================
impl<T: KvMap<RpoDigest, StoreNode>> From<&MerkleTree> for MerkleStore<T> {
fn from(value: &MerkleTree) -> Self {
let nodes = combine_nodes_with_empty_hashes(value.inner_nodes()).collect();
Self { nodes }
}
}
impl<T: KvMap<RpoDigest, StoreNode>, const DEPTH: u8> From<&SimpleSmt<DEPTH>> for MerkleStore<T> {
fn from(value: &SimpleSmt<DEPTH>) -> Self {
let nodes = combine_nodes_with_empty_hashes(value.inner_nodes()).collect();
Self { nodes }
}
}
impl<T: KvMap<RpoDigest, StoreNode>> From<&Smt> for MerkleStore<T> {
fn from(value: &Smt) -> Self {
let nodes = combine_nodes_with_empty_hashes(value.inner_nodes()).collect();
Self { nodes }
}
}
impl<T: KvMap<RpoDigest, StoreNode>> From<&Mmr> for MerkleStore<T> {
fn from(value: &Mmr) -> Self {
let nodes = combine_nodes_with_empty_hashes(value.inner_nodes()).collect();
Self { nodes }
}
}
impl<T: KvMap<RpoDigest, StoreNode>> From<&PartialMerkleTree> for MerkleStore<T> {
fn from(value: &PartialMerkleTree) -> Self {
let nodes = combine_nodes_with_empty_hashes(value.inner_nodes()).collect();
Self { nodes }
}
}
impl<T: KvMap<RpoDigest, StoreNode>> From<T> for MerkleStore<T> {
fn from(values: T) -> Self {
let nodes = values.into_iter().chain(empty_hashes()).collect();
Self { nodes }
}
}
impl<T: KvMap<RpoDigest, StoreNode>> FromIterator<InnerNodeInfo> for MerkleStore<T> {
fn from_iter<I: IntoIterator<Item = InnerNodeInfo>>(iter: I) -> Self {
let nodes = combine_nodes_with_empty_hashes(iter).collect();
Self { nodes }
}
}
impl<T: KvMap<RpoDigest, StoreNode>> FromIterator<(RpoDigest, StoreNode)> for MerkleStore<T> {
fn from_iter<I: IntoIterator<Item = (RpoDigest, StoreNode)>>(iter: I) -> Self {
let nodes = iter.into_iter().chain(empty_hashes()).collect();
Self { nodes }
}
}
// ITERATORS
// ================================================================================================
impl<T: KvMap<RpoDigest, StoreNode>> Extend<InnerNodeInfo> for MerkleStore<T> {
fn extend<I: IntoIterator<Item = InnerNodeInfo>>(&mut self, iter: I) {
self.nodes.extend(
iter.into_iter()
.map(|info| (info.value, StoreNode { left: info.left, right: info.right })),
);
}
}
// SERIALIZATION
// ================================================================================================
impl Serializable for StoreNode {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
self.left.write_into(target);
self.right.write_into(target);
}
}
impl Deserializable for StoreNode {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let left = RpoDigest::read_from(source)?;
let right = RpoDigest::read_from(source)?;
Ok(StoreNode { left, right })
}
}
impl<T: KvMap<RpoDigest, StoreNode>> Serializable for MerkleStore<T> {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
target.write_u64(self.nodes.len() as u64);
for (k, v) in self.nodes.iter() {
k.write_into(target);
v.write_into(target);
}
}
}
impl<T: KvMap<RpoDigest, StoreNode>> Deserializable for MerkleStore<T> {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let len = source.read_u64()?;
let mut nodes: Vec<(RpoDigest, StoreNode)> = Vec::with_capacity(len as usize);
for _ in 0..len {
let key = RpoDigest::read_from(source)?;
let value = StoreNode::read_from(source)?;
nodes.push((key, value));
}
Ok(nodes.into_iter().collect())
}
}
// HELPER FUNCTIONS
// ================================================================================================
/// Creates empty hashes for all the subtrees of a tree with a max depth of 255.
fn empty_hashes() -> impl IntoIterator<Item = (RpoDigest, StoreNode)> {
let subtrees = EmptySubtreeRoots::empty_hashes(255);
subtrees
.iter()
.rev()
.copied()
.zip(subtrees.iter().rev().skip(1).copied())
.map(|(child, parent)| (parent, StoreNode { left: child, right: child }))
}
/// Consumes an iterator of [InnerNodeInfo] and returns an iterator of `(value, node)` tuples
/// which includes the nodes associate with roots of empty subtrees up to a depth of 255.
fn combine_nodes_with_empty_hashes(
nodes: impl IntoIterator<Item = InnerNodeInfo>,
) -> impl Iterator<Item = (RpoDigest, StoreNode)> {
nodes
.into_iter()
.map(|info| (info.value, StoreNode { left: info.left, right: info.right }))
.chain(empty_hashes())
}

932
src/merkle/store/tests.rs Normal file
View File

@@ -0,0 +1,932 @@
use super::{
DefaultMerkleStore as MerkleStore, EmptySubtreeRoots, MerkleError, MerklePath, NodeIndex,
PartialMerkleTree, RecordingMerkleStore, Rpo256, RpoDigest,
};
use crate::{
merkle::{
digests_to_words, int_to_leaf, int_to_node, LeafIndex, MerkleTree, SimpleSmt, SMT_MAX_DEPTH,
},
Felt, Word, ONE, WORD_SIZE, ZERO,
};
#[cfg(feature = "std")]
use super::{Deserializable, Serializable};
#[cfg(feature = "std")]
use std::error::Error;
use seq_macro::seq;
// TEST DATA
// ================================================================================================
const KEYS4: [u64; 4] = [0, 1, 2, 3];
const VALUES4: [RpoDigest; 4] = [int_to_node(1), int_to_node(2), int_to_node(3), int_to_node(4)];
const KEYS8: [u64; 8] = [0, 1, 2, 3, 4, 5, 6, 7];
const VALUES8: [RpoDigest; 8] = [
int_to_node(1),
int_to_node(2),
int_to_node(3),
int_to_node(4),
int_to_node(5),
int_to_node(6),
int_to_node(7),
int_to_node(8),
];
// TESTS
// ================================================================================================
#[test]
fn test_root_not_in_store() -> Result<(), MerkleError> {
let mtree = MerkleTree::new(digests_to_words(&VALUES4))?;
let store = MerkleStore::from(&mtree);
assert_eq!(
store.get_node(VALUES4[0], NodeIndex::make(mtree.depth(), 0)),
Err(MerkleError::RootNotInStore(VALUES4[0])),
"Leaf 0 is not a root"
);
assert_eq!(
store.get_path(VALUES4[0], NodeIndex::make(mtree.depth(), 0)),
Err(MerkleError::RootNotInStore(VALUES4[0])),
"Leaf 0 is not a root"
);
Ok(())
}
#[test]
fn test_merkle_tree() -> Result<(), MerkleError> {
let mtree = MerkleTree::new(digests_to_words(&VALUES4))?;
let store = MerkleStore::from(&mtree);
// STORE LEAVES ARE CORRECT -------------------------------------------------------------------
// checks the leaves in the store corresponds to the expected values
assert_eq!(
store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 0)),
Ok(VALUES4[0]),
"node 0 must be in the tree"
);
assert_eq!(
store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 1)),
Ok(VALUES4[1]),
"node 1 must be in the tree"
);
assert_eq!(
store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 2)),
Ok(VALUES4[2]),
"node 2 must be in the tree"
);
assert_eq!(
store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 3)),
Ok(VALUES4[3]),
"node 3 must be in the tree"
);
// STORE LEAVES MATCH TREE --------------------------------------------------------------------
// sanity check the values returned by the store and the tree
assert_eq!(
mtree.get_node(NodeIndex::make(mtree.depth(), 0)),
store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 0)),
"node 0 must be the same for both MerkleTree and MerkleStore"
);
assert_eq!(
mtree.get_node(NodeIndex::make(mtree.depth(), 1)),
store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 1)),
"node 1 must be the same for both MerkleTree and MerkleStore"
);
assert_eq!(
mtree.get_node(NodeIndex::make(mtree.depth(), 2)),
store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 2)),
"node 2 must be the same for both MerkleTree and MerkleStore"
);
assert_eq!(
mtree.get_node(NodeIndex::make(mtree.depth(), 3)),
store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 3)),
"node 3 must be the same for both MerkleTree and MerkleStore"
);
// STORE MERKLE PATH MATCHES ==============================================================
// assert the merkle path returned by the store is the same as the one in the tree
let result = store.get_path(mtree.root(), NodeIndex::make(mtree.depth(), 0)).unwrap();
assert_eq!(
VALUES4[0], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
mtree.get_path(NodeIndex::make(mtree.depth(), 0)),
Ok(result.path),
"merkle path for index 0 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(mtree.root(), NodeIndex::make(mtree.depth(), 1)).unwrap();
assert_eq!(
VALUES4[1], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
mtree.get_path(NodeIndex::make(mtree.depth(), 1)),
Ok(result.path),
"merkle path for index 1 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(mtree.root(), NodeIndex::make(mtree.depth(), 2)).unwrap();
assert_eq!(
VALUES4[2], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
mtree.get_path(NodeIndex::make(mtree.depth(), 2)),
Ok(result.path),
"merkle path for index 0 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(mtree.root(), NodeIndex::make(mtree.depth(), 3)).unwrap();
assert_eq!(
VALUES4[3], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
mtree.get_path(NodeIndex::make(mtree.depth(), 3)),
Ok(result.path),
"merkle path for index 0 must be the same for the MerkleTree and MerkleStore"
);
Ok(())
}
#[test]
fn test_empty_roots() {
let store = MerkleStore::default();
let mut root = RpoDigest::default();
for depth in 0..255 {
root = Rpo256::merge(&[root; 2]);
assert!(
store.get_node(root, NodeIndex::make(0, 0)).is_ok(),
"The root of the empty tree of depth {depth} must be registered"
);
}
}
#[test]
fn test_leaf_paths_for_empty_trees() -> Result<(), MerkleError> {
let store = MerkleStore::default();
// Starts at 1 because leafs are not included in the store.
// Ends at 64 because it is not possible to represent an index of a depth greater than 64,
// because a u64 is used to index the leaf.
seq!(DEPTH in 1_u8..64_u8 {
let smt = SimpleSmt::<DEPTH>::new()?;
let index = NodeIndex::make(DEPTH, 0);
let store_path = store.get_path(smt.root(), index)?;
let smt_path = smt.open(&LeafIndex::<DEPTH>::new(0)?).path;
assert_eq!(
store_path.value,
RpoDigest::default(),
"the leaf of an empty tree is always ZERO"
);
assert_eq!(
store_path.path, smt_path,
"the returned merkle path does not match the computed values"
);
assert_eq!(
store_path.path.compute_root(DEPTH.into(), RpoDigest::default()).unwrap(),
smt.root(),
"computed root from the path must match the empty tree root"
);
});
Ok(())
}
#[test]
fn test_get_invalid_node() {
let mtree =
MerkleTree::new(digests_to_words(&VALUES4)).expect("creating a merkle tree must work");
let store = MerkleStore::from(&mtree);
let _ = store.get_node(mtree.root(), NodeIndex::make(mtree.depth(), 3));
}
#[test]
fn test_add_sparse_merkle_tree_one_level() -> Result<(), MerkleError> {
let keys2: [u64; 2] = [0, 1];
let leaves2: [Word; 2] = [int_to_leaf(1), int_to_leaf(2)];
let smt = SimpleSmt::<1>::with_leaves(keys2.into_iter().zip(leaves2)).unwrap();
let store = MerkleStore::from(&smt);
let idx = NodeIndex::make(1, 0);
assert_eq!(smt.get_node(idx).unwrap(), leaves2[0].into());
assert_eq!(store.get_node(smt.root(), idx).unwrap(), smt.get_node(idx).unwrap());
let idx = NodeIndex::make(1, 1);
assert_eq!(smt.get_node(idx).unwrap(), leaves2[1].into());
assert_eq!(store.get_node(smt.root(), idx).unwrap(), smt.get_node(idx).unwrap());
Ok(())
}
#[test]
fn test_sparse_merkle_tree() -> Result<(), MerkleError> {
let smt =
SimpleSmt::<SMT_MAX_DEPTH>::with_leaves(KEYS4.into_iter().zip(digests_to_words(&VALUES4)))
.unwrap();
let store = MerkleStore::from(&smt);
// STORE LEAVES ARE CORRECT ==============================================================
// checks the leaves in the store corresponds to the expected values
assert_eq!(
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 0)),
Ok(VALUES4[0]),
"node 0 must be in the tree"
);
assert_eq!(
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 1)),
Ok(VALUES4[1]),
"node 1 must be in the tree"
);
assert_eq!(
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 2)),
Ok(VALUES4[2]),
"node 2 must be in the tree"
);
assert_eq!(
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 3)),
Ok(VALUES4[3]),
"node 3 must be in the tree"
);
assert_eq!(
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 4)),
Ok(RpoDigest::default()),
"unmodified node 4 must be ZERO"
);
// STORE LEAVES MATCH TREE ===============================================================
// sanity check the values returned by the store and the tree
assert_eq!(
smt.get_node(NodeIndex::make(SMT_MAX_DEPTH, 0)),
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 0)),
"node 0 must be the same for both SparseMerkleTree and MerkleStore"
);
assert_eq!(
smt.get_node(NodeIndex::make(SMT_MAX_DEPTH, 1)),
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 1)),
"node 1 must be the same for both SparseMerkleTree and MerkleStore"
);
assert_eq!(
smt.get_node(NodeIndex::make(SMT_MAX_DEPTH, 2)),
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 2)),
"node 2 must be the same for both SparseMerkleTree and MerkleStore"
);
assert_eq!(
smt.get_node(NodeIndex::make(SMT_MAX_DEPTH, 3)),
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 3)),
"node 3 must be the same for both SparseMerkleTree and MerkleStore"
);
assert_eq!(
smt.get_node(NodeIndex::make(SMT_MAX_DEPTH, 4)),
store.get_node(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 4)),
"node 4 must be the same for both SparseMerkleTree and MerkleStore"
);
// STORE MERKLE PATH MATCHES ==============================================================
// assert the merkle path returned by the store is the same as the one in the tree
let result = store.get_path(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 0)).unwrap();
assert_eq!(
VALUES4[0], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
smt.open(&LeafIndex::<SMT_MAX_DEPTH>::new(0).unwrap()).path,
result.path,
"merkle path for index 0 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 1)).unwrap();
assert_eq!(
VALUES4[1], result.value,
"Value for merkle path at index 1 must match leaf value"
);
assert_eq!(
smt.open(&LeafIndex::<SMT_MAX_DEPTH>::new(1).unwrap()).path,
result.path,
"merkle path for index 1 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 2)).unwrap();
assert_eq!(
VALUES4[2], result.value,
"Value for merkle path at index 2 must match leaf value"
);
assert_eq!(
smt.open(&LeafIndex::<SMT_MAX_DEPTH>::new(2).unwrap()).path,
result.path,
"merkle path for index 2 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 3)).unwrap();
assert_eq!(
VALUES4[3], result.value,
"Value for merkle path at index 3 must match leaf value"
);
assert_eq!(
smt.open(&LeafIndex::<SMT_MAX_DEPTH>::new(3).unwrap()).path,
result.path,
"merkle path for index 3 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(smt.root(), NodeIndex::make(SMT_MAX_DEPTH, 4)).unwrap();
assert_eq!(
RpoDigest::default(),
result.value,
"Value for merkle path at index 4 must match leaf value"
);
assert_eq!(
smt.open(&LeafIndex::<SMT_MAX_DEPTH>::new(4).unwrap()).path,
result.path,
"merkle path for index 4 must be the same for the MerkleTree and MerkleStore"
);
Ok(())
}
#[test]
fn test_add_merkle_paths() -> Result<(), MerkleError> {
let mtree = MerkleTree::new(digests_to_words(&VALUES4))?;
let i0 = 0;
let p0 = mtree.get_path(NodeIndex::make(2, i0)).unwrap();
let i1 = 1;
let p1 = mtree.get_path(NodeIndex::make(2, i1)).unwrap();
let i2 = 2;
let p2 = mtree.get_path(NodeIndex::make(2, i2)).unwrap();
let i3 = 3;
let p3 = mtree.get_path(NodeIndex::make(2, i3)).unwrap();
let paths = [
(i0, VALUES4[i0 as usize], p0),
(i1, VALUES4[i1 as usize], p1),
(i2, VALUES4[i2 as usize], p2),
(i3, VALUES4[i3 as usize], p3),
];
let mut store = MerkleStore::default();
store.add_merkle_paths(paths.clone()).expect("the valid paths must work");
let pmt = PartialMerkleTree::with_paths(paths).unwrap();
// STORE LEAVES ARE CORRECT ==============================================================
// checks the leaves in the store corresponds to the expected values
assert_eq!(
store.get_node(pmt.root(), NodeIndex::make(pmt.max_depth(), 0)),
Ok(VALUES4[0]),
"node 0 must be in the pmt"
);
assert_eq!(
store.get_node(pmt.root(), NodeIndex::make(pmt.max_depth(), 1)),
Ok(VALUES4[1]),
"node 1 must be in the pmt"
);
assert_eq!(
store.get_node(pmt.root(), NodeIndex::make(pmt.max_depth(), 2)),
Ok(VALUES4[2]),
"node 2 must be in the pmt"
);
assert_eq!(
store.get_node(pmt.root(), NodeIndex::make(pmt.max_depth(), 3)),
Ok(VALUES4[3]),
"node 3 must be in the pmt"
);
// STORE LEAVES MATCH PMT ================================================================
// sanity check the values returned by the store and the pmt
assert_eq!(
pmt.get_node(NodeIndex::make(pmt.max_depth(), 0)),
store.get_node(pmt.root(), NodeIndex::make(pmt.max_depth(), 0)),
"node 0 must be the same for both PartialMerkleTree and MerkleStore"
);
assert_eq!(
pmt.get_node(NodeIndex::make(pmt.max_depth(), 1)),
store.get_node(pmt.root(), NodeIndex::make(pmt.max_depth(), 1)),
"node 1 must be the same for both PartialMerkleTree and MerkleStore"
);
assert_eq!(
pmt.get_node(NodeIndex::make(pmt.max_depth(), 2)),
store.get_node(pmt.root(), NodeIndex::make(pmt.max_depth(), 2)),
"node 2 must be the same for both PartialMerkleTree and MerkleStore"
);
assert_eq!(
pmt.get_node(NodeIndex::make(pmt.max_depth(), 3)),
store.get_node(pmt.root(), NodeIndex::make(pmt.max_depth(), 3)),
"node 3 must be the same for both PartialMerkleTree and MerkleStore"
);
// STORE MERKLE PATH MATCHES ==============================================================
// assert the merkle path returned by the store is the same as the one in the pmt
let result = store.get_path(pmt.root(), NodeIndex::make(pmt.max_depth(), 0)).unwrap();
assert_eq!(
VALUES4[0], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
pmt.get_path(NodeIndex::make(pmt.max_depth(), 0)),
Ok(result.path),
"merkle path for index 0 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(pmt.root(), NodeIndex::make(pmt.max_depth(), 1)).unwrap();
assert_eq!(
VALUES4[1], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
pmt.get_path(NodeIndex::make(pmt.max_depth(), 1)),
Ok(result.path),
"merkle path for index 1 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(pmt.root(), NodeIndex::make(pmt.max_depth(), 2)).unwrap();
assert_eq!(
VALUES4[2], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
pmt.get_path(NodeIndex::make(pmt.max_depth(), 2)),
Ok(result.path),
"merkle path for index 0 must be the same for the MerkleTree and MerkleStore"
);
let result = store.get_path(pmt.root(), NodeIndex::make(pmt.max_depth(), 3)).unwrap();
assert_eq!(
VALUES4[3], result.value,
"Value for merkle path at index 0 must match leaf value"
);
assert_eq!(
pmt.get_path(NodeIndex::make(pmt.max_depth(), 3)),
Ok(result.path),
"merkle path for index 0 must be the same for the MerkleTree and MerkleStore"
);
Ok(())
}
#[test]
fn wont_open_to_different_depth_root() {
let empty = EmptySubtreeRoots::empty_hashes(64);
let a = [ONE; 4];
let b = [Felt::new(2); 4];
// Compute the root for a different depth. We cherry-pick this specific depth to prevent a
// regression to a bug in the past that allowed the user to fetch a node at a depth lower than
// the inserted path of a Merkle tree.
let mut root = Rpo256::merge(&[a.into(), b.into()]);
for depth in (1..=63).rev() {
root = Rpo256::merge(&[root, empty[depth]]);
}
// For this example, the depth of the Merkle tree is 1, as we have only two leaves. Here we
// attempt to fetch a node on the maximum depth, and it should fail because the root shouldn't
// exist for the set.
let mtree = MerkleTree::new(vec![a, b]).unwrap();
let store = MerkleStore::from(&mtree);
let index = NodeIndex::root();
let err = store.get_node(root, index).err().unwrap();
assert_eq!(err, MerkleError::RootNotInStore(root));
}
#[test]
fn store_path_opens_from_leaf() {
let a = [ONE; 4];
let b = [Felt::new(2); 4];
let c = [Felt::new(3); 4];
let d = [Felt::new(4); 4];
let e = [Felt::new(5); 4];
let f = [Felt::new(6); 4];
let g = [Felt::new(7); 4];
let h = [Felt::new(8); 4];
let i = Rpo256::merge(&[a.into(), b.into()]);
let j = Rpo256::merge(&[c.into(), d.into()]);
let k = Rpo256::merge(&[e.into(), f.into()]);
let l = Rpo256::merge(&[g.into(), h.into()]);
let m = Rpo256::merge(&[i, j]);
let n = Rpo256::merge(&[k, l]);
let root = Rpo256::merge(&[m, n]);
let mtree = MerkleTree::new(vec![a, b, c, d, e, f, g, h]).unwrap();
let store = MerkleStore::from(&mtree);
let path = store.get_path(root, NodeIndex::make(3, 1)).unwrap().path;
let expected = MerklePath::new([a.into(), j, n].to_vec());
assert_eq!(path, expected);
}
#[test]
fn test_set_node() -> Result<(), MerkleError> {
let mtree = MerkleTree::new(digests_to_words(&VALUES4))?;
let mut store = MerkleStore::from(&mtree);
let value = int_to_node(42);
let index = NodeIndex::make(mtree.depth(), 0);
let new_root = store.set_node(mtree.root(), index, value)?.root;
assert_eq!(store.get_node(new_root, index), Ok(value), "Value must have changed");
Ok(())
}
#[test]
fn test_constructors() -> Result<(), MerkleError> {
let mtree = MerkleTree::new(digests_to_words(&VALUES4))?;
let store = MerkleStore::from(&mtree);
let depth = mtree.depth();
let leaves = 2u64.pow(depth.into());
for index in 0..leaves {
let index = NodeIndex::make(depth, index);
let value_path = store.get_path(mtree.root(), index)?;
assert_eq!(mtree.get_path(index)?, value_path.path);
}
const DEPTH: u8 = 32;
let smt =
SimpleSmt::<DEPTH>::with_leaves(KEYS4.into_iter().zip(digests_to_words(&VALUES4))).unwrap();
let store = MerkleStore::from(&smt);
for key in KEYS4 {
let index = NodeIndex::make(DEPTH, key);
let value_path = store.get_path(smt.root(), index)?;
assert_eq!(smt.open(&LeafIndex::<DEPTH>::new(key).unwrap()).path, value_path.path);
}
let d = 2;
let paths = [
(0, VALUES4[0], mtree.get_path(NodeIndex::make(d, 0)).unwrap()),
(1, VALUES4[1], mtree.get_path(NodeIndex::make(d, 1)).unwrap()),
(2, VALUES4[2], mtree.get_path(NodeIndex::make(d, 2)).unwrap()),
(3, VALUES4[3], mtree.get_path(NodeIndex::make(d, 3)).unwrap()),
];
let mut store1 = MerkleStore::default();
store1.add_merkle_paths(paths.clone())?;
let mut store2 = MerkleStore::default();
store2.add_merkle_path(0, VALUES4[0], mtree.get_path(NodeIndex::make(d, 0))?)?;
store2.add_merkle_path(1, VALUES4[1], mtree.get_path(NodeIndex::make(d, 1))?)?;
store2.add_merkle_path(2, VALUES4[2], mtree.get_path(NodeIndex::make(d, 2))?)?;
store2.add_merkle_path(3, VALUES4[3], mtree.get_path(NodeIndex::make(d, 3))?)?;
let pmt = PartialMerkleTree::with_paths(paths).unwrap();
for key in [0, 1, 2, 3] {
let index = NodeIndex::make(d, key);
let value_path1 = store1.get_path(pmt.root(), index)?;
let value_path2 = store2.get_path(pmt.root(), index)?;
assert_eq!(value_path1, value_path2);
let index = NodeIndex::make(d, key);
assert_eq!(pmt.get_path(index)?, value_path1.path);
}
Ok(())
}
#[test]
fn node_path_should_be_truncated_by_midtier_insert() {
let key = 0b11010010_11001100_11001100_11001100_11001100_11001100_11001100_11001100_u64;
let mut store = MerkleStore::new();
let root: RpoDigest = EmptySubtreeRoots::empty_hashes(64)[0];
// insert first node - works as expected
let depth = 64;
let node = RpoDigest::from([Felt::new(key); WORD_SIZE]);
let index = NodeIndex::new(depth, key).unwrap();
let root = store.set_node(root, index, node).unwrap().root;
let result = store.get_node(root, index).unwrap();
let path = store.get_path(root, index).unwrap().path;
assert_eq!(node, result);
assert_eq!(path.depth(), depth);
assert!(path.verify(index.value(), result, &root));
// flip the first bit of the key and insert the second node on a different depth
let key = key ^ (1 << 63);
let key = key >> 8;
let depth = 56;
let node = RpoDigest::from([Felt::new(key); WORD_SIZE]);
let index = NodeIndex::new(depth, key).unwrap();
let root = store.set_node(root, index, node).unwrap().root;
let result = store.get_node(root, index).unwrap();
let path = store.get_path(root, index).unwrap().path;
assert_eq!(node, result);
assert_eq!(path.depth(), depth);
assert!(path.verify(index.value(), result, &root));
// attempt to fetch a path of the second node to depth 64
// should fail because the previously inserted node will remove its sub-tree from the set
let key = key << 8;
let index = NodeIndex::new(64, key).unwrap();
assert!(store.get_node(root, index).is_err());
}
// LEAF TRAVERSAL
// ================================================================================================
#[test]
fn get_leaf_depth_works_depth_64() {
let mut store = MerkleStore::new();
let mut root: RpoDigest = EmptySubtreeRoots::empty_hashes(64)[0];
let key = u64::MAX;
// this will create a rainbow tree and test all opening to depth 64
for d in 0..64 {
let k = key & (u64::MAX >> d);
let node = RpoDigest::from([Felt::new(k); WORD_SIZE]);
let index = NodeIndex::new(64, k).unwrap();
// assert the leaf doesn't exist before the insert. the returned depth should always
// increment with the paths count of the set, as they are intersecting one another up to
// the first bits of the used key.
assert_eq!(d, store.get_leaf_depth(root, 64, k).unwrap());
// insert and assert the correct depth
root = store.set_node(root, index, node).unwrap().root;
assert_eq!(64, store.get_leaf_depth(root, 64, k).unwrap());
}
}
#[test]
fn get_leaf_depth_works_with_incremental_depth() {
let mut store = MerkleStore::new();
let mut root: RpoDigest = EmptySubtreeRoots::empty_hashes(64)[0];
// insert some path to the left of the root and assert it
let key = 0b01001011_10110110_00001101_01110100_00111011_10101101_00000100_01000001_u64;
assert_eq!(0, store.get_leaf_depth(root, 64, key).unwrap());
let depth = 64;
let index = NodeIndex::new(depth, key).unwrap();
let node = RpoDigest::from([Felt::new(key); WORD_SIZE]);
root = store.set_node(root, index, node).unwrap().root;
assert_eq!(depth, store.get_leaf_depth(root, 64, key).unwrap());
// flip the key to the right of the root and insert some content on depth 16
let key = 0b11001011_10110110_00000000_00000000_00000000_00000000_00000000_00000000_u64;
assert_eq!(1, store.get_leaf_depth(root, 64, key).unwrap());
let depth = 16;
let index = NodeIndex::new(depth, key >> (64 - depth)).unwrap();
let node = RpoDigest::from([Felt::new(key); WORD_SIZE]);
root = store.set_node(root, index, node).unwrap().root;
assert_eq!(depth, store.get_leaf_depth(root, 64, key).unwrap());
// attempt the sibling of the previous leaf
let key = 0b11001011_10110111_00000000_00000000_00000000_00000000_00000000_00000000_u64;
assert_eq!(16, store.get_leaf_depth(root, 64, key).unwrap());
let index = NodeIndex::new(depth, key >> (64 - depth)).unwrap();
let node = RpoDigest::from([Felt::new(key); WORD_SIZE]);
root = store.set_node(root, index, node).unwrap().root;
assert_eq!(depth, store.get_leaf_depth(root, 64, key).unwrap());
// move down to the next depth and assert correct behavior
let key = 0b11001011_10110100_00000000_00000000_00000000_00000000_00000000_00000000_u64;
assert_eq!(15, store.get_leaf_depth(root, 64, key).unwrap());
let depth = 17;
let index = NodeIndex::new(depth, key >> (64 - depth)).unwrap();
let node = RpoDigest::from([Felt::new(key); WORD_SIZE]);
root = store.set_node(root, index, node).unwrap().root;
assert_eq!(depth, store.get_leaf_depth(root, 64, key).unwrap());
}
#[test]
fn get_leaf_depth_works_with_depth_8() {
let mut store = MerkleStore::new();
let mut root: RpoDigest = EmptySubtreeRoots::empty_hashes(8)[0];
// insert some random, 8 depth keys. `a` diverges from the first bit
let a = 0b01101001_u64;
let b = 0b10011001_u64;
let c = 0b10010110_u64;
let d = 0b11110110_u64;
for k in [a, b, c, d] {
let index = NodeIndex::new(8, k).unwrap();
let node = RpoDigest::from([Felt::new(k); WORD_SIZE]);
root = store.set_node(root, index, node).unwrap().root;
}
// assert all leaves returns the inserted depth
for k in [a, b, c, d] {
assert_eq!(8, store.get_leaf_depth(root, 8, k).unwrap());
}
// flip last bit of a and expect it to return the the same depth, but for an empty node
assert_eq!(8, store.get_leaf_depth(root, 8, 0b01101000_u64).unwrap());
// flip fourth bit of a and expect an empty node on depth 4
assert_eq!(4, store.get_leaf_depth(root, 8, 0b01111001_u64).unwrap());
// flip third bit of a and expect an empty node on depth 3
assert_eq!(3, store.get_leaf_depth(root, 8, 0b01001001_u64).unwrap());
// flip second bit of a and expect an empty node on depth 2
assert_eq!(2, store.get_leaf_depth(root, 8, 0b00101001_u64).unwrap());
// flip fourth bit of c and expect an empty node on depth 4
assert_eq!(4, store.get_leaf_depth(root, 8, 0b10000110_u64).unwrap());
// flip second bit of d and expect an empty node on depth 3 as depth 2 conflicts with b and c
assert_eq!(3, store.get_leaf_depth(root, 8, 0b10110110_u64).unwrap());
// duplicate the tree on `a` and assert the depth is short-circuited by such sub-tree
let index = NodeIndex::new(8, a).unwrap();
root = store.set_node(root, index, root).unwrap().root;
assert_eq!(Err(MerkleError::DepthTooBig(9)), store.get_leaf_depth(root, 8, a));
}
#[test]
fn find_lone_leaf() {
let mut store = MerkleStore::new();
let empty = EmptySubtreeRoots::empty_hashes(64);
let mut root: RpoDigest = empty[0];
// insert a single leaf into the store at depth 64
let key_a = 0b01010101_10101010_00001111_01110100_00111011_10101101_00000100_01000001_u64;
let idx_a = NodeIndex::make(64, key_a);
let val_a = RpoDigest::from([ONE, ONE, ONE, ONE]);
root = store.set_node(root, idx_a, val_a).unwrap().root;
// for every ancestor of A, A should be a long leaf
for depth in 1..64 {
let parent_index = NodeIndex::make(depth, key_a >> (64 - depth));
let parent = store.get_node(root, parent_index).unwrap();
let res = store.find_lone_leaf(parent, parent_index, 64).unwrap();
assert_eq!(res, Some((idx_a, val_a)));
}
// insert another leaf into the store such that it has the same 8 bit prefix as A
let key_b = 0b01010101_01111010_00001111_01110100_00111011_10101101_00000100_01000001_u64;
let idx_b = NodeIndex::make(64, key_b);
let val_b = RpoDigest::from([ONE, ONE, ONE, ZERO]);
root = store.set_node(root, idx_b, val_b).unwrap().root;
// for any node which is common between A and B, find_lone_leaf() should return None as the
// node has two descendants
for depth in 1..9 {
let parent_index = NodeIndex::make(depth, key_a >> (64 - depth));
let parent = store.get_node(root, parent_index).unwrap();
let res = store.find_lone_leaf(parent, parent_index, 64).unwrap();
assert_eq!(res, None);
}
// for other ancestors of A and B, A and B should be lone leaves respectively
for depth in 9..64 {
let parent_index = NodeIndex::make(depth, key_a >> (64 - depth));
let parent = store.get_node(root, parent_index).unwrap();
let res = store.find_lone_leaf(parent, parent_index, 64).unwrap();
assert_eq!(res, Some((idx_a, val_a)));
}
for depth in 9..64 {
let parent_index = NodeIndex::make(depth, key_b >> (64 - depth));
let parent = store.get_node(root, parent_index).unwrap();
let res = store.find_lone_leaf(parent, parent_index, 64).unwrap();
assert_eq!(res, Some((idx_b, val_b)));
}
// for any other node, find_lone_leaf() should return None as they have no leaf nodes
let parent_index = NodeIndex::make(16, 0b01010101_11111111);
let parent = store.get_node(root, parent_index).unwrap();
let res = store.find_lone_leaf(parent, parent_index, 64).unwrap();
assert_eq!(res, None);
}
// SUBSET EXTRACTION
// ================================================================================================
#[test]
fn mstore_subset() {
// add a Merkle tree of depth 3 to the store
let mtree = MerkleTree::new(digests_to_words(&VALUES8)).unwrap();
let mut store = MerkleStore::default();
let empty_store_num_nodes = store.nodes.len();
store.extend(mtree.inner_nodes());
// build 3 subtrees contained within the above Merkle tree; note that subtree2 is a subset
// of subtree1
let subtree1 = MerkleTree::new(digests_to_words(&VALUES8[..4])).unwrap();
let subtree2 = MerkleTree::new(digests_to_words(&VALUES8[2..4])).unwrap();
let subtree3 = MerkleTree::new(digests_to_words(&VALUES8[6..])).unwrap();
// --- extract all 3 subtrees ---------------------------------------------
let substore = store.subset([subtree1.root(), subtree2.root(), subtree3.root()].iter());
// number of nodes should increase by 4: 3 nodes form subtree1 and 1 node from subtree3
assert_eq!(substore.nodes.len(), empty_store_num_nodes + 4);
// make sure paths that all subtrees are in the store
check_mstore_subtree(&substore, &subtree1);
check_mstore_subtree(&substore, &subtree2);
check_mstore_subtree(&substore, &subtree3);
// --- extract subtrees 1 and 3 -------------------------------------------
// this should give the same result as above as subtree2 is nested within subtree1
let substore = store.subset([subtree1.root(), subtree3.root()].iter());
// number of nodes should increase by 4: 3 nodes form subtree1 and 1 node from subtree3
assert_eq!(substore.nodes.len(), empty_store_num_nodes + 4);
// make sure paths that all subtrees are in the store
check_mstore_subtree(&substore, &subtree1);
check_mstore_subtree(&substore, &subtree2);
check_mstore_subtree(&substore, &subtree3);
}
fn check_mstore_subtree(store: &MerkleStore, subtree: &MerkleTree) {
for (i, value) in subtree.leaves() {
let index = NodeIndex::new(subtree.depth(), i).unwrap();
let path1 = store.get_path(subtree.root(), index).unwrap();
assert_eq!(*path1.value, *value);
let path2 = subtree.get_path(index).unwrap();
assert_eq!(path1.path, path2);
}
}
// SERIALIZATION
// ================================================================================================
#[cfg(feature = "std")]
#[test]
fn test_serialization() -> Result<(), Box<dyn Error>> {
let mtree = MerkleTree::new(digests_to_words(&VALUES4))?;
let store = MerkleStore::from(&mtree);
let decoded = MerkleStore::read_from_bytes(&store.to_bytes()).expect("deserialization failed");
assert_eq!(store, decoded);
Ok(())
}
// MERKLE RECORDER
// ================================================================================================
#[test]
fn test_recorder() {
// instantiate recorder from MerkleTree and SimpleSmt
let mtree = MerkleTree::new(digests_to_words(&VALUES4)).unwrap();
const TREE_DEPTH: u8 = 64;
let smtree = SimpleSmt::<TREE_DEPTH>::with_leaves(
KEYS8.into_iter().zip(VALUES8.into_iter().map(|x| x.into()).rev()),
)
.unwrap();
let mut recorder: RecordingMerkleStore =
mtree.inner_nodes().chain(smtree.inner_nodes()).collect();
// get nodes from both trees and make sure they are correct
let index_0 = NodeIndex::new(mtree.depth(), 0).unwrap();
let node = recorder.get_node(mtree.root(), index_0).unwrap();
assert_eq!(node, mtree.get_node(index_0).unwrap());
let index_1 = NodeIndex::new(TREE_DEPTH, 1).unwrap();
let node = recorder.get_node(smtree.root(), index_1).unwrap();
assert_eq!(node, smtree.get_node(index_1).unwrap());
// insert a value and assert that when we request it next time it is accurate
let new_value = [ZERO, ZERO, ONE, ONE].into();
let index_2 = NodeIndex::new(TREE_DEPTH, 2).unwrap();
let root = recorder.set_node(smtree.root(), index_2, new_value).unwrap().root;
assert_eq!(recorder.get_node(root, index_2).unwrap(), new_value);
// construct the proof
let rec_map = recorder.into_inner();
let (_, proof) = rec_map.finalize();
let merkle_store: MerkleStore = proof.into();
// make sure the proof contains all nodes from both trees
let node = merkle_store.get_node(mtree.root(), index_0).unwrap();
assert_eq!(node, mtree.get_node(index_0).unwrap());
let node = merkle_store.get_node(smtree.root(), index_1).unwrap();
assert_eq!(node, smtree.get_node(index_1).unwrap());
let node = merkle_store.get_node(smtree.root(), index_2).unwrap();
assert_eq!(
node,
smtree.get_leaf(&LeafIndex::<TREE_DEPTH>::try_from(index_2).unwrap()).into()
);
// assert that is doesnt contain nodes that were not recorded
let not_recorded_index = NodeIndex::new(TREE_DEPTH, 4).unwrap();
assert!(merkle_store.get_node(smtree.root(), not_recorded_index).is_err());
assert!(smtree.get_node(not_recorded_index).is_ok());
}

19
src/rand/mod.rs Normal file
View File

@@ -0,0 +1,19 @@
//! Pseudo-random element generation.
pub use winter_crypto::{DefaultRandomCoin as WinterRandomCoin, RandomCoin, RandomCoinError};
use crate::{Felt, FieldElement, Word, ZERO};
mod rpo;
pub use rpo::RpoRandomCoin;
/// Pseudo-random element generator.
///
/// An instance can be used to draw, uniformly at random, base field elements as well as [Word]s.
pub trait FeltRng {
/// Draw, uniformly at random, a base field element.
fn draw_element(&mut self) -> Felt;
/// Draw, uniformly at random, a [Word].
fn draw_word(&mut self) -> Word;
}

267
src/rand/rpo.rs Normal file
View File

@@ -0,0 +1,267 @@
use super::{Felt, FeltRng, FieldElement, Word, ZERO};
use crate::{
hash::rpo::{Rpo256, RpoDigest},
utils::{
collections::Vec, string::ToString, vec, ByteReader, ByteWriter, Deserializable,
DeserializationError, Serializable,
},
};
pub use winter_crypto::{RandomCoin, RandomCoinError};
// CONSTANTS
// ================================================================================================
const STATE_WIDTH: usize = Rpo256::STATE_WIDTH;
const RATE_START: usize = Rpo256::RATE_RANGE.start;
const RATE_END: usize = Rpo256::RATE_RANGE.end;
const HALF_RATE_WIDTH: usize = (Rpo256::RATE_RANGE.end - Rpo256::RATE_RANGE.start) / 2;
// RPO RANDOM COIN
// ================================================================================================
/// A simplified version of the `SPONGE_PRG` reseedable pseudo-random number generator algorithm
/// described in <https://eprint.iacr.org/2011/499.pdf>.
///
/// The simplification is related to the following facts:
/// 1. A call to the reseed method implies one and only one call to the permutation function.
/// This is possible because in our case we never reseed with more than 4 field elements.
/// 2. As a result of the previous point, we don't make use of an input buffer to accumulate seed
/// material.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct RpoRandomCoin {
state: [Felt; STATE_WIDTH],
current: usize,
}
impl RpoRandomCoin {
/// Returns a new [RpoRandomCoin] initialize with the specified seed.
pub fn new(seed: Word) -> Self {
let mut state = [ZERO; STATE_WIDTH];
for i in 0..HALF_RATE_WIDTH {
state[RATE_START + i] += seed[i];
}
// Absorb
Rpo256::apply_permutation(&mut state);
RpoRandomCoin { state, current: RATE_START }
}
/// Returns an [RpoRandomCoin] instantiated from the provided components.
///
/// # Panics
/// Panics if `current` is smaller than 4 or greater than or equal to 12.
pub fn from_parts(state: [Felt; STATE_WIDTH], current: usize) -> Self {
assert!(
(RATE_START..RATE_END).contains(&current),
"current value outside of valid range"
);
Self { state, current }
}
/// Returns components of this random coin.
pub fn into_parts(self) -> ([Felt; STATE_WIDTH], usize) {
(self.state, self.current)
}
fn draw_basefield(&mut self) -> Felt {
if self.current == RATE_END {
Rpo256::apply_permutation(&mut self.state);
self.current = RATE_START;
}
self.current += 1;
self.state[self.current - 1]
}
}
// RANDOM COIN IMPLEMENTATION
// ------------------------------------------------------------------------------------------------
impl RandomCoin for RpoRandomCoin {
type BaseField = Felt;
type Hasher = Rpo256;
fn new(seed: &[Self::BaseField]) -> Self {
let digest: Word = Rpo256::hash_elements(seed).into();
Self::new(digest)
}
fn reseed(&mut self, data: RpoDigest) {
// Reset buffer
self.current = RATE_START;
// Add the new seed material to the first half of the rate portion of the RPO state
let data: Word = data.into();
self.state[RATE_START] += data[0];
self.state[RATE_START + 1] += data[1];
self.state[RATE_START + 2] += data[2];
self.state[RATE_START + 3] += data[3];
// Absorb
Rpo256::apply_permutation(&mut self.state);
}
fn check_leading_zeros(&self, value: u64) -> u32 {
let value = Felt::new(value);
let mut state_tmp = self.state;
state_tmp[RATE_START] += value;
Rpo256::apply_permutation(&mut state_tmp);
let first_rate_element = state_tmp[RATE_START].as_int();
first_rate_element.trailing_zeros()
}
fn draw<E: FieldElement<BaseField = Felt>>(&mut self) -> Result<E, RandomCoinError> {
let ext_degree = E::EXTENSION_DEGREE;
let mut result = vec![ZERO; ext_degree];
for r in result.iter_mut().take(ext_degree) {
*r = self.draw_basefield();
}
let result = E::slice_from_base_elements(&result);
Ok(result[0])
}
fn draw_integers(
&mut self,
num_values: usize,
domain_size: usize,
nonce: u64,
) -> Result<Vec<usize>, RandomCoinError> {
assert!(domain_size.is_power_of_two(), "domain size must be a power of two");
assert!(num_values < domain_size, "number of values must be smaller than domain size");
// absorb the nonce
let nonce = Felt::new(nonce);
self.state[RATE_START] += nonce;
Rpo256::apply_permutation(&mut self.state);
// reset the buffer
self.current = RATE_START;
// determine how many bits are needed to represent valid values in the domain
let v_mask = (domain_size - 1) as u64;
// draw values from PRNG until we get as many unique values as specified by num_queries
let mut values = Vec::new();
for _ in 0..1000 {
// get the next pseudo-random field element
let value = self.draw_basefield().as_int();
// use the mask to get a value within the range
let value = (value & v_mask) as usize;
values.push(value);
if values.len() == num_values {
break;
}
}
if values.len() < num_values {
return Err(RandomCoinError::FailedToDrawIntegers(num_values, values.len(), 1000));
}
Ok(values)
}
}
// FELT RNG IMPLEMENTATION
// ------------------------------------------------------------------------------------------------
impl FeltRng for RpoRandomCoin {
fn draw_element(&mut self) -> Felt {
self.draw_basefield()
}
fn draw_word(&mut self) -> Word {
let mut output = [ZERO; 4];
for o in output.iter_mut() {
*o = self.draw_basefield();
}
output
}
}
// SERIALIZATION
// ------------------------------------------------------------------------------------------------
impl Serializable for RpoRandomCoin {
fn write_into<W: ByteWriter>(&self, target: &mut W) {
self.state.iter().for_each(|v| v.write_into(target));
// casting to u8 is OK because `current` is always between 4 and 12.
target.write_u8(self.current as u8);
}
}
impl Deserializable for RpoRandomCoin {
fn read_from<R: ByteReader>(source: &mut R) -> Result<Self, DeserializationError> {
let state = [
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
Felt::read_from(source)?,
];
let current = source.read_u8()? as usize;
if !(RATE_START..RATE_END).contains(&current) {
return Err(DeserializationError::InvalidValue(
"current value outside of valid range".to_string(),
));
}
Ok(Self { state, current })
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::{Deserializable, FeltRng, RpoRandomCoin, Serializable, ZERO};
use crate::ONE;
#[test]
fn test_feltrng_felt() {
let mut rpocoin = RpoRandomCoin::new([ZERO; 4]);
let output = rpocoin.draw_element();
let mut rpocoin = RpoRandomCoin::new([ZERO; 4]);
let expected = rpocoin.draw_basefield();
assert_eq!(output, expected);
}
#[test]
fn test_feltrng_word() {
let mut rpocoin = RpoRandomCoin::new([ZERO; 4]);
let output = rpocoin.draw_word();
let mut rpocoin = RpoRandomCoin::new([ZERO; 4]);
let mut expected = [ZERO; 4];
for o in expected.iter_mut() {
*o = rpocoin.draw_basefield();
}
assert_eq!(output, expected);
}
#[test]
fn test_feltrng_serialization() {
let coin1 = RpoRandomCoin::from_parts([ONE; 12], 5);
let bytes = coin1.to_bytes();
let coin2 = RpoRandomCoin::read_from_bytes(&bytes).unwrap();
assert_eq!(coin1, coin2);
}
}

404
src/utils/kv_map.rs Normal file
View File

@@ -0,0 +1,404 @@
use core::cell::RefCell;
use winter_utils::{
collections::{btree_map::IntoIter, BTreeMap, BTreeSet},
Box,
};
// KEY-VALUE MAP TRAIT
// ================================================================================================
/// A trait that defines the interface for a key-value map.
pub trait KvMap<K: Ord + Clone, V: Clone>:
Extend<(K, V)> + FromIterator<(K, V)> + IntoIterator<Item = (K, V)>
{
fn get(&self, key: &K) -> Option<&V>;
fn contains_key(&self, key: &K) -> bool;
fn len(&self) -> usize;
fn is_empty(&self) -> bool {
self.len() == 0
}
fn insert(&mut self, key: K, value: V) -> Option<V>;
fn remove(&mut self, key: &K) -> Option<V>;
fn iter(&self) -> Box<dyn Iterator<Item = (&K, &V)> + '_>;
}
// BTREE MAP `KvMap` IMPLEMENTATION
// ================================================================================================
impl<K: Ord + Clone, V: Clone> KvMap<K, V> for BTreeMap<K, V> {
fn get(&self, key: &K) -> Option<&V> {
self.get(key)
}
fn contains_key(&self, key: &K) -> bool {
self.contains_key(key)
}
fn len(&self) -> usize {
self.len()
}
fn insert(&mut self, key: K, value: V) -> Option<V> {
self.insert(key, value)
}
fn remove(&mut self, key: &K) -> Option<V> {
self.remove(key)
}
fn iter(&self) -> Box<dyn Iterator<Item = (&K, &V)> + '_> {
Box::new(self.iter())
}
}
// RECORDING MAP
// ================================================================================================
/// A [RecordingMap] that records read requests to the underlying key-value map.
///
/// The data recorder is used to generate a proof for read requests.
///
/// The [RecordingMap] is composed of three parts:
/// - `data`: which contains the current set of key-value pairs in the map.
/// - `updates`: which tracks keys for which values have been changed since the map was
/// instantiated. updates include both insertions, removals and updates of values under existing
/// keys.
/// - `trace`: which contains the key-value pairs from the original data which have been accesses
/// since the map was instantiated.
#[derive(Debug, Default, Clone, Eq, PartialEq)]
pub struct RecordingMap<K, V> {
data: BTreeMap<K, V>,
updates: BTreeSet<K>,
trace: RefCell<BTreeMap<K, V>>,
}
impl<K: Ord + Clone, V: Clone> RecordingMap<K, V> {
// CONSTRUCTOR
// --------------------------------------------------------------------------------------------
/// Returns a new [RecordingMap] instance initialized with the provided key-value pairs.
/// ([BTreeMap]).
pub fn new(init: impl IntoIterator<Item = (K, V)>) -> Self {
RecordingMap {
data: init.into_iter().collect(),
updates: BTreeSet::new(),
trace: RefCell::new(BTreeMap::new()),
}
}
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
pub fn inner(&self) -> &BTreeMap<K, V> {
&self.data
}
// FINALIZER
// --------------------------------------------------------------------------------------------
/// Consumes the [RecordingMap] and returns a ([BTreeMap], [BTreeMap]) tuple. The first
/// element of the tuple is a map that represents the state of the map at the time `.finalize()`
/// is called. The second element contains the key-value pairs from the initial data set that
/// were read during recording.
pub fn finalize(self) -> (BTreeMap<K, V>, BTreeMap<K, V>) {
(self.data, self.trace.take())
}
// TEST HELPERS
// --------------------------------------------------------------------------------------------
#[cfg(test)]
pub fn trace_len(&self) -> usize {
self.trace.borrow().len()
}
#[cfg(test)]
pub fn updates_len(&self) -> usize {
self.updates.len()
}
}
impl<K: Ord + Clone, V: Clone> KvMap<K, V> for RecordingMap<K, V> {
// PUBLIC ACCESSORS
// --------------------------------------------------------------------------------------------
/// Returns a reference to the value associated with the given key if the value exists.
///
/// If the key is part of the initial data set, the key access is recorded.
fn get(&self, key: &K) -> Option<&V> {
self.data.get(key).map(|value| {
if !self.updates.contains(key) {
self.trace.borrow_mut().insert(key.clone(), value.clone());
}
value
})
}
/// Returns a boolean to indicate whether the given key exists in the data set.
///
/// If the key is part of the initial data set, the key access is recorded.
fn contains_key(&self, key: &K) -> bool {
self.get(key).is_some()
}
/// Returns the number of key-value pairs in the data set.
fn len(&self) -> usize {
self.data.len()
}
// MUTATORS
// --------------------------------------------------------------------------------------------
/// Inserts a key-value pair into the data set.
///
/// If the key already exists in the data set, the value is updated and the old value is
/// returned.
fn insert(&mut self, key: K, value: V) -> Option<V> {
let new_update = self.updates.insert(key.clone());
self.data.insert(key.clone(), value).map(|old_value| {
if new_update {
self.trace.borrow_mut().insert(key, old_value.clone());
}
old_value
})
}
/// Removes a key-value pair from the data set.
///
/// If the key exists in the data set, the old value is returned.
fn remove(&mut self, key: &K) -> Option<V> {
self.data.remove(key).map(|old_value| {
let new_update = self.updates.insert(key.clone());
if new_update {
self.trace.borrow_mut().insert(key.clone(), old_value.clone());
}
old_value
})
}
// ITERATION
// --------------------------------------------------------------------------------------------
/// Returns an iterator over the key-value pairs in the data set.
fn iter(&self) -> Box<dyn Iterator<Item = (&K, &V)> + '_> {
Box::new(self.data.iter())
}
}
impl<K: Clone + Ord, V: Clone> Extend<(K, V)> for RecordingMap<K, V> {
fn extend<T: IntoIterator<Item = (K, V)>>(&mut self, iter: T) {
iter.into_iter().for_each(move |(k, v)| {
self.insert(k, v);
});
}
}
impl<K: Clone + Ord, V: Clone> FromIterator<(K, V)> for RecordingMap<K, V> {
fn from_iter<T: IntoIterator<Item = (K, V)>>(iter: T) -> Self {
Self::new(iter)
}
}
impl<K: Clone + Ord, V: Clone> IntoIterator for RecordingMap<K, V> {
type Item = (K, V);
type IntoIter = IntoIter<K, V>;
fn into_iter(self) -> Self::IntoIter {
self.data.into_iter()
}
}
// TESTS
// ================================================================================================
#[cfg(test)]
mod tests {
use super::*;
const ITEMS: [(u64, u64); 5] = [(0, 0), (1, 1), (2, 2), (3, 3), (4, 4)];
#[test]
fn test_get_item() {
// instantiate a recording map
let map = RecordingMap::new(ITEMS.to_vec());
// get a few items
let get_items = [0, 1, 2];
for key in get_items.iter() {
map.get(key);
}
// convert the map into a proof
let (_, proof) = map.finalize();
// check that the proof contains the expected values
for (key, value) in ITEMS.iter() {
match get_items.contains(key) {
true => assert_eq!(proof.get(key), Some(value)),
false => assert_eq!(proof.get(key), None),
}
}
}
#[test]
fn test_contains_key() {
// instantiate a recording map
let map = RecordingMap::new(ITEMS.to_vec());
// check if the map contains a few items
let get_items = [0, 1, 2];
for key in get_items.iter() {
map.contains_key(key);
}
// convert the map into a proof
let (_, proof) = map.finalize();
// check that the proof contains the expected values
for (key, _) in ITEMS.iter() {
match get_items.contains(key) {
true => assert!(proof.contains_key(key)),
false => assert!(!proof.contains_key(key)),
}
}
}
#[test]
fn test_len() {
// instantiate a recording map
let mut map = RecordingMap::new(ITEMS.to_vec());
// length of the map should be equal to the number of items
assert_eq!(map.len(), ITEMS.len());
// inserting entry with key that already exists should not change the length, but it does
// add entries to the trace and update sets
map.insert(4, 5);
assert_eq!(map.len(), ITEMS.len());
assert_eq!(map.trace_len(), 1);
assert_eq!(map.updates_len(), 1);
// inserting entry with new key should increase the length; it should also record the key
// as an updated key, but the trace length does not change since old values were not touched
map.insert(5, 5);
assert_eq!(map.len(), ITEMS.len() + 1);
assert_eq!(map.trace_len(), 1);
assert_eq!(map.updates_len(), 2);
// get some items so that they are saved in the trace; this should record original items
// in the trace, but should not affect the set of updates
let get_items = [0, 1, 2];
for key in get_items.iter() {
map.contains_key(key);
}
assert_eq!(map.trace_len(), 4);
assert_eq!(map.updates_len(), 2);
// read the same items again, this should not have any effect on either length, trace, or
// the set of updates
let get_items = [0, 1, 2];
for key in get_items.iter() {
map.contains_key(key);
}
assert_eq!(map.trace_len(), 4);
assert_eq!(map.updates_len(), 2);
// read a newly inserted item; this should not affect either length, trace, or the set of
// updates
let _val = map.get(&5).unwrap();
assert_eq!(map.trace_len(), 4);
assert_eq!(map.updates_len(), 2);
// update a newly inserted item; this should not affect either length, trace, or the set
// of updates
map.insert(5, 11);
assert_eq!(map.trace_len(), 4);
assert_eq!(map.updates_len(), 2);
// Note: The length reported by the proof will be different to the length originally
// reported by the map.
let (_, proof) = map.finalize();
// length of the proof should be equal to get_items + 1. The extra item is the original
// value at key = 4u64
assert_eq!(proof.len(), get_items.len() + 1);
}
#[test]
fn test_iter() {
let mut map = RecordingMap::new(ITEMS.to_vec());
assert!(map.iter().all(|(x, y)| ITEMS.contains(&(*x, *y))));
// when inserting entry with key that already exists the iterator should return the new value
let new_value = 5;
map.insert(4, new_value);
assert_eq!(map.iter().count(), ITEMS.len());
assert!(map.iter().all(|(x, y)| if x == &4 {
y == &new_value
} else {
ITEMS.contains(&(*x, *y))
}));
}
#[test]
fn test_is_empty() {
// instantiate an empty recording map
let empty_map: RecordingMap<u64, u64> = RecordingMap::default();
assert!(empty_map.is_empty());
// instantiate a non-empty recording map
let map = RecordingMap::new(ITEMS.to_vec());
assert!(!map.is_empty());
}
#[test]
fn test_remove() {
let mut map = RecordingMap::new(ITEMS.to_vec());
// remove an item that exists
let key = 0;
let value = map.remove(&key).unwrap();
assert_eq!(value, ITEMS[0].1);
assert_eq!(map.len(), ITEMS.len() - 1);
assert_eq!(map.trace_len(), 1);
assert_eq!(map.updates_len(), 1);
// add the item back and then remove it again
let key = 0;
let value = 0;
map.insert(key, value);
let value = map.remove(&key).unwrap();
assert_eq!(value, 0);
assert_eq!(map.len(), ITEMS.len() - 1);
assert_eq!(map.trace_len(), 1);
assert_eq!(map.updates_len(), 1);
// remove an item that does not exist
let key = 100;
let value = map.remove(&key);
assert_eq!(value, None);
assert_eq!(map.len(), ITEMS.len() - 1);
assert_eq!(map.trace_len(), 1);
assert_eq!(map.updates_len(), 1);
// insert a new item and then remove it
let key = 100;
let value = 100;
map.insert(key, value);
let value = map.remove(&key).unwrap();
assert_eq!(value, 100);
assert_eq!(map.len(), ITEMS.len() - 1);
assert_eq!(map.trace_len(), 1);
assert_eq!(map.updates_len(), 2);
// convert the map into a proof
let (_, proof) = map.finalize();
// check that the proof contains the expected values
for (key, value) in ITEMS.iter() {
match key {
0 => assert_eq!(proof.get(key), Some(value)),
_ => assert_eq!(proof.get(key), None),
}
}
}
}

111
src/utils/mod.rs Normal file
View File

@@ -0,0 +1,111 @@
//! Utilities used in this crate which can also be generally useful downstream.
use super::{utils::string::String, Word};
use core::fmt::{self, Display, Write};
#[cfg(not(feature = "std"))]
pub use alloc::{format, vec};
#[cfg(feature = "std")]
pub use std::{format, vec};
mod kv_map;
// RE-EXPORTS
// ================================================================================================
pub use winter_utils::{
string, uninit_vector, Box, ByteReader, ByteWriter, Deserializable, DeserializationError,
Serializable, SliceReader,
};
pub mod collections {
pub use super::kv_map::*;
pub use winter_utils::collections::*;
}
// UTILITY FUNCTIONS
// ================================================================================================
/// Converts a [Word] into hex.
pub fn word_to_hex(w: &Word) -> Result<String, fmt::Error> {
let mut s = String::new();
for byte in w.iter().flat_map(|e| e.to_bytes()) {
write!(s, "{byte:02x}")?;
}
Ok(s)
}
/// Renders an array of bytes as hex into a String.
pub fn bytes_to_hex_string<const N: usize>(data: [u8; N]) -> String {
let mut s = String::with_capacity(N + 2);
s.push_str("0x");
for byte in data.iter() {
write!(s, "{byte:02x}").expect("formatting hex failed");
}
s
}
/// Defines errors which can occur during parsing of hexadecimal strings.
#[derive(Debug)]
pub enum HexParseError {
InvalidLength { expected: usize, actual: usize },
MissingPrefix,
InvalidChar,
OutOfRange,
}
impl Display for HexParseError {
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
match self {
HexParseError::InvalidLength { expected, actual } => {
write!(f, "Hex encoded RpoDigest must have length 66, including the 0x prefix. expected {expected} got {actual}")
}
HexParseError::MissingPrefix => {
write!(f, "Hex encoded RpoDigest must start with 0x prefix")
}
HexParseError::InvalidChar => {
write!(f, "Hex encoded RpoDigest must contain characters [a-zA-Z0-9]")
}
HexParseError::OutOfRange => {
write!(f, "Hex encoded values of an RpoDigest must be inside the field modulus")
}
}
}
}
#[cfg(feature = "std")]
impl std::error::Error for HexParseError {}
/// Parses a hex string into an array of bytes of known size.
pub fn hex_to_bytes<const N: usize>(value: &str) -> Result<[u8; N], HexParseError> {
let expected: usize = (N * 2) + 2;
if value.len() != expected {
return Err(HexParseError::InvalidLength { expected, actual: value.len() });
}
if !value.starts_with("0x") {
return Err(HexParseError::MissingPrefix);
}
let mut data = value.bytes().skip(2).map(|v| match v {
b'0'..=b'9' => Ok(v - b'0'),
b'a'..=b'f' => Ok(v - b'a' + 10),
b'A'..=b'F' => Ok(v - b'A' + 10),
_ => Err(HexParseError::InvalidChar),
});
let mut decoded = [0u8; N];
#[allow(clippy::needless_range_loop)]
for pos in 0..N {
// These `unwrap` calls are okay because the length was checked above
let high: u8 = data.next().unwrap()?;
let low: u8 = data.next().unwrap()?;
decoded[pos] = (high << 4) + low;
}
Ok(decoded)
}