From bf1743bbb9cae23fc6a6221818814f5703d50195 Mon Sep 17 00:00:00 2001 From: Jordi Date: Thu, 29 Nov 2018 18:19:37 +0100 Subject: [PATCH 01/24] Initial commit --- LICENSE | 674 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ README.md | 2 + 2 files changed, 676 insertions(+) create mode 100644 LICENSE create mode 100644 README.md diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..f288702 --- /dev/null +++ b/LICENSE @@ -0,0 +1,674 @@ + GNU GENERAL PUBLIC LICENSE + Version 3, 29 June 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU General Public License is a free, copyleft license for +software and other kinds of works. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +the GNU General Public License is intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. We, the Free Software Foundation, use the +GNU General Public License for most of our software; it applies also to +any other work released this way by its authors. You can apply it to +your programs, too. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + To protect your rights, we need to prevent others from denying you +these rights or asking you to surrender the rights. Therefore, you have +certain responsibilities if you distribute copies of the software, or if +you modify it: responsibilities to respect the freedom of others. + + For example, if you distribute copies of such a program, whether +gratis or for a fee, you must pass on to the recipients the same +freedoms that you received. You must make sure that they, too, receive +or can get the source code. And you must show them these terms so they +know their rights. + + Developers that use the GNU GPL protect your rights with two steps: +(1) assert copyright on the software, and (2) offer you this License +giving you legal permission to copy, distribute and/or modify it. + + For the developers' and authors' protection, the GPL clearly explains +that there is no warranty for this free software. For both users' and +authors' sake, the GPL requires that modified versions be marked as +changed, so that their problems will not be attributed erroneously to +authors of previous versions. + + Some devices are designed to deny users access to install or run +modified versions of the software inside them, although the manufacturer +can do so. This is fundamentally incompatible with the aim of +protecting users' freedom to change the software. The systematic +pattern of such abuse occurs in the area of products for individuals to +use, which is precisely where it is most unacceptable. Therefore, we +have designed this version of the GPL to prohibit the practice for those +products. If such problems arise substantially in other domains, we +stand ready to extend this provision to those domains in future versions +of the GPL, as needed to protect the freedom of users. + + Finally, every program is threatened constantly by software patents. +States should not allow patents to restrict development and use of +software on general-purpose computers, but in those that do, we wish to +avoid the special danger that patents applied to a free program could +make it effectively proprietary. To prevent this, the GPL assures that +patents cannot be used to render the program non-free. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Use with the GNU Affero General Public License. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU Affero General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the special requirements of the GNU Affero General Public License, +section 13, concerning interaction through a network will apply to the +combination as such. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU General Public License from time to time. Such new versions will +be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU General Public License for more details. + + You should have received a copy of the GNU General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If the program does terminal interaction, make it output a short +notice like this when it starts in an interactive mode: + + Copyright (C) + This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'. + This is free software, and you are welcome to redistribute it + under certain conditions; type `show c' for details. + +The hypothetical commands `show w' and `show c' should show the appropriate +parts of the General Public License. Of course, your program's commands +might be different; for a GUI interface, you would use an "about box". + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU GPL, see +. + + The GNU General Public License does not permit incorporating your program +into proprietary programs. If your program is a subroutine library, you +may consider it more useful to permit linking proprietary applications with +the library. If this is what you want to do, use the GNU Lesser General +Public License instead of this License. But first, please read +. diff --git a/README.md b/README.md new file mode 100644 index 0000000..56fcf18 --- /dev/null +++ b/README.md @@ -0,0 +1,2 @@ +# dVote-process +Javascript library to work with the creation and management of voting processes From 43c95cab9bc17191c02aec50120c2a7b34734c0e Mon Sep 17 00:00:00 2001 From: Jordi Date: Thu, 29 Nov 2018 18:32:00 +0100 Subject: [PATCH 02/24] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 56fcf18..f621cea 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,2 @@ # dVote-process -Javascript library to work with the creation and management of voting processes +Go library to work with the creation and management of voting processes From 1a8a1c8d7ce97040e8fb73c0ccd406306caecb17 Mon Sep 17 00:00:00 2001 From: Pau Date: Thu, 29 Nov 2018 18:33:26 +0100 Subject: [PATCH 03/24] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index f621cea..819415b 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,2 @@ # dVote-process -Go library to work with the creation and management of voting processes +Library to work with the creation and management of voting processes From 19326669db623e0bd8be90e33da6b8bd77d9ade5 Mon Sep 17 00:00:00 2001 From: p4u Date: Thu, 13 Dec 2018 18:43:45 +0100 Subject: [PATCH 04/24] First implementation of tree package Signed-off-by: p4u --- tree/README.md | 22 ++++++++++++++++++ tree/tree.go | 62 ++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 84 insertions(+) create mode 100644 tree/README.md create mode 100644 tree/tree.go diff --git a/tree/README.md b/tree/README.md new file mode 100644 index 0000000..3e9af9d --- /dev/null +++ b/tree/README.md @@ -0,0 +1,22 @@ +#### Dvote Tree + +Implementation of dvote tree structure. Currently based on iden3 merkle tree. + +Example of usage: + +``` + T := tree.Tree {namespace: "vocdoni"} + if T.init() != nil { fmt.Println("Cannot create tree database") } + err := T.addClaim([]byte("Hello you!")) + if err != nil { + fmt.Println("Claim already exist") + } + mpHex, err := T.genProof([]byte("Hello you!")) + fmt.Println(mpHex) + fmt.Println(T.checkProof([]byte("Hello you!"), mpHex)) + T.close() +``` + +### To-Do + ++ Add export/import methods diff --git a/tree/tree.go b/tree/tree.go new file mode 100644 index 0000000..d1578d2 --- /dev/null +++ b/tree/tree.go @@ -0,0 +1,62 @@ +package tree + +import ( + "github.com/iden3/go-iden3/db" + "github.com/iden3/go-iden3/merkletree" + mkcore "github.com/iden3/go-iden3/core" + common3 "github.com/iden3/go-iden3/common" + "os/user" +) + +type Tree struct { + namespace string + storage string + tree *merkletree.MerkleTree +} + +func (t *Tree) init() error { + if len(t.storage) < 1 { + usr, err := user.Current() + if err == nil { + t.storage = usr.HomeDir + "/.dvote/Tree" + } else { t.storage = "./dvoteTree" } + } + mtdb, err := db.NewLevelDbStorage(t.storage, false) + if err != nil { + return err + } + mt, err := merkletree.New(mtdb, 140) + if err != nil { + return err + } + t.tree = mt + return nil +} + +func (t *Tree) close() { + defer t.tree.Storage().Close() +} + +func (t *Tree) addClaim(data []byte) error { + claim := mkcore.NewGenericClaim(t.namespace, "default", data, nil) + return t.tree.Add(claim) +} + +func (t *Tree) genProof(data []byte) (string, error) { + claim := mkcore.NewGenericClaim(t.namespace, "default", data, nil) + mp, err := t.tree.GenerateProof(claim.Hi()) + if err!=nil { + return "", err + } + mpHex := common3.BytesToHex(mp) + return mpHex, nil +} + +func (t *Tree) checkProof(data []byte, mpHex string) (bool, error) { + mp, err := common3.HexToBytes(mpHex) + if err != nil { + return false, err + } + claim := mkcore.NewGenericClaim(t.namespace, "default", data, nil) + return merkletree.CheckProof(t.tree.Root(), mp, claim.Hi(), claim.Ht(), t.tree.NumLevels()), nil +} From 5fd182549f6dcb4a3ad4c717673766bb48cc8807 Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 14 Dec 2018 14:11:28 +0100 Subject: [PATCH 05/24] Add processHttp implementation Exposes a HTTP API to manage merkle trees Signed-off-by: p4u --- service/README.md | 40 ++++++++++++ service/processHttp.go | 138 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 178 insertions(+) create mode 100644 service/README.md create mode 100644 service/processHttp.go diff --git a/service/README.md b/service/README.md new file mode 100644 index 0000000..15d0bf0 --- /dev/null +++ b/service/README.md @@ -0,0 +1,40 @@ +## dvot-process http service + +#### start http server + +``` +processHttp.T.namespace = "vocdoni" +T.Init() +processHttp.Listen(1500, "http") +``` + +#### add claims + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim +{"error":false,"response":""} +``` + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim +{"error":false,"response":""} +``` + +#### generate proof + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof +{"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} +``` + +#### check proof + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof +{"error":false,"response":"invalid"} +``` + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof +{"error":false,"response":"valid"} +``` diff --git a/service/processHttp.go b/service/processHttp.go new file mode 100644 index 0000000..7e4c6d1 --- /dev/null +++ b/service/processHttp.go @@ -0,0 +1,138 @@ +package processHttp + +import ( + "encoding/json" + "fmt" + "net/http" + "time" + "log" + "github.com/vocdoni/dvote-process/tree" +) + +var T tree.Tree + +type Claim struct { + ProcessID string `json:"processID"` + ClaimData string `json:"claimData"` + ProofData string `json:"proofData"` +} + +type Result struct { + Error bool `json:"error"` + Response string `json:"response"` +} + +func reply(resp *Result, w http.ResponseWriter) { + err := json.NewEncoder(w).Encode(resp) + if err != nil { + http.Error(w, err.Error(), 500) + } else { + w.Header().Set("content-type", "application/json") + } +} + +func checkRequest(w http.ResponseWriter,req *http.Request) bool { + if req.Body == nil { + http.Error(w, "Please send a request body", 400) + return false + } + return true +} + +func claimHandler(w http.ResponseWriter, req *http.Request, op string) { + var c Claim + var resp Result + if ok := checkRequest(w, req); !ok { return } + // Decode JSON + err := json.NewDecoder(req.Body).Decode(&c) + if err != nil { + http.Error(w, err.Error(), 400) + return + } + + // Process data + log.Printf("Received: %s,%s,%s ", c.ProcessID, c.ClaimData, c.ProofData) + resp.Error = false + resp.Response = "" + + if len(c.ProcessID) > 0 { + T.Namespace = c.ProcessID + } else { + resp.Error = true + resp.Response = "processID is not valid" + reply(&resp, w) + return + } + + if len(c.ClaimData) < 0 { + resp.Error = true + resp.Response = "data not valid" + reply(&resp, w) + return + } + + if op == "add" { + err = T.AddClaim([]byte(c.ClaimData)) + } + + if op == "gen" { + resp.Response, err = T.GenProof([]byte(c.ClaimData)) + } + + if op == "check" { + if len(c.ProofData) < 1 { + resp.Error = true + resp.Response = "proofData not provided" + reply(&resp, w) + return + } + var validProof bool + validProof, err = T.CheckProof([]byte(c.ClaimData), c.ProofData) + if validProof { + resp.Response = "valid" + } else { + resp.Response = "invalid" + } + } + + if err != nil { + resp.Error = true + resp.Response = fmt.Sprint(err) + log.Print(err) + reply(&resp, w) + return + } + + reply(&resp, w) +} + +func Listen(port int, proto string) { + srv := &http.Server{ + Addr: fmt.Sprintf(":%d", port), + ReadHeaderTimeout: 4 * time.Second, + ReadTimeout: 4 * time.Second, + WriteTimeout: 4 * time.Second, + IdleTimeout: 3 * time.Second, + } + + http.HandleFunc("/addClaim", func(w http.ResponseWriter, r *http.Request) { + claimHandler(w, r, "add")}) + http.HandleFunc("/genProof", func(w http.ResponseWriter, r *http.Request) { + claimHandler(w, r, "gen")}) + http.HandleFunc("/checkProof", func(w http.ResponseWriter, r *http.Request) { + claimHandler(w, r, "check")}) + + if proto == "https" { + log.Print("Starting server in https mode") + if err := srv.ListenAndServeTLS("server.crt", "server.key"); err != nil { + panic(err) + } + } + if proto == "http" { + log.Print("Starting server in http mode") + srv.SetKeepAlivesEnabled(false) + if err := srv.ListenAndServe(); err != nil { + panic(err) + } + } +} From a504f461415c96920ee3fbb9a6c9c84c5e1ea8e8 Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 14 Dec 2018 14:13:13 +0100 Subject: [PATCH 06/24] Fix names and update README --- tree/README.md | 16 ++++++++-------- tree/tree.go | 40 ++++++++++++++++++++-------------------- 2 files changed, 28 insertions(+), 28 deletions(-) diff --git a/tree/README.md b/tree/README.md index 3e9af9d..7a75955 100644 --- a/tree/README.md +++ b/tree/README.md @@ -1,22 +1,22 @@ -#### Dvote Tree +## dvote Tree Implementation of dvote tree structure. Currently based on iden3 merkle tree. Example of usage: ``` - T := tree.Tree {namespace: "vocdoni"} - if T.init() != nil { fmt.Println("Cannot create tree database") } - err := T.addClaim([]byte("Hello you!")) + T := tree.Tree + if T.Init() != nil { fmt.Println("Cannot create tree database") } + err := T.AddClaim([]byte("Hello you!")) if err != nil { fmt.Println("Claim already exist") } - mpHex, err := T.genProof([]byte("Hello you!")) + mpHex, err := T.GenProof([]byte("Hello you!")) fmt.Println(mpHex) - fmt.Println(T.checkProof([]byte("Hello you!"), mpHex)) - T.close() + fmt.Println(T.CheckProof([]byte("Hello you!"), mpHex)) + T.Close() ``` -### To-Do +#### To-Do + Add export/import methods diff --git a/tree/tree.go b/tree/tree.go index d1578d2..7445859 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -9,19 +9,19 @@ import ( ) type Tree struct { - namespace string - storage string - tree *merkletree.MerkleTree + Namespace string + Storage string + Tree *merkletree.MerkleTree } -func (t *Tree) init() error { - if len(t.storage) < 1 { +func (t *Tree) Init() error { + if len(t.Storage) < 1 { usr, err := user.Current() if err == nil { - t.storage = usr.HomeDir + "/.dvote/Tree" - } else { t.storage = "./dvoteTree" } + t.Storage = usr.HomeDir + "/.dvote/Tree" + } else { t.Storage = "./dvoteTree" } } - mtdb, err := db.NewLevelDbStorage(t.storage, false) + mtdb, err := db.NewLevelDbStorage(t.Storage, false) if err != nil { return err } @@ -29,22 +29,22 @@ func (t *Tree) init() error { if err != nil { return err } - t.tree = mt + t.Tree = mt return nil } -func (t *Tree) close() { - defer t.tree.Storage().Close() +func (t *Tree) Close() { + defer t.Tree.Storage().Close() } -func (t *Tree) addClaim(data []byte) error { - claim := mkcore.NewGenericClaim(t.namespace, "default", data, nil) - return t.tree.Add(claim) +func (t *Tree) AddClaim(data []byte) error { + claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) + return t.Tree.Add(claim) } -func (t *Tree) genProof(data []byte) (string, error) { - claim := mkcore.NewGenericClaim(t.namespace, "default", data, nil) - mp, err := t.tree.GenerateProof(claim.Hi()) +func (t *Tree) GenProof(data []byte) (string, error) { + claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) + mp, err := t.Tree.GenerateProof(claim.Hi()) if err!=nil { return "", err } @@ -52,11 +52,11 @@ func (t *Tree) genProof(data []byte) (string, error) { return mpHex, nil } -func (t *Tree) checkProof(data []byte, mpHex string) (bool, error) { +func (t *Tree) CheckProof(data []byte, mpHex string) (bool, error) { mp, err := common3.HexToBytes(mpHex) if err != nil { return false, err } - claim := mkcore.NewGenericClaim(t.namespace, "default", data, nil) - return merkletree.CheckProof(t.tree.Root(), mp, claim.Hi(), claim.Ht(), t.tree.NumLevels()), nil + claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) + return merkletree.CheckProof(t.Tree.Root(), mp, claim.Hi(), claim.Ht(), t.Tree.NumLevels()), nil } From 4bd4f33f434b6c9b998e0e7e76edfa43ed5af128 Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 14 Dec 2018 14:17:17 +0100 Subject: [PATCH 07/24] Update README --- service/README.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/service/README.md b/service/README.md index 15d0bf0..7fc3ec8 100644 --- a/service/README.md +++ b/service/README.md @@ -3,8 +3,7 @@ #### start http server ``` -processHttp.T.namespace = "vocdoni" -T.Init() +processHttp.T.Init() processHttp.Listen(1500, "http") ``` From a503c6555e6636b6472ee2977316e8a6cc2c3e00 Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 28 Dec 2018 14:52:55 +0100 Subject: [PATCH 08/24] Change go-iden3 dependencies to vocdoni/go-iden3 New iden3 merkleTree implementation is not longer compatible. Until new documentation is availabl and vocdoni code is adapted, use frozen version of go-iden3 implementation. --- README.md | 2 +- service/processHttp.go | 2 +- tree/tree.go | 8 ++++---- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 819415b..9536351 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,2 @@ # dVote-process -Library to work with the creation and management of voting processes +Library to work with the creation and management of vocdoni census diff --git a/service/processHttp.go b/service/processHttp.go index 7e4c6d1..4c5f460 100644 --- a/service/processHttp.go +++ b/service/processHttp.go @@ -6,7 +6,7 @@ import ( "net/http" "time" "log" - "github.com/vocdoni/dvote-process/tree" + "github.com/vocdoni/dvote-census/tree" ) var T tree.Tree diff --git a/tree/tree.go b/tree/tree.go index 7445859..390af5b 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -1,10 +1,10 @@ package tree import ( - "github.com/iden3/go-iden3/db" - "github.com/iden3/go-iden3/merkletree" - mkcore "github.com/iden3/go-iden3/core" - common3 "github.com/iden3/go-iden3/common" + "github.com/vocdoni/go-iden3/db" + "github.com/vocdoni/go-iden3/merkletree" + mkcore "github.com/vocdoni/go-iden3/core" + common3 "github.com/vocdoni/go-iden3/common" "os/user" ) From 6fd9aea6c8989000210b15034cdde91bf1837917 Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 11 Jan 2019 17:59:38 +0100 Subject: [PATCH 09/24] Add getRoot method --- service/processHttp.go | 15 +++++++++++---- tree/tree.go | 4 ++++ 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/service/processHttp.go b/service/processHttp.go index 4c5f460..df19a7b 100644 --- a/service/processHttp.go +++ b/service/processHttp.go @@ -79,6 +79,10 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { resp.Response, err = T.GenProof([]byte(c.ClaimData)) } + if op == "root" { + resp.Response = T.GetRoot() + } + if op == "check" { if len(c.ProofData) < 1 { resp.Error = true @@ -109,18 +113,21 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { func Listen(port int, proto string) { srv := &http.Server{ Addr: fmt.Sprintf(":%d", port), - ReadHeaderTimeout: 4 * time.Second, + ReadHeaderTimeout: 4 * time.Second, ReadTimeout: 4 * time.Second, WriteTimeout: 4 * time.Second, IdleTimeout: 3 * time.Second, } http.HandleFunc("/addClaim", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "add")}) + claimHandler(w, r, "add")}) http.HandleFunc("/genProof", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "gen")}) + claimHandler(w, r, "gen")}) http.HandleFunc("/checkProof", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "check")}) + claimHandler(w, r, "check")}) + http.HandleFunc("/getRoot", func(w http.ResponseWriter, r *http.Request) { + claimHandler(w, r, "root")}) + if proto == "https" { log.Print("Starting server in https mode") diff --git a/tree/tree.go b/tree/tree.go index 390af5b..74263a3 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -60,3 +60,7 @@ func (t *Tree) CheckProof(data []byte, mpHex string) (bool, error) { claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) return merkletree.CheckProof(t.Tree.Root(), mp, claim.Hi(), claim.Ht(), t.Tree.NumLevels()), nil } + +func (t *Tree) GetRoot() (string) { + return t.Tree.Root().Hex() +} From 1922d46dc94311e212fd7c744abc6d3fe44bf411 Mon Sep 17 00:00:00 2001 From: Aleix Date: Thu, 17 Jan 2019 17:00:10 +0100 Subject: [PATCH 10/24] ProcessID => CensusID --- service/README.md | 10 +++++----- service/processHttp.go | 10 +++++----- 2 files changed, 10 insertions(+), 10 deletions(-) diff --git a/service/README.md b/service/README.md index 7fc3ec8..e95e19e 100644 --- a/service/README.md +++ b/service/README.md @@ -10,30 +10,30 @@ processHttp.Listen(1500, "http") #### add claims ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim {"error":false,"response":""} ``` ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim +curl -d '{"censusID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim {"error":false,"response":""} ``` #### generate proof ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof {"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} ``` #### check proof ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof {"error":false,"response":"invalid"} ``` ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof {"error":false,"response":"valid"} ``` diff --git a/service/processHttp.go b/service/processHttp.go index df19a7b..a6aff90 100644 --- a/service/processHttp.go +++ b/service/processHttp.go @@ -12,7 +12,7 @@ import ( var T tree.Tree type Claim struct { - ProcessID string `json:"processID"` + CensusID string `json:"censusID"` ClaimData string `json:"claimData"` ProofData string `json:"proofData"` } @@ -51,15 +51,15 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } // Process data - log.Printf("Received: %s,%s,%s ", c.ProcessID, c.ClaimData, c.ProofData) + log.Printf("Received: %s,%s,%s ", c.CensusID, c.ClaimData, c.ProofData) resp.Error = false resp.Response = "" - if len(c.ProcessID) > 0 { - T.Namespace = c.ProcessID + if len(c.CensusID) > 0 { + T.Namespace = c.CensusID } else { resp.Error = true - resp.Response = "processID is not valid" + resp.Response = "CensusID is not valid" reply(&resp, w) return } From c150bf570e85bc3192c1324075c27c1c79fa129f Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 18 Jan 2019 13:36:59 +0100 Subject: [PATCH 11/24] Implement authentication method First version, partially tested Signed-off-by: p4u --- service/README.md | 20 +++++++- service/processHttp.go | 109 ++++++++++++++++++++++++++++++----------- 2 files changed, 99 insertions(+), 30 deletions(-) diff --git a/service/README.md b/service/README.md index e95e19e..e8c0c39 100644 --- a/service/README.md +++ b/service/README.md @@ -4,9 +4,17 @@ ``` processHttp.T.Init() -processHttp.Listen(1500, "http") +processHttp.Listen(1500, "http", "") ``` +To enable authentication (using pubKey signature): + +``` +pubK := "39f54ce5293520b689f6658ea7f3401f4ff931fa3d90dea21ff901cdf82bb8aa" +processHttp.Listen(1500, "http", pubK) +``` + + #### add claims ``` @@ -14,6 +22,16 @@ curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:15 {"error":false,"response":""} ``` +``` +curl -d '{"censusID":"GoT_Favorite", +"claimData":"Jon Snow", +"timeStamp":"1547814675", +"signature":"a117c4ce12b29090884112ffe57e664f007e7ef142a1679996e2d34fd2b852fe76966e47932f1e9d3a54610d0f361383afe2d9aab096e15d136c236abb0a0d0e"}' http://localhost:1500/addClaim +{"error":false,"response":""} +``` + +The signature message is a concatenation of the following strings: `censusID, claimData, timeStamp` + ``` curl -d '{"censusID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim {"error":false,"response":""} diff --git a/service/processHttp.go b/service/processHttp.go index a6aff90..ef3dfea 100644 --- a/service/processHttp.go +++ b/service/processHttp.go @@ -3,22 +3,32 @@ package processHttp import ( "encoding/json" "fmt" + "log" "net/http" + "strconv" "time" - "log" + "github.com/vocdoni/dvote-census/tree" + "github.com/vocdoni/dvote-relay/crypto/signature" ) -var T tree.Tree +const authTimeWindow = 10 // Time window (seconds) in which TimeStamp will be accepted if auth enabled + +var authPubKey string + +var T tree.Tree // MerkleTree dvote-census library +var S signature.SignKeys // Signature dvote-relay library type Claim struct { - CensusID string `json:"censusID"` - ClaimData string `json:"claimData"` - ProofData string `json:"proofData"` + CensusID string `json:"censusID"` // References to MerkleTree namespace + ClaimData string `json:"claimData"` // Data to add to the MerkleTree + ProofData string `json:"proofData"` // MerkleProof to check + TimeStamp string `json:"timeStamp"` // Unix TimeStamp in seconds + Signature string `json:"signature"` // Signature as Hexadecimal String } type Result struct { - Error bool `json:"error"` + Error bool `json:"error"` Response string `json:"response"` } @@ -31,7 +41,7 @@ func reply(resp *Result, w http.ResponseWriter) { } } -func checkRequest(w http.ResponseWriter,req *http.Request) bool { +func checkRequest(w http.ResponseWriter, req *http.Request) bool { if req.Body == nil { http.Error(w, "Please send a request body", 400) return false @@ -39,10 +49,33 @@ func checkRequest(w http.ResponseWriter,req *http.Request) bool { return true } +func checkAuth(timestamp, signature, message string) bool { + if len(authPubKey) < 1 { + return true + } + currentTime := int64(time.Now().Unix()) + timeStampRemote, err := strconv.ParseInt(timestamp, 10, 32) + if err != nil { + log.Printf("Cannot parse timestamp data %s\n", err) + return false + } + if timeStampRemote < currentTime+authTimeWindow && + timeStampRemote > currentTime-authTimeWindow { + v, err := S.Verify(message, signature, authPubKey) + if err != nil { + log.Printf("Verification error: %s\n", err) + } + return v + } + return false +} + func claimHandler(w http.ResponseWriter, req *http.Request, op string) { var c Claim var resp Result - if ok := checkRequest(w, req); !ok { return } + if ok := checkRequest(w, req); !ok { + return + } // Decode JSON err := json.NewDecoder(req.Body).Decode(&c) if err != nil { @@ -51,7 +84,8 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } // Process data - log.Printf("Received: %s,%s,%s ", c.CensusID, c.ClaimData, c.ProofData) + log.Printf("Received: %s,%s,%s,%s,%s", c.CensusID, c.ClaimData, c.ProofData, + c.TimeStamp, c.Signature) resp.Error = false resp.Response = "" @@ -59,7 +93,7 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { T.Namespace = c.CensusID } else { resp.Error = true - resp.Response = "CensusID is not valid" + resp.Response = "censusID is not valid" reply(&resp, w) return } @@ -72,7 +106,14 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } if op == "add" { - err = T.AddClaim([]byte(c.ClaimData)) + msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) + log.Printf("Msg to check: %s", msg) + if auth := checkAuth(c.TimeStamp, c.Signature, msg); auth { + err = T.AddClaim([]byte(c.ClaimData)) + } else { + resp.Error = true + resp.Response = "invalid authentication" + } } if op == "gen" { @@ -110,36 +151,46 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { reply(&resp, w) } -func Listen(port int, proto string) { +func Listen(port int, proto string, pubKey string) { srv := &http.Server{ - Addr: fmt.Sprintf(":%d", port), + Addr: fmt.Sprintf(":%d", port), ReadHeaderTimeout: 4 * time.Second, - ReadTimeout: 4 * time.Second, - WriteTimeout: 4 * time.Second, - IdleTimeout: 3 * time.Second, + ReadTimeout: 4 * time.Second, + WriteTimeout: 4 * time.Second, + IdleTimeout: 3 * time.Second, } http.HandleFunc("/addClaim", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "add")}) + claimHandler(w, r, "add") + }) http.HandleFunc("/genProof", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "gen")}) + claimHandler(w, r, "gen") + }) http.HandleFunc("/checkProof", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "check")}) + claimHandler(w, r, "check") + }) http.HandleFunc("/getRoot", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "root")}) + claimHandler(w, r, "root") + }) + if len(pubKey) > 1 { + log.Printf("Enabling signature authentication with %s\n", pubKey) + authPubKey = pubKey + } else { + authPubKey = "" + } if proto == "https" { - log.Print("Starting server in https mode") - if err := srv.ListenAndServeTLS("server.crt", "server.key"); err != nil { - panic(err) - } + log.Print("Starting server in https mode") + if err := srv.ListenAndServeTLS("server.crt", "server.key"); err != nil { + panic(err) + } } if proto == "http" { - log.Print("Starting server in http mode") - srv.SetKeepAlivesEnabled(false) - if err := srv.ListenAndServe(); err != nil { - panic(err) - } + log.Print("Starting server in http mode") + srv.SetKeepAlivesEnabled(false) + if err := srv.ListenAndServe(); err != nil { + panic(err) + } } } From 4f72b436007f33807077eb224fc2fe0af2e21089 Mon Sep 17 00:00:00 2001 From: p4u Date: Tue, 22 Jan 2019 16:41:46 +0100 Subject: [PATCH 12/24] Add snapshot and dump methods Signed-off-by: p4u --- service/README.md | 29 ++++++++++ service/processHttp.go | 39 ++++++++++++- tree/README.md | 2 +- tree/tree.go | 124 +++++++++++++++++++++++++++++------------ 4 files changed, 156 insertions(+), 38 deletions(-) diff --git a/service/README.md b/service/README.md index e8c0c39..b8def01 100644 --- a/service/README.md +++ b/service/README.md @@ -19,6 +19,7 @@ processHttp.Listen(1500, "http", pubK) ``` curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim + {"error":false,"response":""} ``` @@ -27,6 +28,7 @@ curl -d '{"censusID":"GoT_Favorite", "claimData":"Jon Snow", "timeStamp":"1547814675", "signature":"a117c4ce12b29090884112ffe57e664f007e7ef142a1679996e2d34fd2b852fe76966e47932f1e9d3a54610d0f361383afe2d9aab096e15d136c236abb0a0d0e"}' http://localhost:1500/addClaim + {"error":false,"response":""} ``` @@ -34,6 +36,7 @@ The signature message is a concatenation of the following strings: `censusID, cl ``` curl -d '{"censusID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim + {"error":false,"response":""} ``` @@ -41,6 +44,7 @@ curl -d '{"censusID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500 ``` curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof + {"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} ``` @@ -48,10 +52,35 @@ curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:15 ``` curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof + {"error":false,"response":"invalid"} ``` ``` curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof + {"error":false,"response":"valid"} ``` + +#### make snapshot + +Snapshots are static and unmutable copies of a specific census + +``` +curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/snapshot + +{"error":false,"response":"snaphost.GoT_Favorite.1548169813"} +``` + +The name for the snapshot is "snaphost.GoT_Favorite.1548169813" + +Now you can use it as censusID for checkProof, genProof and dump. But addClaim is not longer allowed. + +#### dump + +Dump contents of a specific censusID (values) + +``` +curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/dump +{"error":false,"response":"[\"Tyrion\",\"Jon Snow\"]"} +``` \ No newline at end of file diff --git a/service/processHttp.go b/service/processHttp.go index ef3dfea..4361efe 100644 --- a/service/processHttp.go +++ b/service/processHttp.go @@ -107,7 +107,6 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { if op == "add" { msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) - log.Printf("Msg to check: %s", msg) if auth := checkAuth(c.TimeStamp, c.Signature, msg); auth { err = T.AddClaim([]byte(c.ClaimData)) } else { @@ -124,6 +123,38 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { resp.Response = T.GetRoot() } + if op == "dump" { + values, err := T.Dump() + if err != nil { + resp.Error = true + resp.Response = fmt.Sprint(err) + } else { + jValues, err := json.Marshal(values) + if err != nil { + resp.Error = true + resp.Response = fmt.Sprint(err) + } else { + resp.Response = string(jValues) + } + } + } + + if op == "snapshot" { + msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) + if auth := checkAuth(c.TimeStamp, c.Signature, msg); auth { + snapshotNamespace, err := T.Snapshot() + if err != nil { + resp.Error = true + resp.Response = fmt.Sprint(err) + } else { + resp.Response = snapshotNamespace + } + } else { + resp.Error = true + resp.Response = "invalid authentication" + } + } + if op == "check" { if len(c.ProofData) < 1 { resp.Error = true @@ -172,6 +203,12 @@ func Listen(port int, proto string, pubKey string) { http.HandleFunc("/getRoot", func(w http.ResponseWriter, r *http.Request) { claimHandler(w, r, "root") }) + http.HandleFunc("/snapshot", func(w http.ResponseWriter, r *http.Request) { + claimHandler(w, r, "snapshot") + }) + http.HandleFunc("/dump", func(w http.ResponseWriter, r *http.Request) { + claimHandler(w, r, "dump") + }) if len(pubKey) > 1 { log.Printf("Enabling signature authentication with %s\n", pubKey) diff --git a/tree/README.md b/tree/README.md index 7a75955..8904de6 100644 --- a/tree/README.md +++ b/tree/README.md @@ -19,4 +19,4 @@ Example of usage: #### To-Do -+ Add export/import methods +Avoid duplicates on dump/snapshot \ No newline at end of file diff --git a/tree/tree.go b/tree/tree.go index 74263a3..618f86e 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -1,36 +1,45 @@ package tree import ( - "github.com/vocdoni/go-iden3/db" - "github.com/vocdoni/go-iden3/merkletree" - mkcore "github.com/vocdoni/go-iden3/core" - common3 "github.com/vocdoni/go-iden3/common" - "os/user" + "errors" + "fmt" + "os/user" + "strings" + "time" + + common3 "github.com/vocdoni/go-iden3/common" + mkcore "github.com/vocdoni/go-iden3/core" + "github.com/vocdoni/go-iden3/db" + "github.com/vocdoni/go-iden3/merkletree" ) type Tree struct { - Namespace string - Storage string - Tree *merkletree.MerkleTree + Namespace string + Storage string + Tree *merkletree.MerkleTree + DbStorage *db.LevelDbStorage } func (t *Tree) Init() error { - if len(t.Storage) < 1 { - usr, err := user.Current() - if err == nil { - t.Storage = usr.HomeDir + "/.dvote/Tree" - } else { t.Storage = "./dvoteTree" } - } - mtdb, err := db.NewLevelDbStorage(t.Storage, false) + if len(t.Storage) < 1 { + usr, err := user.Current() + if err == nil { + t.Storage = usr.HomeDir + "/.dvote/Tree" + } else { + t.Storage = "./dvoteTree" + } + } + mtdb, err := db.NewLevelDbStorage(t.Storage, false) if err != nil { - return err + return err } mt, err := merkletree.New(mtdb, 140) if err != nil { - return err + return err } - t.Tree = mt - return nil + t.DbStorage = mtdb + t.Tree = mt + return nil } func (t *Tree) Close() { @@ -38,29 +47,72 @@ func (t *Tree) Close() { } func (t *Tree) AddClaim(data []byte) error { - claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) - return t.Tree.Add(claim) + isSnapshot := strings.Contains(t.Namespace, "snapshot.") + if isSnapshot { + return errors.New("No new claims can be added to a Snapshot") + } + claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) + return t.Tree.Add(claim) } func (t *Tree) GenProof(data []byte) (string, error) { - claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) - mp, err := t.Tree.GenerateProof(claim.Hi()) - if err!=nil { - return "", err - } - mpHex := common3.BytesToHex(mp) - return mpHex, nil + claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) + mp, err := t.Tree.GenerateProof(claim.Hi()) + if err != nil { + return "", err + } + mpHex := common3.BytesToHex(mp) + return mpHex, nil } func (t *Tree) CheckProof(data []byte, mpHex string) (bool, error) { - mp, err := common3.HexToBytes(mpHex) - if err != nil { - return false, err - } - claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) - return merkletree.CheckProof(t.Tree.Root(), mp, claim.Hi(), claim.Ht(), t.Tree.NumLevels()), nil + mp, err := common3.HexToBytes(mpHex) + if err != nil { + return false, err + } + claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) + return merkletree.CheckProof(t.Tree.Root(), mp, claim.Hi(), claim.Ht(), t.Tree.NumLevels()), nil +} + +func (t *Tree) GetRoot() string { + return t.Tree.Root().Hex() +} + +/* Dump, Export and Snapshot functions are a bit tricky. + Since go-iden3 does not provide the necessary tools. Low level operations must be performed. + Once go-iden3 API is mature enough, these functions must be adapted. + + To explore: Values are stored twice in the BD? +*/ +func (t *Tree) Dump() ([]string, error) { + var response []string + substorage := t.DbStorage.WithPrefix([]byte(t.Namespace)) + nsHash := merkletree.HashBytes([]byte(t.Namespace)) + substorage.Iterate(func(key, value []byte) { + nsValue := value[5:37] + if fmt.Sprint(nsHash) == fmt.Sprint(nsValue) { + response = append(response, string(value[69:])) + } + }) + return response, nil } -func (t *Tree) GetRoot() (string) { - return t.Tree.Root().Hex() +func (t *Tree) Snapshot() (string, error) { + substorage := t.DbStorage.WithPrefix([]byte(t.Namespace)) + nsHash := merkletree.HashBytes([]byte(t.Namespace)) + currentTime := int64(time.Now().Unix()) + snapshotNamespace := fmt.Sprintf("snapshot.%s.%d", t.Namespace, currentTime) + substorage.Iterate(func(key, value []byte) { + nsValue := value[5:37] + if fmt.Sprint(nsHash) == fmt.Sprint(nsValue) { + data := value[69:] + //fmt.Printf(" Adding value: %s\n", data) + claim := mkcore.NewGenericClaim(snapshotNamespace, "default", data, nil) + err := t.Tree.Add(claim) + if err != nil { + fmt.Println(err) + } + } + }) + return snapshotNamespace, nil } From 408457d65bca54a54648c5b8fce8a5803afbf8f9 Mon Sep 17 00:00:00 2001 From: p4u Date: Thu, 24 Jan 2019 14:40:13 +0100 Subject: [PATCH 13/24] Fix auth error return Signed-off-by: p4u --- service/processHttp.go | 22 ++++++---------------- 1 file changed, 6 insertions(+), 16 deletions(-) diff --git a/service/processHttp.go b/service/processHttp.go index 4361efe..2b19a8b 100644 --- a/service/processHttp.go +++ b/service/processHttp.go @@ -159,26 +159,16 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { if len(c.ProofData) < 1 { resp.Error = true resp.Response = "proofData not provided" - reply(&resp, w) - return - } - var validProof bool - validProof, err = T.CheckProof([]byte(c.ClaimData), c.ProofData) - if validProof { - resp.Response = "valid" } else { - resp.Response = "invalid" + validProof, _ := T.CheckProof([]byte(c.ClaimData), c.ProofData) + if validProof { + resp.Response = "valid" + } else { + resp.Response = "invalid" + } } } - if err != nil { - resp.Error = true - resp.Response = fmt.Sprint(err) - log.Print(err) - reply(&resp, w) - return - } - reply(&resp, w) } From 04dd103bb1040faaad92d2f9a16d2b0af67b9553 Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 25 Jan 2019 19:04:58 +0100 Subject: [PATCH 14/24] Add censushttp cmd. Rename processHttp to censusmanager Signed-off-by: p4u --- cmd/censushttp/README.md | 51 ++++++++++++++++++++ cmd/censushttp/censushttp.go | 30 ++++++++++++ service/{processHttp.go => censusmanager.go} | 2 +- 3 files changed, 82 insertions(+), 1 deletion(-) create mode 100644 cmd/censushttp/README.md create mode 100644 cmd/censushttp/censushttp.go rename service/{processHttp.go => censusmanager.go} (99%) diff --git a/cmd/censushttp/README.md b/cmd/censushttp/README.md new file mode 100644 index 0000000..2a06617 --- /dev/null +++ b/cmd/censushttp/README.md @@ -0,0 +1,51 @@ +# censusService +Reference implementation of a voting census service running on the Vocdoni platform + +## Census HTTP service + +#### Compile + +``` +go get github.com/vocdoni/censusService +go build -o httpService http_census_service.go +``` + +#### Usage + +``` +./httpService 1500 +2018/12/17 09:54:20 Starting process HTTP service on port 1500 +2018/12/17 09:54:20 Starting server in http mode +``` + +##### add claims + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim +{"error":false,"response":""} +``` + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim +{"error":false,"response":""} +``` + +##### generate proof + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof +{"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} +``` + +##### check proof + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof +{"error":false,"response":"invalid"} +``` + +``` +curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof +{"error":false,"response":"valid"} +``` + diff --git a/cmd/censushttp/censushttp.go b/cmd/censushttp/censushttp.go new file mode 100644 index 0000000..68d545a --- /dev/null +++ b/cmd/censushttp/censushttp.go @@ -0,0 +1,30 @@ +package main + +import ( + "log" + "os" + "strconv" + + censusmanager "github.com/vocdoni/dvote-census/service" +) + +func main() { + if len(os.Args) < 2 { + log.Fatal("Usage: " + os.Args[0] + " [pubKey]") + os.Exit(2) + } + port, err := strconv.Atoi(os.Args[1]) + if err != nil { + log.Fatal(err) + os.Exit(2) + } + pubK := "" + if len(os.Args) > 2 { + log.Print("Public key authentication enabled") + pubK = os.Args[2] + } + log.Print("Starting process HTTP service on port " + os.Args[1]) + censusmanager.T.Init() + censusmanager.Listen(port, "http", pubK) + +} diff --git a/service/processHttp.go b/service/censusmanager.go similarity index 99% rename from service/processHttp.go rename to service/censusmanager.go index 2b19a8b..911dae4 100644 --- a/service/processHttp.go +++ b/service/censusmanager.go @@ -1,4 +1,4 @@ -package processHttp +package censusmanager import ( "encoding/json" From 6551f598cfc1d568eae1a807358f11d84d224161 Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 25 Jan 2019 21:20:38 +0100 Subject: [PATCH 15/24] Use Root Hash as censusID when creating a snapshot. Add more authentication control. Fix snapshot namespace issue. Improve documentation. Signed-off-by: p4u --- cmd/censushttp/README.md | 101 +++++++++++++++++++++++++++++++++++---- service/README.md | 75 +---------------------------- service/censusmanager.go | 25 +++++++--- tree/tree.go | 14 +++--- 4 files changed, 119 insertions(+), 96 deletions(-) diff --git a/cmd/censushttp/README.md b/cmd/censushttp/README.md index 2a06617..f00433d 100644 --- a/cmd/censushttp/README.md +++ b/cmd/censushttp/README.md @@ -1,23 +1,106 @@ -# censusService -Reference implementation of a voting census service running on the Vocdoni platform - ## Census HTTP service +Reference implementation of a voting census service running on the Vocdoni platform + #### Compile +In a GO ready environment: + ``` -go get github.com/vocdoni/censusService -go build -o httpService http_census_service.go +go get -u github.com/vocdoni/dvote-census +go build -o censusHttpService github.com/vocdoni/dvote-census/cmd/censushttp ``` #### Usage +`./censusHttpService [publicKey]` + +Example: + ``` -./httpService 1500 +./censusHttpService 1500 2018/12/17 09:54:20 Starting process HTTP service on port 1500 2018/12/17 09:54:20 Starting server in http mode ``` +#### add claims + +``` +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim + +{"error":false,"response":""} +``` + +If public key authentication enabled, `signature` field is required using Nancl signature. + +The signed message is expected to be a concatenation of the following fields: `censusID, claimData, timeStamp` + +``` +curl -d '{"censusID":"GoT_Favorite", +"claimData":"Jon Snow", +"timeStamp":"1547814675", +"signature":"a117c4ce12b29090884112ffe57e664f007e7ef142a1679996e2d34fd2b852fe76966e47932f1e9d3a54610d0f361383afe2d9aab096e15d136c236abb0a0d0e"}' http://localhost:1500/addClaim + +{"error":false,"response":""} +``` + + +#### generate proof + +``` +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof + +{"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} +``` + +#### check proof + +``` +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof + +{"error":false,"response":"invalid"} +``` + +``` +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof + +{"error":false,"response":"valid"} +``` + +#### make snapshot + +Snapshots are static and unmutable copies of a specific census + +``` +curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/snapshot + +{"error":false,"response":"0x8647692e073a10980d821764c65ca3fddbc606bb936f9812a7a806bfa97df152"} +``` + +The name for the snapshot is "0x8647692e073a10980d821764c65ca3fddbc606bb936f9812a7a806bfa97df152" which represents the Root Hash. + +Now you can use it as censusID for checkProof, getRoot, genProof and dump. But addClaim is not longer allowed. + +#### dump + +Dump contents of a specific censusID (values) + +``` +curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/dump +{"error":false,"response":"[\"Tyrion\",\"Jon Snow\"]"} +``` + + + + + + + + + + + + ##### add claims ``` @@ -33,19 +116,19 @@ curl -d '{"processID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:150 ##### generate proof ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof {"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} ``` ##### check proof ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof {"error":false,"response":"invalid"} ``` ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof {"error":false,"response":"valid"} ``` diff --git a/service/README.md b/service/README.md index b8def01..8263538 100644 --- a/service/README.md +++ b/service/README.md @@ -1,6 +1,6 @@ -## dvot-process http service +## dvot-census http service library -#### start http server +#### How to use it ``` processHttp.T.Init() @@ -13,74 +13,3 @@ To enable authentication (using pubKey signature): pubK := "39f54ce5293520b689f6658ea7f3401f4ff931fa3d90dea21ff901cdf82bb8aa" processHttp.Listen(1500, "http", pubK) ``` - - -#### add claims - -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim - -{"error":false,"response":""} -``` - -``` -curl -d '{"censusID":"GoT_Favorite", -"claimData":"Jon Snow", -"timeStamp":"1547814675", -"signature":"a117c4ce12b29090884112ffe57e664f007e7ef142a1679996e2d34fd2b852fe76966e47932f1e9d3a54610d0f361383afe2d9aab096e15d136c236abb0a0d0e"}' http://localhost:1500/addClaim - -{"error":false,"response":""} -``` - -The signature message is a concatenation of the following strings: `censusID, claimData, timeStamp` - -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim - -{"error":false,"response":""} -``` - -#### generate proof - -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof - -{"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} -``` - -#### check proof - -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof - -{"error":false,"response":"invalid"} -``` - -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof - -{"error":false,"response":"valid"} -``` - -#### make snapshot - -Snapshots are static and unmutable copies of a specific census - -``` -curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/snapshot - -{"error":false,"response":"snaphost.GoT_Favorite.1548169813"} -``` - -The name for the snapshot is "snaphost.GoT_Favorite.1548169813" - -Now you can use it as censusID for checkProof, genProof and dump. But addClaim is not longer allowed. - -#### dump - -Dump contents of a specific censusID (values) - -``` -curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/dump -{"error":false,"response":"[\"Tyrion\",\"Jon Snow\"]"} -``` \ No newline at end of file diff --git a/service/censusmanager.go b/service/censusmanager.go index 911dae4..6b56b85 100644 --- a/service/censusmanager.go +++ b/service/censusmanager.go @@ -6,6 +6,7 @@ import ( "log" "net/http" "strconv" + "strings" "time" "github.com/vocdoni/dvote-census/tree" @@ -84,8 +85,8 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } // Process data - log.Printf("Received: %s,%s,%s,%s,%s", c.CensusID, c.ClaimData, c.ProofData, - c.TimeStamp, c.Signature) + log.Printf("Received censusID:{%s} claimData:{%s} proofData:{%s} timeStamp:{%s} signature:{%s}\n", + c.CensusID, c.ClaimData, c.ProofData, c.TimeStamp, c.Signature) resp.Error = false resp.Response = "" @@ -108,7 +109,12 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { if op == "add" { msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) if auth := checkAuth(c.TimeStamp, c.Signature, msg); auth { - err = T.AddClaim([]byte(c.ClaimData)) + if strings.HasPrefix(c.CensusID, "0x") { + resp.Error = true + resp.Response = "add claim to snapshot is not allowed" + } else { + err = T.AddClaim([]byte(c.ClaimData)) + } } else { resp.Error = true resp.Response = "invalid authentication" @@ -142,12 +148,17 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { if op == "snapshot" { msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) if auth := checkAuth(c.TimeStamp, c.Signature, msg); auth { - snapshotNamespace, err := T.Snapshot() - if err != nil { + if strings.HasPrefix(c.CensusID, "0x") { resp.Error = true - resp.Response = fmt.Sprint(err) + resp.Response = "snapshot an snapshot makes no sense" } else { - resp.Response = snapshotNamespace + snapshotNamespace, err := T.Snapshot() + if err != nil { + resp.Error = true + resp.Response = fmt.Sprint(err) + } else { + resp.Response = snapshotNamespace + } } } else { resp.Error = true diff --git a/tree/tree.go b/tree/tree.go index 618f86e..88de057 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -5,7 +5,6 @@ import ( "fmt" "os/user" "strings" - "time" common3 "github.com/vocdoni/go-iden3/common" mkcore "github.com/vocdoni/go-iden3/core" @@ -47,7 +46,7 @@ func (t *Tree) Close() { } func (t *Tree) AddClaim(data []byte) error { - isSnapshot := strings.Contains(t.Namespace, "snapshot.") + isSnapshot := strings.HasPrefix(t.Namespace, "0x") if isSnapshot { return errors.New("No new claims can be added to a Snapshot") } @@ -100,15 +99,16 @@ func (t *Tree) Dump() ([]string, error) { func (t *Tree) Snapshot() (string, error) { substorage := t.DbStorage.WithPrefix([]byte(t.Namespace)) nsHash := merkletree.HashBytes([]byte(t.Namespace)) - currentTime := int64(time.Now().Unix()) - snapshotNamespace := fmt.Sprintf("snapshot.%s.%d", t.Namespace, currentTime) + snapshotNamespace := t.GetRoot() + fmt.Printf("Snapshoting %s\n", snapshotNamespace) + t.Namespace = snapshotNamespace substorage.Iterate(func(key, value []byte) { nsValue := value[5:37] if fmt.Sprint(nsHash) == fmt.Sprint(nsValue) { + fmt.Printf("%x\n", value) data := value[69:] - //fmt.Printf(" Adding value: %s\n", data) - claim := mkcore.NewGenericClaim(snapshotNamespace, "default", data, nil) - err := t.Tree.Add(claim) + //claim := mkcore.NewGenericClaim(snapshotNamespace, "default", data, nil) + err := t.AddClaim(data) if err != nil { fmt.Println(err) } From abb712d2b647b337fe571000410868445ddb78d1 Mon Sep 17 00:00:00 2001 From: p4u Date: Thu, 31 Jan 2019 18:54:02 +0100 Subject: [PATCH 16/24] Adapt merkletree to new go-iden3 implementation First iteration, more work needs to be done. Signed-off-by: p4u --- tree/tree.go | 96 ++++++++++++++++++++++++++-------------------------- 1 file changed, 48 insertions(+), 48 deletions(-) diff --git a/tree/tree.go b/tree/tree.go index 88de057..ae1d5fa 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -4,12 +4,11 @@ import ( "errors" "fmt" "os/user" - "strings" - common3 "github.com/vocdoni/go-iden3/common" - mkcore "github.com/vocdoni/go-iden3/core" - "github.com/vocdoni/go-iden3/db" - "github.com/vocdoni/go-iden3/merkletree" + common3 "github.com/iden3/go-iden3/common" + mkcore "github.com/iden3/go-iden3/core" + db "github.com/iden3/go-iden3/db" + merkletree "github.com/iden3/go-iden3/merkletree" ) type Tree struct { @@ -32,7 +31,7 @@ func (t *Tree) Init() error { if err != nil { return err } - mt, err := merkletree.New(mtdb, 140) + mt, err := merkletree.NewMerkleTree(mtdb, 140) if err != nil { return err } @@ -45,74 +44,75 @@ func (t *Tree) Close() { defer t.Tree.Storage().Close() } +func (t *Tree) GetClaim(data []byte) (*mkcore.ClaimBasic, error) { + if len(data) > 496/8 { + return nil, errors.New("claim data too large") + } + for i := len(data); i <= 496/8; i++ { + data = append(data, byte('0')) + } + var indexSlot [400 / 8]byte + var dataSlot [496 / 8]byte + copy(indexSlot[:], data[:400/8]) + copy(dataSlot[:], data[:496/8]) + e := mkcore.NewClaimBasic(indexSlot, dataSlot) + return e, nil +} + func (t *Tree) AddClaim(data []byte) error { - isSnapshot := strings.HasPrefix(t.Namespace, "0x") - if isSnapshot { - return errors.New("No new claims can be added to a Snapshot") + e, err := t.GetClaim(data) + if err != nil { + return err } - claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) - return t.Tree.Add(claim) + return t.Tree.Add(e.Entry()) } func (t *Tree) GenProof(data []byte) (string, error) { - claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) - mp, err := t.Tree.GenerateProof(claim.Hi()) + e, err := t.GetClaim(data) if err != nil { return "", err } - mpHex := common3.BytesToHex(mp) + mp, err := t.Tree.GenerateProof(e.Entry().HIndex()) + if err != nil { + return "", err + } + mpHex := common3.HexEncode(mp.Bytes()) return mpHex, nil } func (t *Tree) CheckProof(data []byte, mpHex string) (bool, error) { - mp, err := common3.HexToBytes(mpHex) + mpBytes, err := common3.HexDecode(mpHex) if err != nil { return false, err } - claim := mkcore.NewGenericClaim(t.Namespace, "default", data, nil) - return merkletree.CheckProof(t.Tree.Root(), mp, claim.Hi(), claim.Ht(), t.Tree.NumLevels()), nil + mp, err := merkletree.NewProofFromBytes(mpBytes) + if err != nil { + return false, err + } + e, err := t.GetClaim(data) + if err != nil { + return false, err + } + return merkletree.VerifyProof(t.Tree.RootKey(), mp, + e.Entry().HIndex(), e.Entry().HValue()), nil } func (t *Tree) GetRoot() string { - return t.Tree.Root().Hex() + return t.Tree.RootKey().String() } -/* Dump, Export and Snapshot functions are a bit tricky. - Since go-iden3 does not provide the necessary tools. Low level operations must be performed. - Once go-iden3 API is mature enough, these functions must be adapted. - - To explore: Values are stored twice in the BD? -*/ func (t *Tree) Dump() ([]string, error) { var response []string - substorage := t.DbStorage.WithPrefix([]byte(t.Namespace)) - nsHash := merkletree.HashBytes([]byte(t.Namespace)) - substorage.Iterate(func(key, value []byte) { - nsValue := value[5:37] - if fmt.Sprint(nsHash) == fmt.Sprint(nsValue) { - response = append(response, string(value[69:])) + + err := t.Tree.Walk(t.Tree.RootKey(), func(n *merkletree.Node) { + if n.Type == merkletree.NodeTypeLeaf { + response = append(response, fmt.Sprintf("|%s", n.Entry.Data)) } }) - return response, nil + return response, err } func (t *Tree) Snapshot() (string, error) { - substorage := t.DbStorage.WithPrefix([]byte(t.Namespace)) - nsHash := merkletree.HashBytes([]byte(t.Namespace)) - snapshotNamespace := t.GetRoot() - fmt.Printf("Snapshoting %s\n", snapshotNamespace) - t.Namespace = snapshotNamespace - substorage.Iterate(func(key, value []byte) { - nsValue := value[5:37] - if fmt.Sprint(nsHash) == fmt.Sprint(nsValue) { - fmt.Printf("%x\n", value) - data := value[69:] - //claim := mkcore.NewGenericClaim(snapshotNamespace, "default", data, nil) - err := t.AddClaim(data) - if err != nil { - fmt.Println(err) - } - } - }) + snapshotNamespace := t.Tree.RootKey().String() return snapshotNamespace, nil } From dad369ebff6b79c4a2de01b13348c96188321502 Mon Sep 17 00:00:00 2001 From: p4u Date: Fri, 1 Feb 2019 20:20:34 +0100 Subject: [PATCH 17/24] Keep adapting the code to new merkletree implementation Use different database storages and merkletree for each censusID Signed-off-by: p4u --- cmd/censushttp/README.md | 10 +++-- cmd/censushttp/censushttp.go | 23 +++++++---- service/README.md | 15 -------- service/censusmanager.go | 74 ++++++++++++++++++------------------ tree/tree.go | 22 ++++++++--- 5 files changed, 74 insertions(+), 70 deletions(-) delete mode 100644 service/README.md diff --git a/cmd/censushttp/README.md b/cmd/censushttp/README.md index f00433d..b52242e 100644 --- a/cmd/censushttp/README.md +++ b/cmd/censushttp/README.md @@ -13,14 +13,14 @@ go build -o censusHttpService github.com/vocdoni/dvote-census/cmd/censushttp #### Usage -`./censusHttpService [publicKey]` +`./censusHttpService [:pubKey] [[:pubKey] ...]` Example: ``` -./censusHttpService 1500 -2018/12/17 09:54:20 Starting process HTTP service on port 1500 -2018/12/17 09:54:20 Starting server in http mode +./censusHttpService 1500 Got_Favorite +2019/02/01 20:01:15 Starting process HTTP service on port 1500 for namespace GoT_Favorite +2019/02/01 20:01:15 Starting server in http mode ``` #### add claims @@ -35,6 +35,8 @@ If public key authentication enabled, `signature` field is required using Nancl The signed message is expected to be a concatenation of the following fields: `censusID, claimData, timeStamp` +There is a time window of 10 seconds while the signature is considered valid. + ``` curl -d '{"censusID":"GoT_Favorite", "claimData":"Jon Snow", diff --git a/cmd/censushttp/censushttp.go b/cmd/censushttp/censushttp.go index 68d545a..b80c64c 100644 --- a/cmd/censushttp/censushttp.go +++ b/cmd/censushttp/censushttp.go @@ -4,13 +4,15 @@ import ( "log" "os" "strconv" + "strings" censusmanager "github.com/vocdoni/dvote-census/service" ) func main() { if len(os.Args) < 2 { - log.Fatal("Usage: " + os.Args[0] + " [pubKey]") + log.Fatal("Usage: " + os.Args[0] + + " [:pubKey] [[:pubKey]]...") os.Exit(2) } port, err := strconv.Atoi(os.Args[1]) @@ -18,13 +20,18 @@ func main() { log.Fatal(err) os.Exit(2) } - pubK := "" - if len(os.Args) > 2 { - log.Print("Public key authentication enabled") - pubK = os.Args[2] + for i := 2; i < len(os.Args); i++ { + s := strings.Split(os.Args[i], ":") + ns := s[0] + pubK := "" + if len(s) > 1 { + pubK = s[1] + log.Printf("Public Key authentication enabled on namespace %s\n", ns) + } + censusmanager.AddNamespace(ns, pubK) + log.Printf("Starting process HTTP service on port %d for namespace %s\n", + port, ns) } - log.Print("Starting process HTTP service on port " + os.Args[1]) - censusmanager.T.Init() - censusmanager.Listen(port, "http", pubK) + censusmanager.Listen(port, "http") } diff --git a/service/README.md b/service/README.md deleted file mode 100644 index 8263538..0000000 --- a/service/README.md +++ /dev/null @@ -1,15 +0,0 @@ -## dvot-census http service library - -#### How to use it - -``` -processHttp.T.Init() -processHttp.Listen(1500, "http", "") -``` - -To enable authentication (using pubKey signature): - -``` -pubK := "39f54ce5293520b689f6658ea7f3401f4ff931fa3d90dea21ff901cdf82bb8aa" -processHttp.Listen(1500, "http", pubK) -``` diff --git a/service/censusmanager.go b/service/censusmanager.go index 6b56b85..4bb6da1 100644 --- a/service/censusmanager.go +++ b/service/censusmanager.go @@ -9,16 +9,14 @@ import ( "strings" "time" - "github.com/vocdoni/dvote-census/tree" - "github.com/vocdoni/dvote-relay/crypto/signature" + tree "github.com/vocdoni/dvote-census/tree" + signature "github.com/vocdoni/dvote-relay/crypto/signature" ) -const authTimeWindow = 10 // Time window (seconds) in which TimeStamp will be accepted if auth enabled - -var authPubKey string - -var T tree.Tree // MerkleTree dvote-census library -var S signature.SignKeys // Signature dvote-relay library +const authTimeWindow = 10 // Time window (seconds) in which TimeStamp will be accepted if auth enabled +var MkTrees map[string]*tree.Tree // MerkleTree dvote-census library +var Signatures map[string]string +var Signature signature.SignKeys // Signature dvote-relay library type Claim struct { CensusID string `json:"censusID"` // References to MerkleTree namespace @@ -33,6 +31,20 @@ type Result struct { Response string `json:"response"` } +func AddNamespace(name, pubKey string) { + if len(MkTrees) == 0 { + MkTrees = make(map[string]*tree.Tree) + } + if len(Signatures) == 0 { + Signatures = make(map[string]string) + } + + mkTree := tree.Tree{} + mkTree.Init(name) + MkTrees[name] = &mkTree + Signatures[name] = pubKey +} + func reply(resp *Result, w http.ResponseWriter) { err := json.NewEncoder(w).Encode(resp) if err != nil { @@ -50,8 +62,8 @@ func checkRequest(w http.ResponseWriter, req *http.Request) bool { return true } -func checkAuth(timestamp, signature, message string) bool { - if len(authPubKey) < 1 { +func checkAuth(timestamp, signature, pubKey, message string) bool { + if len(pubKey) < 1 { return true } currentTime := int64(time.Now().Unix()) @@ -62,7 +74,7 @@ func checkAuth(timestamp, signature, message string) bool { } if timeStampRemote < currentTime+authTimeWindow && timeStampRemote > currentTime-authTimeWindow { - v, err := S.Verify(message, signature, authPubKey) + v, err := Signature.Verify(message, signature, pubKey) if err != nil { log.Printf("Verification error: %s\n", err) } @@ -74,6 +86,7 @@ func checkAuth(timestamp, signature, message string) bool { func claimHandler(w http.ResponseWriter, req *http.Request, op string) { var c Claim var resp Result + if ok := checkRequest(w, req); !ok { return } @@ -89,31 +102,25 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { c.CensusID, c.ClaimData, c.ProofData, c.TimeStamp, c.Signature) resp.Error = false resp.Response = "" - + censusFound := false if len(c.CensusID) > 0 { - T.Namespace = c.CensusID - } else { - resp.Error = true - resp.Response = "censusID is not valid" - reply(&resp, w) - return + _, censusFound = MkTrees[c.CensusID] } - - if len(c.ClaimData) < 0 { + if !censusFound { resp.Error = true - resp.Response = "data not valid" + resp.Response = "censusID not valid or not found" reply(&resp, w) return } if op == "add" { msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) - if auth := checkAuth(c.TimeStamp, c.Signature, msg); auth { + if auth := checkAuth(c.TimeStamp, c.Signature, Signatures[c.CensusID], msg); auth { if strings.HasPrefix(c.CensusID, "0x") { resp.Error = true resp.Response = "add claim to snapshot is not allowed" } else { - err = T.AddClaim([]byte(c.ClaimData)) + err = MkTrees[c.CensusID].AddClaim([]byte(c.ClaimData)) } } else { resp.Error = true @@ -122,15 +129,15 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } if op == "gen" { - resp.Response, err = T.GenProof([]byte(c.ClaimData)) + resp.Response, err = MkTrees[c.CensusID].GenProof([]byte(c.ClaimData)) } if op == "root" { - resp.Response = T.GetRoot() + resp.Response = MkTrees[c.CensusID].GetRoot() } if op == "dump" { - values, err := T.Dump() + values, err := MkTrees[c.CensusID].Dump() if err != nil { resp.Error = true resp.Response = fmt.Sprint(err) @@ -147,12 +154,12 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { if op == "snapshot" { msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) - if auth := checkAuth(c.TimeStamp, c.Signature, msg); auth { + if auth := checkAuth(c.TimeStamp, c.Signature, Signatures[c.CensusID], msg); auth { if strings.HasPrefix(c.CensusID, "0x") { resp.Error = true resp.Response = "snapshot an snapshot makes no sense" } else { - snapshotNamespace, err := T.Snapshot() + snapshotNamespace, err := MkTrees[c.CensusID].Snapshot() if err != nil { resp.Error = true resp.Response = fmt.Sprint(err) @@ -171,7 +178,7 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { resp.Error = true resp.Response = "proofData not provided" } else { - validProof, _ := T.CheckProof([]byte(c.ClaimData), c.ProofData) + validProof, _ := MkTrees[c.CensusID].CheckProof([]byte(c.ClaimData), c.ProofData) if validProof { resp.Response = "valid" } else { @@ -183,7 +190,7 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { reply(&resp, w) } -func Listen(port int, proto string, pubKey string) { +func Listen(port int, proto string) { srv := &http.Server{ Addr: fmt.Sprintf(":%d", port), ReadHeaderTimeout: 4 * time.Second, @@ -211,13 +218,6 @@ func Listen(port int, proto string, pubKey string) { claimHandler(w, r, "dump") }) - if len(pubKey) > 1 { - log.Printf("Enabling signature authentication with %s\n", pubKey) - authPubKey = pubKey - } else { - authPubKey = "" - } - if proto == "https" { log.Print("Starting server in https mode") if err := srv.ListenAndServeTLS("server.crt", "server.key"); err != nil { diff --git a/tree/tree.go b/tree/tree.go index ae1d5fa..4ebeac6 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -12,19 +12,21 @@ import ( ) type Tree struct { - Namespace string Storage string Tree *merkletree.MerkleTree DbStorage *db.LevelDbStorage } -func (t *Tree) Init() error { +func (t *Tree) Init(namespace string) error { if len(t.Storage) < 1 { + if len(namespace) < 1 { + return errors.New("namespace not valid") + } usr, err := user.Current() if err == nil { - t.Storage = usr.HomeDir + "/.dvote/Tree" + t.Storage = usr.HomeDir + "/.dvote/census/" + namespace } else { - t.Storage = "./dvoteTree" + t.Storage = "./dvoteTree/" + namespace } } mtdb, err := db.NewLevelDbStorage(t.Storage, false) @@ -49,7 +51,7 @@ func (t *Tree) GetClaim(data []byte) (*mkcore.ClaimBasic, error) { return nil, errors.New("claim data too large") } for i := len(data); i <= 496/8; i++ { - data = append(data, byte('0')) + data = append(data, byte('.')) } var indexSlot [400 / 8]byte var dataSlot [496 / 8]byte @@ -106,7 +108,15 @@ func (t *Tree) Dump() ([]string, error) { err := t.Tree.Walk(t.Tree.RootKey(), func(n *merkletree.Node) { if n.Type == merkletree.NodeTypeLeaf { - response = append(response, fmt.Sprintf("|%s", n.Entry.Data)) + rawValue := n.Value() + var cleanValue []byte + for i := 0; i < len(rawValue); i++ { + if rawValue[i] == byte('.') { + break + } + cleanValue = append(cleanValue, rawValue[i]) + } + response = append(response, fmt.Sprintf("%s", cleanValue)) } }) return response, err From 144221908b085a113a148de767d7317e22e59308 Mon Sep 17 00:00:00 2001 From: Jordi Date: Tue, 5 Feb 2019 13:42:11 +0100 Subject: [PATCH 18/24] Adding CORS headers --- cmd/censushttp/README.md | 2 +- service/censusmanager.go | 12 ++++++++++++ 2 files changed, 13 insertions(+), 1 deletion(-) diff --git a/cmd/censushttp/README.md b/cmd/censushttp/README.md index f00433d..1625546 100644 --- a/cmd/censushttp/README.md +++ b/cmd/censushttp/README.md @@ -7,7 +7,7 @@ Reference implementation of a voting census service running on the Vocdoni platf In a GO ready environment: ``` -go get -u github.com/vocdoni/dvote-census +go get -u github.com/vocdoni/dvote-census/... go build -o censusHttpService github.com/vocdoni/dvote-census/cmd/censushttp ``` diff --git a/service/censusmanager.go b/service/censusmanager.go index 6b56b85..ffbed77 100644 --- a/service/censusmanager.go +++ b/service/censusmanager.go @@ -71,7 +71,19 @@ func checkAuth(timestamp, signature, message string) bool { return false } +func addCorsHeaders(w *http.ResponseWriter, req *http.Request) { + (*w).Header().Set("Access-Control-Allow-Origin", "*") + (*w).Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS") + (*w).Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization") +} + func claimHandler(w http.ResponseWriter, req *http.Request, op string) { + addCorsHeaders(&w, req) + + if (*req).Method == "OPTIONS" { + return + } + var c Claim var resp Result if ok := checkRequest(w, req); !ok { From d26711678257d58a4c9fcdb2b399c1a6b1a2cfa9 Mon Sep 17 00:00:00 2001 From: Jordi Date: Tue, 5 Feb 2019 14:26:46 +0100 Subject: [PATCH 19/24] Handling CORS headers as soon as possible and returning 404 on unkown requests --- service/censusmanager.go | 66 +++++++++++++++++++++++++++++----------- 1 file changed, 48 insertions(+), 18 deletions(-) diff --git a/service/censusmanager.go b/service/censusmanager.go index ffbed77..9a855fd 100644 --- a/service/censusmanager.go +++ b/service/censusmanager.go @@ -71,19 +71,7 @@ func checkAuth(timestamp, signature, message string) bool { return false } -func addCorsHeaders(w *http.ResponseWriter, req *http.Request) { - (*w).Header().Set("Access-Control-Allow-Origin", "*") - (*w).Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS") - (*w).Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization") -} - func claimHandler(w http.ResponseWriter, req *http.Request, op string) { - addCorsHeaders(&w, req) - - if (*req).Method == "OPTIONS" { - return - } - var c Claim var resp Result if ok := checkRequest(w, req); !ok { @@ -195,6 +183,12 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { reply(&resp, w) } +func addCorsHeaders(w *http.ResponseWriter, req *http.Request) { + (*w).Header().Set("Access-Control-Allow-Origin", "*") + (*w).Header().Set("Access-Control-Allow-Methods", "POST, OPTIONS") + (*w).Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization") +} + func Listen(port int, proto string, pubKey string) { srv := &http.Server{ Addr: fmt.Sprintf(":%d", port), @@ -205,22 +199,58 @@ func Listen(port int, proto string, pubKey string) { } http.HandleFunc("/addClaim", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "add") + addCorsHeaders(&w, r) + + if r.Method == http.MethodPost { + claimHandler(w, r, "add") + } else if r.Method != http.MethodOptions { + http.Error(w, "Not found", http.StatusNotFound) + } }) http.HandleFunc("/genProof", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "gen") + addCorsHeaders(&w, r) + + if r.Method == http.MethodPost { + claimHandler(w, r, "gen") + } else if r.Method != http.MethodOptions { + http.Error(w, "Not found", http.StatusNotFound) + } }) http.HandleFunc("/checkProof", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "check") + addCorsHeaders(&w, r) + + if r.Method == http.MethodPost { + claimHandler(w, r, "check") + } else if r.Method != http.MethodOptions { + http.Error(w, "Not found", http.StatusNotFound) + } }) http.HandleFunc("/getRoot", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "root") + addCorsHeaders(&w, r) + + if r.Method == http.MethodPost { + claimHandler(w, r, "root") + } else if r.Method != http.MethodOptions { + http.Error(w, "Not found", http.StatusNotFound) + } }) http.HandleFunc("/snapshot", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "snapshot") + addCorsHeaders(&w, r) + + if r.Method == http.MethodPost { + claimHandler(w, r, "snapshot") + } else if r.Method != http.MethodOptions { + http.Error(w, "Not found", http.StatusNotFound) + } }) http.HandleFunc("/dump", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "dump") + addCorsHeaders(&w, r) + + if r.Method == http.MethodPost { + claimHandler(w, r, "dump") + } else if r.Method != http.MethodOptions { + http.Error(w, "Not found", http.StatusNotFound) + } }) if len(pubKey) > 1 { From 0c3587bb9c26b3a2953487c0b2d92cd6ca50bcc7 Mon Sep 17 00:00:00 2001 From: p4u Date: Thu, 7 Feb 2019 17:16:28 +0100 Subject: [PATCH 20/24] Keep adapting code to new merkletree Add RootHash JSON parameter. Remove Spanshopt API method. Add support on genProof/checkProof/dump to specify the RootHash Refactory of some code Signed-off-by: p4u --- service/censusmanager.go | 127 +++++++++++++++++++++++++-------------- tree/tree.go | 30 +++++++-- 2 files changed, 109 insertions(+), 48 deletions(-) diff --git a/service/censusmanager.go b/service/censusmanager.go index 4bb6da1..aad9dc5 100644 --- a/service/censusmanager.go +++ b/service/censusmanager.go @@ -6,13 +6,13 @@ import ( "log" "net/http" "strconv" - "strings" "time" tree "github.com/vocdoni/dvote-census/tree" signature "github.com/vocdoni/dvote-relay/crypto/signature" ) +const hashSize = 32 const authTimeWindow = 10 // Time window (seconds) in which TimeStamp will be accepted if auth enabled var MkTrees map[string]*tree.Tree // MerkleTree dvote-census library var Signatures map[string]string @@ -20,6 +20,7 @@ var Signature signature.SignKeys // Signature dvote-relay library type Claim struct { CensusID string `json:"censusID"` // References to MerkleTree namespace + RootHash string `json:"rootHash"` // References to MerkleTree rootHash ClaimData string `json:"claimData"` // Data to add to the MerkleTree ProofData string `json:"proofData"` // MerkleProof to check TimeStamp string `json:"timeStamp"` // Unix TimeStamp in seconds @@ -98,8 +99,9 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } // Process data - log.Printf("Received censusID:{%s} claimData:{%s} proofData:{%s} timeStamp:{%s} signature:{%s}\n", - c.CensusID, c.ClaimData, c.ProofData, c.TimeStamp, c.Signature) + log.Printf("censusID:{%s} rootHash:{%s} claimData:{%s} proofData:{%s} timeStamp:{%s} signature:{%s}\n", + c.CensusID, c.RootHash, c.ClaimData, c.ProofData, c.TimeStamp, c.Signature) + authString := fmt.Sprintf("%s%s%s%s", c.CensusID, c.RootHash, c.ClaimData, c.TimeStamp) resp.Error = false resp.Response = "" censusFound := false @@ -114,14 +116,8 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } if op == "add" { - msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) - if auth := checkAuth(c.TimeStamp, c.Signature, Signatures[c.CensusID], msg); auth { - if strings.HasPrefix(c.CensusID, "0x") { - resp.Error = true - resp.Response = "add claim to snapshot is not allowed" - } else { - err = MkTrees[c.CensusID].AddClaim([]byte(c.ClaimData)) - } + if auth := checkAuth(c.TimeStamp, c.Signature, Signatures[c.CensusID], authString); auth { + err = MkTrees[c.CensusID].AddClaim([]byte(c.ClaimData)) } else { resp.Error = true resp.Response = "invalid authentication" @@ -129,61 +125,107 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } if op == "gen" { - resp.Response, err = MkTrees[c.CensusID].GenProof([]byte(c.ClaimData)) + var t *tree.Tree + var err error + if len(c.RootHash) > 1 { //if rootHash specified + t, err = MkTrees[c.CensusID].Snapshot(c.RootHash) + if err != nil { + log.Printf("Snapshot error: %s", err.Error()) + resp.Error = true + resp.Response = "invalid root hash" + reply(&resp, w) + return + } + } else { //if rootHash not specified use current tree + t = MkTrees[c.CensusID] + } + resp.Response, err = t.GenProof([]byte(c.ClaimData)) + if err != nil { + resp.Error = true + resp.Response = err.Error() + reply(&resp, w) + return + } } if op == "root" { resp.Response = MkTrees[c.CensusID].GetRoot() } + if op == "idx" { + + } + if op == "dump" { - values, err := MkTrees[c.CensusID].Dump() + var t *tree.Tree + if auth := checkAuth(c.TimeStamp, c.Signature, Signatures[c.CensusID], authString); !auth { + resp.Error = true + resp.Response = "invalid authentication" + reply(&resp, w) + return + } + + if len(c.RootHash) > 1 { //if rootHash specified + t, err = MkTrees[c.CensusID].Snapshot(c.RootHash) + if err != nil { + log.Printf("Snapshot error: %s", err.Error()) + resp.Error = true + resp.Response = "invalid root hash" + reply(&resp, w) + return + } + } else { //if rootHash not specified use current merkletree + t = MkTrees[c.CensusID] + } + + //dump the claim data and return it + values, err := t.Dump() if err != nil { resp.Error = true - resp.Response = fmt.Sprint(err) + resp.Response = err.Error() } else { jValues, err := json.Marshal(values) if err != nil { resp.Error = true - resp.Response = fmt.Sprint(err) + resp.Response = err.Error() } else { resp.Response = string(jValues) } } } - if op == "snapshot" { - msg := fmt.Sprintf("%s%s%s", c.CensusID, c.ClaimData, c.TimeStamp) - if auth := checkAuth(c.TimeStamp, c.Signature, Signatures[c.CensusID], msg); auth { - if strings.HasPrefix(c.CensusID, "0x") { + if op == "check" { + if len(c.ProofData) < 1 { + resp.Error = true + resp.Response = "proofData not provided" + reply(&resp, w) + return + } + var t *tree.Tree + if len(c.RootHash) > 1 { //if rootHash specified + t, err = MkTrees[c.CensusID].Snapshot(c.RootHash) + if err != nil { + log.Printf("Snapshot error: %s", err.Error()) resp.Error = true - resp.Response = "snapshot an snapshot makes no sense" - } else { - snapshotNamespace, err := MkTrees[c.CensusID].Snapshot() - if err != nil { - resp.Error = true - resp.Response = fmt.Sprint(err) - } else { - resp.Response = snapshotNamespace - } + resp.Response = "invalid root hash" + reply(&resp, w) + return } - } else { - resp.Error = true - resp.Response = "invalid authentication" + } else { //if rootHash not specified use current merkletree + t = MkTrees[c.CensusID] } - } - if op == "check" { - if len(c.ProofData) < 1 { + validProof, err := t.CheckProof([]byte(c.ClaimData), c.ProofData) + if err != nil { resp.Error = true - resp.Response = "proofData not provided" + resp.Response = err.Error() + reply(&resp, w) + return + } + if validProof { + resp.Response = "valid" } else { - validProof, _ := MkTrees[c.CensusID].CheckProof([]byte(c.ClaimData), c.ProofData) - if validProof { - resp.Response = "valid" - } else { - resp.Response = "invalid" - } + resp.Response = "invalid" } } @@ -211,9 +253,6 @@ func Listen(port int, proto string) { http.HandleFunc("/getRoot", func(w http.ResponseWriter, r *http.Request) { claimHandler(w, r, "root") }) - http.HandleFunc("/snapshot", func(w http.ResponseWriter, r *http.Request) { - claimHandler(w, r, "snapshot") - }) http.HandleFunc("/dump", func(w http.ResponseWriter, r *http.Request) { claimHandler(w, r, "dump") }) diff --git a/tree/tree.go b/tree/tree.go index 4ebeac6..d3bf3e3 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -1,8 +1,8 @@ package tree import ( + "bytes" "errors" - "fmt" "os/user" common3 "github.com/iden3/go-iden3/common" @@ -103,6 +103,16 @@ func (t *Tree) GetRoot() string { return t.Tree.RootKey().String() } +func (t *Tree) GetIndex(data []byte) (string, error) { + e, err := t.GetClaim(data) + if err != nil { + return "", err + } + index, err := t.Tree.GetDataByIndex(e.Entry().HIndex()) + return index.String(), err +} + +/* func (t *Tree) Dump() ([]string, error) { var response []string @@ -121,8 +131,20 @@ func (t *Tree) Dump() ([]string, error) { }) return response, err } +*/ + +func (t *Tree) Dump() (string, error) { + w := bytes.NewBufferString("") + err := t.Tree.DumpClaims(w, nil) // as rootKey we can pass a nil pointer, and it will use the current RootKey + + return w.String(), err +} -func (t *Tree) Snapshot() (string, error) { - snapshotNamespace := t.Tree.RootKey().String() - return snapshotNamespace, nil +func (t *Tree) Snapshot(root string) (*Tree, error) { + var rootHash merkletree.Hash + copy(rootHash[:32], root) + mt, err := t.Tree.Snapshot(&rootHash) + snapshotTree := new(Tree) + snapshotTree.Tree = mt + return snapshotTree, err } From c258a9ad9cf309ab2cd71b4f4c5ee34179c62c03 Mon Sep 17 00:00:00 2001 From: p4u Date: Mon, 11 Feb 2019 10:36:33 +0100 Subject: [PATCH 21/24] Using camelCase on censusID Signed-off-by: p4u --- service/censusmanager.go | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/service/censusmanager.go b/service/censusmanager.go index aad9dc5..c84a9f1 100644 --- a/service/censusmanager.go +++ b/service/censusmanager.go @@ -19,7 +19,7 @@ var Signatures map[string]string var Signature signature.SignKeys // Signature dvote-relay library type Claim struct { - CensusID string `json:"censusID"` // References to MerkleTree namespace + CensusID string `json:"censusId"` // References to MerkleTree namespace RootHash string `json:"rootHash"` // References to MerkleTree rootHash ClaimData string `json:"claimData"` // Data to add to the MerkleTree ProofData string `json:"proofData"` // MerkleProof to check @@ -99,7 +99,7 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } // Process data - log.Printf("censusID:{%s} rootHash:{%s} claimData:{%s} proofData:{%s} timeStamp:{%s} signature:{%s}\n", + log.Printf("censusId:{%s} rootHash:{%s} claimData:{%s} proofData:{%s} timeStamp:{%s} signature:{%s}\n", c.CensusID, c.RootHash, c.ClaimData, c.ProofData, c.TimeStamp, c.Signature) authString := fmt.Sprintf("%s%s%s%s", c.CensusID, c.RootHash, c.ClaimData, c.TimeStamp) resp.Error = false @@ -110,7 +110,7 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { } if !censusFound { resp.Error = true - resp.Response = "censusID not valid or not found" + resp.Response = "censusId not valid or not found" reply(&resp, w) return } From 6d3bf7c18d3dcf2929bcb98dce88e16593e6f68c Mon Sep 17 00:00:00 2001 From: p4u Date: Mon, 11 Feb 2019 18:17:00 +0100 Subject: [PATCH 22/24] Fix dump function --- service/censusmanager.go | 2 +- tree/tree.go | 26 +++++--------------------- 2 files changed, 6 insertions(+), 22 deletions(-) diff --git a/service/censusmanager.go b/service/censusmanager.go index c84a9f1..aa9bfad 100644 --- a/service/censusmanager.go +++ b/service/censusmanager.go @@ -189,7 +189,7 @@ func claimHandler(w http.ResponseWriter, req *http.Request, op string) { resp.Error = true resp.Response = err.Error() } else { - resp.Response = string(jValues) + resp.Response = fmt.Sprintf("%s", jValues) } } } diff --git a/tree/tree.go b/tree/tree.go index d3bf3e3..7c466c0 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -3,6 +3,7 @@ package tree import ( "bytes" "errors" + "fmt" "os/user" common3 "github.com/iden3/go-iden3/common" @@ -51,7 +52,7 @@ func (t *Tree) GetClaim(data []byte) (*mkcore.ClaimBasic, error) { return nil, errors.New("claim data too large") } for i := len(data); i <= 496/8; i++ { - data = append(data, byte('.')) + data = append(data, '\x00') } var indexSlot [400 / 8]byte var dataSlot [496 / 8]byte @@ -112,33 +113,16 @@ func (t *Tree) GetIndex(data []byte) (string, error) { return index.String(), err } -/* func (t *Tree) Dump() ([]string, error) { var response []string - - err := t.Tree.Walk(t.Tree.RootKey(), func(n *merkletree.Node) { + err := t.Tree.Walk(nil, func(n *merkletree.Node) { if n.Type == merkletree.NodeTypeLeaf { - rawValue := n.Value() - var cleanValue []byte - for i := 0; i < len(rawValue); i++ { - if rawValue[i] == byte('.') { - break - } - cleanValue = append(cleanValue, rawValue[i]) - } - response = append(response, fmt.Sprintf("%s", cleanValue)) + data := bytes.Trim(n.Value()[65:], "\x00") + response = append(response, fmt.Sprintf("%s", data)) } }) return response, err } -*/ - -func (t *Tree) Dump() (string, error) { - w := bytes.NewBufferString("") - err := t.Tree.DumpClaims(w, nil) // as rootKey we can pass a nil pointer, and it will use the current RootKey - - return w.String(), err -} func (t *Tree) Snapshot(root string) (*Tree, error) { var rootHash merkletree.Hash From b8dccadfaaad6d3c50489a4ecfc5f7d3e2b7a17f Mon Sep 17 00:00:00 2001 From: p4u Date: Mon, 11 Feb 2019 20:03:49 +0100 Subject: [PATCH 23/24] Fix getRoot Signed-off-by: p4u --- tree/tree.go | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/tree/tree.go b/tree/tree.go index 7c466c0..a164d32 100644 --- a/tree/tree.go +++ b/tree/tree.go @@ -101,7 +101,7 @@ func (t *Tree) CheckProof(data []byte, mpHex string) (bool, error) { } func (t *Tree) GetRoot() string { - return t.Tree.RootKey().String() + return common3.HexEncode(t.Tree.RootKey().Bytes()) } func (t *Tree) GetIndex(data []byte) (string, error) { @@ -126,9 +126,13 @@ func (t *Tree) Dump() ([]string, error) { func (t *Tree) Snapshot(root string) (*Tree, error) { var rootHash merkletree.Hash - copy(rootHash[:32], root) - mt, err := t.Tree.Snapshot(&rootHash) snapshotTree := new(Tree) + rootBytes, err := common3.HexDecode(root) + if err != nil { + return snapshotTree, err + } + copy(rootHash[:32], rootBytes) + mt, err := t.Tree.Snapshot(&rootHash) snapshotTree.Tree = mt return snapshotTree, err } From 3e2b4d1cfa7904c2723115b3cb845bd8f9e1ef40 Mon Sep 17 00:00:00 2001 From: p4u Date: Tue, 12 Feb 2019 10:53:01 +0100 Subject: [PATCH 24/24] Adapt documentation to new merkle tree Signed-off-by: p4u --- cmd/censushttp/README.md | 128 ++++++++++++++++++--------------------- 1 file changed, 60 insertions(+), 68 deletions(-) diff --git a/cmd/censushttp/README.md b/cmd/censushttp/README.md index b52242e..c0c3d1e 100644 --- a/cmd/censushttp/README.md +++ b/cmd/censushttp/README.md @@ -2,7 +2,7 @@ Reference implementation of a voting census service running on the Vocdoni platform -#### Compile +## Compile In a GO ready environment: @@ -11,126 +11,118 @@ go get -u github.com/vocdoni/dvote-census go build -o censusHttpService github.com/vocdoni/dvote-census/cmd/censushttp ``` -#### Usage +## Usage -`./censusHttpService [:pubKey] [[:pubKey] ...]` +`./censusHttpService [:pubKey] [[:pubKey] ...]` -Example: +Example ``` ./censusHttpService 1500 Got_Favorite -2019/02/01 20:01:15 Starting process HTTP service on port 1500 for namespace GoT_Favorite -2019/02/01 20:01:15 Starting server in http mode +2019/02/12 10:20:16 Starting process HTTP service on port 1500 for namespace GoT_Favorite +2019/02/12 10:20:16 Starting server in http mode ``` -#### add claims +## API -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim +A HTTP jSON endpoint is available with the following possible fields: `censusId`, `claimData`, `rootHash` and `proofData`. -{"error":false,"response":""} -``` +If `pubKey` has been configured for a specific `censusId`, then two more methods are available (`timeStamp` and `signature`) to provide authentication. -If public key authentication enabled, `signature` field is required using Nancl signature. +The next table shows the available methods and its relation with the fields. -The signed message is expected to be a concatenation of the following fields: `censusID, claimData, timeStamp` +| method | censusId | claimData | rootHash | proofData | protected? | description | +|------------|-----------|-----------|----------|-----------|------------|------------| +| `addCLaim` | mandatory | mandatory | none | none | yes | adds a new claim to the merkle tree | +| `getRoot` | mandatory | none | none | none | no | get the current merkletree root hash +| `genProof` | mandatory | mandatory | optional | none | no | generate the merkle proof for a given claim +| `checkProof` | mandatory | mandatory | optional | mandatory | no | check a claim and its merkle proof +| `getIdx` | mandatory | mandatory | optional | none | no | get the merkletree data index of a given claim +| `dump` | mandatory | none | optional | none | yes | list the contents of the census for a given hash -There is a time window of 10 seconds while the signature is considered valid. -``` -curl -d '{"censusID":"GoT_Favorite", -"claimData":"Jon Snow", -"timeStamp":"1547814675", -"signature":"a117c4ce12b29090884112ffe57e664f007e7ef142a1679996e2d34fd2b852fe76966e47932f1e9d3a54610d0f361383afe2d9aab096e15d136c236abb0a0d0e"}' http://localhost:1500/addClaim - -{"error":false,"response":""} -``` +## Signature +The signature provides authentication by signing a concatenation of the following strings (even if empty) without spaces: `censusId rootHash claimData timeStamp`. -#### generate proof - -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof +The `timeStamp` when received on the server side must not differ more than 10 seconds from the current UNIX time. -{"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} -``` +## Examples -#### check proof +#### add claims +Add two new claims, one for `Jon Snow` and another for `Tyrion`. ``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim -{"error":false,"response":"invalid"} +{"error":false,"response":""} ``` ``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof +curl -d '{"censusID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim -{"error":false,"response":"valid"} +{"error":false,"response":""} ``` -#### make snapshot - -Snapshots are static and unmutable copies of a specific census +In case signature is enabled: ``` -curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/snapshot +curl -d '{ +"censusID":"GoT_Favorite", +"claimData":"Jon Snow", +"timeStamp":"1547814675", +"signature":"a117c4ce12b29090884112ffe57e664f007e7ef142a1679996e2d34fd2b852fe76966e47932f1e9d3a54610d0f361383afe2d9aab096e15d136c236abb0a0d0e" }' http://localhost:1500/addClaim -{"error":false,"response":"0x8647692e073a10980d821764c65ca3fddbc606bb936f9812a7a806bfa97df152"} +{"error":false,"response":""} ``` -The name for the snapshot is "0x8647692e073a10980d821764c65ca3fddbc606bb936f9812a7a806bfa97df152" which represents the Root Hash. - -Now you can use it as censusID for checkProof, getRoot, genProof and dump. But addClaim is not longer allowed. -#### dump +#### generate proof -Dump contents of a specific censusID (values) +Generate a merkle proof for the claim `Jon Snow` ``` -curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/dump -{"error":false,"response":"[\"Tyrion\",\"Jon Snow\"]"} -``` - - - +curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof +{"error":false,"response":"0x000200000000000000000000000000000000000000000000000000000000000212f8134039730791388a9bd0460f9fbd0757327212a64b3a2b0f0841ce561ee3"} +``` +If `rootHash` is specified, the proof will be calculated for the given root hash. +#### get root +The previous merkle proof is valid only for the current root hash. Let's get it +``` +curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/getRoot +{"error":false,"response":"0x2f0ddde5cb995eae23dc3b75a5c0333f1cc89b73f3a00b0fe71996fb90fef04b"} +``` +#### check proof -##### add claims +Now let's check if the proof is valid ``` -curl -d '{"processID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/addClaim -{"error":false,"response":""} -``` +curl -d '{ +"censusID":"GoT_Favorite","claimData":"Jon Snow", +"rootHash":"0x2f0ddde5cb995eae23dc3b75a5c0333f1cc89b73f3a00b0fe71996fb90fef04b", +"proofData":"0x000200000000000000000000000000000000000000000000000000000000000212f8134039730791388a9bd0460f9fbd0757327212a64b3a2b0f0841ce561ee3"}' http://localhost:1500/checkProof -``` -curl -d '{"processID":"GoT_Favorite","claimData":"Tyrion"}' http://localhost:1500/addClaim -{"error":false,"response":""} +{"error":false,"response":"valid"} ``` -##### generate proof +If `rootHash` is not specified, the current root hash is used. -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow"}' http://localhost:1500/genProof -{"error":false,"response":"0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"} -``` +#### dump -##### check proof +Dump contents of a specific censusId (values) ``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x0000000000000000000000000000000000000000000000000000000000000000000123"}' http://localhost:1500/checkProof -{"error":false,"response":"invalid"} -``` +curl -d '{"censusID":"GoT_Favorite"}' http://localhost:1500/dump -``` -curl -d '{"censusID":"GoT_Favorite","claimData":"Jon Snow", "proofData": "0x000000000000000000000000000000000000000000000000000000000000000352f3ca2aaf635ec2ae4452f6a65be7bca72678287a8bb08ad4babfcccd76c2fef1aac7675261bf6d12c746fb7907beea6d1f1635af93ba931eec0c6a747ecc37"}' http://localhost:1500/checkProof -{"error":false,"response":"valid"} +{"error":false,"response":"[\"Tyrion\",\"Jon Snow\"]"} ``` +If `rootHash` is specified, dump will return the values for the merkle tree with the given root hash. \ No newline at end of file