* core/vm: remove function call for stack validation from evm runloop
* core/vm: separate gas calc into static + dynamic
* core/vm: optimize push1
* core/vm: reuse pooled bigints for ADDRESS, ORIGIN and CALLER
* core/vm: use generic error message for jump/jumpi, to avoid string interpolation
* testdata: fix tests for new error message
* core/vm: use 64-bit memory calculations
* core/vm: fix error in memory calculation
* core/vm: address review concerns
* core/vm: avoid unnecessary use of big.Int:BitLen()
dummyHook's fields were concurrently written by nodes and read by
the test. The simplest solution is to protect all fields with a mutex.
Enable: TestMultiplePeersDropSelf, TestMultiplePeersDropOther as they
seemingly accidentally stayed disabled during a refactor/rewrite
since 1836366ac1.
resolvesethersphere/go-ethereum#1286
* swarm/network: propagate span with ctx
* swarm/network: try to stop stream.send.request spans on time
* swarm/storage: add chunk ref as a log to netstore.fetcher span
Although current two implementations(ledgerDriver, trezorDriver) of interface driver.Close do not actually return any error. Instead, they only return nil.
But since the declaration of Close function returns error, it is better to check the returned error in case in future some new implementation of Close function returns error and we may forget to modify the function which invokes Close function at that time.
This PR will will break existing UIs, since it changes all calls like ApproveSignTransaction to be on the form ui_approveSignTransaction.
This is to make it possible for the UI to reuse the json-rpc library from go-ethereum, which uses this convention.
Also, this PR removes some unused structs, after import/export were removed from the external api (so no longer needs internal methods for approval)
One more breaking change is introduced, removing passwords from the ApproveSignTxResponse and the likes. This makes the manual interface more like the rulebased interface, and integrates nicely with the credential storage. Thus, the way it worked before, it would be tempting for the UI to implement 'remember password' functionality. The way it is now, it will be easy instead to tell clef to store passwords and use them.
If a pw is not found in the credential store, the user is prompted to provide the password.
These tests never run as the build tag excluded them from the CI
execution. As a results the (dead) code got out of sync with other
parts of Swarm and now they would not even compile. => Removed.
resolvesethersphere/go-ethereum#1238
* p2p/discover: remove unused function
* p2p/enode: use localItemKey for local sequence number
I added localItemKey for this purpose in #18963, but then
forgot to actually use it. This changes the database layout
yet again and requires bumping the version number.
* node: require LocalAppData variable
This avoids path inconsistencies on Windows XP.
Hat tip to @MicahZoltu for catching this so quickly.
* node: fix typo
* travis, build: switch to NDK 19b, fix gomobile builds
* travis, build: move NDK into its final bundle location
* travis: disable Android build on PRs once again
This change
- implements concurrent LES request serving even for a single peer.
- replaces the request cost estimation method with a cost table based on
benchmarks which gives much more consistent results. Until now the
allowed number of light peers was just a guess which probably contributed
a lot to the fluctuating quality of available service. Everything related
to request cost is implemented in a single object, the 'cost tracker'. It
uses a fixed cost table with a global 'correction factor'. Benchmark code
is included and can be run at any time to adapt costs to low-level
implementation changes.
- reimplements flowcontrol.ClientManager in a cleaner and more efficient
way, with added capabilities: There is now control over bandwidth, which
allows using the flow control parameters for client prioritization.
Target utilization over 100 percent is now supported to model concurrent
request processing. Total serving bandwidth is reduced during block
processing to prevent database contention.
- implements an RPC API for the LES servers allowing server operators to
assign priority bandwidth to certain clients and change prioritized
status even while the client is connected. The new API is meant for
cases where server operators charge for LES using an off-protocol mechanism.
- adds a unit test for the new client manager.
- adds an end-to-end test using the network simulator that tests bandwidth
control functions through the new API.
* swarm/pss: fix data race on HandshakeController.symKeyIndex
The HandshakeController.symKeyIndex map was accessed concurrently.
Since insufficient test coverage the race is not detected every time.
However, running TestClientHandshake a 100 times seems to be enough to
reproduce the race.
Note: I've chosen HandshakeController.lock to protect
HandshakeController.symKeyIndex as that was already protected in a few
functions by that lock.
Additionally:
- removed unused testStore
- enabled tests in handshake_test.go as they pass
- removed code duplication by adding getSymKey()
* swarm/pss: fix a data race on HandshakeController.keyC
* swarm/pss: fix data races with on Pss.symKeyPool
* swarm/storage/mock: implement listings methods for mem and rpc stores
* swarm/storage/mock/rpc: add comments and newTestStore helper function
* swarm/storage/mock/mem: add missing comments
* swarm/storage/mock: add comments to new types and constants
* swarm/storage/mock/db: implement listings for mock/db global store
* swarm/storage/mock/test: add comments for MockStoreListings
* swarm/storage/mock/explorer: initial implementation
* cmd/swarm/global-store: add chunk explorer
* cmd/swarm/global-store: add chunk explorer tests
* swarm/storage/mock/explorer: add tests
* swarm/storage/mock/explorer: add swagger api definition
* swarm/storage/mock/explorer: not-zero test values for invalid addr and key
* swarm/storage/mock/explorer: test wildcard cors origin
* swarm/storage/mock/db: renames based on Fabio's suggestions
* swarm/storage/mock/explorer: add more comments to testHandler function
* cmd/swarm/global-store: terminate subprocess with Kill in tests
* swarm/storage/localstore: close localstore in two tests
* swarm/storage/localstore: fix a possible deadlock in tests
* swarm/storage/localstore: re-enable pull subs tests for travis race
* swarm/storage/localstore: stop sending to errChan on context done in tests
* swarm/storage/localstore: better want check in readPullSubscriptionBin
* swarm/storage/localstore: protect chunk put with addr lock in tests
* swamr/storage/localstore: wait for gc and writeGCSize workers on Close
* swarm/storage/localstore: more correct testDB_collectGarbageWorker
* swarm/storage/localstore: set DB Close timeout to 5s
The current loop continuation condition is always true as a uint8
is always being checked whether it is less than 255 (its maximum
value). Since the loop starts with the value 1, the loop termination
can be guarranteed to exit once the value overflows to 0.
* swarm/storage: increase mget timeout in common_test.go
TestDbStoreCorrect_1k sometimes timed out with -race on Travis.
--- FAIL: TestDbStoreCorrect_1k (24.63s)
common_test.go:194: testStore failed: timed out after 10s
* swarm: remove unused vars from TestSnapshotSyncWithServer
nodeCount and chunkCount is returned from setupSim and those values
we use.
* swarm: move race/norace helpers from stream to testutil
As we will need to use the flag in other packages, too.
* swarm: refactor TestSwarmNetwork case
Extract long running test cases for better visibility.
* swarm/network: skip TestSyncingViaGlobalSync with -race
As panics on Travis.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x7e351b]
* swarm: run TestSwarmNetwork with fewer nodes with -race
As otherwise we always get test failure with `network_test.go:374:
context deadline exceeded` even with raised `Timeout`.
* swarm/network: run TestDeliveryFromNodes with fewer nodes with -race
Test on Travis times out with 8 or more nodes if -race flag is present.
* swarm/network: smaller node count for discovery tests with -race
TestDiscoveryPersistenceSimulationSimAdapters failed on Travis with
`-race` flag present. The failure was due to extensive memory usage,
coming from the CGO runtime. Using a smaller node count resolves the
issue.
=== RUN TestDiscoveryPersistenceSimulationSimAdapter
==7227==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes of clock allocator (error code: 12)
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:6976 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
FAIL github.com/ethereum/go-ethereum/swarm/network/simulations/discovery 804.826s
* swarm/network: run TestFileRetrieval with fewer nodes with -race
Otherwise we get a failure due to extensive memory usage, as the CGO
runtime cannot allocate more bytes.
=== RUN TestFileRetrieval
==7366==ERROR: ThreadSanitizer failed to allocate 0x80000 (524288) bytes of clock allocator (error code: 12)
FATAL: ThreadSanitizer CHECK failed: ./gotsan.cc:6976 "((0 && "unable to mmap")) != (0)" (0x0, 0x0)
FAIL github.com/ethereum/go-ethereum/swarm/network/stream 155.165s
* swarm/network: run TestRetrieval with fewer nodes with -race
Otherwise we get a failure due to extensive memory usage, as the CGO
runtime cannot allocate more bytes ("ThreadSanitizer failed to
allocate").
* swarm/network: skip flaky TestGetSubscriptionsRPC on Travis w/ -race
Test fails a lot with something like:
streamer_test.go:1332: Real subscriptions and expected amount don't match; real: 0, expected: 20
* swarm/storage: skip TestDB_SubscribePull* tests on Travis w/ -race
Travis just hangs...
ok github.com/ethereum/go-ethereum/swarm/storage/feed/lookup 1.307s
keepalive
keepalive
keepalive
or panics after a while.
Without these tests the race detector job is now stable. Let's
invetigate these tests in a separate issue:
https://github.com/ethersphere/go-ethereum/issues/1245
* swarm/newtork: WIP Span request span until delivery and put
* swarm/storage: Introduce new trace across single fetcher lifespan
* swarm/network: Put span ids for sendpriority in context value
* swarm: Add global span store in tracing
* swarm/tracing: Add context key constants
* swarm/tracing: Add comments
* swarm/storage: Remove redundant fix for filestore
* swarm/tracing: Elaborate constants comments
* swarm/network, swarm/storage, swarm:tracing: Minor cleanup
* swarm/network/stream: fix a goroutine leak in Registry
* swarm/network, swamr/network/stream: Kademlia close addr count and depth change chans
* swarm/network/stream: rename close channel to quit
* swarm/network/stream: fix sync between NewRegistry goroutine and Close method
Simplifies the transaction presense check to use a function to
determine if the transaction is present in the block provided
to trace, which originally had a redundant parenthesis and used
a `exist` flag to dictate control flow.
I added localItemKey for this purpose in #18963, but then
forgot to actually use it. This changes the database layout
yet again and requires bumping the version number.
Package crypto works with or without cgo, which is great. However, to make it
work without cgo required setting the build tag `nocgo`. It's common to disable
cgo by instead just setting the environment variable `CGO_ENABLED=0`. Setting
this environment variable does _not_ implicitly set the build tag `nocgo`. So
projects that try to build the crypto package with `CGO_ENABLED=0` will fail. I
have done this myself several times. Until today, I had just assumed that this
meant that this package requires cgo.
But a small build tag change will make this case work. Instead of using `nocgo`
and `!nocgo`, we can use `!cgo` and `cgo`, respectively. The `cgo` build tag is
automatically set if cgo is enabled, and unset if it is disabled.
This changes default location of the data directory to use the LOCALAPPDATA
environment variable, resolving issues with remote home directories an improving
compatibility with Cygwin.
Fixes#2239Fixes#2237Fixes#16437
* swarm/network: DRY out repeated giga comment
I not necessarily agree with the way we wait for event propagation.
But I truly disagree with having duplicated giga comments.
* p2p/simulations: encapsulate Node.Up field so we avoid data races
The Node.Up field was accessed concurrently without "proper" locking.
There was a lock on Network and that was used sometimes to access
the field. Other times the locking was missed and we had
a data race.
For example: https://github.com/ethereum/go-ethereum/pull/18464
The case above was solved, but there were still intermittent/hard to
reproduce races. So let's solve the issue permanently.
resolves: ethersphere/go-ethereum#1146
* p2p/simulations: fix unmarshal of simulations.Node
Making Node.Up field private in 13292ee897e345045fbfab3bda23a77589a271c1
broke TestHTTPNetwork and TestHTTPSnapshot. Because the default
UnmarshalJSON does not handle unexported fields.
Important: The fix is partial and not proper to my taste. But I cut
scope as I think the fix may require a change to the current
serialization format. New ticket:
https://github.com/ethersphere/go-ethereum/issues/1177
* p2p/simulations: Add a sanity test case for Node.Config UnmarshalJSON
* p2p/simulations: revert back to defer Unlock() pattern for Network
It's a good patten to call `defer Unlock()` right after `Lock()` so
(new) error cases won't miss to unlock. Let's get back to that pattern.
The patten was abandoned in 85a79b3ad3,
while fixing a data race. That data race does not exist anymore,
since the Node.Up field got hidden behind its own lock.
* p2p/simulations: consistent naming for test providers Node.UnmarshalJSON
* p2p/simulations: remove JSON annotation from private fields of Node
As unexported fields are not serialized.
* p2p/simulations: fix deadlock in Network.GetRandomDownNode()
Problem: GetRandomDownNode() locks -> getDownNodeIDs() ->
GetNodes() tries to lock -> deadlock
On Network type, unexported functions must assume that `net.lock`
is already acquired and should not call exported functions which
might try to lock again.
* p2p/simulations: ensure method conformity for Network
Connect* methods were moved to p2p/simulations.Network from
swarm/network/simulation. However these new methods did not follow
the pattern of Network methods, i.e., all exported method locks
the whole Network either for read or write.
* p2p/simulations: fix deadlock during network shutdown
`TestDiscoveryPersistenceSimulationSimAdapter` often got into deadlock.
The execution was stuck on two locks, i.e, `Kademlia.lock` and
`p2p/simulations.Network.lock`. Usually the test got stuck once in each
20 executions with high confidence.
`Kademlia` was stuck in `Kademlia.EachAddr()` and `Network` in
`Network.Stop()`.
Solution: in `Network.Stop()` `net.lock` must be released before
calling `node.Stop()` as stopping a node (somehow - I did not find
the exact code path) causes `Network.InitConn()` to be called from
`Kademlia.SuggestPeer()` and that blocks on `net.lock`.
Related ticket: https://github.com/ethersphere/go-ethereum/issues/1223
* swarm/state: simplify if statement in DBStore.Put()
* p2p/simulations: remove faulty godoc from private function
The comment started with the wrong method name.
The method is simple and self explanatory. Also, it's private.
=> Let's just remove the comment.
* swarm/network: new saturation for implementation
* swarm/network: re-added saturation func in Kademlia as it is used elsewhere
* swarm/network: saturation with higher MinBinSize
* swarm/network: PeersPerBin with depth check
* swarm/network: edited tests to pass new saturated check
* swarm/network: minor fix saturated check
* swarm/network/simulations/discovery: fixed renamed RPC call
* swarm/network: renamed to isSaturated and returns bool
* swarm/network: early depth check
* signer/clef: make use of json-rpc notification
* signer: tidy up output of OnApprovedTx
* accounts/external, signer: implement remote signing of text, make accounts_sign take hexdata
* clef: added basic testscript
* signer, external, api: add clique signing test to debug rpc, fix clique signing in clef
* signer: fix clique interoperability between geth and clef
* clef: rename networkid switch to chainid
* clef: enable chainid flag
* clef, signer: minor changes from review
* clef: more tests for signer
* common/fdlimit: cap on MacOS file limits, fixes#18994
* common/fdlimit: fix Maximum-check to respect OPEN_MAX
* common/fdlimit: return error if OPEN_MAX is exceeded in Raise()
* common/fdlimit: goimports
* common/fdlimit: check value after setting fdlimit
* common/fdlimit: make comment a bit more descriptive
* cmd/utils: make fdlimit happy path a bit cleaner
* node: close AccountsManager in new Close method
* p2p/simulations, p2p/simulations/adapters: handle node close on shutdown
* node: move node ephemeralKeystore cleanup to stop method
* node: call Stop in Node.Close method
* cmd/geth: close node.Node created with makeFullNode in cli commands
* node: close Node instances in tests
* cmd/geth, node: minor code style fixes
* cmd, console, miner, mobile: proper node Close() termination
* Named functions and defined a basic EIP191 content type list
* Written basic content type functions
* Added ecRecover method in the clef api
* Updated the extapi changelog and addded indications in the README
* Changed the version of the external API
* Added tests for 0x45
* Implementing UnmarshalJSON() for TypedData
* Working on TypedData
* Solved the auditlog issue
* Changed method to signTypedData
* Changed mimes and implemented the 'encodeType' function for EIP-712
* Polished docstrings, ran goimports and swapped fmt.Errorf with errors.New where possible
* Drafted recursive encodeData
* Ran goimports and gofmt
* Drafted first version of EIP-712, including tests
* Temporarily switched to using common.Address in tests
* Drafted text/validator and and rewritten []byte as hexutil.Bytes
* Solved stringified address encoding issue
* Changed the property type required by signData from bytes to interface{}
* Fixed bugs in 'data/typed' signs
* Brought legal warning back after temporarily disabling it for development
* Added example RPC calls for account_signData and account_signTypedData
* Named functions and defined a basic EIP191 content type list
* Written basic content type functions
* Added ecRecover method in the clef api
* Updated the extapi changelog and addded indications in the README
* Added tests for 0x45
* Implementing UnmarshalJSON() for TypedData
* Working on TypedData
* Solved the auditlog issue
* Changed method to signTypedData
* Changed mimes and implemented the 'encodeType' function for EIP-712
* Polished docstrings, ran goimports and swapped fmt.Errorf with errors.New where possible
* Drafted recursive encodeData
* Ran goimports and gofmt
* Drafted first version of EIP-712, including tests
* Temporarily switched to using common.Address in tests
* Drafted text/validator and and rewritten []byte as hexutil.Bytes
* Solved stringified address encoding issue
* Changed the property type required by signData from bytes to interface{}
* Fixed bugs in 'data/typed' signs
* Brought legal warning back after temporarily disabling it for development
* Added example RPC calls for account_signData and account_signTypedData
* Polished and fixed PR
* Polished and fixed PR
* Solved malformed data panics and also wrote tests
* Solved malformed data panics and also wrote tests
* Added alphabetical sorting to type dependencies
* Added alphabetical sorting to type dependencies
* Added pretty print to data/typed UI
* Added pretty print to data/typed UI
* signer: more tests for typed data
* signer: more tests for typed data
* Fixed TestMalformedData4 errors and renamed IsValid to Validate
* Fixed TestMalformedData4 errors and renamed IsValid to Validate
* Fixed more new failing tests and deanonymised some functions
* Fixed more new failing tests and deanonymised some functions
* Added types to EIP712 output in cliui
* Added types to EIP712 output in cliui
* Fixed regexp issues
* Fixed regexp issues
* Added pseudo-failing test
* Added pseudo-failing test
* Fixed false positive test
* Fixed false positive test
* Added PrettyPrint method
* Added PrettyPrint method
* signer: refactor formatting and UI
* signer: make ui use new message format for signing
* Fixed breaking changes
* Fixed rules_test failing test
* Added extra regexp for reference types
* signer: more hard types
* Fixed failing test, formatted files
* signer: use golang/x keccak
* Fixed goimports error
* clef, signer: address some review concerns
* Implemented latest recommendations
* Fixed comments and uintint256 issue
* accounts, signer: fix mimetypes, add interface to sign data with passphrase
* signer, accounts: remove duplicated code, pass hash preimages to signing
* signer: prevent panic in type assertions, make cliui print rawdata as quotable-safe
* signer: linter fixes, remove deprecated crypto dependency
* accounts: fix goimport
New APIs added:
client.RegisterName(namespace, service) // makes service available to server
client.Notify(ctx, method, args...) // sends a notification
ClientFromContext(ctx) // to get a client in handler method
This is essentially a rewrite of the server-side code. JSON-RPC
processing code is now the same on both server and client side. Many
minor issues were fixed in the process and there is a new test suite for
JSON-RPC spec compliance (and non-compliance in some cases).
List of behavior changes:
- Method handlers are now called with a per-request context instead of a
per-connection context. The context is canceled right after the method
returns.
- Subscription error channels are always closed when the connection
ends. There is no need to also wait on the Notifier's Closed channel
to detect whether the subscription has ended.
- Client now omits "params" instead of sending "params": null when there
are no arguments to a call. The previous behavior was not compliant
with the spec. The server still accepts "params": null.
- Floating point numbers are allowed as "id". The spec doesn't allow
them, but we handle request "id" as json.RawMessage and guarantee that
the same number will be sent back.
- Logging is improved significantly. There is now a message at DEBUG
level for each RPC call served.
This change clears up confusion around the two ways in which nodes
can be added to the table.
When a neighbors packet is received as a reply to findnode, the nodes
contained in the reply are added as 'seen' entries if sufficient space
is available.
When a ping is received and the endpoint verification has taken place,
the remote node is added as a 'verified' entry or moved to the front of
the bucket if present. This also updates the node's IP address and port
if they have changed.
* cmd, eth: Added in the flag to step geth once sync based on input
* cmd, eth: 16400 Add an option to stop geth once in sync.
* cmd: 16400 Add an option to stop geth once in sync. WIP
* cmd/geth/main, les/fletcher: added in light mode support
* cmd/geth/main, les/fletcher: Cleaned Comments and code for light mode
* cmd: 16400 Fixed formatting issue and cleaned code
* cmd, eth, les: 16400 Fixed formatting issues
* cmd, eth, les: Performed gofmt to update formatting
* cmd, eth, les: Fixed bugs resulting formatting
* cmd/geth, eth/, les: switched to downloader event
* eth: Fixed styling and gen_config
* eth/: Fix nil error in config file
* cmd/geth: Updated countdown log
* les/fetcher.go: Removed depcreated channel
* eth/downloader.go: Removed deprecated select
* cmd/geth, cmd/utils: Fixed minor issues
* eth: Reverted config files to proper format
* eth: Fixed typo in config file
* cmd/geth, eth/down: Updated code to use header time stamp
* eth/downloader: Changed the time threshold to 10 minutes
* cmd/geth, eth/downloader: Updated downloading event to pass latest header
* cmd/geth: Updated main to use right timer object
* cmd/geth: Removed unused failed event
* cmd/geth: added in correct time field with type assertion
* cmd/geth, cmd/utils: Updated flag to use boolean
* cmd/geth, cmd/utils, eth/downloader: Cleaned up code based on recommendations
* cmd/geth: Removed unneeded import
* cmd/geth, eth/downloader: fixed event field and suggested changes
* cmd/geth, cmd/utils: Updated flag and linting issue
This change resolves multiple issues around handling of endpoint proofs.
The proof is now done separately for each IP and completing the proof
requires a matching ping hash.
Also remove waitping because it's equivalent to sleep. waitping was
slightly more efficient, but that may cause issues with findnode if
packets are reordered and the remote end sees findnode before pong.
Logging of received packets was hitherto done after handling the packet,
which meant that sent replies were logged before the packet that
generated them. This change splits up packet handling into 'preverify'
and 'handle'. The error from 'preverify' is logged, but 'handle' happens
after the message is logged. This fixes the order. Packet logs now
contain the node ID.
dput --passive should make repo pushes from Travis work again.
dput --no-upload-log works around an issue I had while uploading locally.
debuild -d says that debuild shouldn't check for build dependencies when
creating the source package. This option is needed to make builds work
in environments where the installed Go version doesn't match the
declared dependency in the source package.
This PR is replacing the metrics.influxdb.host.tag cmd-line flag with metrics.influxdb.tags - a comma-separated key/value tags, that are passed to the InfluxDB reporter, so that we can index measurements with multiple tags, and not just one host tag.
This will be useful for Swarm, where we want to index measurements not just with the host tag, but also with bzzkey and git commit version (for long-running deployments).
When some of the same messages are redefined anywhere in a Go project,
the protobuf package panics (see
https://github.com/golang/protobuf/issues/178).
Since this package is internal, there is no way to work around it, as
one cannot use it directly, but also cannot define the same messages.
There is no downside in making the package accessible.
This change unbreaks the build and removes racy access to
disableCheckFreq. Even though the field is set while holding
the lock, it was read outside of the protected section.
When opening the wallet, ask for passphrase as well as for the PIN
and return the relevant error (PIN/passphrase required). Open must then
be called again with either PIN or passphrase to advance the process.
This also updates the console bridge to support passphrase authentication.
This PR adds a new fork which disables EIP-1283. Internally it's called Petersburg,
but the genesis/config field is ConstantinopleFix.
The block numbers are:
7280000 for Constantinople on Mainnet
7280000 for ConstantinopleFix on Mainnet
4939394 for ConstantinopleFix on Ropsten
9999999 for ConstantinopleFix on Rinkeby (real number decided later)
This PR also defaults to using the same ConstantinopleFix number as whatever
Constantinople is set to. That is, it will default to mainnet behaviour if ConstantinopleFix
is not set.This means that for private networks which have already transitioned
to Constantinople, this PR will break the network unless ConstantinopleFix is
explicitly set!
* Initial work on a graphql API
* Added receipts, and more transaction fields.
* Finish receipts, add logs
* Add transactionCount to block
* Add types and .
* Update Block type to be compatible with ethql
* Rename nonce to transactionCount in Account, to be compatible with ethql
* Update transaction, receipt and log to match ethql
* Add query operator, for a range of blocks
* Added ommerCount to Block
* Add transactionAt and ommerAt to Block
* Added sendRawTransaction mutation
* Add Call and EstimateGas to graphQL API
* Refactored to use hexutil.Bytes instead of HexBytes
* Replace BigNum with hexutil.Big
* Refactor call and estimateGas to use ethapi struct type
* Replace ethgraphql.Address with common.Address
* Replace ethgraphql.Hash with common.Hash
* Converted most quantities to Long instead of Int
* Add support for logs
* Fix bug in runFilter
* Restructured Transaction to work primarily with headers, so uncle data is reported properly
* Add gasPrice API
* Add protocolVersion API
* Add syncing API
* Moved schema into its own source file
* Move some single use args types into anonymous structs
* Add doc-comments
* Fixed backend fetching to use context
* Added (very) basic tests
* Add documentation to the graphql schema
* Fix reversion for formatting of big numbers
* Correct spelling error
* s/BigInt/Long/
* Update common/types.go
* Fixes in response to review
* Fix lint error
* Updated calls on private functions
* Fix typo in graphql.go
* Rollback ethapi breaking changes for graphql support
Co-Authored-By: Arachnid <arachnid@notdot.net>
receipts may be null for very short time in some condition. For this case, we should not add the null value into cache. Because you will not get the right result if you keep requesting that receipt.
* swarm/network: fix data race in stream.(*Peer).handleOfferedHashesMsg()
handleOfferedHashesMsg() contained a data race:
- read => in a goroutine, call to c.batchDone()
- write => in the main thread, write to c.sessionAt
c.batchDone() contained a call to c.AddInterval(). Client was a value
receiver for AddInterval. So on c.AddInterval() call the whole client
struct got copied (read) while one of its field was modified in
handleOfferedHashesMsg() (write).
fixesethersphere/go-ethereum#1086
* swarm/network: simplify some trivial statements
* accounts/abi: fix name styling when unpacking abi fields w/ underscores
ABI fields with underscores that are being unpacked
into structs expect structs with following form:
int_one -> Int_one
whereas in abigen the generated structs are camelcased
int_one -> IntOne
so updated the unpack method to expect camelcased structs as well.
* swarm/network: Revised depth calculation with tests
* swarm/network: WIP remove redundant "full" function
* swarm/network: WIP peerpot refactor
* swarm/network: Make test methods submethod of peerpot and embed kad
* swarm/network: Remove commented out code
* swarm/network: Rename health test functions
* swarm/network: Too many n's
* swarm/network: Change hive Healthy func to accept addresses
* swarm/network: Add Healthy proxy method for api in hive
* swarm/network: Skip failing test out of scope for PR
* swarm/network: Skip all tests dependent on SuggestPeers
* swarm/network: Remove commented code and useless kad Pof member
* swarm/network: Remove more unused code, add counter on depth test errors
* swarm/network: WIP Create Healthy assertion tests
* swarm/network: Roll back health related methods receiver change
* swarm/network: Hardwire network minproxbinsize in swarm sim
* swarm/network: Rework Health test to strict
Pending add test for saturation
And add test for as many as possible up to saturation
* swarm/network: Skip discovery tests (dependent on SuggestPeer)
* swarm/network: Remove useless minProxBinSize in stream
* swarm/network: Remove unnecessary testing.T param to assert health
* swarm/network: Implement t.Helper() in checkHealth
* swarm/network: Rename check back to assert now that we have helper magic
* swarm/network: Revert WaitTillHealthy change (deferred to nxt PR)
* swarm/network: Kademlia tests GotNN => ConnectNN
* swarm/network: Renames and comments
* swarm/network: Add comments
* p2p/simulation: WIP minimal snapshot test
* p2p/simulation: Add snapshot create, load and verify to snapshot test
* build: add test tag for tests
* p2p/simulations, build: Revert travis change, build test sym always
* p2p/simulations: Add comments, timeout check on additional events
* p2p/simulation: Add benchmark template for minimal peer protocol init
* p2p/simulations: Remove unused code
* p2p/simulation: Correct timer reset
* p2p/simulations: Put snapshot check events in buffer and call blocking
* p2p/simulations: TestSnapshot fail if Load function returns early
* p2p/simulations: TestSnapshot wait for all connections before returning
* p2p/simulation: Revert to before wait for snap load (5e75594)
* p2p/simulations: add "conns after load" subtest to TestSnapshot
and nudge
* swarm/shed: add metrics to each shed db
* swarm/shed: push metrics prefix up
* swarm/shed: rename prefix to metricsPrefix
* swarm/shed: unexport Meter, remove Mutex for quit channel
* geth/core/eth: implement constantinople override flag
* les: implemnent constantinople override flag for les clients
* cmd/geth, eth, les: fix typo, move flag to experimentals
* Rejects peers that respond with a different hash for any of the passed in block numbers.
* Meant for emergency situations when the network forks unexpectedly.
* swarm/network: Hive - do not notify peer if discovery is disabled
* p2p/simulations: validate all connections on loading a snapshot
* p2p/simulations: track all connections in on snapshot loading
* p2p/simulations: add snapshotLoadTimeout variable
* p2p/simulations: ignore control events in snapshot load
* p2p/simulations: simplify event loop synchronization
* p2p/simulations: return already connected error from Load function
* p2p/simulations: log warning on snapshot loading disconnection
Until this commit, when sending an RPC request that called `NewEVM`, a blank `vm.Config`
would be taken so as to set some options, based on the default configuration. If some extra
configuration switches were passed to the blockchain, those would be ignored.
This PR adds a function to get the config from the blockchain, and this is what is now used
for RPC calls.
Some subsequent changes need to be made, see https://github.com/ethereum/go-ethereum/pull/17955#pullrequestreview-182237244
for the details of the discussion.
* swarm/network/stream/: added stream protocol version match tests
* Increase BZZ version due to streamer version change; version tests
* swarm/network: increased hive and test protocol version
TryUpdate does not call t.trie.TryUpdate(key, value) and calls t.trie.TryDelete
instead. The update operation simply deletes the corresponding entry, though
it could retrieve later by odr. However, it adds further network overhead.
- Have `${DataDir}/bzzd.ipc` as IPC path default.
- Respect the `--datadir` flag.
- Keep only the global `--ipcpath` flag and drop the local `--ipcpath` flag
as flags might overwrite each other. (Note: before global `--ipcpath`
was ignored even if it was set)
fixes ethersphere#795
- Replace "crypto/rand" to "math/rand" for files content generation
- Remove swarm/network_test.go.Shuffle and swarm/btm/btm_test.go.Shuffle - because go1.9 support dropped (see https://github.com/ethereum/go-ethereum/pull/17807 and comments to swarm/network_test.go.Shuffle)
On file access LDBStore's tryAccessIdx() function created a faulty
GC Index Data entry, because not indexing the ikey correctly.
That caused the chunk addresses/hashes to start with '00' and the last
two digits were dropped. => Incorrect chunk address.
Besides the fix, the commit also contains a schema change which will
run the CleanGCIndex() function to clean the GC index from erroneous
entries.
Note: CleanGCIndex() rebuilds the index from scratch which can take
a really-really long time with a huge DB (possibly an hour).
Access count was not incremented when chunk was retrieved
from cache. So the garbage collector might have deleted the most
frequently accessed chunk from disk.
Co-authored-by: Ferenc Szabo <ferenc.szabo@ethereum.org>
ethereum/go-ethereum#16734 introduced BlockHash to the FilterQuery
struct. However, ethclient was not updated to include BlockHash in the actual
RPC request.
* RequestFromPeers does not use peers marked as lightnode
* fix warning about variable name
* write tests for RequestFromPeers
* lightnodes should be omitted from the addressbook
* resolve pr comments regarding logging, formatting and comments
* resolve pr comments regarding comments and added a missing newline
* add assertions to check peers in live connections
* core: speed up GenerateChain
Use a mock implementation of ChainReader instead of creating
and destroying a BlockChain object for each generated block.
* eth/downloader: speed up tests by generating chain only once
This change reworks the downloader tests so they share a common test
blockchain instead of generating a chain in every test. The tests are
roughly twice as fast now.
* swarm: clean up unused private types and functions
Those that were identified by code inspection tool.
* swarm/storage: move/add Proximity GoDoc from deleted private function
The mentioned proximity() private function was deleted in:
1ca8fc1e6fa0ab4ab1aaca06d6fb32e173cd5f2f
* cmd/clef: replace password arg with prompt (#17829)
Entering passwords on the command line is not secure as it is easy to recover from bash_history or the process table.
1. The clef command addpw was renamed to setpw to better describe the functionality
2. The <password> argument was removed and replaced with an interactive prompt
* cmd/clef: remove undeclared variable
This adds the global accumulated refund counter to the standard
json output as a numeric json value. Previously this was not very
interesting since it was not used much, but with the new sstore
gas changes the value is a lot more interesting from a consensus
investigation perspective.
* swarm/network/stream: disambiguate chunk delivery messages (retrieval vs syncing)
* swarm/network/stream: addressed PR comments
* swarm/network/stream: stream protocol version change due to new message types in this PR
* first impl of eth_getProof
* fixed docu
* added comments and refactored based on comments from holiman
* created structs
* handle errors correctly
* change Value to *hexutil.Big in order to have the same output as parity
* use ProofList as return type
This change extends the peer metrics collection:
- traces the life-cycle of the peers
- meters the peer traffic separately for every peer
- creates event feed for the peer events
- emits the peer events
* swarm/network/stream: generalize SetNextBatch and add Server SessionIndex
* swarm/network/stream: fix a typo in comment
* swarm/network/stream: remove live argument from NewSwarmSyncerServer
This PR adds enode.LocalNode and integrates it into the p2p
subsystem. This new object is the keeper of the local node
record. For now, a new version of the record is produced every
time the client restarts. We'll make it smarter to avoid that in
the future.
There are a couple of other changes in this commit: discovery now
waits for all of its goroutines at shutdown and the p2p server
now closes the node database after discovery has shut down. This
fixes a leveldb crash in tests. p2p server startup is faster
because it doesn't need to wait for the external IP query
anymore.
This fixes a rare deadlock with the inproc adapter:
- A node is stopped, which acquires Network.lock.
- The protocol code being simulated (swarm/network in my case)
waits for its goroutines to shut down.
- One of those goroutines calls into the simulation to add a peer,
which waits for Network.lock.
The fix for the deadlock is really simple, just release the lock
before stopping the simulation node.
Other changes in this PR clean up the exec adapter so it reports
node startup errors better and remove the docker adapter because
it just adds overhead.
In the exec adapter, node information is now posted to a one-shot
server. This avoids log parsing and allows reporting startup
errors to the simulation host.
A small change in package node was needed because simulation
nodes use port zero. Node.{HTTP,WS}Endpoint now return the live
endpoints after startup by checking the TCP listener.
Notifier tracks whether subscription are 'active'. A subscription
becomes active when the subscription ID has been sent to the client. If
the client sends notifications in the request handler before the
subscription becomes active they are dropped. The tests tried to work
around this problem by always waiting 5s before sending the first
notification.
Fix it by buffering notifications until the subscription becomes active.
This speeds up all subscription tests.
Also fix TestSubscriptionMultipleNamespaces to wait for three messages
per subscription instead of six. The test now finishes just after all
notifications have been received and doesn't hit the 30s timeout anymore.
* travis: exclude non-test jobs for PRs
We don't usually look at these builders and not starting them
removes ~15min of build time.
* build: don't run vet before tests
Recent versions of Go run vet during 'go test' and we have
a dedicated lint job.
* build: use -timeout 5m for tests
Tests sometimes hang on Travis. CI runs are aborted after 10min with no
output. Adding the timeout means we get to see the stack trace for
timeouts.
- Mime types generator (Standard "mime" package rely on system-settings, see mime.osInitMime)
- Changed swarm/api.Upload:
- simplify I/O throttling by semaphore primitive and use file name where possible
- f.Close() must be called in Defer - otherwise panic or future added early return will cause leak of file descriptors
- one error was suppressed
When CLI tests were spanning new nodes, the log level verbosity was
hard coded as 6. So the Swarm process was always polluting the test
output with TRACE level logs.
Now `go test -v ./cmd/swarm -loglevel 0` works as expected.
* swarm/storage/mru: Adaptive Frequency
swarm/storage/mru/lookup: fixed getBaseTime
Added NewEpoch constructor
swarm/api/client: better error handling in GetResource()
swarm/storage/mru: Renamed structures.
Renamed ResourceMetadata to ResourceID.
Renamed ResourceID.Name to ResourceID.Topic
swarm/storage/mru: Added binarySerializer interface and test tools
swarm/storage/mru/lookup: Changed base time to time and + marshallers
swarm/storage/mru: Added ResourceID (former resourceMetadata)
swarm/storage/mru: Added ResourceViewId and serialization tests
swarm/storage/mru/lookup: fixed epoch unmarshaller. Added Epoch Equals
swarm/storage/mru: Fixes as per review comments
cmd/swarm: reworded resource create/update help text regarding topic
swarm/storage/mru: Added UpdateLookup and serializer tests
swarm/storage/mru: Added UpdateHeader, serializers and tests
swarm/storage/mru: changed UpdateAddr / epoch to Base()
swarm/storage/mru: Added resourceUpdate serializer and tests
swarm/storage/mru: Added SignedResourceUpdate tests and serializers
swarm/storage/mru/lookup: fixed GetFirstEpoch bug
swarm/storage/mru: refactor, comments, cleanup
Also added tests for Topic
swarm/storage/mru: handler tests pass
swarm/storage/mru: all resource package tests pass
swarm/storage/mru: resource test pass after adding
timestamp checking support
swarm/storage/mru: Added JSON serializers to ResourceIDView structures
swarm/storage/mru: Sever, client, API test pass
swarm/storage/mru: server test pass
swarm/storage/mru: Added topic length check
swarm/storage/mru: removed some literals,
improved "previous lookup" test case
swarm/storage/mru: some fixes and comments as per review
swarm/storage/mru: first working version without metadata chunk
swarm/storage/mru: Various fixes as per review
swarm/storage/mru: client test pass
swarm/storage/mru: resource query strings and manifest-less queries
swarm/storage/mru: simplify naming
swarm/storage/mru: first autofreq working version
swarm/storage/mru: renamed ToValues to AppendValues
swarm/resource/mru: Added ToValues / FromValues for URL query strings
swarm/storage/mru: Changed POST resource to work with query strings.
No more JSON.
swarm/storage/mru: removed resourceid
swarm/storage/mru: Opened up structures
swarm/storage/mru: Merged Request and SignedResourceUpdate
swarm/storage/mru: removed initial data from CLI resource create
swarm/storage/mru: Refactor Topic as a direct fixed-length array
swarm/storage/mru/lookup: Comprehensive GetNextLevel tests
swarm/storage/mru: Added comments
Added length checks in Topic
swarm/storage/mru: fixes in tests and some code comments
swarm/storage/mru/lookup: new optimized lookup algorithm
swarm/api: moved getResourceView to api out of server
swarm/storage/mru: Lookup algorithm working
swarm/storage/mru: comments and renamed NewLookupParams
Deleted commented code
swarm/storage/mru/lookup: renamed Epoch.LaterThan to After
swarm/storage/mru/lookup: Comments and tidying naming
swarm/storage/mru: fix lookup algorithm
swarm/storage/mru: exposed lookup hint
removed updateheader
swarm/storage/mru/lookup: changed GetNextEpoch for initial values
swarm/storage/mru: resource tests pass
swarm/storage/mru: valueSerializer interface and tests
swarm/storage/mru/lookup: Comments, improvements, fixes, more tests
swarm/storage/mru: renamed UpdateLookup to ID, LookupParams to Query
swarm/storage/mru: renamed query receiver var
swarm/cmd: MRU CLI tests
* cmd/swarm: remove rogue fmt
* swarm/storage/mru: Add version / header for future use
* swarm/storage/mru: Fixes/comments as per review
cmd/swarm: remove rogue fmt
swarm/storage/mru: Add version / header for future use-
* swarm/storage/mru: fix linter errors
* cmd/swarm: Speeded up TestCLIResourceUpdate
* signer: remove local path disclosure from extapi
* signer: show more data in cli ui
* rpc: make http server forward UA and Origin via Context
* signer, clef/core: ui changes + display UA and Origin
* signer: cliui - indicate less trust in remote headers, see https://github.com/ethereum/go-ethereum/issues/17637
* signer: prevent possibility swap KV-entries in aes_gcm storage, fixes#17635
* signer: remove ecrecover from external API
* signer,clef: default reject instead of warn + valideate new passwords. fixes#17632 and #17631
* signer: check calldata length even if no ABI signature is present
* signer: fix failing testcase
* clef: remove account import from external api
* signer: allow space in passwords, improve error messsage
* signer/storage: fix typos
The contributing instructions in the README are not in the GitHub contributing
guide, which means that people coming from the GitHub issues are less likely to
see them.
Package p2p/enode provides a generalized representation of p2p nodes
which can contain arbitrary information in key/value pairs. It is also
the new home for the node database. The "v4" identity scheme is also
moved here from p2p/enr to remove the dependency on Ethereum crypto from
that package.
Record signature handling is changed significantly. The identity scheme
registry is removed and acceptable schemes must be passed to any method
that needs identity. This means records must now be validated explicitly
after decoding.
The enode API is designed to make signature handling easy and safe: most
APIs around the codebase work with enode.Node, which is a wrapper around
a valid record. Going from enr.Record to enode.Node requires a valid
signature.
* p2p/discover: port to p2p/enode
This ports the discovery code to the new node representation in
p2p/enode. The wire protocol is unchanged, this can be considered a
refactoring change. The Kademlia table can now deal with nodes using an
arbitrary identity scheme. This requires a few incompatible API changes:
- Table.Lookup is not available anymore. It used to take a public key
as argument because v4 protocol requires one. Its replacement is
LookupRandom.
- Table.Resolve takes *enode.Node instead of NodeID. This is also for
v4 protocol compatibility because nodes cannot be looked up by ID
alone.
- Types Node and NodeID are gone. Further commits in the series will be
fixes all over the the codebase to deal with those removals.
* p2p: port to p2p/enode and discovery changes
This adapts package p2p to the changes in p2p/discover. All uses of
discover.Node and discover.NodeID are replaced by their equivalents from
p2p/enode.
New API is added to retrieve the enode.Node instance of a peer. The
behavior of Server.Self with discovery disabled is improved. It now
tries much harder to report a working IP address, falling back to
127.0.0.1 if no suitable address can be determined through other means.
These changes were needed for tests of other packages later in the
series.
* p2p/simulations, p2p/testing: port to p2p/enode
No surprises here, mostly replacements of discover.Node, discover.NodeID
with their new equivalents. The 'interesting' API changes are:
- testing.ProtocolSession tracks complete nodes, not just their IDs.
- adapters.NodeConfig has a new method to create a complete node.
These changes were needed to make swarm tests work.
Note that the NodeID change makes the code incompatible with old
simulation snapshots.
* whisper/whisperv5, whisper/whisperv6: port to p2p/enode
This port was easy because whisper uses []byte for node IDs and
URL strings in the API.
* eth: port to p2p/enode
Again, easy to port because eth uses strings for node IDs and doesn't
care about node information in any way.
* les: port to p2p/enode
Apart from replacing discover.NodeID with enode.ID, most changes are in
the server pool code. It now deals with complete nodes instead
of (Pubkey, IP, Port) triples. The database format is unchanged for now,
but we should probably change it to use the node database later.
* node: port to p2p/enode
This change simply replaces discover.Node and discover.NodeID with their
new equivalents.
* swarm/network: port to p2p/enode
Swarm has its own node address representation, BzzAddr, containing both
an overlay address (the hash of a secp256k1 public key) and an underlay
address (enode:// URL).
There are no changes to the BzzAddr format in this commit, but certain
operations such as creating a BzzAddr from a node ID are now impossible
because node IDs aren't public keys anymore.
Most swarm-related changes in the series remove uses of
NewAddrFromNodeID, replacing it with NewAddr which takes a complete node
as argument. ToOverlayAddr is removed because we can just use the node
ID directly.
Interpreter initialization is left to the PRs implementing them.
Options for external interpreters are passed after a colon in the
`--vm.ewasm` and `--vm.evm` switches.
* ethdb: unified code comment style.
* puppeth: it is unnecessary to alloc pre-funded to 256 addresses
* Revert "puppeth: it is unnecessary to alloc pre-funded to 256 addresses"
This reverts commit 5e04fbccf0b8aca85030af1779bb7a949033d9d8.
* puppeth: fix comment typo
* Revert "ethdb: unified code comment style."
This reverts commit a581efb3f06a96fc7aec0bfae03c7b6d5a0c1a77.
* swarm/network: simplify kademlia/hive; rid interfaces
* swarm, swarm/network/stream, swarm/netork/simulations,, swarm/pss: adapt to new Kad API
* swarm/network: minor changes re review; add missing lock to NeighbourhoodDepthC
* swarm/storage/encryption: async segmentwise encryption/decryption
* swarm/storage: adapt hasherstore to encryption API change
* swarm/api: adapt RefEncryption for AC to new Encryption API
* swarm/storage/encryption: address review comments
Makes Interface interface a bit more stateless and abstract.
Obviously this change is dictated by EVMC design. The EVMC tries to keep the responsibility for EVM features totally inside the VMs, if feasible. This makes VM "stateless" because VM does not need to pass any information between executions, all information is included in parameters of the execute function.
This commit does a few things at once:
- Updates the tests to contain the latest data from ethereum/tests repo.
- Enables Constantinople state tests. This is needed to be able to
fuzz-test the evm with constantinople rules.
- Fixes the error in opSAR that we've known about for some time. I was
kind of saving it to see if we hit upon it with the random test
generator, but it's difficult to both enable the tests and have the
bug there -- we don't want to forget about it, so maybe it's better
to just fix it.
* miner: commit state which is relative with sealing result
* consensus, core, miner, mobile: introduce sealHash interface
* miner: evict pending task with threshold
* miner: go fmt
* les: fix crasher in NodeInfo when running as server
The ProtocolManager computes CHT and Bloom trie roots by asking the
indexers for their current head. It tried to get the indexers from
LesOdr, but no LesOdr instance is created in server mode.
Attempt to fix this by moving the indexers, protocol creation and
NodeInfo to a new lesCommons struct which is embedded into both server
and client.
All this setup code should really be cleaned up, but this is just a
hotfix so we have to do that some other time.
* les: fix commons protocol maker
This PR enables the indexers to work in light client mode by
downloading a part of these tries (the Merkle proofs of the last
values of the last known section) in order to be able to add new
values and recalculate subsequent hashes. It also adds CHT data to
NodeInfo.
* backends: increase gaslimit in order to allow tests of large contracts
* backends: increase gaslimit in order to allow tests of large contracts
* backends: increase gaslimit in order to allow tests of large contracts
This PR implements les.freeClientPool. It also adds a simulated clock
in common/mclock, which enables time-sensitive tests to run quickly
and still produce accurate results, and package common/prque which is
a generalised variant of prque that enables removing elements other
than the top one from the queue.
les.freeClientPool implements a client database that limits the
connection time of each client and manages accepting/rejecting
incoming connections and even kicking out some connected clients. The
pool calculates recent usage time for each known client (a value that
increases linearly when the client is connected and decreases
exponentially when not connected). Clients with lower recent usage are
preferred, unknown nodes have the highest priority. Already connected
nodes receive a small bias in their favor in order to avoid accepting
and instantly kicking out clients.
Note: the pool can use any string for client identification. Using
signature keys for that purpose would not make sense when being known
has a negative value for the client. Currently the LES protocol
manager uses IP addresses (without port address) to identify clients.
* swarm: Added lightnode flag
Added --lightnode command line parameter
Added LightNode to Handshake message
* swarm/config: Fixed variable naming
* cmd/swarm: Changed BoolTFlag to BoolFlag for SwarmLightNodeEnabled
* swarm/network: Changed logging
* swarm/network: Changed protocol version testing
* swarm/network: Renamed DefaultNetworkID variable to TestProtocolNetworkID
* swarm/network: Bumped protocol version
* swarm/network: Changed LightNode handhsake test to table driven
* swarm/network: Changed back TestProtocolVersion to 5 for now
* swarm/network: Moved the test configuration inside the test function scope
* swarm/network/simulation: increase the sleep duration for TestRun
* cmd/swarm, swarm: fix failing tests on mac
* cmd/swarm: update TestCLISwarmFs skip comment
* swarm/network/simulation: adjust disconnections on simulation close
* swarm/network/simulation: call cleanups after net shutdown
* consensus/ethash: start remote ggoroutine to handle remote mining
* consensus/ethash: expose remote miner api
* consensus/ethash: expose submitHashrate api
* miner, ethash: push empty block to sealer without waiting execution
* consensus, internal: add getHashrate API for ethash
* consensus: add three method for consensus interface
* miner: expose consensus engine running status to miner
* eth, miner: specify etherbase when miner created
* miner: commit new work when consensus engine is started
* consensus, miner: fix some logics
* all: delete useless interfaces
* consensus: polish a bit
- Update benchmarks to use a pool of int pools.
Unless benchmarks are aborted with segmentation fault.
Signed-off-by: Hyung-Kyu Choi <hqueue@users.noreply.github.com>
* rpc: Make HTTP server timeout values configurable
* rpc: Remove flags for setting HTTP Timeouts, configuring via .toml is sufficient.
* rpc: Replace separate constants with a single default struct.
* rpc: Update HTTP Server Read and Write Timeouts to 30s.
* rpc: Remove redundant NewDefaultHTTPTimeouts function.
* rpc: document HTTPTimeouts.
* rpc: sanitize timeout values for library use
* build: add support for different package and binary names
* build: bump up copyright date
* build: change default PackageName to empty string
* build, internal, swarm: enhance build/release process
* build: hack ethereum-swarm as a "depends" in deb package
* build/ci: remove redundant variables
* build, cmd, mobile, params, swarm: remove VERSION file; rename Version to VersionMeta;
* internal: remove VERSION() method which reads VERSION file
* build: fix VersionFilePath to Version
* Makefile: remove clean_go_build_cache.sh until it works
* Makefile: revert removal of clean_go_build_cache.sh
- Define an Interpreter interface
- One contract can call contracts from other interpreter types.
- Pass the interpreter to the operands instead of the evm.
This is meant to prevent type assertions in operands.
Our original wrapper code had two parts. One taken from a third
party repository (who took it from upstream Go) licensed under
BSD-3. The second written by Jeff, Felix and Gustav, licensed
under LGPL. This made this package problematic to use from the
outside.
With the agreement of the original copyright holders, this commit
changes the license of the LGPL portions of the code to BSD-3:
---
I agree changing from LGPL to a BSD style license.
Jeff
---
Hey guys,
My preference would be to relicense to GNUBL, but I'm also OK with BSD.
Cheers,
Gustav
---
Felix Lange (fjl):
I would approve anything that makes our licensing less complicated
---
* swarm/storage/mru: Add embedded publickey and remove ENS dep
This commit breaks swarm, swarm/api...
but tests in swarm/storage/mru pass
* swarm: Refactor swarm, swarm/api to mru changes, make tests pass
* swarm/storage/mru: Remove self from recv, remove test ens vldtr
* swarm/storage/mru: Remove redundant test, expose ResourceHash mthd
* swarm/storage/mru: Make HeaderGetter mandatory + godoc fixes
* swarm/storage: Remove validator prefix for metadata chunk
* swarm/storage/mru: Use Address instead of PublicKey
* swarm/storage/mru: Change index from name to metadata chunk addr
* swarm/storage/mru: Refactor swarm/api/... to MRU index changes
* swarm/storage/mru: Refactor cleanup
* swarm/storage/mru: Rebase cleanup
* swarm: Use constructor for GenericSigner MRU in swarm.go
* swarm/storage: Change to BMTHash for MRU hashing
* swarm/storage: Reduce loglevel on chunk validator logs
* swarm/storage/mru: Delint
* swarm: MRU Rebase cleanup
* swarm/storage/mru: client-side mru signatures
Rebase to PR #668 and fix all conflicts
* swarm/storage/mru: refactor and documentation
* swarm/resource/mru: error-checking tests for parseUpdate/newUpdateChunk
* swarm/storage/mru: Added resourcemetadata tests
* swarm/storage/mru: Added tests for UpdateRequest
* swarm/storage/mru: more test coverage for UpdateRequest and comments
* swarm/storage/mru: Avoid fake chunks in parseUpdate()
* swarm/storage/mru: Documented resource.go extensively
moved some functions where they make most sense
* swarm/storage/mru: increase test coverage for UpdateRequest and
variable name changes throughout to increase consistency
* swarm/storage/mru: moved default timestamp to NewCreateRequest-
* swarm/storage/mru: lookup refactor
* swarm/storage/mru: added comments and renamed raw flag to rawmru
* swarm/storage/mru: fix receiver typo
* swarm/storage/mru: refactored update chunk new/create
* swarm/storage/mru: refactored signature digest to avoid malleability
* swarm/storage/mru: optimize update data serialization
* swarm/storage/mru: refactor and cleanup
* swarm/storage/mru: add timestamp struct and serialization
* swarm/storage/mru: fix lint error and mark some old code for deletion
* swarm/storage/mru: remove unnecessary variable
* swarm/storage/mru: Added more comments throughout
* swarm/storage/mru: Refactored metadata chunk layout + extensive error...
* swarm/storage/mru: refactor cli parser
Changed resource info output to JSON
* swarm/storage/mru: refactor serialization for extensibility
refactored error messages to NewErrorf
* swarm/storage/mru: Moved Signature to resource_sign.
Check Sign errors in server tests
* swarm/storage/mru: Remove isSafeName() checks
* swarm/storage/mru: scrubbed off all references to "block" for time
* swarm/storage/mru: removed superfluous isSynced() call.
* swarm/storage/mru: remove isMultihash() and ToSafeName functions
* swarm/storage/mru: various fixes and comments
* swarm/storage/mru: decoupled cli for independent create/update
* Made resource name optional
* Removed unused LookupPrevious
* swarm/storage/mru: Decoupled resource create / update & refactor
* swarm/storage/mru: Fixed some comments as per issues raised in PR #743
* swarm/storage/mru: Cosmetic changes as per #743 comments
* swarm/storage/mru: refct request encoder/decoder > marshal/unmarshal
* swarm/storage/mru: Cosmetic changes as per review in #748
* swarm/storage/mru: removed timestamp proof placeholder
* swarm/storage/mru: cosmetic/doc/fixes changes as per comments in #704
* swarm/storage/mru: removed unnecessary check in Handler.update
* swarm/storage/mru: Implemented Marshaler/Unmarshaler iface in Request
* swarm/storage/mru: Fixed linter error
* swarm/storage/mru: removed redundant address in signature digest
* swarm/storage/mru: fixed bug: LookupLatestVersionInPeriod not working
* swarm/storage/mru: Unfold Request creation API for create or update+create
set common time source for mru package
* swarm/api/http: fix HandleGetResource error variable shadowed
when requesting a resource that does not exist
* swarm/storage/mru: Add simple check to detect duplicate updates
* swarm/storage/mru: moved Multihash() to the right place.
* cmd/swarm: remove unneeded clientaccountmanager.go
* swarm/storage/mru: Changed some comments as per reviews in #784
* swarm/storage/mru: Made SignedResourceUpdate.GetDigest() public
* swarm/storage/mru: cosmetic changes as per comments in #784
* cmd/swarm: Inverted --multihash flag default
* swarm/storage/mru: removed Verify from SignedResourceUpdate.fromChunk
* swarm/storage/mru: Moved validation code out of serializer
Cosmetic / comment changes
* swarm/storage/mru: Added unit tests for UpdateLookup
* swarm/storage/mru: Increased coverage of metadata serialization
* swarm/storage/mru: Increased test coverage of updateHeader serializers
* swarm/storage/mru: Add resourceUpdate serializer test
* cmd/swarm: minor cli flag text adjustments
* cmd/swarm, swarm/storage, swarm: fix mingw on windows test issues
* cmd/swarm: support for smoke tests on the production swarm cluster
* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion
* changed colour of landing page
* landing page reacts to enter keypress
* swarm/api/http: sticky footer for swarm landing page using flex
* swarm/api/http: sticky footer for error pages and fix for multiple choices
* swarm: propagate ctx to internal apis (#754)
* swarm/simnet: add basic node/service functions
* swarm/netsim: add buckets for global state and kademlia health check
* swarm/netsim: Use sync.Map as bucket and provide cleanup function for...
* swarm, swarm/netsim: adjust SwarmNetworkTest
* swarm/netsim: fix tests
* swarm: added visualization option to sim net redesign
* swarm/netsim: support multiple services per node
* swarm/netsim: remove redundant return statement
* swarm/netsim: add comments
* swarm: shutdown HTTP in Simulation.Close
* swarm: sim HTTP server timeout
* swarm/netsim: add more simulation methods and peer events examples
* swarm/netsim: add WaitKademlia example
* swarm/netsim: fix comments
* swarm/netsim: terminate peer events goroutines on simulation done
* swarm, swarm/netsim: naming updates
* swarm/netsim: return not healthy kademlias on WaitTillHealthy
* swarm: fix WaitTillHealthy call in testSwarmNetwork
* swarm/netsim: allow bucket to have any type for a key
* swarm: Added snapshots to new netsim
* swarm/netsim: add more tests for bucket
* swarm/netsim: move http related things into separate files
* swarm/netsim: add AddNodeWithService option
* swarm/netsim: add more tests and Start* methods
* swarm/netsim: add peer events and kademlia tests
* swarm/netsim: fix some tests flakiness
* swarm/netsim: improve random nodes selection, fix TestStartStop* tests
* swarm/netsim: remove time measurement from TestClose to avoid flakiness
* swarm/netsim: builder pattern for netsim HTTP server (#773)
* swarm/netsim: add connect related tests
* swarm/netsim: add comment for TestPeerEvents
* swarm: rename netsim package to network/simulation
- AsyncHasher implements AsyncWriter interface
- add extra level for zerohashes in pool to lookup empty data hash
- remove unused segment, hash and depth fields from Tree
- Hash pkg function -> syncHash moved to test
- add asyncHash helper func to tests using shuffle
- add TestAsyncCorrectness to tests
- add BenchmarkBMTAsync to tests
- refactor benchmarks using subbenchmarks
- improved comments
- preinitialise base hashers on the nodes
* cmd/swarm: minor cli flag text adjustments
* swarm/api/http: sticky footer for swarm landing page using flex
* swarm/api/http: sticky footer for error pages and fix for multiple choices
* cmd/swarm, swarm/storage, swarm: fix mingw on windows test issues
* cmd/swarm: update description of swarm cmd
* swarm: added network ID test
* cmd/swarm: support for smoke tests on the production swarm cluster
* cmd/swarm/swarm-smoke: simplify cluster logic as per suggestion
* swarm: propagate ctx to internal apis (#754)
* swarm/metrics: collect disk measurements
* swarm/bmt: fix io.Writer interface
* Write now tolerates arbitrary variable buffers
* added variable buffer tests
* Write loop and finalise optimisation
* refactor / rename
* add tests for empty input
* swarm/pss: (UPDATE) Generic notifications package (#744)
swarm/pss: Generic package for creating pss notification svcs
* swarm: Adding context to more functions
* swarm/api: change colour of landing page in templates
* swarm/api: change landing page to react to enter keypress
* core: fix func TxDifference
fix a typo in func comment;
change named return to unnamed as there's explicit return in the body
* fix another typo in TxDifference
* p2p/discover: move bond logic from table to transport
This commit moves node endpoint verification (bonding) from the table to
the UDP transport implementation. Previously, adding a node to the table
entailed pinging the node if needed. With this change, the ping-back
logic is embedded in the packet handler at a lower level.
It is easy to verify that the basic protocol is unchanged: we still
require a valid pong reply from the node before findnode is accepted.
The node database tracked the time of last ping sent to the node and
time of last valid pong received from the node. Node endpoints are
considered verified when a valid pong is received and the time of last
pong was called 'bond time'. The time of last ping sent was unused. In
this commit, the last ping database entry is repurposed to mean last
ping _received_. This entry is now used to track whether the node needs
to be pinged back.
The other big change is how nodes are added to the table. We used to add
nodes in Table.bond, which ran when a remote node pinged us or when we
encountered the node in a neighbors reply. The transport now adds to the
table directly after the endpoint is verified through ping. To ensure
that the Table can't be filled just by pinging the node repeatedly, we
retain the isInitDone check. During init, only nodes from neighbors
replies are added.
* p2p/discover: reduce findnode failure counter on success
* p2p/discover: remove unused parameter of loadSeedNodes
* p2p/discover: improve ping-back check and comments
* p2p/discover: add neighbors reply nodes always, not just during init
These RPC calls are analogous to Parity's parity_addReservedPeer and
parity_removeReservedPeer.
They are useful for adjusting the trusted peer set during runtime,
without requiring restarting the server.
The current trie memory database/cache that we do pruning on stores
trie nodes as binary rlp encoded blobs, and also stores the node
relationships/references for GC purposes. However, most of the trie
nodes (everything apart from a value node) is in essence just a
collection of references.
This PR switches out the RLP encoded trie blobs with the
collapsed-but-not-serialized trie nodes. This permits most of the
references to be recovered from within the node data structure,
avoiding the need to track them a second time (expensive memory wise).
* debug: Use pprof goroutine writer in debug.Stacks() to ensure all goroutines are captured.
* Up to 64MB limit, previous code only captured first 1MB of goroutines.
* internal/debug: simplify stacks handler
* fix typo
* fix pointer receiver
This package was meant to hold an improved 256 bit integer library, but
the effort was abandoned in 2015. AFAIK nothing ever used this package.
Time to say goodbye.
Prior to this change, when geth was started with `geth -dev -rpc`,
it would report a network id of `1` in response to the `net_version` RPC
request. But the actual network id it used to verify transactions
was `1337`.
This change causes geth instead respond with `1337` to the `net_version`
RPC when geth is started with `geth -dev -rpc`.
* vm/test: add tests+benchmarks for mstore
* core/vm: less alloc and copying for mstore
* core/vm: less allocs in sload
* vm: check for errors more correctly
This commit adds all changes needed for the merge of swarm-network-rewrite.
The changes:
- build: increase linter timeout
- contracts/ens: export ensNode
- log: add Output method and enable fractional seconds in format
- metrics: relax test timeout
- p2p: reduced some log levels, updates to simulation packages
- rpc: increased maxClientSubscriptionBuffer to 20000
Improves test portability by resolving 127.0.0.1:0
to get a random free port instead of the hard coded one. Now
the test works if you have a running node on the same
interface already.
Fixes#15685
This PR fixes a retriever logic bug. When a peer had a soft timeout
and then a response arrived, it always assumed it was the same peer
even though it could have been a later requested one that did not time
out at all yet. In this case the logic went to an illegal state and
deadlocked, causing a goroutine leak.
Fixes#16243 and replaces #16359.
Thanks to @riceke for finding the bug in the logic.
ToECDSAPub was unsafe because it returned a non-nil key with nil X, Y in
case of invalid input. This change replaces ToECDSAPub with
UnmarshalPubkey across the codebase.
The error produced when using a Parity RPC was the following:
ERROR: transaction did not get mined: failed to get tx for txid 0xbdeb094b3278019383c8da148ff1cb5b5dbd61bf8731bc2310ac1b8ed0235226: json: cannot unmarshal non-string into Go struct field txExtraInfo.blockHash of type common.Hash
Allow the --abi flag to be given - to indicate that it should read the
ABI information from standard input. It expects to read the solc output
with the --combined-json flag providing bin, abi, userdoc, devdoc, and
metadata, and works very similarly to the internal invocation of solc,
except it allows external invocation of solc.
This facilitates integration with more complex solc invocations, such
as invocations that require path remapping or --allow-paths tweaks.
Simple usage example:
solc --combined-json bin,abi,userdoc,devdoc,metadata *.sol | abigen --abi -
* metrics: expvar support for ResettingTimer
* metrics: use integers for percentiles; remove Overall
* metrics: fix edge-case panic for index-out-of-range
This removes a golint warning: type name will be used as trie.TrieSync by
other packages, and that stutters; consider calling this Sync.
In hexToKeybytes len(hex) is even and (even+1)/2 == even/2, remove the +1.
* core: use a wrapped `map` and `sync.RWMutex` for `TxPool.all` to remove contention in `TxPool.Get`.
* core: Remove redundant `txLookup.Find` and improve comments on txLookup methods.
This applies spec changes from ethereum/EIPs#1049 and adds support for
pluggable identity schemes.
Some care has been taken to make the "v4" scheme standalone. It uses
public APIs only and could be moved out of package enr at any time.
A couple of minor changes were needed to make identity schemes work:
- The sequence number is now updated in Set instead of when signing.
- Record is now copy-safe, i.e. calling Set on a shallow copy doesn't
modify the record it was copied from.
Feed keeps active subscription channels in a slice called 'f.sendCases'.
The Send method tracks the active cases in a local variable 'cases'
whose value is f.sendCases initially. 'cases' shrinks to a shorter
prefix of f.sendCases every time a send succeeds, moving the successful
case out of range of the active case list.
This can be confusing because the two slices share a backing array. Add
more comments to document what is going on. Also add a test for removing
a case that is in 'f.sentCases' but not 'cases'.
* whisper/mailserver: pass init error to the caller
* whisper/mailserver: add returns to fmt.Errorf
* whisper/mailserver: check err in mailserver init test
Building geth requires both a Go (version 1.7 or later) and a C compiler.
Building geth requires both a Go (version 1.10 or later) and a C compiler.
You can install them using your favourite package manager.
Once the dependencies are installed, run
@@ -34,13 +34,13 @@ The go-ethereum project comes with several wrappers/executables found in the `cm
| Command | Description |
|:----------:|-------------|
| **`geth`** | Our main Ethereum CLI client. It is the entry point into the Ethereum network (main-, test- or private net), capable of running as a full node (default) archive node (retaining all historical state) or a light node (retrieving data live). It can be used by other processes as a gateway into the Ethereum network via JSON RPC endpoints exposed on top of HTTP, WebSocket and/or IPC transports. `geth --help` and the [CLI Wiki page](https://github.com/ethereum/go-ethereum/wiki/Command-Line-Options) for command line options. |
| `abigen` | Source code generator to convert Ethereum contract definitions into easy to use, compile-time type-safe Go packages. It operates on plain [Ethereum contract ABIs](https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI) with expanded functionality if the contract bytecode is also available. However it also accepts Solidity source files, making development much more streamlined. Please see our [Native DApps](https://github.com/ethereum/go-ethereum/wiki/Native-DApps:-Go-bindings-to-Ethereum-contracts) wiki page for details. |
| **`geth`** | Our main Ethereum CLI client. It is the entry point into the Ethereum network (main-, test- or private net), capable of running as a full node (default), archive node (retaining all historical state) or a light node (retrieving data live). It can be used by other processes as a gateway into the Ethereum network via JSON RPC endpoints exposed on top of HTTP, WebSocket and/or IPC transports. `geth --help` and the [CLI Wiki page](https://github.com/ethereum/go-ethereum/wiki/Command-Line-Options) for command line options. |
| `abigen` | Source code generator to convert Ethereum contract definitions into easy to use, compile-time type-safe Go packages. It operates on plain [Ethereum contract ABIs](https://github.com/ethereum/wiki/wiki/Ethereum-Contract-ABI) with expanded functionality if the contract bytecode is also available. However, it also accepts Solidity source files, making development much more streamlined. Please see our [Native DApps](https://github.com/ethereum/go-ethereum/wiki/Native-DApps:-Go-bindings-to-Ethereum-contracts) wiki page for details. |
| `bootnode` | Stripped down version of our Ethereum client implementation that only takes part in the network node discovery protocol, but does not run any of the higher level application protocols. It can be used as a lightweight bootstrap node to aid in finding peers in private networks. |
| `evm` | Developer utility version of the EVM (Ethereum Virtual Machine) that is capable of running bytecode snippets within a configurable environment and execution mode. Its purpose is to allow isolated, fine-grained debugging of EVM opcodes (e.g. `evm --code 60ff60ff --debug`). |
| `gethrpctest` | Developer utility tool to support our [ethereum/rpc-test](https://github.com/ethereum/rpc-tests) test suite which validates baseline conformity to the [Ethereum JSON RPC](https://github.com/ethereum/wiki/wiki/JSON-RPC) specs. Please see the [test suite's readme](https://github.com/ethereum/rpc-tests/blob/master/README.md) for details. |
| `rlpdump` | Developer utility tool to convert binary RLP ([Recursive Length Prefix](https://github.com/ethereum/wiki/wiki/RLP)) dumps (data encoding used by the Ethereum protocol both network as well as consensus wise) to userfriendlier hierarchical representation (e.g. `rlpdump --hex CE0183FFFFFFC4C304050583616263`). |
| `swarm` | swarm daemon and tools. This is the entrypoint for the swarm network. `swarm --help` for command line options and subcommands. See https://swarm-guide.readthedocs.io for swarm documentation. |
| `rlpdump` | Developer utility tool to convert binary RLP ([Recursive Length Prefix](https://github.com/ethereum/wiki/wiki/RLP)) dumps (data encoding used by the Ethereum protocol both network as well as consensus wise) to user-friendlier hierarchical representation (e.g. `rlpdump --hex CE0183FFFFFFC4C304050583616263`). |
| `swarm` | Swarm daemon and tools. This is the entrypoint for the Swarm network. `swarm --help` for command line options and subcommands. See [Swarm README](https://github.com/ethereum/go-ethereum/tree/master/swarm) for more information. |
| `puppeth` | a CLI wizard that aids in creating a new Ethereum network. |
## Running geth
@@ -69,10 +69,10 @@ This command will:
* Start up Geth's built-in interactive [JavaScript console](https://github.com/ethereum/go-ethereum/wiki/JavaScript-Console),
(via the trailing `console` subcommand) through which you can invoke all official [`web3` methods](https://github.com/ethereum/wiki/wiki/JavaScript-API)
as well as Geth's own [management APIs](https://github.com/ethereum/go-ethereum/wiki/Management-APIs).
This too is optional and if you leave it out you can always attach to an already running Geth instance
This tool is optional and if you leave it out you can always attach to an already running Geth instance
with `geth attach`.
### Full node on the Ethereum test network
### A Full node on the Ethereum test network
Transitioning towards developers, if you'd like to play around with creating Ethereum contracts, you
almost certainly would like to do that without any real money involved until you get the hang of the
@@ -83,10 +83,10 @@ network with your node, which is fully equivalent to the main network, but with
$ geth --testnet console
```
The `console` subcommand have the exact same meaning as above and they are equally useful on the
testnet too. Please see above for their explanations if you've skipped to here.
The `console` subcommand has the exact same meaning as above and they are equally useful on the
testnet too. Please see above for their explanations if you've skipped here.
Specifying the `--testnet` flag however will reconfigure your Geth instance a bit:
Specifying the `--testnet` flag, however, will reconfigure your Geth instance a bit:
* Instead of using the default data directory (`~/.ethereum` on Linux for example), Geth will nest
itself one level deeper into a `testnet` subfolder (`~/.ethereum/testnet` on Linux). Note, on OSX
@@ -103,7 +103,7 @@ separate the two networks and will not make any accounts available between them.
### Full node on the Rinkeby test network
The above test network is a crossclient one based on the ethash proof-of-work consensus algorithm. As such, it has certain extra overhead and is more susceptible to reorganization attacks due to the network's low difficulty / security. Go Ethereum also supports connecting to a proof-of-authority based test network called [*Rinkeby*](https://www.rinkeby.io) (operated by members of the community). This network is lighter, more secure, but is only supported by go-ethereum.
The above test network is a cross-client one based on the ethash proof-of-work consensus algorithm. As such, it has certain extra overhead and is more susceptible to reorganization attacks due to the network's low difficulty/security. Go Ethereum also supports connecting to a proof-of-authority based test network called [*Rinkeby*](https://www.rinkeby.io) (operated by members of the community). This network is lighter, more secure, but is only supported by go-ethereum.
```
$ geth --rinkeby console
@@ -139,13 +139,13 @@ This will start geth in fast-sync mode with a DB memory allowance of 1GB just as
Do not forget `--rpcaddr 0.0.0.0`, if you want to access RPC from other containers and/or hosts. By default, `geth` binds to the local interface and RPC endpoints is not accessible from the outside.
### Programatically interfacing Geth nodes
### Programmatically interfacing Geth nodes
As a developer, sooner rather than later you'll want to start interacting with Geth and the Ethereum
network via your own programs and not manually through the console. To aid this, Geth has built-in
support for a JSON-RPC based APIs ([standard APIs](https://github.com/ethereum/wiki/wiki/JSON-RPC) and
[Geth specific APIs](https://github.com/ethereum/go-ethereum/wiki/Management-APIs)). These can be
exposed via HTTP, WebSockets and IPC (unix sockets on unix based platforms, and named pipes on Windows).
exposed via HTTP, WebSockets and IPC (UNIX sockets on UNIX based platforms, and named pipes on Windows).
The IPC interface is enabled by default and exposes all the APIs supported by Geth, whereas the HTTP
and WS interfaces need to manually be enabled and only expose a subset of APIs due to security reasons.
@@ -168,12 +168,12 @@ HTTP based JSON-RPC API options:
*`--ipcpath` Filename for IPC socket/pipe within the datadir (explicit paths escape it)
You'll need to use your own programming environments' capabilities (libraries, tools, etc) to connect
via HTTP, WS or IPC to a Geth node configured with the above flags and you'll need to speak [JSON-RPC](http://www.jsonrpc.org/specification)
via HTTP, WS or IPC to a Geth node configured with the above flags and you'll need to speak [JSON-RPC](https://www.jsonrpc.org/specification)
on all transports. You can reuse the same connection for multiple requests!
**Note: Please understand the security implications of opening up an HTTP/WS based transport before
doing so! Hackers on the internet are actively trying to subvert Ethereum nodes with exposed APIs!
Further, all browser tabs can access locally running webservers, so malicious webpages could try to
Further, all browser tabs can access locally running webservers, so malicious webpages could try to
subvert locally available APIs!**
### Operating a private network
@@ -241,7 +241,7 @@ that other nodes can use to connect to it and exchange peer information. Make su
displayed IP address information (most probably `[::]`) with your externally accessible IP to get the
actual `enode` URL.
*Note: You could also use a fullfledged Geth node as a bootnode, but it's the less recommended way.*
*Note: You could also use a full-fledged Geth node as a bootnode, but it's the less recommended way.*
#### Starting up your member nodes
@@ -264,7 +264,7 @@ an OpenCL or CUDA enabled `ethminer` instance. For information on such a setup,
[EtherMining subreddit](https://www.reddit.com/r/EtherMining/) and the [Genoil miner](https://github.com/Genoil/cpp-ethereum)
repository.
In a private network setting however, a single CPU miner instance is more than enough for practical
In a private network setting, however a single CPU miner instance is more than enough for practical
purposes as it can produce a stable stream of blocks at the correct intervals without needing heavy
resources (consider running on a single thread, no need for multiple ones either). To start a Geth
instance for mining, run it with all your usual flags, extended by:
@@ -298,7 +298,7 @@ Please make sure your contributions adhere to our coding guidelines:
* E.g. "eth, rpc: make trace configs optional"
Please see the [Developers' Guide](https://github.com/ethereum/go-ethereum/wiki/Developers'-Guide)
for more details on configuring your environment, managing project dependencies and testing procedures.
for more details on configuring your environment, managing project dependencies, and testing procedures.
debsrc [ -signer key-id ] [ -upload dest ] -- creates a debian source package
nsis -- creates a Windows NSIS installer
@@ -59,6 +59,8 @@ import (
"time"
"github.com/ethereum/go-ethereum/internal/build"
"github.com/ethereum/go-ethereum/params"
sv"github.com/ethereum/go-ethereum/swarm/version"
)
var(
@@ -77,52 +79,89 @@ var (
executablePath("geth"),
executablePath("puppeth"),
executablePath("rlpdump"),
executablePath("swarm"),
executablePath("wnode"),
executablePath("clef"),
}
// Files that end up in the swarm*.zip archive.
swarmArchiveFiles=[]string{
"COPYING",
executablePath("swarm"),
}
// A debian package is created for all executables listed here.
debExecutables=[]debExecutable{
{
Name:"abigen",
BinaryName:"abigen",
Description:"Source code generator to convert Ethereum contract definitions into easy to use, compile-time type-safe Go packages.",
},
{
Name:"bootnode",
BinaryName:"bootnode",
Description:"Ethereum bootnode.",
},
{
Name:"evm",
BinaryName:"evm",
Description:"Developer utility version of the EVM (Ethereum Virtual Machine) that is capable of running bytecode snippets within a configurable environment and execution mode.",
},
{
Name:"geth",
BinaryName:"geth",
Description:"Ethereum CLI client.",
},
{
Name:"puppeth",
BinaryName:"puppeth",
Description:"Ethereum private network manager.",
},
{
Name:"rlpdump",
BinaryName:"rlpdump",
Description:"Developer utility tool that prints RLP structures.",
},
{
Name:"swarm",
Description:"Ethereum Swarm daemon and tools",
},
{
Name:"wnode",
BinaryName:"wnode",
Description:"Ethereum Whisper diagnostic tool",
},
{
BinaryName:"clef",
Description:"Ethereum account management tool.",
},
}
// A debian package is created for all executables listed here.
debSwarmExecutables=[]debExecutable{
{
BinaryName:"swarm",
PackageName:"ethereum-swarm",
Description:"Ethereum Swarm daemon and tools",
},
}
debEthereum=debPackage{
Name:"ethereum",
Version:params.Version,
Executables:debExecutables,
}
debSwarm=debPackage{
Name:"ethereum-swarm",
Version:sv.Version,
Executables:debSwarmExecutables,
}
// Debian meta packages to build and push to Ubuntu PPA
debPackages=[]debPackage{
debSwarm,
debEthereum,
}
// Packages to be cross-compiled by the xgo command
--configdir value Directory for Clef configuration (default: "$HOME/.clef")
--chainid value Chain id to use for signing (1=mainnet, 3=ropsten, 4=rinkeby, 5=Goerli) (default: 1)
--lightkdf Reduce key-derivation RAM & CPU usage at some expense of KDF strength
--nousb Disables monitoring for and managing USB hardware wallets
--rpcaddr value HTTP-RPC server listening interface (default: "localhost")
--rpcvhosts value Comma separated list of virtual hostnames from which to accept requests (server enforced). Accepts '*' wildcard. (default: "localhost")
--ipcdisable Disable the IPC-RPC server
--ipcpath Filename for IPC socket/pipe within the datadir (explicit paths escape it)
--rpc Enable the HTTP-RPC server
--rpcport value HTTP-RPC server listening port (default: 8550)
--signersecret value A file containing the password used to encrypt signer credentials, e.g. keystore credentials and ruleset hash
--signersecret value A file containing the (encrypted) master seed to encrypt Clef data, e.g. keystore credentials and ruleset hash
--4bytedb value File containing 4byte-identifiers (default: "./4byte.json")
--4bytedb-custom value File used for writing new 4byte-identifiers submitted via API (default: "./4byte-custom.json")
--auditlog value File used to emit audit logs. Set to "" to disable (default: "audit.log")
--rules value Enable rule-engine (default: "rules.json")
--stdio-ui Use STDIN/STDOUT as a channel for an external UI. This means that an STDIN/STDOUT is used for RPC-communication with a e.g. a graphical user interface, and can be used when the signer is started by an external process.
--stdio-ui-test Mechanism to test interface between signer and UI. Requires 'stdio-ui'.
--stdio-ui Use STDIN/STDOUT as a channel for an external UI. This means that an STDIN/STDOUT is used for RPC-communication with a e.g. a graphical user interface, and can be used when Clef is started by an external process.
--stdio-ui-test Mechanism to test interface between Clef and UI. Requires 'stdio-ui'.
--advanced If enabled, issues warnings instead of rejections for suspicious requests. Default off
--help, -h show help
--version, -v print the version
```
@@ -91,7 +98,7 @@ invoking methods with the following info:
* [x] Version info about the signer
* [x] Address of API (http/ipc)
* [ ] List of known accounts
* [ ] Have a default timeout on signing operations, so that if the user has not answered withing e.g. 60 seconds, the request is rejected.
* [ ] Have a default timeout on signing operations, so that if the user has not answered within e.g. 60 seconds, the request is rejected.
* [ ]`account_signRawTransaction`
* [ ]`account_bulkSignTransactions([] transactions)` should
* only exist if enabled via config/flag
@@ -129,7 +136,7 @@ The signer listens to HTTP requests on `rpcaddr`:`rpcport`, with the same JSONRP
expected to be JSON [jsonrpc 2.0 standard](http://www.jsonrpc.org/specification).
Some of these call can require user interaction. Clients must be aware that responses
may be delayed significanlty or may never be received if a users decides to ignore the confirmation request.
may be delayed significantly or may never be received if a users decides to ignore the confirmation request.
The External API is **untrusted** : it does not accept credentials over this api, nor does it expect
-`text/plain`: simple hex data validated by `account_ecRecover`
- account [address]: account to sign with
- data [data]: data to sign
- data [object]: data to sign
#### Result
- calculated signature [data]
@@ -364,8 +378,9 @@ Bash example:
{
"id":3,
"jsonrpc":"2.0",
"method":"account_sign",
"method":"account_signData",
"params":[
"data/plain",
"0x1923f626bb8dc025849e00f99c25fe2b2f7fb0db",
"0xaabbccdd"
]
@@ -381,11 +396,109 @@ Response
}
```
### account_signTypedData
#### Sign data
Signs a chunk of structured data conformant to [EIP712]([EIP-712](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-712.md)) and returns the calculated signature.
@@ -773,7 +860,7 @@ Invoked when a request for account listing has been made.
```
### ShowInfo
### ShowInfo / `ui_showInfo`
The UI should show the info to the user. Does not expect response.
@@ -783,7 +870,7 @@ The UI should show the info to the user. Does not expect response.
{
"jsonrpc":"2.0",
"id":9,
"method":"ShowInfo",
"method":"ui_showInfo",
"params":[
{
"text":"Tests completed"
@@ -793,7 +880,7 @@ The UI should show the info to the user. Does not expect response.
```
### ShowError
### ShowError / `ui_showError`
The UI should show the info to the user. Does not expect response.
@@ -812,7 +899,7 @@ The UI should show the info to the user. Does not expect response.
```
### OnApproved
### OnApprovedTx / `ui_onApprovedTx`
`OnApprovedTx` is called when a transaction has been approved and signed. The call contains the return value that will be sent to the external caller. The return value from this method is ignored - the reason for having this callback is to allow the ruleset to keep track of approved transactions.
@@ -820,7 +907,7 @@ When implementing rate-limited rules, this callback should be used.
TLDR; Use this method to keep track of signed transactions, instead of using the data in `ApproveTx`.
### OnSignerStartup
### OnSignerStartup / `ui_onSignerStartup`
This method provide the UI with information about what API version the signer uses (both internal and external) aswell as build-info and external api,
in k/v-form.
@@ -831,7 +918,7 @@ Example call:
{
"jsonrpc":"2.0",
"id":1,
"method":"OnSignerStartup",
"method":"ui_onSignerStartup",
"params":[
{
"info":{
@@ -862,7 +949,7 @@ A UI should conform to the following rules.
* A UI SHOULD inform the user about the `SHA256` or `MD5` hash of the binary being executed
* A UI SHOULD NOT maintain a secondary storage of data, e.g. list of accounts
* The signer provides accounts
* A UI SHOULD, to the best extent possible, use static linking / bundling, so that requried libraries are bundled
* A UI SHOULD, to the best extent possible, use static linking / bundling, so that required libraries are bundled
along with the UI.
@@ -875,3 +962,4 @@ There are a couple of implementation for a UI. We'll try to keep this list up to
These data types are defined in the channel between clef and the UI
### SignDataRequest
SignDataRequest contains information about a pending request to sign some data. The data to be signed can be of various types, defined by content-type. Clef has done most of the work in canonicalizing and making sense of the data, and it's up to the UI to presentthe user with the contents of the `message`
SignTxRequest contains information about a pending request to sign a transaction. Aside from the transaction itself, there is also a `call_info`-struct. That struct contains messages of various types, that the user should be informed of.
As in any request, it's important to consider that the `meta` info also contains untrusted data.
The `transaction` (on input into clef) can have either `data` or `input` -- if both are set, they must be identical, otherwise an error is generated. However, Clef will always use `data` when passing this struct on (if Clef does otherwise, please file a ticket)
"message":"Something looks odd, show this message as a warning"
},
{
"type":"Info",
"message":"User should see this aswell"
}
],
"meta":{
"remote":"localhost:9999",
"local":"localhost:8545",
"scheme":"http",
"User-Agent":"Firefox 3.2",
"Origin":"www.malicious.ru"
}
}
```
### SignTxResponse - approve
Response to request to sign a transaction. This response needs to contain the `transaction`, because the UI is free to make modifications to the transaction.
Response to SignTxRequest. When denying a request, there's no need to provide the transaction in return
Example:
```json
{
"transaction":{
"from":"0x",
"to":null,
"gas":"0x0",
"gasPrice":"0x0",
"value":"0x0",
"nonce":"0x0",
"data":null
},
"approved":false
}
```
### OnApproved - SignTransactionResult
SignTransactionResult is used in the call `clef` -> `OnApprovedTx(result)`
This occurs _after_ successful completion of the entire signing procedure, but right before the signed transaction is passed to the external caller. This method (and data) can be used by the UI to signal to the user that the transaction was signed, but it is primarily useful for ruleset implementations.
A ruleset that implements a rate limitation needs to know what transactions are sent out to the external interface. By hooking into this methods, the ruleset can maintain track of that count.
**OBS:** Note that if an attacker can restore your `clef` data to a previous point in time (e.g through a backup), the attacker can reset such windows, even if he/she is unable to decrypt the content.
The `OnApproved` method cannot be responded to, it's purely informative
Sent when clef needs the user to provide data. If 'password' is true, the input field should be treated accordingly (echo-free)
Example:
```json
{
"prompt":"The question to ask the user",
"title":"The title here",
"isPassword":true
}
```
### UserInputResponse
Response to UserInputRequest
Example:
```json
{
"text":"The textual response from user"
}
```
### ListRequest
Sent when a request has been made to list addresses. The UI is provided with the full `account`s, including local directory names. Note: this information is not passed back to the external caller, who only sees the `address`es.
Response to list request. The response contains a list of all addresses to show to the caller. Note: the UI is free to respond with any address the caller, regardless of whether it exists or not
*`New` was changed to deliver only an address, not the full `Account` data
*`Export` was moved from External API to the UI Server API
#### 5.0.0
* The external `account_EcRecover`-method was reimplemented.
* The external method `account_sign(address, data)` was replaced with `account_signData(contentType, address, data)`.
The addition of `contentType` makes it possible to use the method for different types of objects, such as:
* signing data with an intended validator (not yet implemented)
* signing clique headers,
* signing plain personal messages,
* The external method `account_signTypedData` implements [EIP-712](https://github.com/ethereum/EIPs/blob/master/EIPS/eip-712.md) and makes it possible to sign typed data.
#### 4.0.0
* The external `account_Ecrecover`-method was removed.
* The external `account_Import`-method was removed.
#### 3.0.0
* The external `account_List`-method was changed to not expose `url`, which contained info about the local filesystem. It now returns only a list of addresses.
#### 2.0.0
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.