We are proud to present the v1.16.0 release, introducing path-based archive node and several other improvements to the state database implementation. There is also the usual amount of bug fixes and small improvements.
Archive Node
This release includes a new, experimental implementation of the archive node (--gcmode=archive). As an archive node, Geth stores all historical states, and can thus retrieve account balances, nonces and storage values at old blocks.
In previous versions of Geth, running as an archive node was only possible using the legacy 'hash-based' state storage scheme (--state.scheme=hash). When syncing mainnet the resulting database would reach sizes larger than 20TB, and it all had to be stored on a high-speed SSD.
The new implementation provides the state archive in combination with 'path-based' storage (--state.scheme=path). Comparing to hash-based storage, there are three basic advantages:
Geth can be configured to store historical states on a dedicated disk, and it doesn't have to be an SSD (it will be ~3x faster to access when stored on SSD though).
For the full state history of mainnet, the state database will have a size of ~1.9TB (this is without the tx-by-hash index and log index).
The amount of historical state kept by the node is configurable as well. By default, Geth will keep historical state for the last 90k blocks, and older state is pruned from disk automatically as the chain advances. You can set the number of states to keep with the --history.state flag.
Notably, there is also one drawback with the new archive node: eth_getProof is not supported for historical blocks. With --state.scheme=path, Geth can only serve state proofs for the latest 128 blocks. We believe this is an acceptable tradeoff. If you need older proofs, you can still a hash-based archive node as before, but we'd also really like to hear about your use case. Please contact us via email at geth@ethereum.org so we can learn more, and possibly find a solution that can be implemented within Geth.
How it works
With 'path-based' state storage, Geth keeps exactly one full state in the database. Specifically, the persistent state is 128 blocks in the past. For newer blocks up to the head, forward diffs are kept in memory. In order to support rolling back to blocks older than head-128, Geth also keeps 'state history' in the form of reverse key-value diffs. When the chain is reset to an old block, these diffs are applied to the persistent state, going backwards one diff at a time until the target block is reached.
A reverse state diff contains the previous values of all modified accounts and storage locations for a specific block. There is a reverse diff for each historical block. This makes reverse diffs suitable for storage into the 'freezer', i.e. they do not need to live within Pebble/LevelDB.
The new archive mode is built on the realization that reverse state diffs contain all necessary data to serve historical state queries. For example, in order to get the historical balance of an account X at block B, we can search forward through diffs starting at block B until we find a block where account X is modified. This diff will contain the balance of the account, since it stores the all modified pre-values.
To accelerate the search for a suitable diff, Geth creates a database index storing the block numbers in which an account was modified. This index is small compared to the overall state history, but it is crucial for correct operation of the archive node. The state index is stored in PebbleDB and will be generated automatically while geth is syncing the chain. It takes ~30h to build the archive state index for mainnet, and historical state will only be available when the index is fully built. Geth will report a syncing status through eth_syncing while the indexing happens.
Other state implementation changes
In v1.16.0, the implementation of the 'state snapshot' has been rewritten for improved integration with path-based state storage. As you may recall, Geth stores the state in two different ways:
The 'snapshot' contains the flat keys and values of the state, in a layer-based struture.
The trie database contains state trie nodes, necessary for the state root computation.
When the implementation of the snapshot and its layer tree was introduced in Geth v1.10.0, it was built to be entirely separate from the trie database. We added path-based state storage in v1.13.0, and that system contains a layer tree as well, with slight differences in semantics and implementation.
As of this release, the layer trees of the state snapshot and trie database have been merged into a single implementation (#30661). In path-based mode, the trie database now holds a reference to the corresponding snapshot layer for each tree state. The new layer tree has also been enhanced with an additional index of layers in which an account was modified (#30971). Note this is similar in spirit to the historical state index. The new index will enable us to increase the number of in-memory state layers in the future.
The pathdb layer tree has been rewritten for tight integration with the state snapshot, and has an improved lookup structure that helps it scale to more than 128 layers. (#30661, #32086, #32097)
The state prefetcher has been improved to run for all inserted blocks, not just during full sync. (#31557, #32075, #32071)
Metrics for state cache utilization and the prefetcher have been added. (#31998)
We have also added metrics for gas usage and mgas/s (#31753, #32004)
The Pebble database is now synced to disk at explicit safepoints. In the beginning of the v1.15.x release cycle, we added a change that disabled fsync on the database level, then spent a bunch more time throughout the cycle triaging issues related to that. We eventually had to row back and enable fsync again in v1.15.8, causing a drop in performance. The new approach of syncing explicitly gives us the ability to disable fsync for most writes, improving performance. (#31703)
Pebble parameters have been tweaked to ensure better compaction behavior. Specifically, some changes were made to smooth out compaction-related CPU and I/O spikes. (#31970, #31988, #31782, #30464)
RPC
Logs returned by eth_getLogs and similar requests will now return the blockTimestamp. (#31887)
eth_getBlockReceipts will now return null instead of an error for blocks which are not found. (#31949)
The prestate tracer has seen some tweaks related to empty accounts. (#31855, #31427)
Geth now provides a built-in tracer for ERC-7562. (#31006)
Geth no longer provides clique-related RPC APIs. (#31875)
The ABI library has been changed to raise an error when trying to encode a negative integer as an unsigned ABI integer type. (#31790)
Networking
Geth now implements p2p protocol version eth/69. (#29158, #31952, #31843)
Peer discovery performance has been improved. (#31944)
Geth will now start syncing when a ForkchoiceUpdated call is received from the consensus layer. This is a defense-in-depth measure that would've helped a bit with the recent Hoodi chain fault recovery, since we do not have to wait for the CL client to fully sync before starting sync on the EL. (#31928)
Peer-to-peer connection error metrics have been improved. (#31652)
The blob transaction pool now limits transactions to at most 7 blobs. Note this change has no practical impact since there was an overall size limit applied to transactions. (#31837)
For a full rundown of the changes please consult the Geth 1.16.0 release milestone
As with all our previous releases, you can find the:
Pre-built binaries for all platforms on our downloads page.