diff --git a/CLAUDE.md b/CLAUDE.md index e358cac3a..c12762d50 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -284,6 +284,17 @@ actual_slot = finalized_slot + 1 + relative_index **XMSS (eXtended Merkle Signature Scheme):** - Post-quantum signature scheme +- Two schemes share the name. Lean consensus signs with leanVM's XMSS + (BLAKE2s, below). The Beacon Chain's PQ fork signs with leanXMSS over + Keccak-256 (`ethlambda_crypto::lean_xmss`, 32-byte keys, 2404-byte + signatures, a port of leanVM `4af2cddd`'s Rust implementation pinned by + consensus-specs' and leanVM's known answers), and proves many signatures at + once through `ethlambda_crypto::stark`: the specification's reference (the + signatures concatenated) by default, or real leanVM proofs from a + `leanvm-sidecar` process (`--pq-prover-socket`, leanVM `283686575c`, the + revision Lighthouse's PQ branch pins), which is what the PQ devnet runs. + The two encodings do not verify each other's proofs. Never pass one + scheme's key to the other - Wire sizes: `PUBLIC_KEY_SIZE` (`common/types/src/state.rs`) and `SIGNATURE_SIZE` (`common/types/src/attestation.rs`), static-asserted against leanVM's scheme constants in `common/crypto/src/signature.rs` @@ -633,7 +644,7 @@ existing once per chain. silent wrong answer. - **`ForkName::Lean` is deliberately absent from `ForkName::ALL`.** `ALL` is what `parse`, `previous` and `next` search, so its absence keeps - `parse("lean")` at `None` and `Gloas.next()` at `None`, meaning a fork upgrade + `parse("lean")` at `None` and `Pq.next()` at `None`, meaning a fork upgrade cannot walk off the end into lean. Lean is not a point on the Beacon Chain's fork timeline. `Lean` is declared *last* so the derived `Ord` puts it after every beacon fork, which is what `fork >= ForkName::X` gating reads. @@ -812,6 +823,36 @@ transitions are in `ethlambda-types`, per the section above. Nothing above Fork choice is fixture-verified too, including bellatrix's terminal-PoW validation, deneb's and fulu's data availability, and gloas's payload-aware suites. +- **EIP-8483 and PQ** (consensus-specs' `pq` branch feature forks, pinned at + `a4d16c5b`) are `ForkName::Eip8483` and `ForkName::Pq` after gloas, selectors + 8 and 9, implemented in the state transition and fork choice + (`stf/{eip8483,pq}.rs`, `stf/epoch/pq.rs`, `helpers/pq.rs`, `upgrade_to_*`) + and PQ's gossip rules (`gossip/pq.rs`, `p2p/src/beacon/pq.rs`: a PQ digest's + reshaped topics decode by the digest and are judged with XMSS signatures + and STARK proofs). **The node follows PQ** (`ForkName::is_followed`): PQ + blocks and envelopes import from gossip and req/resp, accepted PQ + aggregates and payload attestations reach fork choice, and envelopes of + every fork travel as the fork-tagged `containers::SignedExecutionPayloadEnvelope` + (storage decodes one by its slot's fork, req/resp by the chunk's context + bytes, the engine gets each fork's typed requests list). It was run next to + Lighthouse's PQ client on a PQ-at-genesis devnet (`tooling/pq-devnet/`), + with its own validators: the validator client derives each validator's + leanXMSS key and VRF tree from its BLS key (`ethlambda_crypto::pq_keys`) and + attests, aggregates, casts PTC votes and proposes, each signature at its + leaf. The beacon node builds PQ blocks through `POST + /eth/v1/validator/pq_blocks/{slot}` (`rpc/src/beacon/pq_proposal.rs`, + `pq_block_production.rs`), packing attestations from the validated + aggregates in `Store::pq_aggregate_pool` with their proofs merged on the + sidecar under a deadline (one unmerged attestation when it misses), and + proves its aggregators' committees ahead (`rpc/src/beacon/pq_aggregation.rs`, + `Store::pq_signature_pool`). EIP-8483 is not followed yet, and PQ blocks + carry no blobs. The + gloas rules run on all three gloas-family states through `GloasFamily(Mut)` + (`containers/gloas_family.rs`, returned by `helpers::gloas::gloas_state(_ref)`), + and the PQ registry holds each XMSS key in its 48-byte request form so the + registry accessors keep handing out the shared `Validator`. No release ships + their fixtures; see [`docs/beacon_pq.md`](docs/beacon_pq.md) for the design, + how to generate them, and what remains. - **Gloas** lives in `stf/gloas.rs`, `stf/epoch/gloas.rs`, `helpers/gloas.rs`, the gloas section of `fork_choice.rs`, and `containers/gloas.rs` in `ethlambda-types`. **The live follower follows gloas.** The chain actor's @@ -1061,10 +1102,13 @@ behavior. in-memory map stays as a write-through cache in front of the table. Pruned on finalization to the same horizon as `LiveChain` (the finalized block's own slot, keeping that block's own entry). -- `DB_VERSION` is 7: `States` and `StateDiffs` moved from root keys to - slot||root keys (the block's slot, looked up from `BlockHeaders` by readers) - so `Store::prune_finalized_states` can range-delete finalized states; a - directory written by an earlier version holds rows no lookup finds. (6 was +- `DB_VERSION` is 8: `Config` gained the EIP-8483 and PQ fork versions and + epochs, `UNREGISTERED_VALIDATOR_WITHDRAWABILITY_DELAY` and + `ATTESTATION_SUBNET_COUNT_PQ`, so a version 7 + directory's stored config decodes into the wrong fields. (7 moved `States` + and `StateDiffs` from root keys to slot||root keys, the block's slot looked + up from `BlockHeaders` by readers, so `Store::prune_finalized_states` can + range-delete finalized states; 6 was the gloas schedule, timing and churn keys in `Config`, which is SSZ-encoded under `KEY_CONFIG`; 5 was the unrealized-justifications table, since a directory written before it has no row there for any block it already diff --git a/Cargo.lock b/Cargo.lock index bbac135c4..aa60f93dc 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -1089,6 +1089,15 @@ dependencies = [ "zeroize", ] +[[package]] +name = "claims" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=283686575c4e81790591d2c6e3ed21c69c0e3bbe#283686575c4e81790591d2c6e3ed21c69c0e3bbe" +dependencies = [ + "lean_sphincs", + "lean_xmss", +] + [[package]] name = "clang-sys" version = "1.8.1" @@ -2054,13 +2063,18 @@ name = "ethlambda-crypto" version = "1.0.0" dependencies = [ "blst", + "claims", "ethlambda-metrics", "ethlambda-types", "hex", + "lean_xmss", "leanvm", "postcard", "rand 0.10.1", "rayon", + "sha2", + "sha3 0.10.9", + "sidecar", "thiserror 2.0.18", "tracing", "zk_alloc", @@ -2294,6 +2308,7 @@ dependencies = [ "axum", "blst", "ctr", + "ethlambda-crypto", "ethlambda-metrics", "ethlambda-types", "futures-util", @@ -3808,6 +3823,11 @@ dependencies = [ "sha3-asm", ] +[[package]] +name = "keccak256" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=283686575c4e81790591d2c6e3ed21c69c0e3bbe#283686575c4e81790591d2c6e3ed21c69c0e3bbe" + [[package]] name = "konst" version = "0.2.20" @@ -3879,6 +3899,14 @@ dependencies = [ "tracing", ] +[[package]] +name = "lean_sphincs" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=283686575c4e81790591d2c6e3ed21c69c0e3bbe#283686575c4e81790591d2c6e3ed21c69c0e3bbe" +dependencies = [ + "keccak256", +] + [[package]] name = "lean_vm" version = "0.1.0" @@ -3893,6 +3921,14 @@ dependencies = [ "zk_alloc", ] +[[package]] +name = "lean_xmss" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=283686575c4e81790591d2c6e3ed21c69c0e3bbe#283686575c4e81790591d2c6e3ed21c69c0e3bbe" +dependencies = [ + "keccak256", +] + [[package]] name = "leanvm" version = "0.1.0" @@ -6479,6 +6515,17 @@ version = "1.3.0" source = "registry+https://github.com/rust-lang/crates.io-index" checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64" +[[package]] +name = "sidecar" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=283686575c4e81790591d2c6e3ed21c69c0e3bbe#283686575c4e81790591d2c6e3ed21c69c0e3bbe" +dependencies = [ + "claims", + "lean_sphincs", + "lean_xmss", + "verifier", +] + [[package]] name = "signal-hook-registry" version = "1.4.8" @@ -7355,6 +7402,11 @@ dependencies = [ "rustversion", ] +[[package]] +name = "verifier" +version = "0.1.0" +source = "git+https://github.com/leanEthereum/leanVM.git?rev=283686575c4e81790591d2c6e3ed21c69c0e3bbe#283686575c4e81790591d2c6e3ed21c69c0e3bbe" + [[package]] name = "version_check" version = "0.9.5" diff --git a/Cargo.toml b/Cargo.toml index cb80e6e39..751f9681b 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -116,6 +116,24 @@ secp256k1 = { version = "0.30.0", default-features = false, features = ["global- # build remains portable to hardware lacking the extensions. sha2 = { version = "0.10.9", features = ["asm"] } +# Keccak-256 (Ethereum's, with Keccak's original padding), the hash of leanXMSS, +# the post-quantum signature scheme of the Beacon Chain's PQ fork +# (`ethlambda_crypto::lean_xmss`). Not leanVM's XMSS: that one hashes with +# BLAKE2s and signs lean consensus messages only. +sha3 = "0.10.9" + +# The client of leanVM's `leanvm-sidecar` (`crates/sidecar` without its +# `server` feature, which builds no prover) and its claim encoding: the PQ +# fork's attestation proofs as the PQ devnet makes them, real leanVM STARKs +# over leanXMSS signatures rather than the specification's concatenation +# (`ethlambda_crypto::stark`). Pinned to the revision Lighthouse's PQ branch +# and its `leanvm-sidecar` use: nothing on the wire detects a mismatch, a proof +# from another revision just fails to verify. Not the `leanvm` revision below, +# which lean consensus proves with. +leanvm_sidecar = { package = "sidecar", git = "https://github.com/leanEthereum/leanVM.git", rev = "283686575c4e81790591d2c6e3ed21c69c0e3bbe" } +leanvm_claims = { package = "claims", git = "https://github.com/leanEthereum/leanVM.git", rev = "283686575c4e81790591d2c6e3ed21c69c0e3bbe" } +leanvm_lean_xmss = { package = "lean_xmss", git = "https://github.com/leanEthereum/leanVM.git", rev = "283686575c4e81790591d2c6e3ed21c69c0e3bbe" } + # XMSS signatures + recursive aggregation, through leanVM's own facade crate: # it re-exports the aggregation API, the `xmss` module (keys, signing, and the # SSZ codec for the two wire types) and the `rand` that signing draws from, so diff --git a/bin/ethlambda/pq/ethlambda-pq b/bin/ethlambda/pq/ethlambda-pq new file mode 100755 index 000000000..ee5faaeab --- /dev/null +++ b/bin/ethlambda/pq/ethlambda-pq @@ -0,0 +1,64 @@ +#!/bin/bash +# Runs `ethlambda` with a leanVM prover sidecar beside it, for a network that +# schedules the PQ fork: `ethlambda-pq beacon ` starts `leanvm-sidecar` +# on a private socket, waits for it to listen, then starts +# `ethlambda --pq-prover-socket `. Either exiting stops the +# other. The sidecar must be built from the leanVM revision ethlambda pins +# (`ethlambda_crypto::stark::sidecar::LEANVM_REV`). `LEANVM_THREADS` caps its +# threads (0, the default, is one per core). +set -eu + +socket_dir=$(mktemp -d "${TMPDIR:-/tmp}/leanvm.XXXXXXXX") +socket_path="$socket_dir/prover.sock" +prover_pid= +node_pid= + +cleanup() { + if [[ -n "$node_pid" ]]; then + kill "$node_pid" 2>/dev/null || true + wait "$node_pid" 2>/dev/null || true + fi + if [[ -n "$prover_pid" ]]; then + kill "$prover_pid" 2>/dev/null || true + wait "$prover_pid" 2>/dev/null || true + fi + rm -f -- "$socket_path" "$socket_path.lock" + rmdir -- "$socket_dir" 2>/dev/null || true +} + +trap cleanup EXIT +trap 'exit 143' TERM +trap 'exit 130' INT + +leanvm-sidecar --socket "$socket_path" --threads "${LEANVM_THREADS:-0}" & +prover_pid=$! +ready_deadline=$((SECONDS + 120)) +until [[ -S "$socket_path" ]]; do + if ! kill -0 "$prover_pid" 2>/dev/null; then + echo 'leanvm-sidecar exited before creating its socket' >&2 + exit 1 + fi + if (( SECONDS >= ready_deadline )); then + echo 'leanvm-sidecar did not create its socket within 120 seconds' >&2 + exit 1 + fi + sleep 0.1 +done + +ethlambda "$@" --pq-prover-socket "$socket_path" & +node_pid=$! + +while kill -0 "$prover_pid" 2>/dev/null && kill -0 "$node_pid" 2>/dev/null; do + sleep 0.2 +done + +if kill -0 "$prover_pid" 2>/dev/null; then + status=0 + wait "$node_pid" || status=$? + node_pid= + exit "$status" +fi + +prover_pid= +echo 'leanvm-sidecar exited while ethlambda was running' >&2 +exit 1 diff --git a/bin/ethlambda/src/beacon.rs b/bin/ethlambda/src/beacon.rs index 6cc600924..1ba5da13f 100644 --- a/bin/ethlambda/src/beacon.rs +++ b/bin/ethlambda/src/beacon.rs @@ -177,8 +177,8 @@ pub fn wire_params( // // A fork the node does not follow keeps its own warning: crossing its // digest works like any other, but the node stops tracking the chain - // there regardless. No fork is in that state today, since gloas is - // followed, so the warning is the guard for the next one. + // there regardless. EIP-8483 and PQ are in that state, though no built-in + // network schedules either. let schedule = ForkSchedule::new(&chain, genesis.genesis_validators_root); match schedule.next_boundary_after(epoch) { Some(next) if !next.fork.is_followed() => warn!( @@ -227,6 +227,11 @@ pub fn wire_params( let attestation_subnets = ethlambda_p2p::beacon::subnets::compute_subscribed_subnets(node_id, epoch, &chain) .map_err(|err| eyre::eyre!("computing the attestation subnet backbone: {err:?}"))?; + // A PQ digest's backbone, the same function over `ATTESTATION_SUBNET_COUNT_PQ`: + // held only while a PQ digest is. + let attestation_subnets_pq = + ethlambda_p2p::beacon::subnets::compute_subscribed_subnets_pq(node_id, epoch, &chain) + .map_err(|err| eyre::eyre!("computing the PQ attestation subnet backbone: {err:?}"))?; info!( subnets_per_node = chain.subnets_per_node, subnets = ?attestation_subnets, @@ -250,6 +255,7 @@ pub fn wire_params( genesis_validators_root: genesis.genesis_validators_root, custody_columns, attestation_subnets, + attestation_subnets_pq, }, fork_id, }) diff --git a/bin/ethlambda/src/checkpoint_sync.rs b/bin/ethlambda/src/checkpoint_sync.rs index 8c4467b21..21c3a4f96 100644 --- a/bin/ethlambda/src/checkpoint_sync.rs +++ b/bin/ethlambda/src/checkpoint_sync.rs @@ -25,7 +25,10 @@ use std::time::Duration; use ethlambda_p2p::beacon::decode; use ethlambda_types::beacon::config::Config; -use ethlambda_types::beacon::containers::{BeaconState, SignedBeaconBlock, gloas}; +use ethlambda_types::beacon::containers::{ + BeaconState, SignedBeaconBlock, SignedExecutionPayloadEnvelope, +}; +use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::preset; use ethlambda_types::block::SignedBlock; use ethlambda_types::genesis::{GenesisMismatch, verify_state_genesis}; @@ -717,8 +720,8 @@ enum EnvelopeFetchError { Unknown, #[error("request failed: {0}")] Request(#[from] reqwest::Error), - #[error("SSZ deserialization failed: {0:?}")] - Decode(DecodeError), + #[error("SSZ deserialization failed: {0}")] + Decode(String), #[error("the envelope is for another block")] WrongBlock, } @@ -729,7 +732,8 @@ async fn fetch_envelope( client: &Client, base_url: &str, anchor_root: H256, -) -> Result { + fork: ForkName, +) -> Result { let url = format!("{base_url}{}", beacon_envelope_path(anchor_root)); let response = client .get(&url) @@ -740,9 +744,9 @@ async fn fetch_envelope( return Err(EnvelopeFetchError::Unknown); } let bytes = response.error_for_status()?.bytes().await?; - let envelope = gloas::SignedExecutionPayloadEnvelope::from_ssz_bytes(&bytes) - .map_err(EnvelopeFetchError::Decode)?; - if envelope.message.beacon_block_root != anchor_root { + let envelope = SignedExecutionPayloadEnvelope::from_ssz(fork, &bytes) + .map_err(|err| EnvelopeFetchError::Decode(format!("{err:?}")))?; + if envelope.beacon_block_root() != anchor_root { return Err(EnvelopeFetchError::WrongBlock); } Ok(envelope) @@ -760,13 +764,14 @@ async fn fetch_envelope( pub async fn fetch_anchor_envelope( urls: &[String], anchor_root: H256, -) -> Option { + fork: ForkName, +) -> Option { let client = build_client() .inspect_err(|err| info!(%err, "Skipping the anchor envelope fetch")) .ok()?; for url in urls { let base_url = trim_trailing_slash(url); - match fetch_envelope(&client, base_url, anchor_root).await { + match fetch_envelope(&client, base_url, anchor_root, fork).await { Ok(envelope) => { info!(%url, "Fetched the anchor's execution payload envelope"); return Some(envelope); @@ -1406,7 +1411,10 @@ mod anchor_envelope_tests { (url, rx) } - fn envelope_for(root: H256) -> gloas::SignedExecutionPayloadEnvelope { + fn envelope_for( + root: H256, + ) -> ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope { + use ethlambda_types::beacon::containers::gloas; let payload = gloas::ExecutionPayload { parent_hash: Default::default(), fee_recipient: Default::default(), @@ -1448,9 +1456,9 @@ mod anchor_envelope_tests { let envelope = envelope_for(root); let (url, mut requests) = serve("200 OK", envelope.to_ssz()).await; - let fetched = fetch_anchor_envelope(&[url], root).await; + let fetched = fetch_anchor_envelope(&[url], root, ForkName::Gloas).await; - assert_eq!(fetched, Some(envelope)); + assert_eq!(fetched, Some(envelope.into())); let line = requests.recv().await.unwrap(); assert!( line.starts_with(&format!( @@ -1468,13 +1476,13 @@ mod anchor_envelope_tests { let (knows, _) = serve("200 OK", envelope.to_ssz()).await; assert_eq!( - fetch_anchor_envelope(std::slice::from_ref(&unknown), root).await, + fetch_anchor_envelope(std::slice::from_ref(&unknown), root, ForkName::Gloas).await, None, "a 404 is not an envelope" ); assert_eq!( - fetch_anchor_envelope(&[unknown, knows], root).await, - Some(envelope) + fetch_anchor_envelope(&[unknown, knows], root, ForkName::Gloas).await, + Some(envelope.into()) ); } @@ -1485,10 +1493,16 @@ mod anchor_envelope_tests { let (garbage, _) = serve("200 OK", vec![1, 2, 3]).await; let (other, _) = serve("200 OK", envelope_for(H256::from([8u8; 32])).to_ssz()).await; - assert_eq!(fetch_anchor_envelope(&[broken], root).await, None); - assert_eq!(fetch_anchor_envelope(&[garbage], root).await, None); assert_eq!( - fetch_anchor_envelope(&[other], root).await, + fetch_anchor_envelope(&[broken], root, ForkName::Gloas).await, + None + ); + assert_eq!( + fetch_anchor_envelope(&[garbage], root, ForkName::Gloas).await, + None + ); + assert_eq!( + fetch_anchor_envelope(&[other], root, ForkName::Gloas).await, None, "an envelope for another block must not be handed over" ); diff --git a/bin/ethlambda/src/cli.rs b/bin/ethlambda/src/cli.rs index 43b186b34..3887d7055 100644 --- a/bin/ethlambda/src/cli.rs +++ b/bin/ethlambda/src/cli.rs @@ -391,6 +391,17 @@ pub(crate) struct MainnetOptions { #[arg(long, default_value = crate::network::DEFAULT_NETWORK)] pub(crate) network: String, + /// The Unix socket of a `leanvm-sidecar` (built from leanVM + /// `283686575c4e81790591d2c6e3ed21c69c0e3bbe`), which makes and checks + /// the PQ fork's attestation proofs as leanVM STARKs, the encoding the PQ + /// devnet's clients use. + /// + /// Without it the node uses the specification's reference encoding (a + /// proof is the signatures concatenated), which only another node on the + /// reference accepts. Only a network scheduling the PQ fork reads either. + #[arg(long)] + pub(crate) pq_prover_socket: Option, + /// Base URL of a builder API endpoint, normally MEV-Boost (for example /// `http://127.0.0.1:18550`) or a relay directly. Without it every block /// is built locally, and `register_validator` answers 500 so a validator diff --git a/bin/ethlambda/src/main.rs b/bin/ethlambda/src/main.rs index 7be4bd243..8ab878ca8 100644 --- a/bin/ethlambda/src/main.rs +++ b/bin/ethlambda/src/main.rs @@ -62,7 +62,8 @@ use ethlambda_types::{ aggregator::AggregatorController, beacon::config::Config, beacon::containers::{ - BeaconState, SignedBeaconBlock, altair, bellatrix, capella, deneb, electra, gloas, phase0, + BeaconState, SignedBeaconBlock, altair, bellatrix, capella, deneb, eip8483, electra, gloas, + phase0, pq, }, beacon::fork::ForkName, beacon::primitives::Epoch, @@ -385,6 +386,17 @@ async fn run_node(options: Options) -> eyre::Result<()> { ); ethlambda_crypto::init_leanvm(lean.prover_arena); } + // The PQ fork's attestation proofs: leanVM proofs from a sidecar when one + // is named, the specification's reference encoding otherwise. Before + // anything can verify a proof, since the first one used stays installed. + if let Network::Mainnet { mainnet, .. } = &network + && let Some(socket) = &mainnet.pq_prover_socket + { + ethlambda_crypto::stark::install_sidecar( + ethlambda_crypto::stark::sidecar::SidecarConfig::new(socket.clone()), + ) + .wrap_err("cannot use the leanVM prover sidecar")?; + } // Initialize metrics ethlambda_blockchain::metrics::init(); @@ -959,7 +971,7 @@ async fn run_node(options: Options) -> eyre::Result<()> { let _ = served.inspect_err(|err| error!(%err, "RPC server failed")); }); - let mut anchor_envelope_fetch: Option<(Vec, H256)> = None; + let mut anchor_envelope_fetch: Option<(Vec, H256, ForkName)> = None; let blockchain = match setup.chain { ChainActor::Lean(validator_keys, config) => { BlockChain::spawn(setup.store, validator_keys, config, events) @@ -973,7 +985,7 @@ async fn run_node(options: Options) -> eyre::Result<()> { check_custody_set(&custody_columns)?; anchor_envelope_fetch = missing_anchor_envelope_root(&setup.store) .filter(|_| !anchor_envelope_urls.is_empty()) - .map(|root| (anchor_envelope_urls, root)); + .map(|(root, fork)| (anchor_envelope_urls, root, fork)); BlockChain::spawn_beacon( setup.store, sync_status, @@ -1009,10 +1021,11 @@ async fn run_node(options: Options) -> eyre::Result<()> { // delay the node. The envelope goes through the same chain-actor entry a // gossiped one does, so the bid check, the column check and the // verification all apply to it. - if let Some((urls, anchor_root)) = anchor_envelope_fetch { + if let Some((urls, anchor_root, anchor_fork)) = anchor_envelope_fetch { let chain = blockchain.actor_ref().to_p2p_to_block_chain_ref(); tokio::spawn(async move { - if let Some(envelope) = checkpoint_sync::fetch_anchor_envelope(&urls, anchor_root).await + if let Some(envelope) = + checkpoint_sync::fetch_anchor_envelope(&urls, anchor_root, anchor_fork).await { let _ = chain .new_execution_payload_envelope(Box::new(envelope), BlockArrival::now()) @@ -1092,7 +1105,7 @@ fn hostname() -> String { /// gloas block has the block and state but not the payload its children may /// build on; asking the checkpoint source for it saves waiting for a FULL /// child to make the by-root request fetch it. -fn missing_anchor_envelope_root(store: &Store) -> Option { +fn missing_anchor_envelope_root(store: &Store) -> Option<(H256, ForkName)> { let root = store.latest_finalized().ok()?.root; let block = store.get_signed_block(&root).ok()??; // Gloas or any later fork (lean sorts last, and is not a beacon fork). @@ -1101,7 +1114,7 @@ fn missing_anchor_envelope_root(store: &Store) -> Option { return None; } let held = store.get_execution_payload_envelope(&root).ok()?.is_some(); - (!held).then_some(root) + (!held).then_some((root, fork)) } /// Reject a beacon node with no custody set before it spawns. @@ -1872,6 +1885,10 @@ fn first_config_difference( fulu_fork_epoch: _, gloas_fork_version: _, gloas_fork_epoch: _, + eip8483_fork_version: _, + eip8483_fork_epoch: _, + pq_fork_version: _, + pq_fork_epoch: _, // Time parameters: `seconds_per_slot`/`slot_duration_ms` (compared // below) move every slot boundary. The rest are operator-visible @@ -1900,6 +1917,9 @@ fn first_config_difference( // Compared below, alongside `min_validator_withdrawability_delay`: // the builder-registry counterpart, also a state-transition rule. min_builder_withdrawability_delay: _, + // Compared below: when an unregistered validator becomes withdrawable + // at the PQ fork is a state-transition rule. + unregistered_validator_withdrawability_delay: _, // Validator cycle: compared below. Churn and inactivity-leak // parameters change which exits, activations and inactivity scores a @@ -1947,6 +1967,7 @@ fn first_config_difference( attestation_propagation_slot_range: _, attestation_subnet_count: _, attestation_subnet_extra_bits: _, + attestation_subnet_count_pq: _, blob_sidecar_subnet_count: _, blob_sidecar_subnet_count_electra: _, data_column_sidecar_subnet_count: _, @@ -2052,6 +2073,7 @@ fn first_config_difference( slot_duration_ms, min_validator_withdrawability_delay, min_builder_withdrawability_delay, + unregistered_validator_withdrawability_delay, shard_committee_period, inactivity_score_bias, inactivity_score_recovery_rate, @@ -2344,7 +2366,39 @@ async fn fetch_initial_beacon_state( /// rebuilding it here and letting `get_forkchoice_store` check the header /// hash is what proves this reconstruction matches what the state actually /// describes, rather than assuming it. +/// +/// From gloas on the genesis body is not quite empty: the specification's +/// genesis (`create_genesis_state` and `get_genesis_block` in its test +/// helpers, which ethpandaops' generator follows) puts the state's +/// `latest_execution_payload_bid` in the body's bid, so the first block's +/// `parent_block_hash` check has a genesis payload to build on. fn genesis_anchor_block(state: &BeaconState) -> SignedBeaconBlock { + let with_bid = genesis_block_with(state, true); + // A generator built against an older gloas release commits to the fully + // empty body instead. Whichever body the header names is the genesis + // block; with neither, `get_forkchoice_store` refuses the bid form. + if state.fork_name() >= ForkName::Gloas + && with_bid.body_root() != state.latest_block_header().body_root + { + let empty = genesis_block_with(state, false); + if empty.body_root() == state.latest_block_header().body_root { + return empty; + } + } + with_bid +} + +/// [`genesis_anchor_block`]'s block, its body's bid the state's own when +/// `bid` (the gloas-family genesis the specification builds) and empty +/// otherwise. Pre-gloas forks have no bid, so `bid` changes nothing for them. +fn genesis_block_with(state: &BeaconState, bid: bool) -> SignedBeaconBlock { + let body_bid = |state: &BeaconState| { + if bid { + genesis_bid(state) + } else { + Default::default() + } + }; let header = state.latest_block_header(); let slot = header.slot; let proposer_index = header.proposer_index; @@ -2431,10 +2485,48 @@ fn genesis_anchor_block(state: &BeaconState) -> SignedBeaconBlock { proposer_index, parent_root, state_root, - body: gloas::BeaconBlockBody::empty(), + body: gloas::BeaconBlockBody { + signed_execution_payload_bid: gloas::SignedExecutionPayloadBid { + message: body_bid(state), + ..Default::default() + }, + ..gloas::BeaconBlockBody::empty() + }, }, signature: Default::default(), }), + ForkName::Eip8483 => SignedBeaconBlock::Eip8483(eip8483::SignedBeaconBlock { + message: eip8483::BeaconBlock { + slot, + proposer_index, + parent_root, + state_root, + body: eip8483::BeaconBlockBody { + signed_execution_payload_bid: gloas::SignedExecutionPayloadBid { + message: body_bid(state), + ..Default::default() + }, + ..eip8483::BeaconBlockBody::empty() + }, + }, + signature: Default::default(), + }), + ForkName::Pq => SignedBeaconBlock::Pq(Box::new(pq::SignedBeaconBlock { + message: pq::BeaconBlock { + slot, + proposer_index, + parent_root, + state_root, + body: pq::BeaconBlockBody { + signed_execution_payload_bid: pq::SignedExecutionPayloadBid { + message: body_bid(state), + ..Default::default() + }, + ..pq::BeaconBlockBody::empty() + }, + }, + signature: Default::default(), + })), // Never reached: this is only called on a network's own genesis // state, and every `NetworkSource` decodes a beacon fork there // (`NetworkDir::load` resolves the fork from `Config::fork_at_epoch`, @@ -2445,6 +2537,14 @@ fn genesis_anchor_block(state: &BeaconState) -> SignedBeaconBlock { } } +/// The bid a gloas-family genesis block carries: the state's own. +fn genesis_bid(state: &BeaconState) -> gloas::ExecutionPayloadBid { + state + .gloas_family() + .map(|inner| inner.latest_execution_payload_bid.clone()) + .unwrap_or_default() +} + /// Reject an `--aggregate-subnet-ids` value that names no subnet. /// /// The flag feeds two consumers that read an out-of-range value differently: @@ -2698,14 +2798,15 @@ mod tests { assert!(validate_aggregate_subnet_ids(Some(&[]), 4).is_ok()); } - /// Gloas is followed: the envelopes and payload attestations a gloas chain - /// needs are delivered, so no fork is refused as an anchor. + /// Every beacon fork is followed but EIP-8483, so only its anchor is + /// refused for its fork. #[test] - fn startup_accepts_an_anchor_at_every_beacon_fork() { + fn startup_accepts_an_anchor_at_every_followed_fork() { for fork in ForkName::ALL { - assert!( + assert_eq!( refuse_unfollowable_fork(fork).is_ok(), - "{fork} must be followable" + fork != ForkName::Eip8483, + "{fork}" ); } } @@ -3266,10 +3367,17 @@ validators: state = upgrade_state(&state, fork, &config).unwrap(); } assert_eq!(state.fork_name(), ForkName::Gloas); - // A real gloas genesis commits to gloas's empty body; the upgrade - // chain above leaves phase0's in the header. - state.latest_block_header_mut().body_root = - gloas::BeaconBlockBody::empty().hash_tree_root(); + // A real gloas genesis commits to the specification's genesis body + // (empty but for the state's own bid); the upgrade chain above leaves + // phase0's in the header. + state.latest_block_header_mut().body_root = gloas::BeaconBlockBody { + signed_execution_payload_bid: gloas::SignedExecutionPayloadBid { + message: genesis_bid(&state), + ..Default::default() + }, + ..gloas::BeaconBlockBody::empty() + } + .hash_tree_root(); let dir = tempfile::tempdir().unwrap(); std::fs::write(dir.path().join("config.yaml"), config_text).unwrap(); @@ -3569,6 +3677,34 @@ validators: /// Chains all the way through gloas: `upgrade_state` handles every fork /// from altair to gloas, so this loop exercises `genesis_anchor_block`'s /// gloas arm the same way it does every earlier fork's. + /// A gloas genesis whose header names the fully empty body (a generator + /// built against an older gloas release) still anchors: the empty body + /// is tried when the bid form does not match. + #[test] + fn a_gloas_genesis_with_an_empty_body_still_anchors() { + let mut config = Config::mainnet(); + let mut state = beacon::mainnet_genesis_state().unwrap(); + for fork in ForkName::ALL + .into_iter() + .skip(1) + .take_while(|f| *f <= ForkName::Gloas) + { + config = config.with_fork_epoch(fork, 0); + state = + ethlambda_state_transition::beacon::upgrade::upgrade_state(&state, fork, &config) + .unwrap(); + } + // An upgraded state's bid is not the default one, so the two bodies differ. + assert_ne!(genesis_bid(&state), Default::default()); + state.latest_block_header_mut().body_root = + gloas::BeaconBlockBody::empty().hash_tree_root(); + let block = genesis_anchor_block(&state); + assert_eq!( + block.body_root(), + gloas::BeaconBlockBody::empty().hash_tree_root() + ); + } + #[test] fn a_genesis_anchor_block_hashes_to_the_states_own_header_at_every_fork() { let config = Config::mainnet(); @@ -3591,7 +3727,32 @@ validators: ForkName::Electra | ForkName::Fulu => { electra::BeaconBlockBody::empty().hash_tree_root() } - ForkName::Gloas => gloas::BeaconBlockBody::empty().hash_tree_root(), + // From gloas on, the specification's genesis body carries the + // state's own bid (`get_genesis_block` in its test helpers). + ForkName::Gloas => gloas::BeaconBlockBody { + signed_execution_payload_bid: gloas::SignedExecutionPayloadBid { + message: genesis_bid(&state), + ..Default::default() + }, + ..gloas::BeaconBlockBody::empty() + } + .hash_tree_root(), + ForkName::Eip8483 => eip8483::BeaconBlockBody { + signed_execution_payload_bid: gloas::SignedExecutionPayloadBid { + message: genesis_bid(&state), + ..Default::default() + }, + ..eip8483::BeaconBlockBody::empty() + } + .hash_tree_root(), + ForkName::Pq => pq::BeaconBlockBody { + signed_execution_payload_bid: pq::SignedExecutionPayloadBid { + message: genesis_bid(&state), + ..Default::default() + }, + ..pq::BeaconBlockBody::empty() + } + .hash_tree_root(), ForkName::Lean => unreachable!("ForkName::ALL excludes Lean"), }; state.latest_block_header_mut().body_root = empty_body_root; @@ -3874,7 +4035,10 @@ mod anchor_envelope_root_tests { #[test] fn a_gloas_anchor_without_its_envelope_is_asked_about() { let (store, root, _) = anchored_at(gloas_block(64)); - assert_eq!(missing_anchor_envelope_root(&store), Some(root)); + assert_eq!( + missing_anchor_envelope_root(&store), + Some((root, ForkName::Gloas)) + ); } #[test] diff --git a/bin/ethlambda/src/network/mod.rs b/bin/ethlambda/src/network/mod.rs index 8ff79eaf9..8bad13d88 100644 --- a/bin/ethlambda/src/network/mod.rs +++ b/bin/ethlambda/src/network/mod.rs @@ -188,6 +188,7 @@ pub(crate) fn check_constants(config: &Config) -> Result<(), ConstantsMismatch> } check!( attestation_subnet_count == p2p::ATTESTATION_SUBNET_COUNT, + attestation_subnet_count_pq == p2p::ATTESTATION_SUBNET_COUNT_PQ, data_column_sidecar_subnet_count == constants::DATA_COLUMN_SIDECAR_SUBNET_COUNT, number_of_custody_groups == constants::NUMBER_OF_CUSTODY_GROUPS, custody_requirement == constants::CUSTODY_REQUIREMENT, diff --git a/crates/blockchain/src/beacon_aggregates.rs b/crates/blockchain/src/beacon_aggregates.rs index a4d65002e..cdcd2d2ff 100644 --- a/crates/blockchain/src/beacon_aggregates.rs +++ b/crates/blockchain/src/beacon_aggregates.rs @@ -36,7 +36,7 @@ use std::collections::{HashMap, VecDeque}; use ethlambda_state_transition::beacon::aggregate::{ MAX_AGGREGATES_PER_SLOT, is_non_strict_superset, }; -use ethlambda_types::beacon::containers::SignedAggregateAndProof; +use ethlambda_types::beacon::containers::{AttestationData, SignedAggregateAndProof}; use ethlambda_types::beacon::primitives::{ CommitteeIndex, Epoch, HashTreeRoot as _, Root, Slot, ValidatorIndex, }; @@ -100,9 +100,45 @@ pub(crate) struct AggregateGossip { seen_bits: HashMap>>, /// Aggregates waiting for their own slot to pass. deferred: VecDeque, + /// PQ aggregates waiting for their own slot to pass, as the vote and the + /// attesting indices gossip validation resolved. Kept apart from + /// [`Self::deferred`] because the coverage gate reads a BLS-fork + /// aggregate; p2p's seen cache already drops a PQ aggregate whose bits + /// add nothing. + pq_deferred: VecDeque<(AttestationData, Vec)>, } impl AggregateGossip { + /// Holds a PQ aggregate until its slot has passed. The oldest is dropped + /// once the queue holds [`MAX_DEFERRED`]. + pub(crate) fn defer_pq( + &mut self, + data: AttestationData, + attesting_indices: Vec, + ) { + if self.pq_deferred.len() >= MAX_DEFERRED { + self.pq_deferred.pop_front(); + } + self.pq_deferred.push_back((data, attesting_indices)); + } + + /// The PQ aggregates whose slot is before `current_slot`, removed from the + /// queue. + pub(crate) fn take_ready_pq( + &mut self, + current_slot: Slot, + ) -> Vec<(AttestationData, Vec)> { + let mut ready = Vec::new(); + self.pq_deferred.retain(|entry| { + let is_ready = current_slot >= entry.0.slot.saturating_add(1); + if is_ready { + ready.push(entry.clone()); + } + !is_ready + }); + ready + } + /// Whether this aggregate adds nothing that has not already been applied. /// /// Read before [`fork_choice::apply_verified_aggregate`] and written only diff --git a/crates/blockchain/src/beacon_engine.rs b/crates/blockchain/src/beacon_engine.rs index 9707eb5f2..f33579025 100644 --- a/crates/blockchain/src/beacon_engine.rs +++ b/crates/blockchain/src/beacon_engine.rs @@ -25,11 +25,10 @@ use ethlambda_engine::types::PayloadStatusValue; use ethlambda_engine::{EngineClient, EngineError, PayloadStatusV1}; use ethlambda_state_transition::beacon::containers::{self, SignedBeaconBlock}; use ethlambda_state_transition::beacon::fork_choice::PayloadValidity; -use ethlambda_state_transition::beacon::primitives::HashTreeRoot as _; use ethlambda_state_transition::beacon::primitives::{Bytes32, Root}; use ethlambda_state_transition::beacon::stf::deneb::kzg_commitment_to_versioned_hash; use ethlambda_state_transition::beacon::stf::electra::get_execution_requests_list; -use ethlambda_state_transition::beacon::stf::gloas; +use ethlambda_state_transition::beacon::stf::{eip8483, gloas, pq}; /// Everything `engine_newPayloadV4` takes, derived from one block. pub struct NewPayloadRequest<'a> { @@ -70,11 +69,15 @@ pub struct GloasNewPayloadRequest<'a> { /// `engine_newPayloadV4`, and no fork after gloas exists yet to need a V6. pub fn gloas_new_payload_request<'a>( block: &SignedBeaconBlock, - envelope: &'a containers::gloas::SignedExecutionPayloadEnvelope, + envelope: &'a containers::SignedExecutionPayloadEnvelope, ) -> Result, EngineError> { let mismatch = |reason: &str| Err(EngineError::EnvelopeMismatch(reason.to_string())); - let inner = match block { - SignedBeaconBlock::Gloas(inner) => inner, + let bid = match block { + SignedBeaconBlock::Gloas(inner) => &inner.message.body.signed_execution_payload_bid.message, + SignedBeaconBlock::Eip8483(inner) => { + &inner.message.body.signed_execution_payload_bid.message + } + SignedBeaconBlock::Pq(inner) => &inner.message.body.signed_execution_payload_bid.message, SignedBeaconBlock::Phase0(_) | SignedBeaconBlock::Altair(_) | SignedBeaconBlock::Bellatrix(_) @@ -82,14 +85,16 @@ pub fn gloas_new_payload_request<'a>( | SignedBeaconBlock::Deneb(_) | SignedBeaconBlock::Electra(_) | SignedBeaconBlock::Fulu(_) - | SignedBeaconBlock::Lean(_) => return mismatch("the block is not a gloas block"), + | SignedBeaconBlock::Lean(_) => return mismatch("the block has no payload envelope"), }; + if envelope.fork() != block.fork_name() { + return mismatch("the envelope is not of the block's fork"); + } - if envelope.message.beacon_block_root != inner.message.hash_tree_root() { + if envelope.beacon_block_root() != block.message_hash_tree_root() { return mismatch("the envelope's beacon_block_root is not the block's root"); } - let bid = &inner.message.body.signed_execution_payload_bid.message; - if bid.block_hash != envelope.message.payload.block_hash { + if bid.block_hash != envelope.payload().block_hash { return mismatch("the bid's block_hash is not the payload's block_hash"); } @@ -99,13 +104,25 @@ pub fn gloas_new_payload_request<'a>( .map(kzg_commitment_to_versioned_hash) .collect(); + // Each fork's own requests list, with its own request types: the engine + // hashes whatever typed requests it is handed. + let execution_requests = match envelope { + containers::SignedExecutionPayloadEnvelope::Gloas(inner) => { + gloas::get_execution_requests_list(&inner.message.execution_requests) + } + containers::SignedExecutionPayloadEnvelope::Eip8483(inner) => { + eip8483::get_execution_requests_list(&inner.message.execution_requests) + } + containers::SignedExecutionPayloadEnvelope::Pq(inner) => { + pq::get_execution_requests_list(&inner.message.execution_requests) + } + }; + Ok(GloasNewPayloadRequest { - execution_payload: &envelope.message.payload, + execution_payload: envelope.payload(), versioned_hashes, - parent_beacon_block_root: envelope.message.parent_beacon_block_root, - execution_requests: gloas::get_execution_requests_list( - &envelope.message.execution_requests, - ), + parent_beacon_block_root: envelope.parent_beacon_block_root(), + execution_requests, }) } @@ -134,6 +151,8 @@ pub fn new_payload_request(block: &SignedBeaconBlock) -> Option return None, }; @@ -295,7 +314,7 @@ pub async fn ask( pub async fn ask_envelope( client: &EngineClient, block: &SignedBeaconBlock, - envelope: &containers::gloas::SignedExecutionPayloadEnvelope, + envelope: &containers::SignedExecutionPayloadEnvelope, ) -> Result { ask_envelope_status(client, block, envelope) .await @@ -309,7 +328,7 @@ pub async fn ask_envelope( pub async fn ask_envelope_status( client: &EngineClient, block: &SignedBeaconBlock, - envelope: &containers::gloas::SignedExecutionPayloadEnvelope, + envelope: &containers::SignedExecutionPayloadEnvelope, ) -> Result { let request = gloas_new_payload_request(block, envelope)?; let call = client.new_payload_v5( @@ -329,6 +348,7 @@ mod tests { }; use super::*; + use ethlambda_state_transition::beacon::primitives::HashTreeRoot as _; /// An otherwise-zero electra signed block with `parent_root` set. /// @@ -510,6 +530,7 @@ mod tests { .builder_exits .push(Default::default()); + let envelope = envelope.into(); let request = gloas_new_payload_request(&block, &envelope) .unwrap_or_else(|err| panic!("a matching pair has a request: {err}")); @@ -534,7 +555,7 @@ mod tests { let envelope = gloas_envelope(Root::ZERO, Root::ZERO); assert!(matches!( - gloas_new_payload_request(&block, &envelope), + gloas_new_payload_request(&block, &envelope.into()), Err(EngineError::EnvelopeMismatch(_)) )); } @@ -545,7 +566,7 @@ mod tests { let envelope = gloas_envelope(Root::repeat_byte(0xee), Root::ZERO); assert!(matches!( - gloas_new_payload_request(&block, &envelope), + gloas_new_payload_request(&block, &envelope.into()), Err(EngineError::EnvelopeMismatch(_)) )); } @@ -559,7 +580,7 @@ mod tests { envelope.message.payload.block_hash = ExecutionBlockHash::repeat_byte(0xab); assert!(matches!( - gloas_new_payload_request(&block, &envelope), + gloas_new_payload_request(&block, &envelope.into()), Err(EngineError::EnvelopeMismatch(_)) )); } diff --git a/crates/blockchain/src/beacon_envelope.rs b/crates/blockchain/src/beacon_envelope.rs index 8fc805d77..80d7d41ae 100644 --- a/crates/blockchain/src/beacon_envelope.rs +++ b/crates/blockchain/src/beacon_envelope.rs @@ -58,8 +58,10 @@ use ethlambda_state_transition::beacon::fork_choice::{ use ethlambda_state_transition::beacon::stf::ExecutionEngine; use ethlambda_state_transition::beacon::stf::gloas as stf_gloas; use ethlambda_types::ShortRoot; -use ethlambda_types::beacon::containers::{SignedBeaconBlock, gloas}; -use ethlambda_types::primitives::{H256, HashTreeRoot as _}; +use ethlambda_types::beacon::containers::{ + SignedBeaconBlock, SignedExecutionPayloadEnvelope, gloas, +}; +use ethlambda_types::primitives::H256; use tracing::{debug, error, info, trace, warn}; use crate::beacon_engine; @@ -128,7 +130,7 @@ pub(crate) const MAX_REJECTED_ENVELOPES: usize = 64; /// An envelope the actor is holding until it can be verified. pub(crate) struct HeldEnvelope { - envelope: gloas::SignedExecutionPayloadEnvelope, + envelope: SignedExecutionPayloadEnvelope, /// The envelope's own root, which tells a re-delivery from a different /// envelope for the same block. envelope_root: H256, @@ -144,12 +146,13 @@ pub(crate) struct HeldEnvelope { impl HeldEnvelope { pub(crate) fn new( - envelope: gloas::SignedExecutionPayloadEnvelope, + envelope: impl Into, slot: u64, event_ms: u64, ) -> Self { + let envelope = envelope.into(); Self { - envelope_root: envelope.hash_tree_root(), + envelope_root: envelope.signed_root(), envelope, slot, event_ms, @@ -285,16 +288,17 @@ pub(crate) fn missing_columns_for_envelope( /// The checks of `verify_execution_payload_envelope` that need only the block's /// bid, not its post-state: the envelope is the one the bid promised. fn envelope_matches_bid( - envelope: &gloas::ExecutionPayloadEnvelope, + envelope: &SignedExecutionPayloadEnvelope, bid: &gloas::ExecutionPayloadBid, block_slot: u64, ) -> bool { - envelope.builder_index == bid.builder_index - && envelope.payload.block_hash == bid.block_hash - && envelope.payload.prev_randao == bid.prev_randao - && envelope.payload.gas_limit == bid.gas_limit - && envelope.execution_requests.hash_tree_root() == bid.execution_requests_root - && envelope.payload.slot_number == block_slot + let payload = envelope.payload(); + envelope.builder_index() == bid.builder_index + && payload.block_hash == bid.block_hash + && payload.prev_randao == bid.prev_randao + && payload.gas_limit == bid.gas_limit + && envelope.execution_requests_root() == bid.execution_requests_root + && payload.slot_number == block_slot } impl BlockChainServer { @@ -304,7 +308,7 @@ impl BlockChainServer { /// caller then settles the queues. pub(crate) async fn receive_envelope( &mut self, - envelope: gloas::SignedExecutionPayloadEnvelope, + envelope: impl Into, event_ms: u64, ) { let config = self.store.config(); @@ -316,7 +320,7 @@ impl BlockChainServer { /// Verify `held` against its block, or put it back in the queue for what it /// is waiting on. `true` when the envelope was verified and recorded. async fn apply_or_hold_envelope(&mut self, held: HeldEnvelope) -> bool { - let root = held.envelope.message.beacon_block_root; + let root = held.envelope.beacon_block_root(); if self .store .beacon_block_payload_status(root) @@ -395,7 +399,7 @@ impl BlockChainServer { // envelope is parked (see `Self::hold_envelope_awaiting_columns`), or // by `on_execution_payload_envelope` when it is applied: gossip // validation does not reach it for an envelope it queued. - if !envelope_matches_bid(&held.envelope.message, bid, slot) { + if !envelope_matches_bid(&held.envelope, bid, slot) { warn!( %slot, block_root = %ShortRoot(&root.0), @@ -550,18 +554,14 @@ impl BlockChainServer { let seen_ms = held.event_ms.saturating_sub(config.genesis_time_ms()); self.store.insert_beacon_envelope_seen(root, slot, seen_ms); // Lets `latestValidHash` find this FULL node. - self.store.insert_beacon_el_block_hash( - root, - slot, - held.envelope.message.payload.block_hash, - ); + self.store + .insert_beacon_el_block_hash(root, slot, held.envelope.payload().block_hash); self.envelopes.verified.push_back(root); - let message = &held.envelope.message; self.events.publish_envelope_imported( &self.store, slot, - message.builder_index, - message.payload.block_hash, + held.envelope.builder_index(), + held.envelope.payload().block_hash, root, ); true @@ -637,7 +637,7 @@ impl BlockChainServer { ); return None; }; - if !envelope_matches_bid(&held.envelope.message, bid, slot) { + if !envelope_matches_bid(&held.envelope, bid, slot) { warn!( %slot, block_root = %ShortRoot(&root.0), @@ -709,12 +709,13 @@ impl BlockChainServer { fn envelope_signature_holds( &self, root: H256, - envelope: &gloas::SignedExecutionPayloadEnvelope, + envelope: &SignedExecutionPayloadEnvelope, ) -> bool { let Some(state) = self.store.get_state(&root).ok().flatten() else { return false; }; - stf_gloas::verify_execution_payload_envelope_signature(&state, envelope).unwrap_or(false) + stf_gloas::verify_any_execution_payload_envelope_signature(&state, envelope) + .unwrap_or(false) } fn hold_envelope_awaiting_columns( @@ -1171,6 +1172,7 @@ mod tests { use crate::tests::{GENESIS_TIME, beacon_server, beacon_store}; use ethlambda_network_api::BlockSource; use ethlambda_state_transition::beacon::fork::ForkName; + use ethlambda_types::primitives::HashTreeRoot as _; /// The mainnet preset with every fork active from genesis, as the fork /// choice fixtures that carry no `config.yaml` of their own are run. diff --git a/crates/blockchain/src/beacon_payloads.rs b/crates/blockchain/src/beacon_payloads.rs index 1d87e9242..aacbfaea2 100644 --- a/crates/blockchain/src/beacon_payloads.rs +++ b/crates/blockchain/src/beacon_payloads.rs @@ -49,6 +49,12 @@ pub(crate) fn bid_of(block: &SignedBeaconBlock) -> Option<&gloas::ExecutionPaylo SignedBeaconBlock::Gloas(inner) => { Some(&inner.message.body.signed_execution_payload_bid.message) } + SignedBeaconBlock::Eip8483(inner) => { + Some(&inner.message.body.signed_execution_payload_bid.message) + } + SignedBeaconBlock::Pq(inner) => { + Some(&inner.message.body.signed_execution_payload_bid.message) + } SignedBeaconBlock::Phase0(_) | SignedBeaconBlock::Altair(_) | SignedBeaconBlock::Bellatrix(_) diff --git a/crates/blockchain/src/lib.rs b/crates/blockchain/src/lib.rs index 37db88474..967b83b5c 100644 --- a/crates/blockchain/src/lib.rs +++ b/crates/blockchain/src/lib.rs @@ -882,7 +882,9 @@ fn data_availability_for( // block's own import (see `beacon_envelope`). So a gloas block has // nothing to wait for here, and `on_block` reads no availability // evidence for it. - SignedBeaconBlock::Gloas(_) => Some(fork_choice::DataAvailability::NotRequired), + SignedBeaconBlock::Gloas(_) | SignedBeaconBlock::Eip8483(_) | SignedBeaconBlock::Pq(_) => { + Some(fork_choice::DataAvailability::NotRequired) + } // `process_block` dispatches a lean block to `store::on_block` before // this function is ever reached, so this arm is never observed for // one; named on its own rather than folded into the group above so @@ -2528,6 +2530,35 @@ impl BlockChainServer { self.apply_beacon_aggregate(&aggregate, &attesting_indices, &index, Some(arrival)); } + /// Apply one PQ aggregate gossip already verified (its STARK proof + /// included) to fork choice. + fn apply_pq_aggregate( + &mut self, + data: ethlambda_types::beacon::containers::AttestationData, + attesting_indices: &[ValidatorIndex], + index: &HashMap, + ) { + let config = self.store.config(); + let outcome = fork_choice::apply_verified_aggregate( + &mut self.store, + data, + attesting_indices, + &config, + index, + ); + match outcome { + Ok(()) => metrics::inc_beacon_aggregate_outcome("applied"), + Err(err) => { + trace!( + slot = data.slot, + %err, + "PQ aggregate not applied to fork choice" + ); + metrics::inc_beacon_aggregate_outcome("invalid"); + } + } + } + /// Apply one already-verified aggregate to fork choice and record what /// became of it. /// @@ -2632,6 +2663,14 @@ impl BlockChainServer { } } + let ready_pq = self.beacon_aggregates.take_ready_pq(current_slot); + if !ready_pq.is_empty() { + let index = self.store.block_index(); + for (data, attesting_indices) in ready_pq { + self.apply_pq_aggregate(data, &attesting_indices, &index); + } + } + metrics::update_beacon_aggregates_deferred(self.beacon_aggregates.deferred_len()); } @@ -4050,6 +4089,7 @@ impl BlockChainServer { use ethlambda_network_api::p2p_to_block_chain::{ DataColumnSidecarsAwaitingParent, NewAggregatedAttestation, NewAttestation, NewBeaconAggregate, NewBlock, NewDataColumnSidecars, NewExecutionPayloadEnvelope, NewPayloadAttestationMessage, + NewPqAggregate, NewPqPayloadAttestationMessage, }; impl Handler for BlockChainServer { @@ -4204,6 +4244,55 @@ impl Handler for BlockChainServer { } } +impl Handler for BlockChainServer { + async fn handle(&mut self, msg: NewPqAggregate, _ctx: &Context) { + let ChainDuties::Beacon = &self.duties else { + return; + }; + metrics::observe_beacon_aggregate_mailbox_wait(msg.arrival.handed_off.elapsed()); + let config = self.store.config(); + let current_slot = fork_choice::get_current_slot(&self.store, &config); + // `validate_on_attestation`'s `current_slot >= data.slot + 1`, as for + // every other fork's aggregates: held until its slot has passed. + if current_slot < msg.data.slot.saturating_add(1) { + self.beacon_aggregates + .defer_pq(msg.data, msg.attesting_indices); + return; + } + let index = self.store.block_index(); + self.apply_pq_aggregate(msg.data, &msg.attesting_indices, &index); + } +} + +impl Handler for BlockChainServer { + async fn handle(&mut self, msg: NewPqPayloadAttestationMessage, ctx: &Context) { + let ChainDuties::Beacon = &self.duties else { + return; + }; + let slot = msg.message.data.slot; + let config = self.store.config(); + if self.ms_until_slot_start(slot) == 0 + && fork_choice::get_current_slot(&self.store, &config) < slot + { + self.on_tick(unix_now_ms(), ctx).await; + } + self.advance_beacon_clock_to(msg.arrival.wire_ms(Instant::now(), unix_now_ms())); + let validator = msg.message.validator_index; + let block_root = msg.message.data.beacon_block_root; + let _ = + fork_choice::on_pq_payload_attestation_message(&mut self.store, &msg.message, &config) + .inspect_err(|err| { + debug!( + slot, + validator, + block_root = %ShortRoot(&block_root.0), + %err, + "Dropping a PQ payload attestation message" + ) + }); + } +} + impl Handler for BlockChainServer { async fn handle(&mut self, msg: NewAttestation, ctx: &Context) { // Lean-only. A beacon node subscribes to no attestation subnet, so diff --git a/crates/blockchain/state_transition/src/beacon/block_production.rs b/crates/blockchain/state_transition/src/beacon/block_production.rs index 16ec4605d..764427076 100644 --- a/crates/blockchain/state_transition/src/beacon/block_production.rs +++ b/crates/blockchain/state_transition/src/beacon/block_production.rs @@ -93,7 +93,7 @@ pub fn payload_inputs(state: &BeaconState, config: &Config) -> Result { + BeaconState::Gloas(_) | BeaconState::Eip8483(_) | BeaconState::Pq(_) => { return Err(Error::SpecAssert("block production does not support gloas")); } BeaconState::Lean(_) => lean_state_unreachable("payload_inputs"), @@ -624,7 +624,7 @@ fn post_state( | BeaconState::Deneb(_) => { return Err(Error::SpecAssert("block production is electra and later")); } - BeaconState::Gloas(_) => { + BeaconState::Gloas(_) | BeaconState::Eip8483(_) | BeaconState::Pq(_) => { return Err(Error::SpecAssert("block production does not support gloas")); } BeaconState::Lean(_) => lean_state_unreachable("post_state"), diff --git a/crates/blockchain/state_transition/src/beacon/builder_market.rs b/crates/blockchain/state_transition/src/beacon/builder_market.rs index 6eda031bd..4e65c223e 100644 --- a/crates/blockchain/state_transition/src/beacon/builder_market.rs +++ b/crates/blockchain/state_transition/src/beacon/builder_market.rs @@ -16,13 +16,59 @@ use std::{ use lru::LruCache; -use super::containers::gloas; +use super::containers::{gloas, pq}; use super::gossip::IgnoreReason; use super::primitives::{BlsPubkey, ExecutionAddress, ExecutionBlockHash, Root, Slot}; /// Exactly one per node. pub type SharedBuilderMarket = Arc; +/// The PQ fork's market: its bids and preferences carry XMSS signatures, so +/// they are a different container from gloas's, and none of them may reach +/// the gloas market that block production reads. One per node, beside +/// [`SharedBuilderMarket`]. +pub type PqBuilderMarket = + BuilderMarket; + +/// Exactly one per node, beside [`SharedBuilderMarket`]. +pub type SharedPqBuilderMarket = Arc; + +/// A signed builder bid, whichever fork's signature it carries: gloas's BLS +/// or PQ's XMSS. The message is the same `ExecutionPayloadBid` at both. +pub trait SignedBid: Clone + PartialEq + std::fmt::Debug { + fn message(&self) -> &gloas::ExecutionPayloadBid; +} + +impl SignedBid for gloas::SignedExecutionPayloadBid { + fn message(&self) -> &gloas::ExecutionPayloadBid { + &self.message + } +} + +impl SignedBid for pq::SignedExecutionPayloadBid { + fn message(&self) -> &gloas::ExecutionPayloadBid { + &self.message + } +} + +/// Signed proposer preferences, whichever fork's signature they carry. The +/// message is the same `ProposerPreferences` at gloas and PQ. +pub trait SignedPreferences: Clone + std::fmt::Debug { + fn message(&self) -> &gloas::ProposerPreferences; +} + +impl SignedPreferences for gloas::SignedProposerPreferences { + fn message(&self) -> &gloas::ProposerPreferences { + &self.message + } +} + +impl SignedPreferences for pq::SignedProposerPreferences { + fn message(&self) -> &gloas::ProposerPreferences { + &self.message + } +} + /// Bids pooled per `(slot, parent hash, parent root)`, top values kept. pub const MAX_BIDS_PER_PARENT: usize = 16; /// A full slot refuses new keys: `record_bid` answers `false`. @@ -43,31 +89,57 @@ pub struct KnownPayload { } /// Bids for one `(parent_block_hash, parent_block_root)` of a slot. -#[derive(Debug, Default)] -struct ParentBids { +#[derive(Debug)] +struct ParentBids { /// The highest value recorded, which a later bid must strictly beat. Kept /// apart from `bids` because the pool is truncated and the bar is not. best_value: Option, /// Value descending, then builder index ascending. - bids: Vec, + bids: Vec, +} + +impl Default for ParentBids { + fn default() -> Self { + Self { + best_value: None, + bids: Vec::new(), + } + } } type ParentKey = (ExecutionBlockHash, Root); -#[derive(Debug, Default)] -struct SlotBids { +#[derive(Debug)] +struct SlotBids { /// The spec's `seen.execution_payload_bids`: one bid per builder per /// `(slot, parent_hash, parent_root)`. seen: BTreeSet<(ParentKey, u64)>, - parents: BTreeMap, + parents: BTreeMap>, } -#[derive(Debug, Default)] -struct BidPool { - slots: BTreeMap, +impl Default for SlotBids { + fn default() -> Self { + Self { + seen: BTreeSet::new(), + parents: BTreeMap::new(), + } + } +} + +#[derive(Debug)] +struct BidPool { + slots: BTreeMap>, +} + +impl Default for BidPool { + fn default() -> Self { + Self { + slots: BTreeMap::new(), + } + } } -impl BidPool { +impl BidPool { fn check(&self, bid: &gloas::ExecutionPayloadBid) -> Result<(), IgnoreReason> { let Some(slot) = self.slots.get(&bid.slot) else { return Ok(()); @@ -86,16 +158,20 @@ impl BidPool { } /// Preferences by `(proposal_slot, dependent_root)`: the first valid one wins. -type PreferencesCache = BTreeMap<(Slot, Root), gloas::SignedProposerPreferences>; +type PreferencesCache

= BTreeMap<(Slot, Root), P>; +/// Generic over the signed bid and preferences containers so the PQ fork's +/// XMSS-signed ones get a market of their own ([`PqBuilderMarket`]) with the +/// same rules; the defaults are gloas's, the one block production reads. #[derive(Debug)] -pub struct BuilderMarket { - bids: Mutex, - preferences: Mutex, +pub struct BuilderMarket +{ + bids: Mutex>, + preferences: Mutex>, payloads: Mutex>, } -impl Default for BuilderMarket { +impl Default for BuilderMarket { fn default() -> Self { Self { bids: Mutex::default(), @@ -114,7 +190,7 @@ fn lock(mutex: &Mutex) -> MutexGuard<'_, T> { .unwrap_or_else(|poisoned| poisoned.into_inner()) } -impl BuilderMarket { +impl BuilderMarket { // Bids: seen.execution_payload_bids + seen.best_execution_payload_bid + pool. /// The spec's two seen rules, in order: `(slot, parent_hash, parent_root, @@ -127,29 +203,29 @@ impl BuilderMarket { /// Re-runs [`Self::check_bid_seen`] under the lock. If it passes, records /// both seen keys and pools the bid. `false` = not recorded (race, or the /// per-slot key cap). Prunes slots below `bid.slot - 1`. - pub fn record_bid(&self, signed: gloas::SignedExecutionPayloadBid) -> bool { + pub fn record_bid(&self, signed: B) -> bool { let mut pool = lock(&self.bids); - if pool.check(&signed.message).is_err() { + if pool.check(signed.message()).is_err() { return false; } - let bid_slot = signed.message.slot; + let bid_slot = signed.message().slot; let slot = pool.slots.entry(bid_slot).or_default(); if slot.seen.len() >= MAX_SEEN_BID_KEYS_PER_SLOT { return false; } let parent = ( - signed.message.parent_block_hash, - signed.message.parent_block_root, + signed.message().parent_block_hash, + signed.message().parent_block_root, ); - slot.seen.insert((parent, signed.message.builder_index)); + slot.seen.insert((parent, signed.message().builder_index)); let entry = slot.parents.entry(parent).or_default(); - entry.best_value = Some(signed.message.value); + entry.best_value = Some(signed.message().value); entry.bids.push(signed); entry.bids.sort_by(|a, b| { - b.message + b.message() .value - .cmp(&a.message.value) - .then(a.message.builder_index.cmp(&b.message.builder_index)) + .cmp(&a.message().value) + .then(a.message().builder_index.cmp(&b.message().builder_index)) }); entry.bids.truncate(MAX_BIDS_PER_PARENT); let keep_from = bid_slot.saturating_sub(1); @@ -158,14 +234,14 @@ impl BuilderMarket { } /// The identical message and signature is pooled (API idempotency). - pub fn contains_bid(&self, signed: &gloas::SignedExecutionPayloadBid) -> bool { + pub fn contains_bid(&self, signed: &B) -> bool { let pool = lock(&self.bids); pool.slots - .get(&signed.message.slot) + .get(&signed.message().slot) .and_then(|slot| { slot.parents.get(&( - signed.message.parent_block_hash, - signed.message.parent_block_root, + signed.message().parent_block_hash, + signed.message().parent_block_root, )) }) .is_some_and(|parent| parent.bids.contains(signed)) @@ -177,7 +253,7 @@ impl BuilderMarket { slot: Slot, parent_block_root: Root, parent_block_hash: ExecutionBlockHash, - ) -> Vec { + ) -> Vec { lock(&self.bids) .slots .get(&slot) @@ -200,11 +276,7 @@ impl BuilderMarket { // Proposer preferences: seen.proposer_preferences. - pub fn preferences( - &self, - proposal_slot: Slot, - dependent_root: Root, - ) -> Option { + pub fn preferences(&self, proposal_slot: Slot, dependent_root: Root) -> Option

{ lock(&self.preferences) .get(&(proposal_slot, dependent_root)) .cloned() @@ -212,13 +284,12 @@ impl BuilderMarket { /// The first valid preferences per key win. `false` if one is held. Prunes /// `proposal_slot < current_slot`. - pub fn record_preferences( - &self, - signed: gloas::SignedProposerPreferences, - current_slot: Slot, - ) -> bool { + pub fn record_preferences(&self, signed: P, current_slot: Slot) -> bool { let mut cache = lock(&self.preferences); - let key = (signed.message.proposal_slot, signed.message.dependent_root); + let key = ( + signed.message().proposal_slot, + signed.message().dependent_root, + ); if cache.contains_key(&key) { return false; } @@ -249,7 +320,28 @@ impl BuilderMarket { beacon_block_root: envelope.beacon_block_root, builder_exits, }; - lock(&self.payloads).put(envelope.payload.block_hash, known); + self.record_known_payload(envelope.payload.block_hash, known); + } + + /// Record a payload an envelope of the PQ fork's shape revealed, whose + /// execution requests are not gloas's container. + pub fn record_pq_execution_payload(&self, envelope: &pq::ExecutionPayloadEnvelope) { + let builder_exits = envelope + .execution_requests + .builder_exits + .iter() + .map(|exit| (exit.pubkey, exit.source_address)) + .collect(); + let known = KnownPayload { + gas_limit: envelope.payload.gas_limit, + beacon_block_root: envelope.beacon_block_root, + builder_exits, + }; + self.record_known_payload(envelope.payload.block_hash, known); + } + + fn record_known_payload(&self, block_hash: ExecutionBlockHash, known: KnownPayload) { + lock(&self.payloads).put(block_hash, known); } pub fn known_payload(&self, block_hash: ExecutionBlockHash) -> Option { @@ -431,7 +523,7 @@ mod tests { #[test] fn a_builder_bids_once_per_parent() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); assert!(market.record_bid(bid(10, 1, 5))); assert_eq!( market.check_bid_seen(&bid(10, 1, 9).message), @@ -446,7 +538,7 @@ mod tests { #[test] fn a_bid_must_strictly_beat_the_best() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); assert!(market.record_bid(bid(10, 1, 5))); assert_eq!( market.check_bid_seen(&bid(10, 2, 5).message), @@ -464,7 +556,7 @@ mod tests { #[test] fn a_stale_check_fails_the_record() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let racing = bid(10, 2, 7); assert_eq!(market.check_bid_seen(&racing.message), Ok(())); assert!(market.record_bid(bid(10, 1, 8))); @@ -473,7 +565,7 @@ mod tests { #[test] fn a_full_slot_refuses_new_keys() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); for builder in 0..MAX_SEEN_BID_KEYS_PER_SLOT as u64 { assert!(market.record_bid(bid(10, builder, builder + 1))); } @@ -485,7 +577,7 @@ mod tests { #[test] fn bids_for_orders_by_value_and_keeps_the_top() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let total = MAX_BIDS_PER_PARENT as u64 + 4; for builder in 0..total { assert!(market.record_bid(bid(10, builder, builder + 1))); @@ -505,7 +597,7 @@ mod tests { #[test] fn contains_bid_matches_the_exact_message() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let signed = bid(10, 1, 5); assert!(!market.contains_bid(&signed)); assert!(market.record_bid(signed.clone())); @@ -519,7 +611,7 @@ mod tests { #[test] fn bids_prune_below_the_previous_slot() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); assert!(market.record_bid(bid(10, 1, 5))); assert!(market.record_bid(bid(11, 1, 5))); assert!(market.has_bids_for_slot(10)); @@ -533,7 +625,7 @@ mod tests { #[test] fn the_first_preferences_win() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let first = prefs(40, 1, 3); assert!(market.record_preferences(first.clone(), 38)); assert!(!market.record_preferences(prefs(40, 1, 4), 38)); @@ -545,7 +637,7 @@ mod tests { #[test] fn preferences_prune_by_proposal_slot() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); assert!(market.record_preferences(prefs(40, 1, 3), 38)); assert!(market.record_preferences(prefs(41, 1, 3), 41)); assert!(market.preferences(40, Root::repeat_byte(1)).is_none()); @@ -556,7 +648,7 @@ mod tests { #[test] fn preferences_over_the_cap_evict_the_lowest_slot() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); for slot in 0..MAX_PREFERENCES as u64 { assert!(market.record_preferences(prefs(100 + slot, 1, 3), 0)); } @@ -570,7 +662,7 @@ mod tests { #[test] fn a_revealed_payload_is_known_with_its_exits() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let pubkey = BlsPubkey([7; 48]); let source = ExecutionAddress::repeat_byte(9); let envelope = envelope_with_gas_limit( @@ -593,7 +685,7 @@ mod tests { #[test] fn known_payloads_are_a_bounded_lru() { - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let capacity = KNOWN_PAYLOADS_CAPACITY.get(); for index in 0..capacity { let mut block_hash = ExecutionBlockHash::ZERO; diff --git a/crates/blockchain/state_transition/src/beacon/fork_choice.rs b/crates/blockchain/state_transition/src/beacon/fork_choice.rs index d9bf42851..07ce83cff 100644 --- a/crates/blockchain/state_transition/src/beacon/fork_choice.rs +++ b/crates/blockchain/state_transition/src/beacon/fork_choice.rs @@ -179,8 +179,10 @@ use tracing::{debug, error, warn}; use crate::beacon::config::Config; use crate::beacon::constants; use crate::beacon::containers::shared::Validator; -use crate::beacon::containers::{AttestationData, BeaconState, Checkpoint, SignedBeaconBlock}; -use crate::beacon::containers::{bellatrix, deneb, electra, fulu, gloas, phase0}; +use crate::beacon::containers::{ + AttestationData, BeaconState, Checkpoint, SignedBeaconBlock, SignedExecutionPayloadEnvelope, +}; +use crate::beacon::containers::{bellatrix, deneb, electra, fulu, gloas, phase0, pq}; use crate::beacon::error::{Error, Result, verify}; use crate::beacon::fork::ForkName; use crate::beacon::helpers::accessors::{ @@ -193,6 +195,7 @@ use crate::beacon::helpers::gloas as gloas_helpers; use crate::beacon::helpers::misc::{ compute_epoch_at_slot, compute_start_slot_at_epoch, is_valid_merkle_branch, }; +use crate::beacon::helpers::pq as pq_helpers; use crate::beacon::helpers::predicates::{is_active_validator, is_slashable_attestation_data}; use crate::beacon::helpers::target_balances::{ TargetBalanceDelta, block_has_slashings, compute_target_balances, extend_parent_totals, @@ -243,7 +246,11 @@ pub use ethlambda_types::beacon::fork_choice::{ pub enum Attestation { Phase0(phase0::Attestation), Electra(electra::Attestation), + /// Gloas's shape, which EIP-8483 keeps. Gloas(gloas::Attestation), + /// The PQ fork's: no signature, the attesters' signatures being checked by + /// gossip validation or by the block's `proof`. + Pq(pq::Attestation), } /// Which fork's handler rules apply: to an attestation's `data.index` and to @@ -284,7 +291,7 @@ impl ForkRules { | ForkName::Deneb | ForkName::Electra | ForkName::Fulu => ForkRules::PreGloas, - ForkName::Gloas => ForkRules::Gloas, + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => ForkRules::Gloas, ForkName::Lean => lean_fork_unreachable("ForkRules::of"), } } @@ -311,6 +318,7 @@ impl Attestation { Attestation::Phase0(attestation) => attestation.data, Attestation::Electra(attestation) => attestation.data, Attestation::Gloas(attestation) => attestation.data, + Attestation::Pq(attestation) => attestation.data, } } @@ -319,7 +327,7 @@ impl Attestation { pub fn rules(&self) -> ForkRules { match self { Attestation::Phase0(_) | Attestation::Electra(_) => ForkRules::PreGloas, - Attestation::Gloas(_) => ForkRules::Gloas, + Attestation::Gloas(_) | Attestation::Pq(_) => ForkRules::Gloas, } } @@ -401,6 +409,15 @@ impl Attestation { } Ok(indexed.attesting_indices.iter().copied().collect()) } + // [Modified in PQ] An attestation carries no signature, so + // `on_attestation` only checks that it has attesters. + Attestation::Pq(attestation) => { + let indices = pq_helpers::get_attesting_indices(state, attestation, committees)?; + if verify_signature { + verify(!indices.is_empty(), "len(attesting_indices) > 0")?; + } + Ok(indices) + } } } } @@ -413,6 +430,8 @@ pub enum AttesterSlashing { Phase0(phase0::AttesterSlashing), Electra(electra::AttesterSlashing), Gloas(gloas::AttesterSlashing), + /// The PQ fork's, each half carrying a proof of its attesters' signatures. + Pq(pq::AttesterSlashing), } impl AttesterSlashing { @@ -431,6 +450,9 @@ impl AttesterSlashing { AttesterSlashing::Gloas(slashing) => { (slashing.attestation_1.data, slashing.attestation_2.data) } + AttesterSlashing::Pq(slashing) => { + (slashing.attestation_1.data, slashing.attestation_2.data) + } } } @@ -521,6 +543,30 @@ impl AttesterSlashing { .collect(), )) } + AttesterSlashing::Pq(slashing) => { + verify( + pq_helpers::is_valid_indexed_attestation(state, &slashing.attestation_1)?, + "is_valid_indexed_attestation(state, attestation_1)", + )?; + verify( + pq_helpers::is_valid_indexed_attestation(state, &slashing.attestation_2)?, + "is_valid_indexed_attestation(state, attestation_2)", + )?; + Ok(( + slashing + .attestation_1 + .attesting_indices + .iter() + .copied() + .collect(), + slashing + .attestation_2 + .attesting_indices + .iter() + .copied() + .collect(), + )) + } } } } @@ -586,6 +632,18 @@ pub fn block_operations(block: &SignedBeaconBlock) -> (Vec, Vec wrap_operations( + block.message.body.attestations.iter(), + block.message.body.attester_slashings.iter(), + Attestation::Gloas, + AttesterSlashing::Gloas, + ), + SignedBeaconBlock::Pq(block) => wrap_operations( + block.message.body.attestations.iter(), + block.message.body.attester_slashings.iter(), + Attestation::Pq, + AttesterSlashing::Pq, + ), SignedBeaconBlock::Lean(_) => lean_block_unreachable("fork_choice::block_operations"), } } @@ -3351,6 +3409,13 @@ fn gloas_bid(block: &SignedBeaconBlock) -> Result<&gloas::ExecutionPayloadBid> { SignedBeaconBlock::Gloas(inner) => { Ok(&inner.message.body.signed_execution_payload_bid.message) } + // The feature forks after gloas keep its bid. + SignedBeaconBlock::Eip8483(inner) => { + Ok(&inner.message.body.signed_execution_payload_bid.message) + } + SignedBeaconBlock::Pq(inner) => { + Ok(&inner.message.body.signed_execution_payload_bid.message) + } other => Err(Error::UnsupportedForFork { function: "gloas_bid", fork: other.fork_name(), @@ -5435,6 +5500,11 @@ pub fn on_block_speculative( // imported, and its full payload branch stays unreachable, since // building on it requires `is_payload_verified` above. SignedBeaconBlock::Gloas(block) => Some(&block.message.body.payload_attestations), + SignedBeaconBlock::Eip8483(block) => Some(&block.message.body.payload_attestations), + // [Modified in PQ] Blocks no longer carry payload attestations, so + // `notify_ptc_messages` is gone: the committee's votes reach fork + // choice from the wire only. + SignedBeaconBlock::Pq(_) => None, SignedBeaconBlock::Lean(_) => lean_block_unreachable("fork_choice::on_block"), }; @@ -5845,8 +5915,85 @@ fn apply_payload_attestation_message( source: PtcMessageSource, config: &Config, ) -> Result<()> { - let data = ptc_message.data; + apply_ptc_vote( + store, + state, + ptc_message.validator_index, + ptc_message.data, + source, + config, + |state| { + let indexed = gloas::IndexedPayloadAttestation { + attesting_indices: gloas::PayloadTimelinessCommitteeIndices::try_from(vec![ + ptc_message.validator_index, + ])?, + data: ptc_message.data, + signature: ptc_message.signature, + }; + verify( + gloas_helpers::is_valid_indexed_payload_attestation(state, &indexed), + "is_valid_indexed_payload_attestation(state, indexed_payload_attestation)", + ) + }, + ) +} + +/// `on_payload_attestation_message` (modified in PQ): a message always comes +/// from the wire now, blocks carrying no payload attestations, so its slot and +/// its XMSS signature, at the slot's PTC leaf, are always checked. +pub fn on_pq_payload_attestation_message( + store: &mut Store, + ptc_message: &pq::PayloadAttestationMessage, + config: &Config, +) -> Result<()> { + let state = store + .get_state(&ptc_message.data.beacon_block_root) + .expect("get") + .ok_or(Error::SpecAssert( + "data.beacon_block_root in store.block_states", + ))?; + apply_ptc_vote( + store, + &state, + ptc_message.validator_index, + ptc_message.data, + PtcMessageSource::Wire, + config, + |state| { + let data = &ptc_message.data; + let domain = crate::beacon::helpers::accessors::get_domain( + state, + constants::DOMAIN_PTC_ATTESTER, + Some(compute_epoch_at_slot(data.slot)), + ); + let signing_root = + crate::beacon::helpers::misc::compute_signing_root(data.hash_tree_root(), domain); + let leaf_index = pq_helpers::compute_xmss_leaf_index( + data.slot, + constants::LEAF_OFFSET_PTC_ATTESTER, + )?; + let pubkey = state.pq_validator(ptc_message.validator_index)?.pubkey(); + verify( + pq_helpers::xmss_verify(&pubkey, leaf_index, signing_root, &ptc_message.signature), + "xmss.Verify(pubkey, leaf_index, signing_root, ptc_message.signature)", + ) + }, + ) +} +/// The fork-independent body of a payload timeliness committee vote: `data` +/// from `validator_index`, checked against `state` (the state of the block it +/// names), with `verify_signature` run where `source` says the signature is +/// still unchecked. +fn apply_ptc_vote( + store: &mut Store, + state: &BeaconState, + validator_index: ValidatorIndex, + data: gloas::PayloadAttestationData, + source: PtcMessageSource, + config: &Config, + verify_signature: impl FnOnce(&BeaconState) -> Result<()>, +) -> Result<()> { // PTC votes can only change the vote for their assigned beacon block, // return early otherwise. if data.slot != state.slot() { @@ -5858,7 +6005,7 @@ fn apply_payload_attestation_message( let ptc_indices: Vec = ptc .iter() .enumerate() - .filter(|&(_, &validator_index)| validator_index == ptc_message.validator_index) + .filter(|&(_, &member)| member == validator_index) .map(|(ptc_index, _)| ptc_index) .collect(); @@ -5874,17 +6021,7 @@ fn apply_payload_attestation_message( )?; } if source == PtcMessageSource::Wire { - let indexed = gloas::IndexedPayloadAttestation { - attesting_indices: gloas::PayloadTimelinessCommitteeIndices::try_from(vec![ - ptc_message.validator_index, - ])?, - data, - signature: ptc_message.signature, - }; - verify( - gloas_helpers::is_valid_indexed_payload_attestation(state, &indexed), - "is_valid_indexed_payload_attestation(state, indexed_payload_attestation)", - )?; + verify_signature(state)?; } // Update the votes for the block. @@ -5938,7 +6075,7 @@ fn apply_payload_attestation_message( /// verified is checked again and recorded again, which is idempotent. pub fn on_execution_payload_envelope( store: &mut Store, - signed_envelope: &gloas::SignedExecutionPayloadEnvelope, + signed_envelope: &SignedExecutionPayloadEnvelope, config: &Config, sidecars: &[gloas::DataColumnSidecar], engine: &stf::ExecutionEngine, @@ -5960,7 +6097,7 @@ pub fn on_execution_payload_envelope( /// [`stf::ExecutionEngine::valid`] and consults the real engine itself. pub fn check_execution_payload_envelope( store: &Store, - signed_envelope: &gloas::SignedExecutionPayloadEnvelope, + signed_envelope: &SignedExecutionPayloadEnvelope, config: &Config, sidecars: &[gloas::DataColumnSidecar], engine: &stf::ExecutionEngine, @@ -5979,7 +6116,7 @@ pub fn check_execution_payload_envelope( /// taken on trust, so the KZG batch is not repeated on every envelope. pub fn check_execution_payload_envelope_columns_verified( store: &Store, - signed_envelope: &gloas::SignedExecutionPayloadEnvelope, + signed_envelope: &SignedExecutionPayloadEnvelope, config: &Config, engine: &stf::ExecutionEngine, ) -> Result<()> { @@ -5990,13 +6127,12 @@ pub fn check_execution_payload_envelope_columns_verified( /// vouched for every sampled column, which skips only the availability step. fn check_envelope( store: &Store, - signed_envelope: &gloas::SignedExecutionPayloadEnvelope, + signed_envelope: &SignedExecutionPayloadEnvelope, config: &Config, sidecars: Option<&[gloas::DataColumnSidecar]>, engine: &stf::ExecutionEngine, ) -> Result<()> { - let envelope = &signed_envelope.message; - let block_root = envelope.beacon_block_root; + let block_root = signed_envelope.beacon_block_root(); // The corresponding beacon block root needs to be known. let state = store @@ -6036,9 +6172,9 @@ fn check_envelope( /// envelope; a block the store does not hold is the only case this refuses. pub fn accept_execution_payload_envelope( store: &mut Store, - signed_envelope: &gloas::SignedExecutionPayloadEnvelope, + signed_envelope: &SignedExecutionPayloadEnvelope, ) { - let block_root = signed_envelope.message.beacon_block_root; + let block_root = signed_envelope.beacon_block_root(); let Some((slot, _parent)) = store.block_entry(&block_root) else { return; }; diff --git a/crates/blockchain/state_transition/src/beacon/gloas_block_production.rs b/crates/blockchain/state_transition/src/beacon/gloas_block_production.rs index b88f07174..b862a8404 100644 --- a/crates/blockchain/state_transition/src/beacon/gloas_block_production.rs +++ b/crates/blockchain/state_transition/src/beacon/gloas_block_production.rs @@ -462,8 +462,8 @@ pub fn bid_is_includable( && builder.version == constants::PAYLOAD_BUILDER_VERSION && bid.blob_kzg_commitments.len() <= preset::MAX_BLOB_COMMITMENTS_PER_BLOCK && !exited_by_parent - && is_active_builder(inner, bid.builder_index).unwrap_or(false) - && can_builder_cover_bid(inner, bid.builder_index, bid.value).unwrap_or(false) + && is_active_builder(&inner.into(), bid.builder_index).unwrap_or(false) + && can_builder_cover_bid(&inner.into(), bid.builder_index, bid.value).unwrap_or(false) } /// A produced block and the unsigned envelope that reveals its payload. @@ -611,7 +611,7 @@ fn check_envelope( "payload.slot_number == state.slot", )?; verify( - payload.parent_hash == inner.latest_block_hash, + payload.parent_hash == *inner.latest_block_hash, "payload.parent_hash == state.latest_block_hash", )?; verify( @@ -944,7 +944,7 @@ mod gloas_block_production_tests { }; stf::gloas::verify_execution_payload_envelope( &post, - &signed_envelope, + &signed_envelope.clone().into(), &config(), &ExecutionEngine::valid(), ) diff --git a/crates/blockchain/state_transition/src/beacon/gossip/aggregate.rs b/crates/blockchain/state_transition/src/beacon/gossip/aggregate.rs index 3b7feaa1a..dbf69e5d5 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/aggregate.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/aggregate.rs @@ -122,6 +122,17 @@ impl PackedBits { } } +/// What [`SeenAggregates`] keys an aggregate by: the specification's +/// `aggregate_data_roots` key `(hash_tree_root(data), committee_index)` and its +/// `aggregator_epochs` key `(target_epoch, aggregator_index)`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct AggregateKey { + pub data_root: Root, + pub committee_index: CommitteeIndex, + pub target_epoch: Epoch, + pub aggregator_index: ValidatorIndex, +} + /// Accepted aggregates, keyed the way the specification's `Seen` keys them. /// /// Bounded by capacity, like [`super::SeenBlocks`], rather than pruned on @@ -176,13 +187,26 @@ impl SeenAggregates { committee_index: CommitteeIndex, ) -> Result<(), Outcome> { let data = aggregate.data(); - let bits = aggregate.aggregation_bits(); + self.verdict_parts( + &AggregateKey { + data_root: data.hash_tree_root(), + committee_index, + target_epoch: data.target.epoch, + aggregator_index: aggregate.aggregator_index(), + }, + &aggregate.aggregation_bits(), + ) + } + + /// [`Self::verdict`] over the keys alone, for an aggregate of a shape the + /// shared enum does not carry (the PQ fork's, in `super::pq`). + pub(crate) fn verdict_parts(&self, key: &AggregateKey, bits: &[bool]) -> Result<(), Outcome> { // Checked in the specification's own order: the superset rule first, // then the aggregator/epoch rule. - if self.covered(data.hash_tree_root(), committee_index, &bits) { + if self.covered(key.data_root, key.committee_index, bits) { return Err(Outcome::Ignore(IgnoreReason::CoveredBits)); } - if self.aggregator_seen(data.target.epoch, aggregate.aggregator_index()) { + if self.aggregator_seen(key.target_epoch, key.aggregator_index) { return Err(Outcome::Ignore(IgnoreReason::AlreadySeen)); } Ok(()) @@ -206,23 +230,29 @@ impl SeenAggregates { let Some(committee_index) = aggregate.committee_index() else { return false; }; - if self.verdict(aggregate, committee_index).is_err() { + let data = aggregate.data(); + let key = AggregateKey { + data_root: data.hash_tree_root(), + committee_index, + target_epoch: data.target.epoch, + aggregator_index: aggregate.aggregator_index(), + }; + self.record_parts(&key, &aggregate.aggregation_bits()) + } + + /// [`Self::record`] over the keys alone; see [`Self::verdict_parts`]. + pub fn record_parts(&mut self, key: &AggregateKey, bits: &[bool]) -> bool { + if self.verdict_parts(key, bits).is_err() { return false; } - - let data = aggregate.data(); - let bits = aggregate.aggregation_bits(); self.aggregator_epochs - .put((data.target.epoch, aggregate.aggregator_index()), ()); - let packed = PackedBits::from_bits(&bits); - match self - .data_roots - .get_mut(&(data.hash_tree_root(), committee_index)) - { + .put((key.target_epoch, key.aggregator_index), ()); + let packed = PackedBits::from_bits(bits); + let data_key = (key.data_root, key.committee_index); + match self.data_roots.get_mut(&data_key) { Some(existing) => existing.push(packed), None => { - self.data_roots - .put((data.hash_tree_root(), committee_index), vec![packed]); + self.data_roots.put(data_key, vec![packed]); } } true diff --git a/crates/blockchain/state_transition/src/beacon/gossip/attestation.rs b/crates/blockchain/state_transition/src/beacon/gossip/attestation.rs index 711d2acac..19428fc04 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/attestation.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/attestation.rs @@ -76,7 +76,7 @@ impl SeenAttestations { /// Read-only ([`LruCache::contains`] does not touch recency), so /// [`cheap_checks`] can use it without mutating anything on a message /// that turns out invalid. - fn contains(&self, target_epoch: Epoch, attester_index: ValidatorIndex) -> bool { + pub(crate) fn contains(&self, target_epoch: Epoch, attester_index: ValidatorIndex) -> bool { self.0.contains(&(target_epoch, attester_index)) } @@ -86,7 +86,13 @@ impl SeenAttestations { /// two racing `Accept`s here too, and this settles the second as /// `Ignore(AlreadySeen)` rather than double-recording. pub fn record(&mut self, attestation: &SingleAttestation) -> bool { - let key = (attestation.data.target.epoch, attestation.attester_index); + self.record_key(attestation.data.target.epoch, attestation.attester_index) + } + + /// [`Self::record`] by its key, for an attestation of a shape this + /// module's own type does not carry (the PQ fork's, in `super::pq`). + pub fn record_key(&mut self, target_epoch: Epoch, attester_index: ValidatorIndex) -> bool { + let key = (target_epoch, attester_index); if self.0.contains(&key) { return false; } diff --git a/crates/blockchain/state_transition/src/beacon/gossip/block.rs b/crates/blockchain/state_transition/src/beacon/gossip/block.rs index 76bae3c62..39803fcca 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/block.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/block.rs @@ -28,6 +28,13 @@ pub fn cheap_checks( block: &SignedBeaconBlock, now_ms: u64, ) -> Result<(), Outcome> { + if !block.fork_name().is_followed() { + return Err(Outcome::Ignore(IgnoreReason::UnsupportedFork)); + } + // PQ's rules: gloas's with its own operation and request limits. + if let SignedBeaconBlock::Pq(pq_block) = block { + return super::pq::block_cheap_checks(seen, store, pq_block, now_ms); + } if let Some(gloas_block) = as_gloas(block, "block::cheap_checks") { return cheap_checks_gloas(seen, store, gloas_block, now_ms); } @@ -71,6 +78,10 @@ fn as_gloas<'a>( | SignedBeaconBlock::Deneb(_) | SignedBeaconBlock::Electra(_) | SignedBeaconBlock::Fulu(_) => None, + // Not followed (`ForkName::is_followed`): `cheap_checks` ignores + // their blocks before any rule reads them. PQ's rules are + // `super::pq::block_cheap_checks`. + SignedBeaconBlock::Eip8483(_) | SignedBeaconBlock::Pq(_) => None, SignedBeaconBlock::Lean(_) => lean_block_unreachable(function), } } @@ -140,7 +151,9 @@ fn body_operations_within_limits(body: &gloas::BeaconBlockBody) -> bool { /// rebuild the state from diffs, which is far too slow for a verdict that /// gossipsub waits on. A miss queues the block instead. pub fn stateful_checks(store: &Store, block: &SignedBeaconBlock, block_root: Root) -> Outcome { - let gloas_block = as_gloas(block, "block::stateful_checks"); + // Gloas and the feature forks built on it (EIP-8483, PQ) share the + // payload-envelope rules: they read only the bid and the parent. + let bid = payload_bid(block); let config = store.config(); let parent_root = block.parent_root(); let parent_known = store.has_block(&parent_root); @@ -173,7 +186,7 @@ pub fn stateful_checks(store: &Store, block: &SignedBeaconBlock, block_root: Roo // even with an unverified parent payload. The parent's fullness is read // once: it decodes the parent block, which the bounded validation pool // should not do twice. - let parent_full = if gloas_block.is_some() { + let parent_full = if bid.is_some() { match fork_choice::is_parent_node_full(store, block) { Ok(full) => full, Err(err) => { @@ -184,8 +197,7 @@ pub fn stateful_checks(store: &Store, block: &SignedBeaconBlock, block_root: Roo } else { true }; - if gloas_block.is_some() && parent_full && !fork_choice::is_payload_verified(store, parent_root) - { + if bid.is_some() && parent_full && !fork_choice::is_payload_verified(store, parent_root) { return Outcome::Ignore(IgnoreReason::ParentPayloadUnverified); } // [REJECT] The finalized checkpoint is an ancestor of the block. @@ -194,14 +206,9 @@ pub fn stateful_checks(store: &Store, block: &SignedBeaconBlock, block_root: Roo } // A gloas block has no payload to time; earlier forks: [REJECT] the // execution payload's timestamp is the slot's. - if let Some(gloas_block) = gloas_block { + if let Some(bid) = bid { // [REJECT] The bid's parent equals the block's parent. - let bid = &gloas_block - .message - .body - .signed_execution_payload_bid - .message; - if bid.parent_block_root != gloas_block.message.parent_root { + if bid.parent_block_root != block.parent_root() { return Outcome::Reject(RejectReason::BidParentMismatch); } // [REJECT] If the parent is not full, the bid builds on the parent's @@ -225,11 +232,37 @@ pub fn stateful_checks(store: &Store, block: &SignedBeaconBlock, block_root: Roo Outcome::Accept } +/// The execution payload bid of a block of gloas or a feature fork built on +/// it, or `None` for an earlier fork. +fn payload_bid(block: &SignedBeaconBlock) -> Option<&gloas::ExecutionPayloadBid> { + match block { + SignedBeaconBlock::Gloas(inner) => { + Some(&inner.message.body.signed_execution_payload_bid.message) + } + SignedBeaconBlock::Eip8483(inner) => { + Some(&inner.message.body.signed_execution_payload_bid.message) + } + SignedBeaconBlock::Pq(inner) => { + Some(&inner.message.body.signed_execution_payload_bid.message) + } + SignedBeaconBlock::Phase0(_) + | SignedBeaconBlock::Altair(_) + | SignedBeaconBlock::Bellatrix(_) + | SignedBeaconBlock::Capella(_) + | SignedBeaconBlock::Deneb(_) + | SignedBeaconBlock::Electra(_) + | SignedBeaconBlock::Fulu(_) => None, + SignedBeaconBlock::Lean(_) => lean_block_unreachable("block::payload_bid"), + } +} + /// `state.latest_block_hash`, for a gloas state. `None` for an earlier fork, /// which has none: its parent counts as full, so no caller asks. fn latest_block_hash(state: &BeaconState) -> Option { match state { - BeaconState::Gloas(state) => Some(state.latest_block_hash), + BeaconState::Gloas(_) | BeaconState::Eip8483(_) | BeaconState::Pq(_) => { + state.gloas_family().map(|inner| *inner.latest_block_hash) + } BeaconState::Phase0(_) | BeaconState::Altair(_) | BeaconState::Bellatrix(_) diff --git a/crates/blockchain/state_transition/src/beacon/gossip/column.rs b/crates/blockchain/state_transition/src/beacon/gossip/column.rs index 8ebe22e61..19e2f1045 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/column.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/column.rs @@ -752,6 +752,11 @@ fn gloas_before_kzg( | SignedBeaconBlock::Deneb(_) | SignedBeaconBlock::Electra(_) | SignedBeaconBlock::Fulu(_) => return Err(Outcome::Reject(RejectReason::BlockNotGloas)), + // Not followed (`ForkName::is_followed`): no column of theirs is + // judged here. + SignedBeaconBlock::Eip8483(_) | SignedBeaconBlock::Pq(_) => { + return Err(Outcome::Ignore(IgnoreReason::UnsupportedFork)); + } SignedBeaconBlock::Lean(_) => lean_block_unreachable("column::stateful_checks_gloas"), }; // [REJECT] The sidecar passes structural validation. diff --git a/crates/blockchain/state_transition/src/beacon/gossip/envelope.rs b/crates/blockchain/state_transition/src/beacon/gossip/envelope.rs index 8f5d6cfb7..8a7e8330b 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/envelope.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/envelope.rs @@ -135,6 +135,11 @@ pub fn stateful_checks(store: &Store, envelope: &gloas::SignedExecutionPayloadEn | SignedBeaconBlock::Deneb(_) | SignedBeaconBlock::Electra(_) | SignedBeaconBlock::Fulu(_) => return Outcome::Reject(RejectReason::BlockNotGloas), + // Not followed (`ForkName::is_followed`), and their envelopes are not + // gloas's shape. + SignedBeaconBlock::Eip8483(_) | SignedBeaconBlock::Pq(_) => { + return Outcome::Ignore(IgnoreReason::UnsupportedFork); + } SignedBeaconBlock::Lean(_) => lean_block_unreachable("envelope::stateful_checks"), }; let bid = &block.body.signed_execution_payload_bid.message; diff --git a/crates/blockchain/state_transition/src/beacon/gossip/execution_payload_bid.rs b/crates/blockchain/state_transition/src/beacon/gossip/execution_payload_bid.rs index 8ee152be3..35c72c489 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/execution_payload_bid.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/execution_payload_bid.rs @@ -31,19 +31,70 @@ use super::{ IgnoreReason, Outcome, RejectReason, is_current_slot, is_gloas_slot, proposer_preferences::dependent_root_at, }; -use crate::beacon::builder_market::{BuilderMarket, KnownPayload}; +use crate::beacon::builder_market::{BuilderMarket, KnownPayload, SignedBid, SignedPreferences}; use crate::beacon::config::Config; use crate::beacon::constants::PAYLOAD_BUILDER_VERSION; -use crate::beacon::containers::{BeaconState, SignedBeaconBlock, gloas}; +use crate::beacon::containers::{BeaconState, SignedBeaconBlock, gloas, pq}; use crate::beacon::fork_choice::{self, ForkChoiceNode, PayloadStatus, Store}; use crate::beacon::helpers::accessors::{get_current_epoch, get_randao_mix}; use crate::beacon::helpers::gloas::{can_builder_cover_bid, is_active_builder}; use crate::beacon::helpers::misc::{compute_epoch_at_slot, compute_start_slot_at_epoch}; use crate::beacon::lean_boundary::lean_state_unreachable; use crate::beacon::preset; -use crate::beacon::primitives::{Epoch, ExecutionBlockHash, Root, Slot}; +use crate::beacon::primitives::{BlsPubkey, Epoch, ExecutionBlockHash, Root, Slot}; use crate::beacon::stf; -use crate::beacon::stf::gloas::verify_execution_payload_bid_signature; + +/// What a bid's rules need from its fork beyond the shared message: which +/// state shape judges it, how an exit request names a builder, and the +/// signature. Gloas's bid is BLS-signed, PQ's XMSS-signed +/// (`specs/_features/pq/p2p-interface.md`'s modified `execution_payload_bid`). +pub trait BidRules: SignedBid { + /// Whether `state` holds the registry this bid is judged against. + fn judged_by(state: &BeaconState) -> bool; + /// Whether an exit request for `request_pubkey` exits the builder whose + /// registry key is `builder_pubkey`. + fn exit_names(request_pubkey: &BlsPubkey, builder_pubkey: &BlsPubkey) -> bool; + /// The builder's signature over the bid, against `state`'s registry. + fn signature_valid(&self, state: &BeaconState) -> bool; +} + +impl BidRules for gloas::SignedExecutionPayloadBid { + fn judged_by(state: &BeaconState) -> bool { + matches!(state, BeaconState::Gloas(_)) + } + + fn exit_names(request_pubkey: &BlsPubkey, builder_pubkey: &BlsPubkey) -> bool { + request_pubkey == builder_pubkey + } + + fn signature_valid(&self, state: &BeaconState) -> bool { + matches!( + stf::gloas::verify_execution_payload_bid_signature(state, self), + Ok(true) + ) + } +} + +impl BidRules for pq::SignedExecutionPayloadBid { + fn judged_by(state: &BeaconState) -> bool { + matches!(state, BeaconState::Pq(_)) + } + + /// `[Modified in PQ]` `XMSSPubkey(request.pubkey[16:]) == builder.pubkey`. + /// The state holds a PQ builder's key in the 48-byte request form, so its + /// last 32 bytes are the key; the request's first 16 are not compared. + fn exit_names(request_pubkey: &BlsPubkey, builder_pubkey: &BlsPubkey) -> bool { + request_pubkey.0[16..] == builder_pubkey.0[16..] + } + + /// `[Modified in PQ]` an XMSS signature at a bid leaf. + fn signature_valid(&self, state: &BeaconState) -> bool { + matches!( + stf::pq::verify_execution_payload_bid_signature(state, self), + Ok(true) + ) + } +} /// Gloas p2p preset: the largest decompressed `SignedExecutionPayloadBid`. pub const MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE: usize = 196_932; @@ -78,13 +129,13 @@ pub(crate) fn is_current_or_next_slot(config: &Config, slot: Slot, now_ms: u64) /// The rules that read only the message, the market's seen state and the /// clock. -pub fn cheap_checks( - market: &BuilderMarket, +pub fn cheap_checks( + market: &BuilderMarket, store: &Store, - signed: &gloas::SignedExecutionPayloadBid, + signed: &B, now_ms: u64, ) -> Result<(), Outcome> { - let bid = &signed.message; + let bid = signed.message(); let config = store.config(); // [IGNORE] The first bid for this slot, parent and builder, and [IGNORE] // the highest value seen for the slot and parent. @@ -138,7 +189,11 @@ fn pre_gloas_payload(state: &BeaconState) -> Option<(ExecutionBlockHash, u64)> { let header = &s.latest_execution_payload_header; Some((header.block_hash, header.gas_limit)) } - BeaconState::Phase0(_) | BeaconState::Altair(_) | BeaconState::Gloas(_) => None, + BeaconState::Phase0(_) + | BeaconState::Altair(_) + | BeaconState::Gloas(_) + | BeaconState::Eip8483(_) + | BeaconState::Pq(_) => None, BeaconState::Lean(_) => lean_state_unreachable("execution_payload_bid::pre_gloas_payload"), } } @@ -171,43 +226,59 @@ fn head_view(store: &Store) -> Result { _ => fork_choice::get_head_node(store, &config).map_err(|_| internal())?, }; if let Some(state) = store.cached_state(CacheKey::BlockState(node.root)) { - return match &*state { - BeaconState::Gloas(inner) => Ok(HeadView::Gloas { + // A head of gloas or a fork built on it carries the bid its payload + // facts are read from, so a PQ bid is judged against a PQ head. + if let Some(inner) = state.gloas_family() { + return Ok(HeadView::Gloas { node, parent_root: inner.latest_block_header.parent_root, bid_parent_hash: inner.latest_execution_payload_bid.parent_block_hash, bid_block_hash: inner.latest_execution_payload_bid.block_hash, + }); + } + return match pre_gloas_payload(&state) { + Some((block_hash, _)) => Ok(HeadView::PreGloas { + root: node.root, + block_hash, }), - other => match pre_gloas_payload(other) { - Some((block_hash, _)) => Ok(HeadView::PreGloas { - root: node.root, - block_hash, - }), - None => Err(Outcome::Ignore(IgnoreReason::NotOnHeadBranch)), - }, + None => Err(Outcome::Ignore(IgnoreReason::NotOnHeadBranch)), }; } let block = store .get_signed_block(&node.root) .map_err(|_| internal())? .ok_or(Outcome::Ignore(IgnoreReason::StateUnavailable))?; - match block { - SignedBeaconBlock::Gloas(block) => { - let bid = &block.message.body.signed_execution_payload_bid.message; - Ok(HeadView::Gloas { - node, - parent_root: block.message.parent_root, - bid_parent_hash: bid.parent_block_hash, - bid_block_hash: bid.block_hash, - }) - } - other => match other.execution_block_hash() { - Some(block_hash) => Ok(HeadView::PreGloas { - root: node.root, - block_hash, - }), - None => Err(Outcome::Ignore(IgnoreReason::NotOnHeadBranch)), - }, + // Every gloas-family block's bid message is gloas's, whatever its + // signature. + let bid = match &block { + SignedBeaconBlock::Gloas(block) => Some(( + block.message.parent_root, + &block.message.body.signed_execution_payload_bid.message, + )), + SignedBeaconBlock::Eip8483(block) => Some(( + block.message.parent_root, + &block.message.body.signed_execution_payload_bid.message, + )), + SignedBeaconBlock::Pq(block) => Some(( + block.message.parent_root, + &block.message.body.signed_execution_payload_bid.message, + )), + _ => None, + }; + if let Some((parent_root, bid)) = bid { + return Ok(HeadView::Gloas { + node, + parent_root, + bid_parent_hash: bid.parent_block_hash, + bid_block_hash: bid.block_hash, + }); + } + match block.execution_block_hash() { + Some(block_hash) => Ok(HeadView::PreGloas { + root: node.root, + block_hash, + }), + None => Err(Outcome::Ignore(IgnoreReason::NotOnHeadBranch)), } } @@ -283,9 +354,9 @@ pub(crate) fn cached_checkpoint_state( /// The exits a parent payload carried, for rule 21: the market's known payload /// when it is the parent block's own, else the verified envelope in the store. /// `None` when neither is available. -fn parent_payload_exits( +fn parent_payload_exits( store: &Store, - market: &BuilderMarket, + market: &BuilderMarket, bid: &gloas::ExecutionPayloadBid, ) -> Option< Vec<( @@ -302,9 +373,12 @@ fn parent_payload_exits( { return Some(builder_exits); } + // A gloas bid's parent payload is a gloas envelope: the later forks' + // bids are judged against their own markets. let envelope = store .get_execution_payload_envelope(&bid.parent_block_root) .ok()??; + let envelope = envelope.as_gloas()?; Some( envelope .message @@ -317,10 +391,10 @@ fn parent_payload_exits( } /// The rules that need states, then the signature. Runs on a blocking thread. -pub fn stateful_checks( +pub fn stateful_checks( store: &Store, - market: &BuilderMarket, - signed: &gloas::SignedExecutionPayloadBid, + market: &BuilderMarket, + signed: &B, ) -> Outcome { match stateful_rules(store, market, signed) { Ok(()) => Outcome::Accept, @@ -328,12 +402,12 @@ pub fn stateful_checks( } } -fn stateful_rules( +fn stateful_rules( store: &Store, - market: &BuilderMarket, - signed: &gloas::SignedExecutionPayloadBid, + market: &BuilderMarket, + signed: &B, ) -> Result<(), Outcome> { - let bid = &signed.message; + let bid = signed.message(); let ignore = |reason| Outcome::Ignore(reason); let reject = |reason| Outcome::Reject(reason); let proposal_epoch = compute_epoch_at_slot(bid.slot); @@ -364,7 +438,8 @@ fn stateful_rules( let preferences = market .preferences(bid.slot, dependent_root) .ok_or(ignore(IgnoreReason::PreferencesUnseen))? - .message; + .message() + .clone(); // [IGNORE] The fee recipient matches the proposer's preference. if bid.fee_recipient != preferences.fee_recipient { return Err(ignore(IgnoreReason::FeeRecipientMismatch)); @@ -403,11 +478,16 @@ fn stateful_rules( cached_checkpoint_state(store, proposal_epoch, bid.parent_block_root) .ok_or(ignore(IgnoreReason::StateUnavailable))? }; - let BeaconState::Gloas(inner) = &*state else { - // A bid in a gloas epoch is advanced into gloas by the epoch - // transition; anything else is a state this cannot judge. + // A bid in a gloas epoch is advanced into gloas by the epoch transition, + // and a PQ bid's into PQ; anything else is a state this cannot judge. + // EIP-8483's states are left out: its parent envelopes are not gloas's + // shape, which the builder-exit rule below reads. + if !B::judged_by(&state) { return Err(ignore(IgnoreReason::StateUnavailable)); - }; + } + let inner = state + .gloas_family() + .expect("a state a bid is judged by is gloas or PQ"); // [REJECT] The builder index is valid, [REJECT] it is a payload builder // and [REJECT] active. @@ -418,31 +498,31 @@ fn stateful_rules( if builder.version != PAYLOAD_BUILDER_VERSION { return Err(reject(RejectReason::NotPayloadBuilder)); } - if !is_active_builder(inner, bid.builder_index).unwrap_or(false) { + if !is_active_builder(&inner, bid.builder_index).unwrap_or(false) { return Err(reject(RejectReason::InactiveBuilder)); } // [IGNORE] The builder can cover the bid. - if !can_builder_cover_bid(inner, bid.builder_index, bid.value).unwrap_or(false) { + if !can_builder_cover_bid(&inner, bid.builder_index, bid.value).unwrap_or(false) { return Err(ignore(IgnoreReason::BuilderCannotCover)); } // [IGNORE] The parent's payload does not try to exit the builder. Only a - // gloas parent has an envelope to carry the request. - let parent_is_gloas = matches!(&*parent_state, BeaconState::Gloas(_)); - if parent_is_gloas && bid.parent_block_hash == inner.latest_execution_payload_bid.block_hash { + // parent of the bid's own fork has an envelope to carry the request. A + // PQ parent's envelope is never recorded (the node does not follow PQ), + // so a PQ bid on one ends here as unverified. + let parent_has_envelope = B::judged_by(&parent_state); + if parent_has_envelope && bid.parent_block_hash == inner.latest_execution_payload_bid.block_hash + { let exits = parent_payload_exits(store, market, bid) .ok_or(ignore(IgnoreReason::ParentPayloadUnverified))?; if exits.iter().any(|(pubkey, source)| { - *pubkey == builder.pubkey && *source == builder.execution_address + B::exit_names(pubkey, &builder.pubkey) && *source == builder.execution_address }) { return Err(ignore(IgnoreReason::BuilderMayExit)); } } // [REJECT] The signature is valid. An error (an index the state lacks) // is a signature that cannot be. - if !matches!( - verify_execution_payload_bid_signature(&state, signed), - Ok(true) - ) { + if !signed.signature_valid(&state) { return Err(reject(RejectReason::BadSignature)); } Ok(()) @@ -450,10 +530,10 @@ fn stateful_rules( /// Both halves. The caller records the bid on `Accept`: the specification's /// `validate_execution_payload_bid_gossip`. -pub fn validate( - market: &BuilderMarket, +pub fn validate( + market: &BuilderMarket, store: &Store, - signed: &gloas::SignedExecutionPayloadBid, + signed: &B, now_ms: u64, ) -> Outcome { if let Err(outcome) = cheap_checks(market, store, signed, now_ms) { @@ -736,7 +816,7 @@ mod tests { #[test] fn a_bid_without_preferences_is_ignored() { let scene = scene(); - let empty = BuilderMarket::default(); + let empty: BuilderMarket = BuilderMarket::default(); assert_eq!( stateful_checks(&scene.store, &empty, &scene.bid), ignore(IgnoreReason::PreferencesUnseen) @@ -1045,7 +1125,7 @@ mod tests { store.set_head_payload_status(parent, PayloadStatus::Empty); let slot = 2 * SLOTS_PER_EPOCH; - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let proposer = fixed_proposer(&state, slot).expect("in the window"); let dependent_root = dependent_root_at(&state, parent, slot).expect("in the window"); let preferences = sign_preferences( diff --git a/crates/blockchain/state_transition/src/beacon/gossip/mod.rs b/crates/blockchain/state_transition/src/beacon/gossip/mod.rs index 213a92266..167a2c460 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/mod.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/mod.rs @@ -14,6 +14,7 @@ pub mod envelope; pub mod execution_payload_bid; pub mod operations; pub mod payload_attestation; +pub mod pq; pub mod proposer_preferences; pub mod sync_committee; #[cfg(test)] @@ -287,6 +288,8 @@ pub enum RejectReason { AggregatorSignature, /// The aggregate attestation's own signature is invalid. AggregateSignature, + /// A PQ aggregate's STARK proof does not prove its attesters' signatures. + AggregateProof, /// The target is not the voted block's ancestor at the target epoch. TargetNotAncestor, /// A bid promises a payment outside the bid value. @@ -372,6 +375,7 @@ impl RejectReason { Self::SelectionProof => "selection_proof", Self::AggregatorSignature => "aggregator_signature", Self::AggregateSignature => "aggregate_signature", + Self::AggregateProof => "aggregate_proof", Self::TargetNotAncestor => "target_not_ancestor", Self::ExecutionPaymentNonZero => "execution_payment_nonzero", Self::BlockHashEqualsParent => "block_hash_equals_parent", @@ -581,7 +585,7 @@ pub(crate) fn is_current_or_previous_epoch(config: &Config, epoch: Epoch, now_ms /// (A gloas aggregate has its own container, which says so itself.) pub(crate) fn is_gloas_slot(config: &Config, slot: Slot) -> bool { match config.fork_at_epoch(compute_epoch_at_slot(slot)) { - ForkName::Gloas => true, + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => true, ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix diff --git a/crates/blockchain/state_transition/src/beacon/gossip/operations.rs b/crates/blockchain/state_transition/src/beacon/gossip/operations.rs index fa096e666..ae0e708d9 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/operations.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/operations.rs @@ -78,6 +78,37 @@ fn slashed_indices(slashing: &AttesterSlashing) -> Vec { } impl SeenOperations { + /// Whether no slashing of `proposer_index` has been accepted yet. + pub(crate) fn proposer_slashing_is_new(&self, proposer_index: ValidatorIndex) -> bool { + !self.proposer_slashings.contains(&proposer_index) + } + + /// Whether any of `slashed` has not been slashed by an accepted attester + /// slashing yet. + pub(crate) fn attester_slashing_is_new(&self, slashed: &[ValidatorIndex]) -> bool { + slashed + .iter() + .any(|index| !self.attester_slashed.contains(index)) + } + + /// Record an accepted proposer slashing of a shape [`BeaconOperation`] + /// does not carry (the PQ fork's). `false` if one was recorded first. + pub fn record_proposer_slashing(&mut self, proposer_index: ValidatorIndex) -> bool { + self.proposer_slashings.insert(proposer_index) + } + + /// Record an accepted attester slashing of a shape [`BeaconOperation`] + /// does not carry (the PQ fork's), by the indices it slashes. `false` if + /// none of them is new. + pub fn record_attester_slashing(&mut self, slashed: &[ValidatorIndex]) -> bool { + // An explicit loop rather than `any`, which would short-circuit. + let mut any_new = false; + for &index in slashed { + any_new |= self.attester_slashed.insert(index); + } + any_new + } + /// `false` when the specification's IGNORE condition holds. pub fn is_new(&self, operation: &BeaconOperation) -> bool { match operation { diff --git a/crates/blockchain/state_transition/src/beacon/gossip/payload_attestation.rs b/crates/blockchain/state_transition/src/beacon/gossip/payload_attestation.rs index 0b186b85c..184c62653 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/payload_attestation.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/payload_attestation.rs @@ -25,7 +25,7 @@ use lru::LruCache; use super::{IgnoreReason, Outcome, RejectReason, is_current_slot}; use crate::beacon::bls; use crate::beacon::constants::DOMAIN_PTC_ATTESTER; -use crate::beacon::containers::gloas; +use crate::beacon::containers::{BeaconState, gloas}; use crate::beacon::fork_choice::Store; use crate::beacon::helpers::accessors::get_domain; use crate::beacon::helpers::gloas::get_ptc; @@ -66,15 +66,32 @@ pub fn cheap_checks( store: &Store, message: &gloas::PayloadAttestationMessage, now_ms: u64, +) -> Result<(), Outcome> { + cheap_checks_parts( + seen, + store, + message.data.slot, + message.validator_index, + now_ms, + ) +} + +/// [`cheap_checks`] over the two fields it reads, for a message of a shape +/// this module's own type does not carry (the PQ fork's, in `super::pq`). +pub(crate) fn cheap_checks_parts( + seen: &SeenPayloadAttestations, + store: &Store, + slot: Slot, + validator_index: ValidatorIndex, + now_ms: u64, ) -> Result<(), Outcome> { let config = store.config(); - let slot = message.data.slot; // [REJECT] The payload attestation's slot is at or after the gloas fork. if compute_epoch_at_slot(slot) < config.gloas_fork_epoch { return Err(Outcome::Reject(RejectReason::PreGloasSlot)); } // [IGNORE] This is the first valid payload attestation from this validator. - if seen.contains(slot, message.validator_index) { + if seen.contains(slot, validator_index) { return Err(Outcome::Ignore(IgnoreReason::AlreadySeen)); } // [IGNORE] The payload attestation's slot is the current slot. @@ -87,8 +104,42 @@ pub fn cheap_checks( /// The rules that need the voted block and the head state. Runs on a blocking /// thread. pub fn stateful_checks(store: &Store, message: &gloas::PayloadAttestationMessage) -> Outcome { - let data = &message.data; - let validator_index = message.validator_index; + stateful_checks_with( + store, + message.validator_index, + &message.data, + |state| !matches!(state, BeaconState::Pq(_)), + |state, validator_index| { + let Ok(validator) = state.validator(validator_index) else { + return false; + }; + let domain = get_domain( + state, + DOMAIN_PTC_ATTESTER, + Some(compute_epoch_at_slot(message.data.slot)), + ); + let signing_root = compute_signing_root(message.data.hash_tree_root(), domain); + let validator = bls::IndexedPubkey { + index: validator_index, + pubkey: validator.pubkey, + }; + bls::verify(&validator, signing_root, &message.signature) + }, + ) +} + +/// [`stateful_checks`]' rules for any fork's message: the PQ fork changes only +/// the signature (`specs/_features/pq/p2p-interface.md`), so `verify` is the +/// fork's own signature check and `judged_by` says which registry shape +/// holds the key it reads. A head state of another shape cannot judge the +/// message, which is not the sender's fault. +pub(crate) fn stateful_checks_with( + store: &Store, + validator_index: ValidatorIndex, + data: &gloas::PayloadAttestationData, + judged_by: impl FnOnce(&BeaconState) -> bool, + verify: impl FnOnce(&BeaconState, ValidatorIndex) -> bool, +) -> Outcome { // [IGNORE] The voted block has been seen. let Some((block_slot, _)) = store.block_slot_and_state_root(&data.beacon_block_root) else { return Outcome::Ignore(IgnoreReason::UnknownBlock); @@ -113,10 +164,13 @@ pub fn stateful_checks(store: &Store, message: &gloas::PayloadAttestationMessage let Some(state) = store.cached_state(CacheKey::BlockState(head_root)) else { return Outcome::Ignore(IgnoreReason::StateUnavailable); }; + if !judged_by(&state) { + return Outcome::Ignore(IgnoreReason::StateUnavailable); + } // [REJECT] The validator index is valid. - let Ok(validator) = state.validator(validator_index) else { + if state.validator(validator_index).is_err() { return Outcome::Reject(RejectReason::UnknownValidator); - }; + } // [REJECT] The validator is a member of the payload timeliness committee. // A slot the head's committee window cannot answer is not the sender's fault. let Ok(ptc) = get_ptc(&state, data.slot, &store.config()) else { @@ -126,17 +180,7 @@ pub fn stateful_checks(store: &Store, message: &gloas::PayloadAttestationMessage return Outcome::Reject(RejectReason::NotInPtc); } // [REJECT] The signature is valid. - let domain = get_domain( - &state, - DOMAIN_PTC_ATTESTER, - Some(compute_epoch_at_slot(data.slot)), - ); - let signing_root = compute_signing_root(data.hash_tree_root(), domain); - let validator = bls::IndexedPubkey { - index: validator_index, - pubkey: validator.pubkey, - }; - if !bls::verify(&validator, signing_root, &message.signature) { + if !verify(&state, validator_index) { return Outcome::Reject(RejectReason::BadSignature); } Outcome::Accept diff --git a/crates/blockchain/state_transition/src/beacon/gossip/pq.rs b/crates/blockchain/state_transition/src/beacon/gossip/pq.rs new file mode 100644 index 000000000..8e9ea776f --- /dev/null +++ b/crates/blockchain/state_transition/src/beacon/gossip/pq.rs @@ -0,0 +1,1594 @@ +//! Gossip validation for the PQ fork's topics +//! (`specs/_features/pq/p2p-interface.md`, consensus-specs' `pq` branch at +//! `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`). +//! +//! Every topic whose message carries a validator's or builder's signature +//! carries an XMSS one instead, at the leaf the fork assigns that message, and +//! an aggregate proves its attesters' signatures with a STARK proof rather +//! than an aggregate BLS signature. Aggregators come from the state's +//! aggregation committee rather than a selection proof, and attestations +//! travel on `ATTESTATION_SUBNET_COUNT_PQ` subnets. `voluntary_exit`, +//! `bls_to_execution_change` and the sync committee topics are gone. +//! +//! # What is ported +//! +//! The branch restates every rule it touches on a consensus-specs `master` +//! newer than this repository's pin, with head-state reads and checks in a +//! different order. This module applies only the lines marked `[Modified in +//! PQ]` or `[New in PQ]` to the gloas rules the neighbouring modules already +//! implement, keeping their deliberate departures (the vote block's own +//! post-state rather than the head's, the pubkey-only signature before any +//! committee derivation, "post-state is cached" for "passes validation"); see +//! `docs/spec_deviations.md`. The topics with a gloas counterpart that differs +//! only in the signature (bids, preferences, payload attestation messages) +//! reuse that module's rules through its fork trait or `*_with` function. +//! +//! # States +//! +//! A PQ rule reads XMSS keys, which only a PQ state holds. A rule whose state +//! is of another fork answers [`IgnoreReason::StateUnavailable`], never a +//! REJECT: the node does not follow PQ (`ForkName::is_followed`), so on a live +//! node no PQ block is imported and every stateful rule here ends in an +//! IGNORE or a queue, while the cheap rules judge what arrives on a PQ digest +//! that a network schedules. + +use std::sync::Arc; + +use ethlambda_storage::CacheKey; + +use super::aggregate::AggregateKey; +use super::column::judgeable_block_gloas; +use super::execution_payload_bid::cached_checkpoint_state; +use super::operations::SeenOperations; +use super::{ + IgnoreReason, Outcome, QueueReason, RejectReason, SeenAggregates, SeenAttestations, SeenBlocks, + SeenEnvelopes, SeenPayloadAttestations, ancestor_at, finalized_start_slot, + is_current_or_previous_epoch, is_future_slot, verify_attestation_payload_status, +}; +use crate::beacon::config::Config; +use crate::beacon::constants::{ + DOMAIN_AGGREGATE_AND_PROOF, DOMAIN_BEACON_ATTESTER, DOMAIN_PTC_ATTESTER, + LEAF_OFFSET_AGGREGATE_AND_PROOF, LEAF_OFFSET_PTC_ATTESTER, +}; +use crate::beacon::containers::{BeaconState, SignedBeaconBlock, pq}; +use crate::beacon::fork_choice::Store; +use crate::beacon::helpers::accessors::{ + CommitteeCacheExt, get_current_epoch, get_domain, get_domain_from_schedule, +}; +use crate::beacon::helpers::misc::{ + compute_epoch_at_slot, compute_signing_root, compute_start_slot_at_epoch, +}; +use crate::beacon::helpers::pq::{ + compute_xmss_attestation_leaf_index, compute_xmss_leaf_index, get_aggregation_committee, + get_attestation_statements, get_attesting_indices, get_committee_indices, + is_valid_indexed_attestation, verify_aggregates, xmss_verify, +}; +use crate::beacon::helpers::predicates::{is_slashable_attestation_data, is_slashable_validator}; +use crate::beacon::lean_boundary::lean_block_unreachable; +use crate::beacon::preset; +use crate::beacon::primitives::{ + CommitteeIndex, Epoch, HashTreeRoot as _, Root, Slot, ValidatorIndex, +}; +use crate::beacon::stf; + +/// How many epochs past the vote block's own a PQ rule advances its state to +/// reach the attestation's epoch. Advancing runs every slot between, so an +/// unbounded advance is work any peer could ask for: on a node that does not +/// follow PQ, every head is far behind a PQ slot. The aggregation committee +/// window already reaches `MIN_SEED_LOOKAHEAD` epochs ahead, so a vote block +/// that recent needs no advance at all. +pub const MAX_VOTE_STATE_ADVANCE_EPOCHS: u64 = preset::MIN_SEED_LOOKAHEAD + 1; + +/// The most aggregation bits a PQ aggregate may carry: every committee of a +/// slot at full size, under the PQ committee cap. The progressive bitlist has +/// no bound of its own, so the rules enforce this one before reading the bits. +pub const MAX_AGGREGATION_BITS: usize = + preset::MAX_VALIDATORS_PER_COMMITTEE * preset::MAX_COMMITTEES_PER_SLOT_PQ; + +// --------------------------------------------------------------------------- +// Shared +// --------------------------------------------------------------------------- + +/// `compute_subnet_for_attestation` (modified in PQ, `pq/validator.md`): the +/// subnet an attestation travels on, among `ATTESTATION_SUBNET_COUNT_PQ`. +pub fn compute_subnet_for_attestation( + committees_per_slot: u64, + slot: Slot, + committee_index: CommitteeIndex, + config: &Config, +) -> u64 { + let slots_since_epoch_start = slot % preset::SLOTS_PER_EPOCH; + let committees_since_epoch_start = committees_per_slot.saturating_mul(slots_since_epoch_start); + committees_since_epoch_start.saturating_add(committee_index) + % config.attestation_subnet_count_pq.max(1) +} + +/// The vote block's post-state, as a PQ state that can name `attestation_epoch`'s +/// committees and aggregation committees. +/// +/// The vote block's own cached post-state when it is a PQ state whose +/// aggregation-committee window reaches the epoch, else that state advanced to +/// the epoch's start, the specification's `[New in PQ]` `process_slots` for an +/// epoch past `MIN_SEED_LOOKAHEAD` (here also for a vote block of an earlier +/// fork, whose state holds no XMSS keys), at most +/// [`MAX_VOTE_STATE_ADVANCE_EPOCHS`] ahead. Cached-only, as every gossip rule +/// reads states: a miss is an IGNORE, and so is a state that is not PQ. +fn pq_vote_state( + store: &Store, + block_root: Root, + attestation_epoch: Epoch, +) -> Result, Outcome> { + // [IGNORE] The block being voted for has been seen. + if !store.has_block(&block_root) { + return Err(Outcome::Ignore(IgnoreReason::UnknownBlock)); + } + // "Passes validation" is "post-state is cached"; see `super::aggregate`. + let Some(state) = store.cached_state(CacheKey::BlockState(block_root)) else { + return Err(Outcome::Ignore(IgnoreReason::StateUnavailable)); + }; + let state_epoch = get_current_epoch(&state); + let window_reaches = attestation_epoch + 1 >= state_epoch + && attestation_epoch <= state_epoch + preset::MIN_SEED_LOOKAHEAD; + let state = if matches!(*state, BeaconState::Pq(_)) && window_reaches { + state + } else { + // Bounded before anything is advanced: see + // `MAX_VOTE_STATE_ADVANCE_EPOCHS`. + if attestation_epoch < state_epoch + || attestation_epoch - state_epoch > MAX_VOTE_STATE_ADVANCE_EPOCHS + { + return Err(Outcome::Ignore(IgnoreReason::StateUnavailable)); + } + cached_checkpoint_state(store, attestation_epoch, block_root) + .ok_or(Outcome::Ignore(IgnoreReason::StateUnavailable))? + }; + if !matches!(*state, BeaconState::Pq(_)) { + return Err(Outcome::Ignore(IgnoreReason::StateUnavailable)); + } + Ok(state) +} + +/// The two ancestry rules every attestation topic ends with, against the vote +/// state's own history ([`ancestor_at`]). +fn check_ancestry( + store: &Store, + state: &BeaconState, + block_root: Root, + target_epoch: Epoch, + target_root: Root, +) -> Result<(), Outcome> { + // [REJECT] The target is the vote block's ancestor at the target epoch. + let target_start_slot = compute_start_slot_at_epoch(target_epoch); + let Some(checkpoint_block) = ancestor_at(state, block_root, target_start_slot) else { + return Err(Outcome::Ignore(IgnoreReason::AncestryUnknown)); + }; + if checkpoint_block != target_root { + return Err(Outcome::Reject(RejectReason::TargetNotAncestor)); + } + // [IGNORE] The finalized checkpoint is an ancestor of the vote block. + let finalized = store.beacon_finalized_checkpoint(); + let finalized_start_slot = compute_start_slot_at_epoch(finalized.epoch); + let Some(finalized_block) = ancestor_at(state, block_root, finalized_start_slot) else { + return Err(Outcome::Ignore(IgnoreReason::AncestryUnknown)); + }; + if finalized_block != finalized.root { + return Err(Outcome::Ignore(IgnoreReason::FinalizedNotAncestor)); + } + Ok(()) +} + +/// The fork-choice head's post-state when it is a PQ state: what the operation +/// topics check against, as the specification and `super::operations` do. +fn pq_head_state(store: &Store) -> Result, Outcome> { + let Some((_slot, head_root)) = store.beacon_head() else { + return Err(Outcome::Ignore(IgnoreReason::StateUnavailable)); + }; + let Ok(Some(state)) = store.get_state(&head_root) else { + return Err(Outcome::Ignore(IgnoreReason::StateUnavailable)); + }; + if !matches!(*state, BeaconState::Pq(_)) { + return Err(Outcome::Ignore(IgnoreReason::StateUnavailable)); + } + Ok(state) +} + +// --------------------------------------------------------------------------- +// beacon_aggregate_and_proof +// --------------------------------------------------------------------------- + +/// The keys [`SeenAggregates`] records a PQ aggregate under, or `None` when its +/// committee bits do not name exactly one committee. +pub fn aggregate_key(signed: &pq::SignedAggregateAndProof) -> Option { + let aggregate = &signed.message.aggregate; + let [committee_index] = get_committee_indices(&aggregate.committee_bits)[..] else { + return None; + }; + Some(AggregateKey { + data_root: aggregate.data.hash_tree_root(), + committee_index, + target_epoch: aggregate.data.target.epoch, + aggregator_index: signed.message.aggregator_index, + }) +} + +/// The aggregate's bits, expanded. Only after [`aggregate_cheap_checks`] has +/// bounded their length. +pub fn aggregation_bits(signed: &pq::SignedAggregateAndProof) -> Vec { + let bits = &signed.message.aggregate.aggregation_bits; + (0..bits.len()) + .map(|index| bits.get(index).unwrap_or(false)) + .collect() +} + +/// The rules that read only the message, the clock and the seen cache: the +/// gloas aggregate's, with no selection proof to read. +pub fn aggregate_cheap_checks( + seen: &SeenAggregates, + store: &Store, + signed: &pq::SignedAggregateAndProof, + now_ms: u64, +) -> Result<(), Outcome> { + let config = store.config(); + let aggregate = &signed.message.aggregate; + let data = &aggregate.data; + // [REJECT] The bitfield fits every committee of a slot, read without + // expanding it: everything below expands it. + if aggregate.aggregation_bits.len() > MAX_AGGREGATION_BITS { + return Err(Outcome::Reject(RejectReason::BitsLength)); + } + // [REJECT] The aggregate attestation's data index is 0 or 1. + if data.index > 1 { + return Err(Outcome::Reject(RejectReason::DataIndexOutOfRange)); + } + // [REJECT] Exactly one committee is specified by the committee bits. + let Some(key) = aggregate_key(signed) else { + return Err(Outcome::Reject(RejectReason::CommitteeBits)); + }; + let bits = aggregation_bits(signed); + // [IGNORE] Not a covered superset, and [IGNORE] the first for this epoch + // and aggregator. + seen.verdict_parts(&key, &bits)?; + // [IGNORE] Not from a future slot. + if is_future_slot(&config, data.slot, now_ms) { + return Err(Outcome::Ignore(IgnoreReason::FutureSlot)); + } + // [IGNORE] The current or the previous epoch. + let attestation_epoch = compute_epoch_at_slot(data.slot); + if !is_current_or_previous_epoch(&config, attestation_epoch, now_ms) { + return Err(Outcome::Ignore(IgnoreReason::OutsideEpochWindow)); + } + // [REJECT] The epoch matches its target. + if data.target.epoch != attestation_epoch { + return Err(Outcome::Reject(RejectReason::EpochMismatch)); + } + // [REJECT] The aggregate has participants. One named committee, so a set + // bit is a real attester whichever committee it is. + if !bits.iter().any(|&bit| bit) { + return Err(Outcome::Reject(RejectReason::NoParticipants)); + } + Ok(()) +} + +/// The rules that need the vote block's state. `Ok` is `Accept`, carrying the +/// attesting indices the proof covers. +pub fn aggregate_stateful_checks( + store: &Store, + signed: &pq::SignedAggregateAndProof, +) -> Result, Outcome> { + let aggregate_and_proof = &signed.message; + let aggregate = &aggregate_and_proof.aggregate; + let data = &aggregate.data; + let target_epoch = data.target.epoch; + let config = store.config(); + let state = pq_vote_state( + store, + data.beacon_block_root, + compute_epoch_at_slot(data.slot), + )?; + + // The pubkey-only signature first, before any committee derivation. + let aggregator_index = aggregate_and_proof.aggregator_index; + let Ok(aggregator) = state.pq_validator(aggregator_index) else { + return Err(Outcome::Reject(RejectReason::UnknownValidator)); + }; + // [Modified in PQ] [REJECT] The aggregator's XMSS signature at the slot's + // aggregate leaf. + let domain = + get_domain_from_schedule(&config, &state, DOMAIN_AGGREGATE_AND_PROOF, target_epoch); + let signing_root = compute_signing_root(aggregate_and_proof.hash_tree_root(), domain); + let leaf_index = compute_xmss_leaf_index(data.slot, LEAF_OFFSET_AGGREGATE_AND_PROOF) + .map_err(|_| Outcome::Reject(RejectReason::AggregatorSignature))?; + if !xmss_verify( + &aggregator.pubkey(), + leaf_index, + signing_root, + &signed.signature, + ) { + return Err(Outcome::Reject(RejectReason::AggregatorSignature)); + } + + let committees = store.committee_cache(); + let epoch_committees = committees.committees(&state, target_epoch); + let committee_index = aggregate_key(signed) + .expect("aggregate_cheap_checks rejects an aggregate with no single committee") + .committee_index; + // [REJECT] The committee index is within the expected range. + if committee_index >= epoch_committees.committees_per_slot() { + return Err(Outcome::Reject(RejectReason::CommitteeIndex)); + } + let Ok(committee) = epoch_committees.committee(data.slot, committee_index) else { + return Err(Outcome::Ignore(IgnoreReason::Internal)); + }; + // [REJECT] The number of aggregation bits matches the committee size. + if aggregate.aggregation_bits.len() != committee.len() { + return Err(Outcome::Reject(RejectReason::BitsLength)); + } + // [Modified in PQ] [REJECT] The validator is one of the committee's + // aggregators: a member of its aggregation committee. + let aggregators = get_aggregation_committee(&state, data.slot, committee_index) + .map_err(|_| Outcome::Ignore(IgnoreReason::StateUnavailable))?; + if !aggregators.contains(&aggregator_index) { + return Err(Outcome::Reject(RejectReason::NotAggregator)); + } + // [Modified in PQ] [REJECT] The aggregate stark is valid. + let statements = get_attestation_statements(&state, aggregate, &committees) + .map_err(|_| Outcome::Reject(RejectReason::AggregateProof))?; + if !verify_aggregates(&statements, &aggregate_and_proof.proof) { + return Err(Outcome::Reject(RejectReason::AggregateProof)); + } + let attesting_indices = get_attesting_indices(&state, aggregate, &committees) + .map_err(|_| Outcome::Ignore(IgnoreReason::Internal))?; + + check_ancestry( + store, + &state, + data.beacon_block_root, + target_epoch, + data.target.root, + )?; + // The attested payload status is consistent with the block's payload. + verify_attestation_payload_status(store, data)?; + Ok(attesting_indices) +} + +// --------------------------------------------------------------------------- +// beacon_attestation_{subnet_id} +// --------------------------------------------------------------------------- + +/// The rules that read only the message, the clock and the seen cache: the +/// gloas attestation's. +pub fn attestation_cheap_checks( + seen: &SeenAttestations, + store: &Store, + attestation: &pq::SingleAttestation, + now_ms: u64, +) -> Result<(), Outcome> { + let config = store.config(); + let data = &attestation.data; + // [IGNORE] No other valid attestation seen for this target epoch and + // validator. + if seen.contains(data.target.epoch, attestation.attester_index) { + return Err(Outcome::Ignore(IgnoreReason::AlreadySeen)); + } + // [REJECT] The attestation's data index is 0 or 1. + if data.index > 1 { + return Err(Outcome::Reject(RejectReason::DataIndexOutOfRange)); + } + // [IGNORE] Not from a future slot. + if is_future_slot(&config, data.slot, now_ms) { + return Err(Outcome::Ignore(IgnoreReason::FutureSlot)); + } + // [IGNORE] The current or the previous epoch. + let attestation_epoch = compute_epoch_at_slot(data.slot); + if !is_current_or_previous_epoch(&config, attestation_epoch, now_ms) { + return Err(Outcome::Ignore(IgnoreReason::OutsideEpochWindow)); + } + // [REJECT] The epoch matches its target. + if data.target.epoch != attestation_epoch { + return Err(Outcome::Reject(RejectReason::EpochMismatch)); + } + Ok(()) +} + +/// The rules that need the vote block's state. +pub fn attestation_stateful_checks( + store: &Store, + attestation: &pq::SingleAttestation, + subnet_id: u64, +) -> Outcome { + match attestation_stateful_rules(store, attestation, subnet_id) { + Ok(()) => Outcome::Accept, + Err(outcome) => outcome, + } +} + +fn attestation_stateful_rules( + store: &Store, + attestation: &pq::SingleAttestation, + subnet_id: u64, +) -> Result<(), Outcome> { + let data = &attestation.data; + let target_epoch = data.target.epoch; + let config = store.config(); + let state = pq_vote_state( + store, + data.beacon_block_root, + compute_epoch_at_slot(data.slot), + )?; + + // The pubkey-only signature first, before any committee derivation. + let Ok(attester) = state.pq_validator(attestation.attester_index) else { + return Err(Outcome::Reject(RejectReason::UnknownValidator)); + }; + // [Modified in PQ] [REJECT] The attester's XMSS signature at the target + // epoch's attestation leaf. + let domain = get_domain_from_schedule(&config, &state, DOMAIN_BEACON_ATTESTER, target_epoch); + let signing_root = compute_signing_root(data.hash_tree_root(), domain); + let leaf_index = compute_xmss_attestation_leaf_index(target_epoch) + .map_err(|_| Outcome::Reject(RejectReason::BadSignature))?; + if !xmss_verify( + &attester.pubkey(), + leaf_index, + signing_root, + &attestation.signature, + ) { + return Err(Outcome::Reject(RejectReason::BadSignature)); + } + + let committees = store.committee_cache(); + let epoch_committees = committees.committees(&state, target_epoch); + // [REJECT] The committee index is within the expected range. + if attestation.committee_index >= epoch_committees.committees_per_slot() { + return Err(Outcome::Reject(RejectReason::CommitteeIndex)); + } + // [Modified in PQ] [REJECT] The attestation is for the correct subnet, one + // of `ATTESTATION_SUBNET_COUNT_PQ`. + let expected_subnet = compute_subnet_for_attestation( + epoch_committees.committees_per_slot(), + data.slot, + attestation.committee_index, + &config, + ); + if expected_subnet != subnet_id { + return Err(Outcome::Reject(RejectReason::WrongSubnet)); + } + let Ok(committee) = epoch_committees.committee(data.slot, attestation.committee_index) else { + return Err(Outcome::Ignore(IgnoreReason::Internal)); + }; + // [REJECT] The attester is a member of the committee. + if !committee.contains(&attestation.attester_index) { + return Err(Outcome::Reject(RejectReason::NotInCommittee)); + } + + check_ancestry( + store, + &state, + data.beacon_block_root, + target_epoch, + data.target.root, + )?; + // The attested payload status is consistent with the block's payload. + verify_attestation_payload_status(store, data) +} + +// --------------------------------------------------------------------------- +// proposer_slashing, attester_slashing +// --------------------------------------------------------------------------- + +/// The rules that read only the message and the seen record. +pub fn proposer_slashing_cheap_checks( + seen: &SeenOperations, + slashing: &pq::ProposerSlashing, +) -> Result<(), Outcome> { + let header_1 = &slashing.signed_header_1.message; + let header_2 = &slashing.signed_header_2.message; + // [IGNORE] The first valid proposer slashing for this proposer. + if !seen.proposer_slashing_is_new(header_1.proposer_index) { + return Err(Outcome::Ignore(IgnoreReason::AlreadySeen)); + } + // [REJECT] The header slots and proposer indices match, and the headers + // differ. + if header_1.slot != header_2.slot + || header_1.proposer_index != header_2.proposer_index + || header_1 == header_2 + { + return Err(Outcome::Reject(RejectReason::InvalidOperation)); + } + Ok(()) +} + +/// The rules that read the head state. +pub fn proposer_slashing_stateful_checks( + store: &Store, + slashing: &pq::ProposerSlashing, +) -> Outcome { + let state = match pq_head_state(store) { + Ok(state) => state, + Err(outcome) => return outcome, + }; + let proposer_index = slashing.signed_header_1.message.proposer_index; + // [REJECT] The proposer index is a valid validator index. + let Ok(proposer) = state.pq_validator(proposer_index) else { + return Outcome::Reject(RejectReason::UnknownValidator); + }; + // [REJECT] The proposer is slashable. + if !is_slashable_validator(&proposer.record, get_current_epoch(&state)) { + return Outcome::Reject(RejectReason::InvalidOperation); + } + // [Modified in PQ] [REJECT] Both XMSS signatures, each at its header slot's + // proposer leaf. + let pubkey = proposer.pubkey(); + for signed_header in [&slashing.signed_header_1, &slashing.signed_header_2] { + let valid = stf::pq::header_signature_valid( + &state, + &pubkey, + &signed_header.message, + &signed_header.signature, + ); + if !matches!(valid, Ok(true)) { + return Outcome::Reject(RejectReason::BadSignature); + } + } + Outcome::Accept +} + +/// The indices both of a PQ attester slashing's attestations name: the +/// validators it slashes. +pub fn slashed_indices(slashing: &pq::AttesterSlashing) -> Vec { + let first: std::collections::HashSet = slashing + .attestation_1 + .attesting_indices + .iter() + .copied() + .collect(); + slashing + .attestation_2 + .attesting_indices + .iter() + .copied() + .filter(|index| first.contains(index)) + .collect() +} + +/// The rules that read only the message and the seen record. +pub fn attester_slashing_cheap_checks( + seen: &SeenOperations, + slashing: &pq::AttesterSlashing, +) -> Result<(), Outcome> { + // [IGNORE] At least one overlapping index is unseen. + if !seen.attester_slashing_is_new(&slashed_indices(slashing)) { + return Err(Outcome::Ignore(IgnoreReason::AlreadySeen)); + } + // [REJECT] The data is a double or surround vote. + if !is_slashable_attestation_data(&slashing.attestation_1.data, &slashing.attestation_2.data) { + return Err(Outcome::Reject(RejectReason::InvalidOperation)); + } + Ok(()) +} + +/// The rules that read the head state. In `super::operations`'s order: the +/// cheap REJECTs before any proof is verified. +pub fn attester_slashing_stateful_checks( + store: &Store, + slashing: &pq::AttesterSlashing, +) -> Outcome { + let state = match pq_head_state(store) { + Ok(state) => state, + Err(outcome) => return outcome, + }; + let attestations = [&slashing.attestation_1, &slashing.attestation_2]; + let validator_count = state.validator_count() as u64; + // [REJECT] Every index is a validator. + if attestations.iter().any(|attestation| { + attestation + .attesting_indices + .iter() + .any(|index| *index >= validator_count) + }) { + return Outcome::Reject(RejectReason::UnknownValidator); + } + // [REJECT] At least one overlapping validator is slashable. + let current_epoch = get_current_epoch(&state); + let any_slashable = slashed_indices(slashing).into_iter().any(|index| { + state + .validator(index) + .is_ok_and(|validator| is_slashable_validator(validator, current_epoch)) + }); + if !any_slashable { + return Outcome::Reject(RejectReason::InvalidOperation); + } + // [Modified in PQ] [REJECT] Each indexed attestation is non-empty, sorted + // and unique, and its proof proves its attesters' signatures. + if !attestations + .iter() + .all(|attestation| matches!(is_valid_indexed_attestation(&state, attestation), Ok(true))) + { + return Outcome::Reject(RejectReason::InvalidOperation); + } + Outcome::Accept +} + +// --------------------------------------------------------------------------- +// payload_attestation_message +// --------------------------------------------------------------------------- + +/// The gloas rules that read only the message, the clock and the seen cache. +pub fn payload_attestation_cheap_checks( + seen: &SeenPayloadAttestations, + store: &Store, + message: &pq::PayloadAttestationMessage, + now_ms: u64, +) -> Result<(), Outcome> { + super::payload_attestation::cheap_checks_parts( + seen, + store, + message.data.slot, + message.validator_index, + now_ms, + ) +} + +/// The gloas rules that need the head state, with `[Modified in PQ]` the +/// validator's XMSS signature at the slot's PTC leaf. +pub fn payload_attestation_stateful_checks( + store: &Store, + message: &pq::PayloadAttestationMessage, +) -> Outcome { + let data = &message.data; + super::payload_attestation::stateful_checks_with( + store, + message.validator_index, + data, + |state| matches!(state, BeaconState::Pq(_)), + |state, validator_index| { + let Ok(validator) = state.pq_validator(validator_index) else { + return false; + }; + let domain = get_domain( + state, + DOMAIN_PTC_ATTESTER, + Some(compute_epoch_at_slot(data.slot)), + ); + let signing_root = compute_signing_root(data.hash_tree_root(), domain); + let Ok(leaf_index) = compute_xmss_leaf_index(data.slot, LEAF_OFFSET_PTC_ATTESTER) + else { + return false; + }; + xmss_verify( + &validator.pubkey(), + leaf_index, + signing_root, + &message.signature, + ) + }, + ) +} + +// --------------------------------------------------------------------------- +// execution_payload +// --------------------------------------------------------------------------- + +/// `verify_execution_requests_limits` (modified in PQ): gloas's four lists, +/// then the three PQ adds. EIP-8483's registration lists are gone. +pub fn execution_requests_within_limits(requests: &pq::ExecutionRequests) -> bool { + requests.withdrawals.len() <= preset::MAX_WITHDRAWAL_REQUESTS_PER_PAYLOAD + && requests.consolidations.len() <= preset::MAX_CONSOLIDATION_REQUESTS_PER_PAYLOAD + && requests.builder_deposits.len() as u64 + <= preset::MAX_BUILDER_DEPOSIT_REQUESTS_PER_PAYLOAD + && requests.builder_exits.len() as u64 <= preset::MAX_BUILDER_EXIT_REQUESTS_PER_PAYLOAD + && requests.validator_preregistrations.len() as u64 + <= preset::MAX_VALIDATOR_PREREGISTRATION_REQUESTS_PER_PAYLOAD + && requests.builder_preregistrations.len() as u64 + <= preset::MAX_BUILDER_PREREGISTRATION_REQUESTS_PER_PAYLOAD + && requests.voluntary_exits.len() as u64 <= preset::MAX_VOLUNTARY_EXIT_REQUESTS_PER_PAYLOAD +} + +/// The gloas envelope rules that read only the message and the seen cache, +/// with PQ's execution request limits. +pub fn envelope_cheap_checks( + seen: &SeenEnvelopes, + store: &Store, + envelope: &pq::SignedExecutionPayloadEnvelope, +) -> Result<(), Outcome> { + let message = &envelope.message; + // [IGNORE] The first envelope for this block and builder. + if seen.contains(message.beacon_block_root, message.builder_index) { + return Err(Outcome::Ignore(IgnoreReason::AlreadySeen)); + } + // [IGNORE] Not from before the finalized epoch. + if message.payload.slot_number < finalized_start_slot(store) { + return Err(Outcome::Ignore(IgnoreReason::Finalized)); + } + // [REJECT] The withdrawals and execution requests are within their limits. + if message.payload.withdrawals.len() > preset::MAX_WITHDRAWALS_PER_PAYLOAD { + return Err(Outcome::Reject(RejectReason::TooManyWithdrawals)); + } + if !execution_requests_within_limits(&message.execution_requests) { + return Err(Outcome::Reject(RejectReason::OperationLimit)); + } + Ok(()) +} + +/// The gloas envelope rules that need the block, with `[Modified in PQ]` the +/// builder's (or self-building proposer's) XMSS signature. +pub fn envelope_stateful_checks( + store: &Store, + envelope: &pq::SignedExecutionPayloadEnvelope, +) -> Outcome { + let message = &envelope.message; + let payload = &message.payload; + let block = match judgeable_block_gloas(store, &message.beacon_block_root) { + Ok(block) => block, + Err(outcome) => return outcome, + }; + let block = match &block { + SignedBeaconBlock::Pq(block) => &block.message, + SignedBeaconBlock::Phase0(_) + | SignedBeaconBlock::Altair(_) + | SignedBeaconBlock::Bellatrix(_) + | SignedBeaconBlock::Capella(_) + | SignedBeaconBlock::Deneb(_) + | SignedBeaconBlock::Electra(_) + | SignedBeaconBlock::Fulu(_) => return Outcome::Reject(RejectReason::BlockNotGloas), + // A PQ-shaped envelope for a gloas or EIP-8483 block: around the + // boundary, the one digest's topic may carry the other's. Not + // something this envelope's rules can judge. + SignedBeaconBlock::Gloas(_) | SignedBeaconBlock::Eip8483(_) => { + return Outcome::Ignore(IgnoreReason::UnsupportedFork); + } + SignedBeaconBlock::Lean(_) => lean_block_unreachable("pq::envelope_stateful_checks"), + }; + let bid = &block.body.signed_execution_payload_bid.message; + // [REJECT] The envelope matches its block's slot and bid. + if block.slot != payload.slot_number { + return Outcome::Reject(RejectReason::SlotMismatch); + } + if message.builder_index != bid.builder_index { + return Outcome::Reject(RejectReason::BuilderIndexMismatch); + } + if payload.block_hash != bid.block_hash { + return Outcome::Reject(RejectReason::BlockHashMismatch); + } + if message.execution_requests.hash_tree_root() != bid.execution_requests_root { + return Outcome::Reject(RejectReason::ExecutionRequestsRootMismatch); + } + let Some(state) = store.cached_state(CacheKey::BlockState(message.beacon_block_root)) else { + return Outcome::Queue(QueueReason::StateNotCached); + }; + // [Modified in PQ] [REJECT] The signature is valid. + if !matches!( + stf::pq::verify_execution_payload_envelope_signature(&state, envelope), + Ok(true) + ) { + return Outcome::Reject(RejectReason::BadSignature); + } + Outcome::Accept +} + +// --------------------------------------------------------------------------- +// beacon_block +// --------------------------------------------------------------------------- + +/// `verify_block_body_operation_limits` (modified in PQ): the attestation +/// count is `MAX_ATTESTATIONS_PQ`, and exits, BLS changes and payload +/// attestations are no longer in the body. +pub fn body_operations_within_limits(body: &pq::BeaconBlockBody) -> bool { + body.proposer_slashings.len() <= preset::MAX_PROPOSER_SLASHINGS + && body.attester_slashings.len() <= preset::MAX_ATTESTER_SLASHINGS_ELECTRA + && body.attestations.len() as u64 <= preset::MAX_ATTESTATIONS_PQ + && body.deposits.is_empty() +} + +/// The gloas block rules that read only the block, the clock and the store's +/// metadata, with PQ's operation and execution request limits. The proposer's +/// XMSS signature is the precheck's (`crate::beacon::precheck`). +pub fn block_cheap_checks( + seen: &SeenBlocks, + store: &Store, + signed_block: &pq::SignedBeaconBlock, + now_ms: u64, +) -> Result<(), Outcome> { + let config = store.config(); + let block = &signed_block.message; + // [IGNORE] The first block with a valid signature for its slot and + // proposer. + if seen.contains(block.slot, block.proposer_index) { + return Err(Outcome::Ignore(IgnoreReason::AlreadySeen)); + } + // [IGNORE] Not from a future slot. + if is_future_slot(&config, block.slot, now_ms) { + return Err(Outcome::Ignore(IgnoreReason::FutureSlot)); + } + // [IGNORE] From a slot greater than the latest finalized slot. + if block.slot <= finalized_start_slot(store) { + return Err(Outcome::Ignore(IgnoreReason::Finalized)); + } + // [Modified in PQ] [REJECT] The body's operation counts, and the parent + // execution requests' counts, are within their limits. + if !body_operations_within_limits(&block.body) + || !execution_requests_within_limits(&block.body.parent_execution_requests) + { + return Err(Outcome::Reject(RejectReason::OperationLimit)); + } + // [REJECT] The bid's blob KZG commitment count is within the epoch's limit. + let bid = &block.body.signed_execution_payload_bid.message; + let max_blobs = config.max_blobs_per_block(compute_epoch_at_slot(block.slot)); + if bid.blob_kzg_commitments.len() as u64 > max_blobs { + return Err(Outcome::Reject(RejectReason::TooManyBlobs)); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use std::sync::Arc; + + use ethlambda_crypto::{lean_xmss, stark}; + + use super::*; + use crate::beacon::builder_market::SignedBid; + use crate::beacon::constants::{ + DOMAIN_BEACON_PROPOSER, DOMAIN_PROPOSER_PREFERENCES, LEAF_OFFSET_BEACON_PROPOSER, + LEAF_OFFSET_PROPOSER_PREFERENCES, + }; + use crate::beacon::containers::gloas; + use crate::beacon::containers::shared::{AttestationData, BeaconBlockHeader, Checkpoint, Fork}; + use crate::beacon::fork::ForkName; + use crate::beacon::gossip::execution_payload_bid::BidRules; + use crate::beacon::gossip::proposer_preferences::PreferencesRules; + use crate::beacon::gossip::test_support::{ + seen_aggregates, seen_attestations, slot_start_ms, store_with_config, + }; + use crate::beacon::helpers::accessors::get_beacon_committee; + use crate::beacon::helpers::gloas::get_ptc; + use crate::beacon::helpers::misc::compute_domain; + use crate::beacon::helpers::pq::initialize_aggregation_committee_window; + use crate::beacon::helpers::test_state::with_validators_at; + use crate::beacon::primitives::{BlsPubkey, XmssPubkey, XmssSignature}; + + /// The vote block, which is also the head: the anchor root, so the store's + /// head is it without a fork-choice run. + const BLOCK: Root = Root::ZERO; + + fn config() -> Config { + Config::mainnet() + .with_fork_epoch(ForkName::Gloas, 0) + .with_fork_epoch(ForkName::Eip8483, 0) + .with_fork_epoch(ForkName::Pq, 0) + } + + /// A key real over `first..=last` alone: a range key, a few hashes to + /// build. Its public key depends on the range, so a validator signs at the + /// leaves of one range only. + fn key(seed: u64, first: u64, last: u64) -> lean_xmss::SecretKey { + let mut bytes = [0u8; 32]; + bytes[..8].copy_from_slice(&(seed + 1).to_le_bytes()); + lean_xmss::range_key(&bytes, first as u32, last as u32) + } + + fn sign(secret: &lean_xmss::SecretKey, root: Root, leaf: u64) -> XmssSignature { + let signature = secret + .sign(&root.0, leaf as u32) + .expect("a leaf of the range"); + XmssSignature( + signature + .to_bytes() + .try_into() + .expect("a leanXMSS signature is XMSS_SIGNATURE_SIZE bytes"), + ) + } + + fn set_key(state: &mut BeaconState, index: ValidatorIndex, secret: &lean_xmss::SecretKey) { + let BeaconState::Pq(inner) = state else { + panic!("a PQ test state") + }; + inner + .validators + .get_mut(index as usize) + .expect("index is in the registry") + .record + .pubkey = XmssPubkey(secret.public_key().to_bytes()).to_request_key(); + } + + /// A PQ state at slot 32 (epoch 1) with 64 validators, every slot's + /// committee two strong, under the schedule's own versions, its + /// aggregation committee window filled. + fn pq_state(config: &Config) -> BeaconState { + let mut state = with_validators_at(ForkName::Pq, 64); + *state.fork_mut() = Fork { + previous_version: config.eip8483_fork_version, + current_version: config.pq_fork_version, + epoch: 0, + }; + let window = initialize_aggregation_committee_window(&state).expect("a full window"); + let BeaconState::Pq(inner) = &mut state else { + unreachable!("built as a PQ state") + }; + inner.aggregation_committee_window = window; + state.apply_pending_mutations(); + state + } + + fn pq_block(slot: Slot) -> SignedBeaconBlock { + SignedBeaconBlock::Pq(Box::new(pq::SignedBeaconBlock { + message: pq::BeaconBlock { + slot, + proposer_index: 0, + parent_root: Root::repeat_byte(9), + state_root: Root::ZERO, + body: pq::BeaconBlockBody::empty(), + }, + signature: Default::default(), + })) + } + + /// A store whose vote block and head is [`BLOCK`], with `state` as its + /// post-state. + fn store_with(config: &Config, state: &BeaconState) -> Store { + let mut store = store_with_config(0, config.clone()); + store + .insert_pending_block(BLOCK, pq_block(state.slot())) + .expect("insert the vote block"); + // Cached only: every rule reads states through the cache, and + // `has_state` asks it first. + let mut cached = state.clone(); + cached.apply_pending_mutations(); + store.cache_state(CacheKey::BlockState(BLOCK), Arc::new(cached)); + store + } + + /// A vote at the state's slot for [`BLOCK`], its own target. + fn data(state: &BeaconState) -> AttestationData { + AttestationData { + slot: state.slot(), + index: 0, + beacon_block_root: BLOCK, + source: Checkpoint::default(), + target: Checkpoint { + epoch: compute_epoch_at_slot(state.slot()), + root: BLOCK, + }, + } + } + + fn attestation_root(config: &Config, state: &BeaconState, data: &AttestationData) -> Root { + let domain = + get_domain_from_schedule(config, state, DOMAIN_BEACON_ATTESTER, data.target.epoch); + compute_signing_root(data.hash_tree_root(), domain) + } + + #[test] + fn a_pq_attestation_takes_one_of_sixteen_subnets() { + let config = config(); + assert_eq!(config.attestation_subnet_count_pq, 16); + // Slots within the epoch under either preset. + assert_eq!(compute_subnet_for_attestation(5, 3, 0, &config), 15); + assert_eq!(compute_subnet_for_attestation(4, 4, 0, &config), 0); + assert_eq!(compute_subnet_for_attestation(4, 3, 5, &config), 1); + } + + /// A vote block far behind the attestation's epoch is not advanced: that + /// would replay every slot between on any peer's say-so. + #[test] + fn a_pq_vote_for_a_block_far_behind_is_ignored_unadvanced() { + let config = config(); + let state = pq_state(&config); + let store = store_with(&config, &state); + let far = compute_epoch_at_slot(state.slot()) + MAX_VOTE_STATE_ADVANCE_EPOCHS + 1; + let mut data = data(&state); + data.slot = compute_start_slot_at_epoch(far); + data.target.epoch = far; + let attestation = pq::SingleAttestation { + committee_index: 0, + attester_index: 0, + data, + signature: XmssSignature::default(), + }; + assert_eq!( + attestation_stateful_checks(&store, &attestation, 0), + Outcome::Ignore(IgnoreReason::StateUnavailable) + ); + assert!( + store + .cached_state(CacheKey::CheckpointState { + epoch: far, + root: BLOCK + }) + .is_none(), + "nothing was advanced" + ); + } + + #[test] + fn a_pq_attestation_is_checked_at_its_target_epochs_leaf() { + let config = config(); + let mut state = pq_state(&config); + let data = data(&state); + let attester = get_beacon_committee(&state, data.slot, 0).unwrap()[0]; + let leaf = compute_xmss_attestation_leaf_index(data.target.epoch).unwrap(); + let secret = key(attester, leaf, leaf + 1); + set_key(&mut state, attester, &secret); + let store = store_with(&config, &state); + let committees_per_slot = store + .committee_cache() + .committees(&state, data.target.epoch) + .committees_per_slot(); + let subnet = compute_subnet_for_attestation(committees_per_slot, data.slot, 0, &config); + let root = attestation_root(&config, &state, &data); + let attestation = |leaf| pq::SingleAttestation { + committee_index: 0, + attester_index: attester, + data, + signature: sign(&secret, root, leaf), + }; + + assert_eq!( + attestation_stateful_checks(&store, &attestation(leaf), subnet), + Outcome::Accept + ); + assert_eq!( + attestation_stateful_checks(&store, &attestation(leaf + 1), subnet), + Outcome::Reject(RejectReason::BadSignature), + "signed at the next leaf" + ); + assert_eq!( + attestation_stateful_checks(&store, &attestation(leaf), (subnet + 1) % 16), + Outcome::Reject(RejectReason::WrongSubnet) + ); + } + + /// The node does not follow PQ, so on a live node the vote block's state + /// is never a PQ one: the message is ignored, not rejected. + #[test] + fn a_pq_vote_for_a_block_without_a_pq_state_is_ignored() { + let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, 0); + let state = with_validators_at(ForkName::Gloas, 64); + let mut store = store_with_config(0, config); + store + .insert_pending_block(BLOCK, pq_block(state.slot())) + .unwrap(); + store.cache_state(CacheKey::BlockState(BLOCK), Arc::new(state.clone())); + let attestation = pq::SingleAttestation { + committee_index: 0, + attester_index: 0, + data: data(&state), + signature: XmssSignature::default(), + }; + assert_eq!( + attestation_stateful_checks(&store, &attestation, 0), + Outcome::Ignore(IgnoreReason::StateUnavailable) + ); + } + + /// An aggregate of the slot's first committee member's vote, by + /// `aggregator`, whose proof is `proof` and whose own signature is at + /// `aggregator_leaf`. + fn aggregate( + state: &BeaconState, + config: &Config, + aggregator: (ValidatorIndex, &lean_xmss::SecretKey), + aggregator_leaf: u64, + proof: Vec, + ) -> pq::SignedAggregateAndProof { + let data = data(state); + let committee_len = get_beacon_committee(state, data.slot, 0).unwrap().len(); + let mut aggregation_bits = pq::AggregationBits::with_length(committee_len); + aggregation_bits.set(0, true).unwrap(); + let mut committee_bits = pq::CommitteeBits::new(); + committee_bits.set(0, true).unwrap(); + let message = pq::AggregateAndProof { + aggregator_index: aggregator.0, + aggregate: pq::Attestation { + aggregation_bits, + data, + committee_bits, + }, + proof: proof.try_into().expect("far below MAX_STARK_PROOF_SIZE"), + }; + let domain = + get_domain_from_schedule(config, state, DOMAIN_AGGREGATE_AND_PROOF, data.target.epoch); + let root = compute_signing_root(message.hash_tree_root(), domain); + pq::SignedAggregateAndProof { + signature: sign(aggregator.1, root, aggregator_leaf), + message, + } + } + + #[test] + fn a_pq_aggregate_needs_an_aggregator_its_signature_and_its_proof() { + let config = config(); + let mut state = pq_state(&config); + let data = data(&state); + let committee = get_beacon_committee(&state, data.slot, 0).unwrap(); + let aggregators = get_aggregation_committee(&state, data.slot, 0).unwrap(); + let (attester, aggregator) = (committee[0], committee[1]); + assert!(aggregators.contains(&aggregator)); + let outsider = (0..64) + .find(|index| !aggregators.contains(index)) + .expect("a validator outside the aggregation committee"); + + let vote_leaf = compute_xmss_attestation_leaf_index(data.target.epoch).unwrap(); + let attester_key = key(attester, vote_leaf, vote_leaf); + let aggregate_leaf = + compute_xmss_leaf_index(data.slot, LEAF_OFFSET_AGGREGATE_AND_PROOF).unwrap(); + let aggregator_key = key(aggregator, aggregate_leaf, aggregate_leaf + 1); + let outsider_key = key(outsider, aggregate_leaf, aggregate_leaf); + set_key(&mut state, attester, &attester_key); + set_key(&mut state, aggregator, &aggregator_key); + set_key(&mut state, outsider, &outsider_key); + let store = store_with(&config, &state); + + let vote_root = attestation_root(&config, &state, &data); + let vote = sign(&attester_key, vote_root, vote_leaf); + let statement = stark::AggregateStatement { + pubkeys: vec![attester_key.public_key().to_bytes()], + leaf_index: vote_leaf, + message: vote_root.0, + }; + let proof = stark::prove_aggregates(&[statement], &[vec![vote.0.to_vec()]]).unwrap(); + + let valid = aggregate( + &state, + &config, + (aggregator, &aggregator_key), + aggregate_leaf, + proof.clone(), + ); + assert_eq!( + aggregate_stateful_checks(&store, &valid), + Ok(vec![attester]) + ); + + let wrong_leaf = aggregate( + &state, + &config, + (aggregator, &aggregator_key), + aggregate_leaf + 1, + proof.clone(), + ); + assert_eq!( + aggregate_stateful_checks(&store, &wrong_leaf), + Err(Outcome::Reject(RejectReason::AggregatorSignature)) + ); + + let not_aggregator = aggregate( + &state, + &config, + (outsider, &outsider_key), + aggregate_leaf, + proof, + ); + assert_eq!( + aggregate_stateful_checks(&store, ¬_aggregator), + Err(Outcome::Reject(RejectReason::NotAggregator)) + ); + + let unproven = aggregate( + &state, + &config, + (aggregator, &aggregator_key), + aggregate_leaf, + Vec::new(), + ); + assert_eq!( + aggregate_stateful_checks(&store, &unproven), + Err(Outcome::Reject(RejectReason::AggregateProof)) + ); + } + + #[test] + fn a_pq_aggregate_names_one_committee_and_carries_a_payload_flag() { + let config = config(); + let state = pq_state(&config); + let store = store_with(&config, &state); + let now = slot_start_ms(&store, state.slot()); + let secret = key(1, 0, 0); + let valid = aggregate(&state, &config, (1, &secret), 0, Vec::new()); + assert_eq!( + aggregate_cheap_checks(&seen_aggregates(), &store, &valid, now), + Ok(()) + ); + + let mut two_committees = valid.clone(); + two_committees + .message + .aggregate + .committee_bits + .set(1, true) + .unwrap(); + assert_eq!( + aggregate_cheap_checks(&seen_aggregates(), &store, &two_committees, now), + Err(Outcome::Reject(RejectReason::CommitteeBits)) + ); + + let mut flag_two = valid.clone(); + flag_two.message.aggregate.data.index = 2; + assert_eq!( + aggregate_cheap_checks(&seen_aggregates(), &store, &flag_two, now), + Err(Outcome::Reject(RejectReason::DataIndexOutOfRange)) + ); + + // Recorded, a second aggregate with the same bits is covered. + let mut seen = seen_aggregates(); + let key = aggregate_key(&valid).unwrap(); + assert!(seen.record_parts(&key, &aggregation_bits(&valid))); + assert_eq!( + aggregate_cheap_checks(&seen, &store, &valid, now), + Err(Outcome::Ignore(IgnoreReason::CoveredBits)) + ); + } + + #[test] + fn a_pq_attestation_with_a_payload_flag_past_one_is_rejected() { + let config = config(); + let state = pq_state(&config); + let store = store_with(&config, &state); + let mut data = data(&state); + data.index = 2; + let attestation = pq::SingleAttestation { + committee_index: 0, + attester_index: 0, + data, + signature: XmssSignature::default(), + }; + let now = slot_start_ms(&store, state.slot()); + assert_eq!( + attestation_cheap_checks(&seen_attestations(), &store, &attestation, now), + Err(Outcome::Reject(RejectReason::DataIndexOutOfRange)) + ); + } + + #[test] + fn a_pq_proposer_slashing_is_signed_at_the_slots_proposer_leaf() { + let config = config(); + let mut state = pq_state(&config); + let proposer = 3; + let slot = state.slot() - 1; + let leaf = compute_xmss_leaf_index(slot, LEAF_OFFSET_BEACON_PROPOSER).unwrap(); + let secret = key(proposer, leaf, leaf + 1); + set_key(&mut state, proposer, &secret); + let store = store_with(&config, &state); + let header = |state_root: Root| BeaconBlockHeader { + slot, + proposer_index: proposer, + parent_root: Root::ZERO, + state_root, + body_root: Root::ZERO, + }; + let domain = get_domain( + &state, + DOMAIN_BEACON_PROPOSER, + Some(compute_epoch_at_slot(slot)), + ); + let signed = |header: BeaconBlockHeader, leaf| pq::SignedBeaconBlockHeader { + signature: sign( + &secret, + compute_signing_root(header.hash_tree_root(), domain), + leaf, + ), + message: header, + }; + let slashing = |second_leaf| pq::ProposerSlashing { + signed_header_1: signed(header(Root::repeat_byte(1)), leaf), + signed_header_2: signed(header(Root::repeat_byte(2)), second_leaf), + }; + + let seen = SeenOperations::default(); + assert_eq!( + proposer_slashing_cheap_checks(&seen, &slashing(leaf)), + Ok(()) + ); + assert_eq!( + proposer_slashing_stateful_checks(&store, &slashing(leaf)), + Outcome::Accept + ); + assert_eq!( + proposer_slashing_stateful_checks(&store, &slashing(leaf + 1)), + Outcome::Reject(RejectReason::BadSignature) + ); + } + + #[test] + fn a_pq_attester_slashing_proves_both_votes() { + let config = config(); + let mut state = pq_state(&config); + let slashed = 5; + let epoch = compute_epoch_at_slot(state.slot()); + let leaf = compute_xmss_attestation_leaf_index(epoch).unwrap(); + let secret = key(slashed, leaf, leaf); + set_key(&mut state, slashed, &secret); + let store = store_with(&config, &state); + let vote = |root_byte: u8| { + let mut data = data(&state); + data.beacon_block_root = Root::repeat_byte(root_byte); + data + }; + let indexed = |data: AttestationData, proven: bool| { + let domain = get_domain(&state, DOMAIN_BEACON_ATTESTER, Some(epoch)); + let root = compute_signing_root(data.hash_tree_root(), domain); + let proof = if proven { + sign(&secret, root, leaf).0.to_vec() + } else { + Vec::new() + }; + pq::IndexedAttestation { + attesting_indices: vec![slashed].into(), + data, + proof: proof.try_into().unwrap(), + } + }; + let slashing = |proven| pq::AttesterSlashing { + attestation_1: indexed(vote(1), true), + attestation_2: indexed(vote(2), proven), + }; + + let seen = SeenOperations::default(); + assert_eq!( + attester_slashing_cheap_checks(&seen, &slashing(true)), + Ok(()) + ); + assert_eq!( + attester_slashing_stateful_checks(&store, &slashing(true)), + Outcome::Accept + ); + assert_eq!( + attester_slashing_stateful_checks(&store, &slashing(false)), + Outcome::Reject(RejectReason::InvalidOperation) + ); + } + + #[test] + fn a_pq_payload_attestation_is_signed_at_the_slots_ptc_leaf() { + let config = config(); + let mut state = pq_state(&config); + let slot = state.slot(); + let member = get_ptc(&state, slot, &config).unwrap()[0]; + let leaf = compute_xmss_leaf_index(slot, LEAF_OFFSET_PTC_ATTESTER).unwrap(); + let secret = key(member, leaf, leaf + 1); + set_key(&mut state, member, &secret); + let store = store_with(&config, &state); + let data = gloas::PayloadAttestationData { + beacon_block_root: BLOCK, + slot, + payload_present: true, + blob_data_available: true, + }; + let domain = get_domain( + &state, + DOMAIN_PTC_ATTESTER, + Some(compute_epoch_at_slot(slot)), + ); + let root = compute_signing_root(data.hash_tree_root(), domain); + let message = |leaf| pq::PayloadAttestationMessage { + validator_index: member, + data, + signature: sign(&secret, root, leaf), + }; + assert_eq!( + payload_attestation_stateful_checks(&store, &message(leaf)), + Outcome::Accept + ); + assert_eq!( + payload_attestation_stateful_checks(&store, &message(leaf + 1)), + Outcome::Reject(RejectReason::BadSignature) + ); + } + + #[test] + fn pq_preferences_are_signed_at_the_proposal_slots_preferences_leaf() { + let config = config(); + let mut state = pq_state(&config); + let proposer = 7; + let proposal_slot = state.slot() + 1; + let leaf = + compute_xmss_leaf_index(proposal_slot, LEAF_OFFSET_PROPOSER_PREFERENCES).unwrap(); + let secret = key(proposer, leaf, leaf + 1); + set_key(&mut state, proposer, &secret); + let message = gloas::ProposerPreferences { + proposal_slot, + validator_index: proposer, + ..Default::default() + }; + let proposal_epoch = compute_epoch_at_slot(proposal_slot); + let domain = compute_domain( + DOMAIN_PROPOSER_PREFERENCES, + config.fork_version(config.fork_at_epoch(proposal_epoch)), + state.genesis_validators_root(), + ); + let root = compute_signing_root(message.hash_tree_root(), domain); + let signed = |leaf| pq::SignedProposerPreferences { + message: message.clone(), + signature: sign(&secret, root, leaf), + }; + assert!(signed(leaf).signature_valid(&state, &config, proposal_epoch)); + assert!(!signed(leaf + 1).signature_valid(&state, &config, proposal_epoch)); + assert!(::judged_by(&state)); + assert!(!::judged_by(&state)); + } + + /// `XMSSPubkey(request.pubkey[16:]) == builder.pubkey`: a request naming + /// the key with anything in its first 16 bytes still names it. + #[test] + fn a_pq_builder_exit_names_its_builder_by_the_last_32_bytes() { + let builder = XmssPubkey([7; 32]).to_request_key(); + let mut request = builder; + request.0[0] = 1; + assert!(::exit_names( + &request, &builder + )); + assert!(!::exit_names( + &request, &builder + )); + let other = BlsPubkey([9; 48]); + assert!(!::exit_names( + &other, &builder + )); + assert_eq!(pq::SignedExecutionPayloadBid::default().message().slot, 0); + } + + /// A PQ bid on a PQ head runs gloas's whole rule over the PQ market: the + /// head view, the preferences and known payload, the builder, and its + /// XMSS signature, which only a bid leaf may carry. + #[test] + fn a_pq_bid_on_a_pq_head_is_accepted_at_a_bid_leaf() { + use crate::beacon::builder_market::PqBuilderMarket; + use crate::beacon::builder_market::test_support::envelope_with_gas_limit; + use crate::beacon::constants::{ + DOMAIN_BEACON_BUILDER, FAR_FUTURE_EPOCH, PAYLOAD_BUILDER_VERSION, + XMSS_ATTESTATION_LEAVES, XMSS_RESERVED_LEAVES, + }; + use crate::beacon::fork_choice::PayloadStatus; + use crate::beacon::gossip::execution_payload_bid as bid_rules; + use crate::beacon::gossip::proposer_preferences::dependent_root_at; + use crate::beacon::helpers::accessors::get_randao_mix; + use crate::beacon::precheck::fixed_proposer; + use crate::beacon::primitives::{ExecutionAddress, ExecutionBlockHash}; + + let config = config(); + let mut state = pq_state(&config); + let parent_hash = ExecutionBlockHash::repeat_byte(0x44); + let gas_limit = 30_000_000; + // The first bid leaf, and the last attestation leaf just before it. + let bid_leaf = XMSS_RESERVED_LEAVES + XMSS_ATTESTATION_LEAVES; + let secret = key(1_000, bid_leaf - 1, bid_leaf); + { + let BeaconState::Pq(inner) = &mut state else { + unreachable!("a PQ test state") + }; + inner.builders.0.push(gloas::Builder { + pubkey: XmssPubkey(secret.public_key().to_bytes()).to_request_key(), + version: PAYLOAD_BUILDER_VERSION, + execution_address: ExecutionAddress::repeat_byte(1), + balance: 100_000_000_000, + deposit_epoch: 0, + withdrawable_epoch: FAR_FUTURE_EPOCH, + }); + // Active once the finalized epoch passes its deposit epoch. + inner.finalized_checkpoint.epoch = 1; + inner.latest_execution_payload_bid.block_hash = parent_hash; + } + state.apply_pending_mutations(); + let store = store_with(&config, &state); + store.set_head_payload_status(BLOCK, PayloadStatus::Full); + + let bid_slot = state.slot() + 2; + let dependent_root = dependent_root_at(&state, BLOCK, bid_slot).expect("in the window"); + let proposer = fixed_proposer(&state, bid_slot).expect("in the lookahead"); + let fee_recipient = ExecutionAddress::repeat_byte(0x11); + let market = PqBuilderMarket::default(); + assert!(market.record_preferences( + pq::SignedProposerPreferences { + message: gloas::ProposerPreferences { + dependent_root, + proposal_slot: bid_slot, + validator_index: proposer, + fee_recipient, + target_gas_limit: gas_limit, + }, + signature: Default::default(), + }, + bid_slot - 1, + )); + market.record_execution_payload(&envelope_with_gas_limit( + parent_hash, + gas_limit, + BLOCK, + vec![], + )); + + let message = gloas::ExecutionPayloadBid { + parent_block_hash: parent_hash, + parent_block_root: BLOCK, + block_hash: ExecutionBlockHash::repeat_byte(0x33), + prev_randao: get_randao_mix(&state, compute_epoch_at_slot(state.slot())), + fee_recipient, + gas_limit, + builder_index: 0, + slot: bid_slot, + value: 1, + ..Default::default() + }; + let domain = get_domain(&state, DOMAIN_BEACON_BUILDER, None); + let root = compute_signing_root(message.hash_tree_root(), domain); + let signed = |leaf| pq::SignedExecutionPayloadBid { + message: message.clone(), + signature: sign(&secret, root, leaf), + }; + + let now = slot_start_ms(&store, bid_slot) + 100; + assert_eq!( + bid_rules::cheap_checks(&market, &store, &signed(bid_leaf), now), + Ok(()) + ); + assert_eq!( + bid_rules::stateful_checks(&store, &market, &signed(bid_leaf)), + Outcome::Accept + ); + assert_eq!( + bid_rules::stateful_checks(&store, &market, &signed(bid_leaf - 1)), + Outcome::Reject(RejectReason::BadSignature), + "an attestation leaf" + ); + } + + /// The gloas envelope rules against a PQ block, with PQ's request + /// container: a block never seen is queued, and one whose slot or + /// request root differs is refused before any signature. + #[test] + fn a_pq_envelope_is_matched_against_its_pq_block() { + use crate::beacon::builder_market::test_support::envelope_with_gas_limit; + use crate::beacon::primitives::ExecutionBlockHash; + + let config = config(); + let state = pq_state(&config); + let store = store_with(&config, &state); + let envelope = |block_root: Root, slot: Slot| { + let gloas = envelope_with_gas_limit(ExecutionBlockHash::ZERO, 0, block_root, vec![]); + let mut payload = gloas.payload; + payload.slot_number = slot; + pq::SignedExecutionPayloadEnvelope { + message: pq::ExecutionPayloadEnvelope { + payload, + execution_requests: Default::default(), + builder_index: 0, + beacon_block_root: block_root, + parent_beacon_block_root: Root::ZERO, + }, + signature: Default::default(), + } + }; + assert_eq!( + envelope_stateful_checks(&store, &envelope(Root::repeat_byte(3), state.slot())), + Outcome::Queue(QueueReason::BlockUnknown) + ); + assert_eq!( + envelope_stateful_checks(&store, &envelope(BLOCK, state.slot() + 1)), + Outcome::Reject(RejectReason::SlotMismatch) + ); + assert_eq!( + envelope_stateful_checks(&store, &envelope(BLOCK, state.slot())), + Outcome::Reject(RejectReason::ExecutionRequestsRootMismatch), + "the block's bid commits to no requests root" + ); + } + + #[test] + fn pq_operation_and_request_limits_are_enforced() { + let mut requests = pq::ExecutionRequests::default(); + assert!(execution_requests_within_limits(&requests)); + requests.voluntary_exits = + vec![Default::default(); preset::MAX_VOLUNTARY_EXIT_REQUESTS_PER_PAYLOAD as usize + 1] + .into(); + assert!(!execution_requests_within_limits(&requests)); + + let mut body = pq::BeaconBlockBody::empty(); + assert!(body_operations_within_limits(&body)); + let config = config(); + let state = pq_state(&config); + let attestation = aggregate(&state, &config, (0, &key(0, 0, 0)), 0, Vec::new()) + .message + .aggregate; + body.attestations = vec![attestation; preset::MAX_ATTESTATIONS_PQ as usize + 1].into(); + assert!(!body_operations_within_limits(&body)); + } +} diff --git a/crates/blockchain/state_transition/src/beacon/gossip/proposer_preferences.rs b/crates/blockchain/state_transition/src/beacon/gossip/proposer_preferences.rs index 3cbeb9aa7..d18c8d726 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/proposer_preferences.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/proposer_preferences.rs @@ -28,10 +28,10 @@ use super::{ IgnoreReason, Outcome, RejectReason, ancestor_at, is_future_slot, is_gloas_slot, slot_start_ms, }; use crate::beacon::bls; -use crate::beacon::builder_market::BuilderMarket; +use crate::beacon::builder_market::{BuilderMarket, SignedBid, SignedPreferences}; use crate::beacon::config::Config; use crate::beacon::constants::{DOMAIN_PROPOSER_PREFERENCES, MAXIMUM_GOSSIP_CLOCK_DISPARITY}; -use crate::beacon::containers::{BeaconState, gloas}; +use crate::beacon::containers::{BeaconState, gloas, pq}; use crate::beacon::fork_choice::{ Store, compute_shuffling_dependent_slot, compute_shuffling_lookahead_start_slot, }; @@ -43,13 +43,13 @@ use crate::beacon::primitives::{Domain, Epoch, HashTreeRoot as _, Root, Slot}; /// The rules that read only the message, the market's seen state and the /// clock. -pub fn cheap_checks( - market: &BuilderMarket, +pub fn cheap_checks( + market: &BuilderMarket, store: &Store, - signed: &gloas::SignedProposerPreferences, + signed: &P, now_ms: u64, ) -> Result<(), Outcome> { - let preferences = &signed.message; + let preferences = signed.message(); let config = store.config(); // [IGNORE] The first valid preferences for this dependent root and slot. if market @@ -77,15 +77,103 @@ pub fn cheap_checks( /// The rules that need the dependent block and a state, then the signature. /// Runs on a blocking thread. -pub fn stateful_checks(store: &Store, signed: &gloas::SignedProposerPreferences) -> Outcome { +/// What a preferences message's rules need from its fork beyond the shared +/// message: which state shape holds the proposer's key, and the signature. +/// Gloas's preferences are BLS-signed, PQ's XMSS-signed +/// (`specs/_features/pq/p2p-interface.md`'s modified `proposer_preferences`). +pub trait PreferencesRules: SignedPreferences { + /// How many epochs the dependent block's state may be advanced to reach + /// the proposal's lookahead, or `None` for no bound. Advancing runs every + /// slot between, on a peer's say-so. + const MAX_LOOKAHEAD_ADVANCE_EPOCHS: Option; + /// Whether `state` holds the proposer key this message is checked + /// against. + fn judged_by(state: &BeaconState) -> bool; + /// The proposer's signature, against the lookahead `state`. + fn signature_valid(&self, state: &BeaconState, config: &Config, proposal_epoch: Epoch) -> bool; +} + +impl PreferencesRules for gloas::SignedProposerPreferences { + /// Unbounded, as before: a followed chain's dependent block is recent. + const MAX_LOOKAHEAD_ADVANCE_EPOCHS: Option = None; + + /// Any state with a BLS registry: the lookahead state of a proposal at + /// the first gloas epochs may still be fulu's. + fn judged_by(state: &BeaconState) -> bool { + !matches!(state, BeaconState::Pq(_)) + } + + /// Under any candidate domain; see [`proposer_preferences_domains`]. + fn signature_valid(&self, state: &BeaconState, config: &Config, proposal_epoch: Epoch) -> bool { + let preferences = &self.message; + let Ok(validator) = state.validator(preferences.validator_index) else { + return false; + }; + let pubkey = bls::IndexedPubkey { + index: preferences.validator_index, + pubkey: validator.pubkey, + }; + let message_root = preferences.hash_tree_root(); + proposer_preferences_domains(state, config, proposal_epoch) + .into_iter() + .any(|domain| { + bls::verify( + &pubkey, + compute_signing_root(message_root, domain), + &self.signature, + ) + }) + } +} + +impl PreferencesRules for pq::SignedProposerPreferences { + /// Bounded: the node does not follow PQ, so its head, a valid dependent + /// root, can be any distance behind a PQ proposal. + const MAX_LOOKAHEAD_ADVANCE_EPOCHS: Option = Some(preset::MIN_SEED_LOOKAHEAD + 1); + + fn judged_by(state: &BeaconState) -> bool { + matches!(state, BeaconState::Pq(_)) + } + + /// `[Modified in PQ]` an XMSS signature at the proposal slot's + /// preferences leaf, under the domain of the proposal epoch's own fork + /// version, the one domain the specification names. + fn signature_valid(&self, state: &BeaconState, config: &Config, proposal_epoch: Epoch) -> bool { + use crate::beacon::constants::LEAF_OFFSET_PROPOSER_PREFERENCES; + use crate::beacon::helpers::pq::{compute_xmss_leaf_index, xmss_verify}; + let preferences = &self.message; + let Ok(validator) = state.pq_validator(preferences.validator_index) else { + return false; + }; + let domain = compute_domain( + DOMAIN_PROPOSER_PREFERENCES, + config.fork_version(config.fork_at_epoch(proposal_epoch)), + state.genesis_validators_root(), + ); + let signing_root = compute_signing_root(preferences.hash_tree_root(), domain); + let Ok(leaf_index) = + compute_xmss_leaf_index(preferences.proposal_slot, LEAF_OFFSET_PROPOSER_PREFERENCES) + else { + return false; + }; + xmss_verify( + &validator.pubkey(), + leaf_index, + signing_root, + &self.signature, + ) + } +} + +pub fn stateful_checks(store: &Store, signed: &P) -> Outcome { match stateful_rules(store, signed) { Ok(()) => Outcome::Accept, Err(outcome) => outcome, } } -fn stateful_rules(store: &Store, signed: &gloas::SignedProposerPreferences) -> Result<(), Outcome> { - let preferences = &signed.message; +fn stateful_rules(store: &Store, signed: &P) -> Result<(), Outcome> { + let preferences = signed.message(); let config = store.config(); let proposal_epoch = compute_epoch_at_slot(preferences.proposal_slot); let dependent_slot = compute_shuffling_dependent_slot(proposal_epoch); @@ -113,32 +201,25 @@ fn stateful_rules(store: &Store, signed: &gloas::SignedProposerPreferences) -> R return Err(Outcome::Ignore(IgnoreReason::ImpossibleDependentRoot)); } - let state = lookahead_state(store, preferences, proposal_epoch) - .ok_or(Outcome::Ignore(IgnoreReason::StateUnavailable))?; + let state = lookahead_state( + store, + preferences, + proposal_epoch, + P::MAX_LOOKAHEAD_ADVANCE_EPOCHS, + ) + .ok_or(Outcome::Ignore(IgnoreReason::StateUnavailable))?; // [REJECT] The validator is the proposer for the slot in the lookahead. if fixed_proposer(&state, preferences.proposal_slot) != Some(preferences.validator_index) { return Err(Outcome::Reject(RejectReason::WrongProposer)); } - // [REJECT] The signature is valid, under any candidate domain. - let pubkey = bls::IndexedPubkey { - index: preferences.validator_index, - pubkey: state - .validator(preferences.validator_index) - .map_err(|_| Outcome::Reject(RejectReason::WrongProposer))? - .pubkey, - }; - let message_root = preferences.hash_tree_root(); - let valid = proposer_preferences_domains(&state, &config, proposal_epoch) - .into_iter() - .any(|domain| { - bls::verify( - &pubkey, - compute_signing_root(message_root, domain), - &signed.signature, - ) - }); - if !valid { + // [REJECT] The signature is valid. A state of another registry's shape + // (a PQ message checked against a BLS registry, or the reverse) cannot + // judge it, which is not the sender's fault. + if !P::judged_by(&state) { + return Err(Outcome::Ignore(IgnoreReason::StateUnavailable)); + } + if !signed.signature_valid(&state, &config, proposal_epoch) { return Err(Outcome::Reject(RejectReason::BadSignature)); } Ok(()) @@ -146,11 +227,13 @@ fn stateful_rules(store: &Store, signed: &gloas::SignedProposerPreferences) -> R /// A cached state whose proposer lookahead answers for `preferences`: the /// head's when the head shares the dependent root, else the dependent block's -/// state advanced to the epoch before the proposal's. +/// state advanced to the epoch before the proposal's, by at most +/// `max_advance` epochs when one is given. fn lookahead_state( store: &Store, preferences: &gloas::ProposerPreferences, proposal_epoch: Epoch, + max_advance: Option, ) -> Option> { if let Ok(head_root) = store.head() && let Some(head_state) = store.cached_state(CacheKey::BlockState(head_root)) @@ -167,19 +250,24 @@ fn lookahead_state( return Some(head_state); } } - cached_checkpoint_state( - store, - proposal_epoch.saturating_sub(preset::MIN_SEED_LOOKAHEAD), - preferences.dependent_root, - ) + let target_epoch = proposal_epoch.saturating_sub(preset::MIN_SEED_LOOKAHEAD); + if let Some(max_advance) = max_advance { + let dependent_state = + store.cached_state(CacheKey::BlockState(preferences.dependent_root))?; + let state_epoch = compute_epoch_at_slot(dependent_state.slot()); + if target_epoch.saturating_sub(state_epoch) > max_advance { + return None; + } + } + cached_checkpoint_state(store, target_epoch, preferences.dependent_root) } /// Both halves. The caller records the preferences on `Accept`: the /// specification's `validate_proposer_preferences_gossip`. -pub fn validate( - market: &BuilderMarket, +pub fn validate( + market: &BuilderMarket, store: &Store, - signed: &gloas::SignedProposerPreferences, + signed: &P, now_ms: u64, ) -> Outcome { if let Err(outcome) = cheap_checks(market, store, signed, now_ms) { diff --git a/crates/blockchain/state_transition/src/beacon/gossip/test_support.rs b/crates/blockchain/state_transition/src/beacon/gossip/test_support.rs index 82532df69..a0f975869 100644 --- a/crates/blockchain/state_transition/src/beacon/gossip/test_support.rs +++ b/crates/blockchain/state_transition/src/beacon/gossip/test_support.rs @@ -349,7 +349,7 @@ pub(crate) mod builder_scene { let dependent_root = dependent_root_at(&state, PARENT, bid_slot).expect("in the window"); let proposer = crate::beacon::precheck::fixed_proposer(&state, bid_slot).expect("window"); - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let preferences = test_support::sign_preferences( &state, gloas::ProposerPreferences { diff --git a/crates/blockchain/state_transition/src/beacon/helpers/accessors.rs b/crates/blockchain/state_transition/src/beacon/helpers/accessors.rs index fa3ddd152..f9c119326 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/accessors.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/accessors.rs @@ -138,16 +138,39 @@ pub fn get_seed(state: &BeaconState, epoch: Epoch, domain_type: DomainType) -> B /// for p2p's gossipsub scoring, which sizes expected aggregate traffic from /// a validator count rather than from a state. pub fn committee_count_per_slot(active_count: u64) -> u64 { + committee_count_per_slot_for(ForkName::Gloas, active_count) +} + +/// The most committees a slot of `fork` has: `MAX_COMMITTEES_PER_SLOT`, or +/// the PQ fork's `MAX_COMMITTEES_PER_SLOT_PQ`. +/// +/// The PQ fork's cap applies to every epoch a PQ state is asked about, +/// including a previous epoch that ran before the fork: the specification's +/// modified `get_committee_count_per_slot` reads no fork, so those committees +/// are recomputed under the new cap. +pub fn max_committees_per_slot(fork: ForkName) -> u64 { + if fork == ForkName::Pq { + preset::MAX_COMMITTEES_PER_SLOT_PQ as u64 + } else { + preset::MAX_COMMITTEES_PER_SLOT as u64 + } +} + +/// [`committee_count_per_slot`] under `fork`'s cap. +pub fn committee_count_per_slot_for(fork: ForkName, active_count: u64) -> u64 { let ideal = active_count / preset::SLOTS_PER_EPOCH / preset::TARGET_COMMITTEE_SIZE; - ideal.clamp(1, preset::MAX_COMMITTEES_PER_SLOT as u64) + ideal.clamp(1, max_committees_per_slot(fork)) } /// How many committees each slot of `epoch` has. /// /// At least one, so a small chain still produces committees, and at most -/// `MAX_COMMITTEES_PER_SLOT`. +/// `MAX_COMMITTEES_PER_SLOT` (`MAX_COMMITTEES_PER_SLOT_PQ` on a PQ state). pub fn get_committee_count_per_slot(state: &BeaconState, epoch: Epoch) -> u64 { - committee_count_per_slot(get_active_validator_indices(state, epoch).len() as u64) + committee_count_per_slot_for( + state.fork_name(), + get_active_validator_indices(state, epoch).len() as u64, + ) } /// `epoch`'s committees as `state` names them: one scan of the active @@ -189,7 +212,8 @@ pub fn get_committee_count_per_slot(state: &BeaconState, epoch: Epoch) -> u64 { /// what [`ShufflingKey`] does; see [`shuffling_key`]. pub fn build_epoch_committees(state: &BeaconState, epoch: Epoch) -> EpochCommittees { let active_indices = get_active_validator_indices(state, epoch); - let committees_per_slot = committee_count_per_slot(active_indices.len() as u64); + let committees_per_slot = + committee_count_per_slot_for(state.fork_name(), active_indices.len() as u64); let seed = get_seed(state, epoch, constants::DOMAIN_BEACON_ATTESTER); // The active set's own buffer becomes the shuffled set, so a build holds // one validator-sized list at a time, not a permutation and a gathered @@ -244,6 +268,7 @@ fn shuffling_key(state: &BeaconState, epoch: Epoch) -> Option { Some(ShufflingKey { epoch, decision_root, + max_committees_per_slot: max_committees_per_slot(state.fork_name()), }) } @@ -627,7 +652,8 @@ pub fn get_beacon_committee( ) -> Result> { let epoch = compute_epoch_at_slot(slot); let active_indices = get_active_validator_indices(state, epoch); - let committees_per_slot = committee_count_per_slot(active_indices.len() as u64); + let committees_per_slot = + committee_count_per_slot_for(state.fork_name(), active_indices.len() as u64); compute_committee( &active_indices, get_seed(state, epoch, constants::DOMAIN_BEACON_ATTESTER), @@ -664,7 +690,9 @@ pub fn get_beacon_proposer_index(state: &BeaconState) -> Result // Gloas's own `proposer_lookahead` is unchanged from fulu (see // `containers::gloas`'s module doc) and gloas does not redefine // this function, so both forks share fulu's exact same lookup. - ForkName::Fulu | ForkName::Gloas => return super::fulu::get_beacon_proposer_index(state), + ForkName::Fulu | ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => { + return super::fulu::get_beacon_proposer_index(state); + } ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -698,7 +726,9 @@ pub fn get_beacon_proposer_index(state: &BeaconState) -> Result |index| Ok(state.validator(index)?.effective_balance), ), ForkName::Fulu => unreachable!("Fulu returns via the dispatch match above"), - ForkName::Gloas => unreachable!("Gloas returns via the dispatch match above"), + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => { + unreachable!("Gloas returns via the dispatch match above") + } ForkName::Lean => lean_state_unreachable("get_beacon_proposer_index"), } } diff --git a/crates/blockchain/state_transition/src/beacon/helpers/altair.rs b/crates/blockchain/state_transition/src/beacon/helpers/altair.rs index d6404e558..19543ee62 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/altair.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/altair.rs @@ -168,7 +168,16 @@ pub fn get_next_sync_committee(state: &BeaconState) -> Result crate::beacon::helpers::gloas::get_next_sync_committee_indices(state)?, + ForkName::Gloas | ForkName::Eip8483 => { + crate::beacon::helpers::gloas::get_next_sync_committee_indices(state)? + } + // Sync committees are removed in PQ. + ForkName::Pq => { + return Err(crate::beacon::error::Error::UnsupportedForFork { + function: "get_next_sync_committee", + fork: ForkName::Pq, + }); + } ForkName::Lean => lean_state_unreachable("get_next_sync_committee"), }; let mut pubkeys = Vec::with_capacity(indices.len()); diff --git a/crates/blockchain/state_transition/src/beacon/helpers/electra.rs b/crates/blockchain/state_transition/src/beacon/helpers/electra.rs index e54741af9..a00d7ca9a 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/electra.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/electra.rs @@ -127,7 +127,7 @@ use crate::beacon::bls::{AggregatePubkey, IndexedPubkey, SignatureSet}; use crate::beacon::config::Config; use crate::beacon::constants::{self, FAR_FUTURE_EPOCH}; use crate::beacon::containers::shared::{AttestationData, Validator}; -use crate::beacon::containers::{BeaconState, electra, fulu, gloas}; +use crate::beacon::containers::{BeaconState, GloasFamily, GloasFamilyMut, electra, fulu, gloas}; use crate::beacon::error::{Error, Result}; use crate::beacon::fork::ForkName; use crate::beacon::hash::hash; @@ -886,10 +886,12 @@ pub fn compute_exit_epoch_and_update_churn( total_active_balance_or_compute(state, active_balances)?, config, ), - ForkName::Gloas => crate::beacon::helpers::gloas::exit_churn_limit_for( - total_active_balance_or_compute(state, active_balances)?, - config, - ), + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => { + crate::beacon::helpers::gloas::exit_churn_limit_for( + total_active_balance_or_compute(state, active_balances)?, + config, + ) + } fork @ (ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -1033,7 +1035,7 @@ pub fn get_consolidation_churn_limit_for_fork( total_active_balance_or_compute(state, active_balances)?, config, )), - ForkName::Gloas => Ok( + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => Ok( crate::beacon::helpers::gloas::consolidation_churn_limit_for( total_active_balance_or_compute(state, active_balances)?, config, @@ -1157,7 +1159,13 @@ pub fn queue_excess_active_balance(state: &mut BeaconState, index: ValidatorInde // this cannot fail. *state.balance_mut(index)? = preset::MIN_ACTIVATION_BALANCE; - let deposit = placeholder_pending_deposit(state.validator(index)?, excess_balance); + let mut deposit = placeholder_pending_deposit(state.validator(index)?, excess_balance); + // [Modified in PQ] An empty signature field rather than the point at + // infinity; the key is already the 48-byte request form the registry + // holds an XMSS key in. + if state.fork_name() == ForkName::Pq { + deposit.signature = BlsSignature::default(); + } pending_queue_fields(state, "queue_excess_active_balance")?.push_pending_deposit(deposit) } @@ -1263,10 +1271,15 @@ fn placeholder_pending_deposit(validator: &Validator, amount: Gwei) -> electra:: /// exit/consolidation churn cursor fields, which nothing here reads or /// writes, and every gloas caller that needs those reaches them through /// `ChurnCursorsMut` directly, not through this projection. +// The gloas variant holds the gloas-family view, a borrow of each field rather +// than one of the state, so it is far wider than the others. The value lives on +// the stack for one call and is never stored, where a box would only add an +// allocation per use. +#[allow(clippy::large_enum_variant)] pub(crate) enum PendingQueueFields<'a> { Electra(&'a mut electra::BeaconState), Fulu(&'a mut fulu::BeaconState), - Gloas(&'a mut gloas::BeaconState), + Gloas(GloasFamilyMut<'a>), } impl<'a> PendingQueueFields<'a> { @@ -1284,7 +1297,7 @@ impl<'a> PendingQueueFields<'a> { match self { PendingQueueFields::Electra(state) => state.deposit_requests_start_index, PendingQueueFields::Fulu(state) => state.deposit_requests_start_index, - PendingQueueFields::Gloas(state) => state.deposit_requests_start_index, + PendingQueueFields::Gloas(state) => *state.deposit_requests_start_index, } } @@ -1293,7 +1306,7 @@ impl<'a> PendingQueueFields<'a> { match self { PendingQueueFields::Electra(state) => state.deposit_balance_to_consume, PendingQueueFields::Fulu(state) => state.deposit_balance_to_consume, - PendingQueueFields::Gloas(state) => state.deposit_balance_to_consume, + PendingQueueFields::Gloas(state) => *state.deposit_balance_to_consume, } } @@ -1301,7 +1314,7 @@ impl<'a> PendingQueueFields<'a> { match self { PendingQueueFields::Electra(state) => &mut state.deposit_balance_to_consume, PendingQueueFields::Fulu(state) => &mut state.deposit_balance_to_consume, - PendingQueueFields::Gloas(state) => &mut state.deposit_balance_to_consume, + PendingQueueFields::Gloas(state) => &mut *state.deposit_balance_to_consume, } } @@ -1336,7 +1349,7 @@ impl<'a> PendingQueueFields<'a> { core::mem::take(&mut state.pending_deposits).to_vec() } PendingQueueFields::Gloas(state) => { - core::mem::take(&mut state.pending_deposits).to_vec() + core::mem::take(&mut *state.pending_deposits).to_vec() } } } @@ -1356,7 +1369,7 @@ impl<'a> PendingQueueFields<'a> { state.pending_deposits = electra::PendingDeposits::try_from(deposits)?; } PendingQueueFields::Gloas(state) => { - state.pending_deposits = gloas::PendingDeposits::from(deposits); + *state.pending_deposits = gloas::PendingDeposits::from(deposits); } } Ok(()) @@ -1370,7 +1383,7 @@ impl<'a> PendingQueueFields<'a> { match self { PendingQueueFields::Electra(state) => &state.pending_partial_withdrawals, PendingQueueFields::Fulu(state) => &state.pending_partial_withdrawals, - PendingQueueFields::Gloas(state) => &state.pending_partial_withdrawals, + PendingQueueFields::Gloas(state) => &*state.pending_partial_withdrawals, } } @@ -1399,7 +1412,7 @@ impl<'a> PendingQueueFields<'a> { match self { PendingQueueFields::Electra(state) => &state.pending_consolidations, PendingQueueFields::Fulu(state) => &state.pending_consolidations, - PendingQueueFields::Gloas(state) => &state.pending_consolidations, + PendingQueueFields::Gloas(state) => &*state.pending_consolidations, } } @@ -1435,7 +1448,7 @@ impl<'a> PendingQueueFields<'a> { core::mem::take(&mut state.pending_consolidations).into_inner() } PendingQueueFields::Gloas(state) => { - core::mem::take(&mut state.pending_consolidations).into_inner() + core::mem::take(&mut *state.pending_consolidations).into_inner() } } } @@ -1457,7 +1470,7 @@ impl<'a> PendingQueueFields<'a> { electra::PendingConsolidations::try_from(consolidations)?; } PendingQueueFields::Gloas(state) => { - state.pending_consolidations = gloas::PendingConsolidations::from(consolidations); + *state.pending_consolidations = gloas::PendingConsolidations::from(consolidations); } } Ok(()) @@ -1501,7 +1514,9 @@ pub(crate) fn pending_queue_fields<'a>( match state { BeaconState::Electra(state) => Ok(PendingQueueFields::Electra(state)), BeaconState::Fulu(state) => Ok(PendingQueueFields::Fulu(state)), - BeaconState::Gloas(state) => Ok(PendingQueueFields::Gloas(state)), + state @ (BeaconState::Gloas(_) | BeaconState::Eip8483(_) | BeaconState::Pq(_)) => Ok( + PendingQueueFields::Gloas(state.gloas_family_mut().expect("a gloas-family state")), + ), other => Err(Error::UnsupportedForFork { function, fork: other.fork_name(), @@ -1512,10 +1527,15 @@ pub(crate) fn pending_queue_fields<'a>( /// The read-only counterpart of [`PendingQueueFields`], for a caller (like /// [`get_pending_balance_to_withdraw`]) that only ever reads a pending queue /// and so only ever holds a `&BeaconState`. +// The gloas variant holds the gloas-family view, a borrow of each field rather +// than one of the state, so it is far wider than the others. The value lives on +// the stack for one call and is never stored, where a box would only add an +// allocation per use. +#[allow(clippy::large_enum_variant)] pub(crate) enum PendingQueueFieldsRef<'a> { Electra(&'a electra::BeaconState), Fulu(&'a fulu::BeaconState), - Gloas(&'a gloas::BeaconState), + Gloas(GloasFamily<'a>), } impl<'a> PendingQueueFieldsRef<'a> { @@ -1523,7 +1543,7 @@ impl<'a> PendingQueueFieldsRef<'a> { match self { PendingQueueFieldsRef::Electra(state) => &state.pending_partial_withdrawals, PendingQueueFieldsRef::Fulu(state) => &state.pending_partial_withdrawals, - PendingQueueFieldsRef::Gloas(state) => &state.pending_partial_withdrawals, + PendingQueueFieldsRef::Gloas(state) => state.pending_partial_withdrawals, } } } @@ -1536,7 +1556,9 @@ pub(crate) fn pending_queue_fields_ref<'a>( match state { BeaconState::Electra(state) => Ok(PendingQueueFieldsRef::Electra(state)), BeaconState::Fulu(state) => Ok(PendingQueueFieldsRef::Fulu(state)), - BeaconState::Gloas(state) => Ok(PendingQueueFieldsRef::Gloas(state)), + state @ (BeaconState::Gloas(_) | BeaconState::Eip8483(_) | BeaconState::Pq(_)) => Ok( + PendingQueueFieldsRef::Gloas(state.gloas_family().expect("a gloas-family state")), + ), other => Err(Error::UnsupportedForFork { function, fork: other.fork_name(), @@ -1557,10 +1579,15 @@ pub(crate) fn pending_queue_fields_ref<'a>( /// deposit-balance churn cursor and the three pending queues, none of which /// this projection reads or writes. This one carries only the four /// exit/consolidation churn cursor fields. +// The gloas variant holds the gloas-family view, a borrow of each field rather +// than one of the state, so it is far wider than the others. The value lives on +// the stack for one call and is never stored, where a box would only add an +// allocation per use. +#[allow(clippy::large_enum_variant)] pub(crate) enum ChurnCursorsMut<'a> { Electra(&'a mut electra::BeaconState), Fulu(&'a mut fulu::BeaconState), - Gloas(&'a mut gloas::BeaconState), + Gloas(GloasFamilyMut<'a>), } impl<'a> ChurnCursorsMut<'a> { @@ -1568,7 +1595,7 @@ impl<'a> ChurnCursorsMut<'a> { match self { ChurnCursorsMut::Electra(state) => state.earliest_exit_epoch, ChurnCursorsMut::Fulu(state) => state.earliest_exit_epoch, - ChurnCursorsMut::Gloas(state) => state.earliest_exit_epoch, + ChurnCursorsMut::Gloas(state) => *state.earliest_exit_epoch, } } @@ -1576,7 +1603,7 @@ impl<'a> ChurnCursorsMut<'a> { match self { ChurnCursorsMut::Electra(state) => &mut state.earliest_exit_epoch, ChurnCursorsMut::Fulu(state) => &mut state.earliest_exit_epoch, - ChurnCursorsMut::Gloas(state) => &mut state.earliest_exit_epoch, + ChurnCursorsMut::Gloas(state) => &mut *state.earliest_exit_epoch, } } @@ -1584,7 +1611,7 @@ impl<'a> ChurnCursorsMut<'a> { match self { ChurnCursorsMut::Electra(state) => state.exit_balance_to_consume, ChurnCursorsMut::Fulu(state) => state.exit_balance_to_consume, - ChurnCursorsMut::Gloas(state) => state.exit_balance_to_consume, + ChurnCursorsMut::Gloas(state) => *state.exit_balance_to_consume, } } @@ -1592,7 +1619,7 @@ impl<'a> ChurnCursorsMut<'a> { match self { ChurnCursorsMut::Electra(state) => &mut state.exit_balance_to_consume, ChurnCursorsMut::Fulu(state) => &mut state.exit_balance_to_consume, - ChurnCursorsMut::Gloas(state) => &mut state.exit_balance_to_consume, + ChurnCursorsMut::Gloas(state) => &mut *state.exit_balance_to_consume, } } @@ -1600,7 +1627,7 @@ impl<'a> ChurnCursorsMut<'a> { match self { ChurnCursorsMut::Electra(state) => state.earliest_consolidation_epoch, ChurnCursorsMut::Fulu(state) => state.earliest_consolidation_epoch, - ChurnCursorsMut::Gloas(state) => state.earliest_consolidation_epoch, + ChurnCursorsMut::Gloas(state) => *state.earliest_consolidation_epoch, } } @@ -1608,7 +1635,7 @@ impl<'a> ChurnCursorsMut<'a> { match self { ChurnCursorsMut::Electra(state) => &mut state.earliest_consolidation_epoch, ChurnCursorsMut::Fulu(state) => &mut state.earliest_consolidation_epoch, - ChurnCursorsMut::Gloas(state) => &mut state.earliest_consolidation_epoch, + ChurnCursorsMut::Gloas(state) => &mut *state.earliest_consolidation_epoch, } } @@ -1616,7 +1643,7 @@ impl<'a> ChurnCursorsMut<'a> { match self { ChurnCursorsMut::Electra(state) => state.consolidation_balance_to_consume, ChurnCursorsMut::Fulu(state) => state.consolidation_balance_to_consume, - ChurnCursorsMut::Gloas(state) => state.consolidation_balance_to_consume, + ChurnCursorsMut::Gloas(state) => *state.consolidation_balance_to_consume, } } @@ -1624,7 +1651,7 @@ impl<'a> ChurnCursorsMut<'a> { match self { ChurnCursorsMut::Electra(state) => &mut state.consolidation_balance_to_consume, ChurnCursorsMut::Fulu(state) => &mut state.consolidation_balance_to_consume, - ChurnCursorsMut::Gloas(state) => &mut state.consolidation_balance_to_consume, + ChurnCursorsMut::Gloas(state) => &mut *state.consolidation_balance_to_consume, } } } @@ -1638,7 +1665,9 @@ pub(crate) fn churn_cursors_mut<'a>( match state { BeaconState::Electra(state) => Ok(ChurnCursorsMut::Electra(state)), BeaconState::Fulu(state) => Ok(ChurnCursorsMut::Fulu(state)), - BeaconState::Gloas(state) => Ok(ChurnCursorsMut::Gloas(state)), + state @ (BeaconState::Gloas(_) | BeaconState::Eip8483(_) | BeaconState::Pq(_)) => Ok( + ChurnCursorsMut::Gloas(state.gloas_family_mut().expect("a gloas-family state")), + ), other => Err(Error::UnsupportedForFork { function, fork: other.fork_name(), diff --git a/crates/blockchain/state_transition/src/beacon/helpers/fulu.rs b/crates/blockchain/state_transition/src/beacon/helpers/fulu.rs index 1a5bee5bf..1610bf854 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/fulu.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/fulu.rs @@ -141,6 +141,8 @@ fn proposer_lookahead<'a>( match state { BeaconState::Fulu(inner) => Ok(&inner.proposer_lookahead), BeaconState::Gloas(inner) => Ok(&inner.proposer_lookahead), + BeaconState::Eip8483(inner) => Ok(&inner.proposer_lookahead), + BeaconState::Pq(inner) => Ok(&inner.proposer_lookahead), other => Err(Error::UnsupportedForFork { function, fork: other.fork_name(), @@ -161,6 +163,8 @@ pub(crate) fn proposer_lookahead_mut<'a>( match state { BeaconState::Fulu(inner) => Ok(&mut inner.proposer_lookahead), BeaconState::Gloas(inner) => Ok(&mut inner.proposer_lookahead), + BeaconState::Eip8483(inner) => Ok(&mut inner.proposer_lookahead), + BeaconState::Pq(inner) => Ok(&mut inner.proposer_lookahead), other => Err(Error::UnsupportedForFork { function, fork: other.fork_name(), diff --git a/crates/blockchain/state_transition/src/beacon/helpers/gloas.rs b/crates/blockchain/state_transition/src/beacon/helpers/gloas.rs index 9485beda0..b180cfe6d 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/gloas.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/gloas.rs @@ -90,7 +90,7 @@ use crate::beacon::bls::{IndexedPubkey, SignatureSet}; use crate::beacon::config::Config; use crate::beacon::constants; use crate::beacon::containers::shared::AttestationData; -use crate::beacon::containers::{BeaconState, gloas}; +use crate::beacon::containers::{BeaconState, GloasFamily, GloasFamilyMut, gloas}; use crate::beacon::error::{Error, Result}; use crate::beacon::hash::hash; use crate::beacon::preset; @@ -278,7 +278,7 @@ pub fn is_builder_index(validator_index: ValidatorIndex) -> bool { /// Whether the builder at `builder_index` is active: its placement in the /// registry is finalized, and it has not initiated an exit. pub fn is_active_builder( - state: &gloas::BeaconState, + state: &GloasFamily<'_>, builder_index: gloas::BuilderIndex, ) -> Result { let builder = state @@ -401,7 +401,7 @@ pub fn convert_validator_index_to_builder_index( /// checking it can cover a new bid, so a builder cannot bid its way into /// double-spending a balance a withdrawal has already claimed. pub fn get_pending_balance_to_withdraw_for_builder( - state: &gloas::BeaconState, + state: &GloasFamily<'_>, builder_index: gloas::BuilderIndex, ) -> Result { let mut balance: Gwei = 0; @@ -430,7 +430,7 @@ pub fn get_pending_balance_to_withdraw_for_builder( /// after keeping [`preset::MIN_DEPOSIT_AMOUNT`] and every balance it already /// owes ([`get_pending_balance_to_withdraw_for_builder`]) in reserve. pub fn can_builder_cover_bid( - state: &gloas::BeaconState, + state: &GloasFamily<'_>, builder_index: gloas::BuilderIndex, bid_amount: Gwei, ) -> Result { @@ -668,6 +668,11 @@ pub fn get_next_sync_committee_indices(state: &BeaconState) -> Result super::pq::get_latest_block_slot(state, data.slot, parent_slot)?, + _ => parent_slot, + }; let inner = gloas_state_ref(state, "get_attestation_participation_flag_indices")?; - let slot_index = (parent_slot % preset::SLOTS_PER_HISTORICAL_ROOT as u64) as usize; + let slot_index = (block_slot % preset::SLOTS_PER_HISTORICAL_ROOT as u64) as usize; let payload_index = inner.execution_payload_availability.get(slot_index).ok_or( Error::IndexOutOfBounds { index: slot_index, @@ -723,7 +734,8 @@ pub fn get_attestation_participation_flag_indices( if is_matching_target { participation_flag_indices.push(constants::TIMELY_TARGET_FLAG_INDEX); } - if is_matching_head && inclusion_delay == preset::MIN_ATTESTATION_INCLUSION_DELAY { + // [Modified in PQ] The head flag needs the PQ fork's delay there. + if is_matching_head && inclusion_delay == super::pq::min_attestation_inclusion_delay(state) { participation_flag_indices.push(constants::TIMELY_HEAD_FLAG_INDEX); } @@ -936,11 +948,11 @@ pub(crate) fn consolidation_churn_limit_for(total_active_balance: Gwei, config: /// builder-registry counterpart of /// [`crate::beacon::helpers::mutators::initiate_validator_exit`]. pub fn initiate_builder_exit( - state: &mut gloas::BeaconState, + state: &mut GloasFamilyMut<'_>, builder_index: gloas::BuilderIndex, config: &Config, ) -> Result<()> { - let current_epoch = compute_epoch_at_slot(state.slot); + let current_epoch = compute_epoch_at_slot(*state.slot); let len = state.builders.len(); let builder = state @@ -970,7 +982,7 @@ pub fn initiate_builder_exit( /// by weight, against [`get_builder_payment_quorum_threshold`]; this settles /// one parent block's still-live entry unconditionally, the moment that /// block's payload is applied. -pub fn settle_builder_payment(state: &mut gloas::BeaconState, payment_index: u64) -> Result<()> { +pub fn settle_builder_payment(state: &mut GloasFamilyMut<'_>, payment_index: u64) -> Result<()> { let len = state.builder_pending_payments.len() as u64; crate::beacon::verify( payment_index < len, @@ -997,8 +1009,8 @@ pub fn settle_builder_payment(state: &mut gloas::BeaconState, payment_index: u64 /// `set_or_append_list` appends a fresh entry instead of overwriting one. /// Builder indices are reusable this way, unlike the validator registry, /// which never removes an entry. -pub fn get_index_for_new_builder(state: &gloas::BeaconState) -> gloas::BuilderIndex { - let current_epoch = compute_epoch_at_slot(state.slot); +pub fn get_index_for_new_builder(state: &GloasFamily<'_>) -> gloas::BuilderIndex { + let current_epoch = compute_epoch_at_slot(*state.slot); for (index, builder) in state.builders.iter().enumerate() { if builder.withdrawable_epoch <= current_epoch && builder.balance == 0 { return index as gloas::BuilderIndex; @@ -1018,14 +1030,14 @@ pub fn get_index_for_new_builder(state: &gloas::BeaconState) -> gloas::BuilderIn /// second lookup. Called from that one-time fork-boundary onboarding pass /// and from builder deposit requests (EIP-8282's ongoing onboarding path). pub fn add_builder_to_registry( - state: &mut gloas::BeaconState, + state: &mut GloasFamilyMut<'_>, pubkey: BlsPubkey, version: u8, execution_address: ExecutionAddress, amount: Gwei, slot: Slot, ) -> gloas::BuilderIndex { - let index = get_index_for_new_builder(state); + let index = get_index_for_new_builder(&state.as_ref()); let builder = gloas::Builder { pubkey, version, @@ -1046,39 +1058,32 @@ pub fn add_builder_to_registry( // Fork projection // --------------------------------------------------------------------------- -/// The gloas state, mutably, or an error naming the function that needs one. +/// The gloas-family fields of a gloas, EIP-8483 or PQ state, mutably, or an +/// error naming the function that needs them. /// -/// Kept alongside [`gloas_state_ref`] for mutating a gloas-only field -/// through a generic [`BeaconState`] rather than one already known to be -/// [`BeaconState::Gloas`] the way [`initiate_builder_exit`] and -/// [`settle_builder_payment`] are always called with. The fork upgrade -/// (`crate::beacon::upgrade::upgrade_to_gloas`) and -/// [`crate::beacon::stf::epoch::gloas`] are its callers. +/// A [`GloasFamilyMut`] rather than a `&mut gloas::BeaconState`: the two +/// feature forks after gloas keep every field the gloas rules read, at the +/// same types, in states of their own shape, and the view is what lets one +/// copy of those rules serve all three. pub(crate) fn gloas_state<'a>( state: &'a mut BeaconState, function: &'static str, -) -> Result<&'a mut gloas::BeaconState> { - match state { - BeaconState::Gloas(inner) => Ok(inner), - other => Err(Error::UnsupportedForFork { - function, - fork: other.fork_name(), - }), - } +) -> Result> { + let fork = state.fork_name(); + state + .gloas_family_mut() + .ok_or(Error::UnsupportedForFork { function, fork }) } -/// The gloas state, immutably. See [`gloas_state`]. +/// The gloas-family fields, immutably. See [`gloas_state`]. pub(crate) fn gloas_state_ref<'a>( state: &'a BeaconState, function: &'static str, -) -> Result<&'a gloas::BeaconState> { - match state { - BeaconState::Gloas(inner) => Ok(inner), - other => Err(Error::UnsupportedForFork { - function, - fork: other.fork_name(), - }), - } +) -> Result> { + state.gloas_family().ok_or(Error::UnsupportedForFork { + function, + fork: state.fork_name(), + }) } #[cfg(test)] @@ -1275,7 +1280,7 @@ mod tests { let BeaconState::Gloas(state) = gloas_state_with_validators(4) else { panic!("gloas_state_with_validators returns a gloas state"); }; - assert_eq!(get_index_for_new_builder(&state), 0); + assert_eq!(get_index_for_new_builder(&(&state).into()), 0); } #[test] @@ -1285,7 +1290,7 @@ mod tests { }; state.builders.push(active_builder()); state.builders.push(active_builder()); - assert_eq!(get_index_for_new_builder(&state), 2); + assert_eq!(get_index_for_new_builder(&(&state).into()), 2); } #[test] @@ -1299,7 +1304,7 @@ mod tests { // and fully swept (balance zero). state.builders.push(exited_swept_builder(current_epoch)); state.builders.push(active_builder()); - assert_eq!(get_index_for_new_builder(&state), 1); + assert_eq!(get_index_for_new_builder(&(&state).into()), 1); } #[test] @@ -1315,7 +1320,7 @@ mod tests { balance: 1, ..Default::default() }); - assert_eq!(get_index_for_new_builder(&state), 1); + assert_eq!(get_index_for_new_builder(&(&state).into()), 1); } #[test] @@ -1328,7 +1333,7 @@ mod tests { let slot = state.slot; let index = add_builder_to_registry( - &mut state, + &mut (&mut state).into(), pubkey, constants::PAYLOAD_BUILDER_VERSION, ExecutionAddress::ZERO, @@ -1353,7 +1358,7 @@ mod tests { let slot = state.slot; let index = add_builder_to_registry( - &mut state, + &mut (&mut state).into(), pubkey, constants::PAYLOAD_BUILDER_VERSION, ExecutionAddress::ZERO, diff --git a/crates/blockchain/state_transition/src/beacon/helpers/mod.rs b/crates/blockchain/state_transition/src/beacon/helpers/mod.rs index fc1ce3093..bffd1f274 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/mod.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/mod.rs @@ -25,6 +25,7 @@ pub mod mutators; pub mod participation; #[cfg(any(test, debug_assertions))] pub mod participation_reference; +pub mod pq; pub mod predicates; pub mod shuffling; pub mod sync_committee; diff --git a/crates/blockchain/state_transition/src/beacon/helpers/mutators.rs b/crates/blockchain/state_transition/src/beacon/helpers/mutators.rs index 72efd03bc..d97e7e98a 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/mutators.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/mutators.rs @@ -180,7 +180,7 @@ fn initiate_validator_exit_for_fork( // `initiate_validator_exit` does on a gloas state: the churn it // draws through `compute_exit_epoch_and_update_churn` is already // gloas's own by the time it gets here. - ForkName::Electra | ForkName::Fulu | ForkName::Gloas => { + ForkName::Electra | ForkName::Fulu | ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => { crate::beacon::helpers::electra::initiate_validator_exit(state, index, config, None) } ForkName::Phase0 @@ -229,6 +229,20 @@ pub fn slash_validator( whistleblower_index, whistleblower_reward - proposer_reward, )?; + + // From EIP-8483 on (the `pq` branch's newer gloas, see + // `stf::eip8483`'s module doc): remove the pending builder payments for + // blocks the slashed validator proposed, whatever the slashing. + if fork >= ForkName::Eip8483 + && fork != ForkName::Lean + && let Some(inner) = state.gloas_family_mut() + { + for payment in inner.builder_pending_payments.iter_mut() { + if payment.proposer_index == slashed_index { + *payment = crate::beacon::containers::gloas::BuilderPendingPayment::default(); + } + } + } Ok(()) } diff --git a/crates/blockchain/state_transition/src/beacon/helpers/pq.rs b/crates/blockchain/state_transition/src/beacon/helpers/pq.rs new file mode 100644 index 000000000..df0e9cd6d --- /dev/null +++ b/crates/blockchain/state_transition/src/beacon/helpers/pq.rs @@ -0,0 +1,647 @@ +//! Helpers the PQ fork adds or modifies (consensus-specs +//! `specs/_features/pq/beacon-chain.md`, on the `pq` branch at +//! `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`). +//! +//! # Keys +//! +//! A PQ registry record holds its XMSS key in the 48-byte form a request names +//! it by, 16 zero bytes then the key (see `containers::pq`'s module doc). That +//! is why the specification's `XMSSPubkey(request.pubkey[16:])` comparisons +//! appear here as plain 48-byte comparisons against `validator.pubkey` once a +//! request has been checked not to be a legacy one ([`is_legacy_pubkey`]). +//! +//! # Signatures +//! +//! [`xmss_verify`] and [`verify_aggregates`] are the specification's +//! `xmss.Verify` and `stark.VerifyAggregates`, over `ethlambda_crypto`'s +//! leanXMSS. The specification's own test generator runs with XMSS inactive, +//! so its fixtures carry stub signatures and empty proofs that only an +//! inactive verifier accepts; [`disable_xmss_verification`], compiled only for +//! the spec-test binary, is that switch. `GetLeafIndex` and the empty-signature +//! comparison for self-built bids read the bytes either way, as the +//! specification's do. + +use std::sync::atomic::{AtomicBool, Ordering}; + +use ethlambda_crypto::{lean_xmss, stark}; + +use crate::beacon::constants; +use crate::beacon::containers::{BeaconState, electra, pq}; +use crate::beacon::error::{Error, Result}; +use crate::beacon::fork::ForkName; +use crate::beacon::hash::hash; +use crate::beacon::preset; +use crate::beacon::primitives::{ + BlsPubkey, Bytes32, CommitteeIndex, Epoch, Root, Slot, ValidatorIndex, XmssPubkey, + XmssSignature, +}; + +use super::accessors::{ + CommitteeCache, CommitteeCacheExt, get_block_root_at_slot, get_committee_count_per_slot, + get_current_epoch, get_domain, get_seed, +}; +use super::gloas::compute_balance_weighted_selection; +use super::misc::{ + compute_epoch_at_slot, compute_signing_root, compute_start_slot_at_epoch, + is_valid_merkle_branch, +}; + +// --------------------------------------------------------------------------- +// XMSS and STARK verification +// --------------------------------------------------------------------------- + +static XMSS_VERIFICATION: AtomicBool = AtomicBool::new(true); + +/// Turns `xmss.Verify` and `stark.VerifyAggregates` into the specification's +/// inactive stubs, which accept everything, for the rest of the process. +/// +/// The specification's test generator runs with XMSS inactive (key generation +/// takes minutes in its reference implementation), so the fixtures for the PQ +/// fork carry stub signatures and empty proofs. Only the spec-test binary can +/// reach this. +#[cfg(feature = "beacon-spec-tests")] +pub fn disable_xmss_verification() { + XMSS_VERIFICATION.store(false, Ordering::Relaxed); +} + +fn xmss_verification_enabled() -> bool { + XMSS_VERIFICATION.load(Ordering::Relaxed) +} + +/// The specification's `xmss.Verify`. +pub fn xmss_verify( + pubkey: &XmssPubkey, + leaf_index: u64, + message: Root, + signature: &XmssSignature, +) -> bool { + if !xmss_verification_enabled() { + return true; + } + lean_xmss::verify(&pubkey.0, leaf_index, &message.0, &signature.0) +} + +/// The specification's `xmss.GetLeafIndex`: the leaf index a signature carries, +/// read without verifying it. +pub fn get_leaf_index(signature: &XmssSignature) -> u64 { + lean_xmss::signature_index(&signature.0).expect("an XMSSSignature is longer than four bytes") +} + +/// The specification's `XMSSAggregateStatement`. +pub type AggregateStatement = stark::AggregateStatement; + +/// The specification's `stark.ProveAggregates`: one proof of every +/// statement's signatures, `signatures[i]` being those of `statements[i]`'s +/// keys, in their order. +pub fn prove_aggregates>( + statements: &[AggregateStatement], + signatures: &[Vec], +) -> std::result::Result, stark::ProveError> { + stark::prove_aggregates(statements, signatures) +} + +/// The specification's `stark.VerifyAggregates`. +pub fn verify_aggregates(statements: &[AggregateStatement], proof: &[u8]) -> bool { + if !xmss_verification_enabled() { + return true; + } + stark::verify_aggregates(statements, proof) +} + +// --------------------------------------------------------------------------- +// Misc +// --------------------------------------------------------------------------- + +/// `compute_legacy_xmss_pubkey`: the XMSS key derived from a BLS key, which a +/// validator or builder that registered none takes at the fork. No signature +/// verifies against it. +pub fn compute_legacy_xmss_pubkey(pubkey: &BlsPubkey) -> XmssPubkey { + XmssPubkey(legacy_digest(pubkey, constants::LEGACY_XMSS_PUBKEY_DOMAIN).0) +} + +/// `compute_legacy_vrf_root`: the VRF root derived from a BLS key. +pub fn compute_legacy_vrf_root(pubkey: &BlsPubkey) -> Root { + legacy_digest(pubkey, constants::LEGACY_VRF_ROOT_DOMAIN) +} + +fn legacy_digest(pubkey: &BlsPubkey, domain: u8) -> Bytes32 { + let mut input = Vec::with_capacity(pubkey.0.len() + 1); + input.extend_from_slice(&pubkey.0); + input.push(domain); + hash(&input) +} + +/// `is_legacy_pubkey`: whether a 48-byte request key names a BLS key (made +/// before the fork) rather than an XMSS one. A compressed BLS key's first byte +/// has its top bit set, so it never starts with the 16 zero bytes an XMSS key +/// is prefixed with. +pub fn is_legacy_pubkey(pubkey: &BlsPubkey) -> bool { + pubkey.0[..constants::XMSS_DEPOSIT_PREFIX.len()] != constants::XMSS_DEPOSIT_PREFIX +} + +/// `compute_deposit_pubkey`: the XMSS key a deposit naming `pubkey` is for. +pub fn compute_deposit_pubkey(pubkey: &BlsPubkey) -> XmssPubkey { + if is_legacy_pubkey(pubkey) { + compute_legacy_xmss_pubkey(pubkey) + } else { + XmssPubkey::from_request_key(pubkey) + } +} + +/// `compute_deposit_vrf_root`: the VRF root of the validator `deposit` creates, +/// the first 32 bytes of its signature field (or the legacy derivation). +pub fn compute_deposit_vrf_root(deposit: &electra::PendingDeposit) -> Root { + if is_legacy_pubkey(&deposit.pubkey) { + compute_legacy_vrf_root(&deposit.pubkey) + } else { + Root::from_slice(&deposit.signature.0[..32]) + } +} + +/// `compute_xmss_attestation_leaf_index`: the leaf an attestation for target +/// `epoch` is signed at, one per target epoch past the reserved leaves. +pub fn compute_xmss_attestation_leaf_index(epoch: Epoch) -> Result { + constants::XMSS_RESERVED_LEAVES + .checked_add(epoch) + .ok_or(Error::ArithmeticOverflow( + "compute_xmss_attestation_leaf_index: XMSS_RESERVED_LEAVES + epoch", + )) +} + +/// `compute_xmss_leaf_index`: the leaf a message of the type `leaf_offset` +/// names is signed at for `slot`. +pub fn compute_xmss_leaf_index(slot: Slot, leaf_offset: u64) -> Result { + slot.checked_mul(constants::XMSS_LEAVES_PER_SLOT) + .and_then(|slot_leaves| { + (constants::XMSS_RESERVED_LEAVES + + constants::XMSS_ATTESTATION_LEAVES + + constants::XMSS_BUILDER_BID_LEAVES) + .checked_add(slot_leaves) + }) + .and_then(|leaf| leaf.checked_add(leaf_offset)) + .ok_or(Error::ArithmeticOverflow( + "compute_xmss_leaf_index: slot * XMSS_LEAVES_PER_SLOT + offsets", + )) +} + +/// `is_builder_bid_leaf_index`: whether `leaf_index` is one of the leaves +/// reserved for builder bids. +pub fn is_builder_bid_leaf_index(leaf_index: u64) -> bool { + let start = constants::XMSS_RESERVED_LEAVES + constants::XMSS_ATTESTATION_LEAVES; + (start..start + constants::XMSS_BUILDER_BID_LEAVES).contains(&leaf_index) +} + +/// `is_valid_vrf_proof`: whether `proof` opens the leaf at index `slot` of the +/// tree under the VRF root of the validator at `validator_index`. +pub fn is_valid_vrf_proof( + state: &BeaconState, + validator_index: ValidatorIndex, + slot: Slot, + proof: &pq::VrfProof, +) -> Result { + let root = state.pq_validator(validator_index)?.vrf_root; + Ok(is_valid_merkle_branch( + proof[0], + &proof[1..], + constants::VRF_TREE_DEPTH as u64, + slot, + root, + )) +} + +// --------------------------------------------------------------------------- +// Aggregation committees +// --------------------------------------------------------------------------- + +/// `compute_aggregation_committee`: the aggregators of the beacon committee at +/// `slot` for `index`, `AGGREGATORS_PER_COMMITTEE` of them (fewer for a +/// smaller committee) sampled by effective balance without duplicates. +pub fn compute_aggregation_committee( + state: &BeaconState, + slot: Slot, + index: CommitteeIndex, + committees: &CommitteeCache, +) -> Result { + let epoch = compute_epoch_at_slot(slot); + let mut seed_input = Vec::with_capacity(48); + seed_input.extend_from_slice(&get_seed(state, epoch, constants::DOMAIN_SELECTION_PROOF).0); + seed_input.extend_from_slice(&slot.to_le_bytes()); + seed_input.extend_from_slice(&index.to_le_bytes()); + let seed = hash(&seed_input); + + let mut candidates = committees + .committees(state, epoch) + .committee(slot, index)? + .to_vec(); + let mut aggregators: Vec = Vec::new(); + while (aggregators.len() as u64) < constants::AGGREGATORS_PER_COMMITTEE + && !candidates.is_empty() + { + let mut draw_input = Vec::with_capacity(40); + draw_input.extend_from_slice(&seed.0); + draw_input.extend_from_slice(&(aggregators.len() as u64).to_le_bytes()); + let aggregator = + compute_balance_weighted_selection(state, &candidates, hash(&draw_input), 1, false)?[0]; + aggregators.push(aggregator); + candidates.retain(|&candidate| candidate != aggregator); + } + Ok(pq::AggregationCommittee::from(aggregators)) +} + +/// `compute_aggregation_committees`: the aggregation committee of each beacon +/// committee at `slot`, in committee order. +pub fn compute_aggregation_committees( + state: &BeaconState, + slot: Slot, + committees: &CommitteeCache, +) -> Result { + let committees_per_slot = get_committee_count_per_slot(state, compute_epoch_at_slot(slot)); + let mut out = Vec::with_capacity(committees_per_slot as usize); + for index in 0..committees_per_slot { + out.push(compute_aggregation_committee( + state, slot, index, committees, + )?); + } + Ok(pq::AggregationCommittees::from(out)) +} + +/// `get_aggregation_committee`: the aggregation committee of the beacon +/// committee at `slot` for `index`, read from the state's window. Defined for +/// the previous epoch through `MIN_SEED_LOOKAHEAD` epochs ahead. +pub fn get_aggregation_committee( + state: &BeaconState, + slot: Slot, + index: CommitteeIndex, +) -> Result { + let BeaconState::Pq(inner) = state else { + return Err(Error::UnsupportedForFork { + function: "get_aggregation_committee", + fork: state.fork_name(), + }); + }; + let epoch = compute_epoch_at_slot(slot); + let state_epoch = get_current_epoch(state); + let position = if epoch < state_epoch { + crate::beacon::verify( + epoch + 1 == state_epoch, + "get_aggregation_committee: epoch + 1 == state_epoch", + )?; + slot % preset::SLOTS_PER_EPOCH + } else { + crate::beacon::verify( + epoch <= state_epoch + preset::MIN_SEED_LOOKAHEAD, + "get_aggregation_committee: epoch <= state_epoch + MIN_SEED_LOOKAHEAD", + )?; + (epoch - state_epoch + 1) * preset::SLOTS_PER_EPOCH + slot % preset::SLOTS_PER_EPOCH + } as usize; + let slot_committees = &inner.aggregation_committee_window[position]; + slot_committees + .get(index as usize) + .cloned() + .ok_or(Error::IndexOutOfBounds { + index: index as usize, + len: slot_committees.len(), + }) +} + +/// `initialize_aggregation_committee_window`: an empty previous epoch, then +/// the current and lookahead epochs' aggregation committees. What the fork +/// upgrade fills the window with. +pub fn initialize_aggregation_committee_window( + state: &BeaconState, +) -> Result { + let committees = CommitteeCache::default(); + let current_epoch = get_current_epoch(state); + let start_slot = compute_start_slot_at_epoch(current_epoch); + let end_slot = compute_start_slot_at_epoch(current_epoch + preset::MIN_SEED_LOOKAHEAD + 1); + let mut window = vec![pq::AggregationCommittees::default(); preset::SLOTS_PER_EPOCH as usize]; + for slot in start_slot..end_slot { + window.push(compute_aggregation_committees(state, slot, &committees)?); + } + Ok(pq::AggregationCommitteeWindow::try_from(window) + .expect("one empty epoch and MIN_SEED_LOOKAHEAD + 1 computed ones fill the window")) +} + +// --------------------------------------------------------------------------- +// Attestations +// --------------------------------------------------------------------------- + +/// The signing root and leaf an attestation for `data` is signed at: +/// `compute_signing_root(data, get_domain(state, DOMAIN_BEACON_ATTESTER, +/// target.epoch))` and `compute_xmss_attestation_leaf_index(target.epoch)`. +pub fn attestation_message( + state: &BeaconState, + data: &crate::beacon::containers::shared::AttestationData, +) -> Result<(Root, u64)> { + use crate::beacon::primitives::HashTreeRoot as _; + let domain = get_domain( + state, + constants::DOMAIN_BEACON_ATTESTER, + Some(data.target.epoch), + ); + let message = compute_signing_root(data.hash_tree_root(), domain); + Ok(( + message, + compute_xmss_attestation_leaf_index(data.target.epoch)?, + )) +} + +/// The XMSS keys of `indices`, in order. +pub fn pubkeys_of(state: &BeaconState, indices: &[ValidatorIndex]) -> Result> { + indices + .iter() + .map(|&index| Ok(state.pq_validator(index)?.pubkey().0)) + .collect() +} + +/// The committee indices `committee_bits` sets, in order. +pub fn get_committee_indices(committee_bits: &pq::CommitteeBits) -> Vec { + (0..committee_bits.len()) + .filter(|&index| committee_bits.get(index).unwrap_or(false)) + .map(|index| index as CommitteeIndex) + .collect() +} + +/// `get_attestation_statements`: for each committee of `attestation` in +/// order, the statement its attesters' signatures prove. +pub fn get_attestation_statements( + state: &BeaconState, + attestation: &pq::Attestation, + committees: &CommitteeCache, +) -> Result> { + let (message, leaf_index) = attestation_message(state, &attestation.data)?; + let epoch_committees = committees.committees(state, attestation.data.target.epoch); + let mut statements = Vec::new(); + let mut committee_offset = 0usize; + for committee_index in get_committee_indices(&attestation.committee_bits) { + let committee = epoch_committees.committee(attestation.data.slot, committee_index)?; + let attesters: Vec = committee + .iter() + .enumerate() + .filter(|(position, _)| { + attestation + .aggregation_bits + .get(committee_offset + position) + .unwrap_or(false) + }) + .map(|(_, &validator_index)| validator_index) + .collect(); + statements.push(AggregateStatement { + pubkeys: pubkeys_of(state, &attesters)?, + leaf_index, + message: message.0, + }); + committee_offset += committee.len(); + } + Ok(statements) +} + +/// `get_attesting_indices`, sorted: the attesters `attestation`'s bits name in +/// the committees its `committee_bits` select, under the PQ fork's committee +/// count. +pub fn get_attesting_indices( + state: &BeaconState, + attestation: &pq::Attestation, + committees: &CommitteeCache, +) -> Result> { + let epoch_committees = + committees.committees(state, compute_epoch_at_slot(attestation.data.slot)); + let mut indices = Vec::new(); + let mut committee_offset = 0usize; + for committee_index in get_committee_indices(&attestation.committee_bits) { + let committee = epoch_committees.committee(attestation.data.slot, committee_index)?; + for (position, &validator_index) in committee.iter().enumerate() { + if attestation + .aggregation_bits + .get(committee_offset + position) + .unwrap_or(false) + { + indices.push(validator_index); + } + } + committee_offset += committee.len(); + } + indices.sort_unstable(); + indices.dedup(); + Ok(indices) +} + +/// `is_valid_indexed_attestation`: not empty, sorted and unique indices within +/// the PQ bound, and a proof of one statement over the attesters' keys. +pub fn is_valid_indexed_attestation( + state: &BeaconState, + indexed_attestation: &pq::IndexedAttestation, +) -> Result { + let indices: &[ValidatorIndex] = &indexed_attestation.attesting_indices; + if indices.is_empty() + || indices.len() > preset::MAX_VALIDATORS_PER_COMMITTEE * preset::MAX_COMMITTEES_PER_SLOT_PQ + || !super::predicates::are_indices_sorted_and_unique(indices) + { + return Ok(false); + } + let (message, leaf_index) = attestation_message(state, &indexed_attestation.data)?; + // An index outside the registry has no key: the attestation is invalid, + // not the block's processing broken. + let Ok(pubkeys) = pubkeys_of(state, indices) else { + return Ok(false); + }; + let statement = AggregateStatement { + pubkeys, + leaf_index, + message: message.0, + }; + Ok(verify_aggregates(&[statement], &indexed_attestation.proof)) +} + +// --------------------------------------------------------------------------- +// Preregistrations +// --------------------------------------------------------------------------- + +/// The slot of the outstanding parent payload, whose execution requests are +/// the ones being applied: what a preregistration's expiry is compared with. +fn parent_payload_slot(state: &pq::BeaconState) -> Slot { + state.latest_execution_payload_bid.slot +} + +/// `is_active_validator_preregistration`. +pub fn is_active_validator_preregistration( + state: &pq::BeaconState, + preregistration: &pq::ValidatorPreregistration, +) -> bool { + parent_payload_slot(state) < preregistration.expiry_slot +} + +/// `is_active_builder_preregistration`. +pub fn is_active_builder_preregistration( + state: &pq::BeaconState, + preregistration: &pq::BuilderPreregistration, +) -> bool { + parent_payload_slot(state) < preregistration.expiry_slot +} + +/// `get_active_validator_preregistration`. +pub fn get_active_validator_preregistration( + state: &pq::BeaconState, + pubkey: &XmssPubkey, +) -> Option { + state + .validator_preregistrations + .iter() + .find(|preregistration| preregistration.pubkey == *pubkey) + .filter(|preregistration| is_active_validator_preregistration(state, preregistration)) + .cloned() +} + +/// `get_active_builder_preregistration`. +pub fn get_active_builder_preregistration( + state: &pq::BeaconState, + pubkey: &XmssPubkey, +) -> Option { + state + .builder_preregistrations + .iter() + .find(|preregistration| preregistration.pubkey == *pubkey) + .filter(|preregistration| is_active_builder_preregistration(state, preregistration)) + .cloned() +} + +/// The PQ state, mutably, or an error naming the function that needs one. +pub(crate) fn pq_state<'a>( + state: &'a mut BeaconState, + function: &'static str, +) -> Result<&'a mut pq::BeaconState> { + match state { + BeaconState::Pq(inner) => Ok(inner), + other => Err(Error::UnsupportedForFork { + function, + fork: other.fork_name(), + }), + } +} + +/// Whether `fork` signs with leanXMSS. +pub fn is_xmss_fork(fork: ForkName) -> bool { + fork == ForkName::Pq +} + +/// How many slots after its own an attestation can be included at the +/// earliest in a block on `state`, which is also the delay the timely-head +/// flag requires: `MIN_ATTESTATION_INCLUSION_DELAY_PQ` from PQ on, which +/// gives the proposer a slot to merge aggregates' proofs. +pub fn min_attestation_inclusion_delay(state: &BeaconState) -> u64 { + match state { + BeaconState::Pq(_) => preset::MIN_ATTESTATION_INCLUSION_DELAY_PQ, + _ => preset::MIN_ATTESTATION_INCLUSION_DELAY, + } +} + +/// `get_latest_block_slot` (new in PQ): the slot of the latest block at or +/// before `slot`, where `parent_slot` is the slot of the parent of the block +/// being processed. +/// +/// With a two-slot inclusion delay, the block a timely attestation votes for +/// may be the block being processed's grandparent, so its payload +/// availability is read at its own slot rather than the parent's. A skipped +/// slot repeats the previous block's root, which is what the walk back +/// follows; it stops at the oldest root `block_roots` still holds. +pub fn get_latest_block_slot(state: &BeaconState, slot: Slot, parent_slot: Slot) -> Result { + if parent_slot <= slot { + return Ok(parent_slot); + } + let block_root = get_block_root_at_slot(state, slot)?; + let mut slot = slot; + while slot > 0 + && slot + preset::SLOTS_PER_HISTORICAL_ROOT as u64 > state.slot() + && get_block_root_at_slot(state, slot - 1)? == block_root + { + slot -= 1; + } + Ok(slot) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn leaf_ranges_are_disjoint_and_ordered() { + let reserved_end = constants::XMSS_RESERVED_LEAVES; + assert_eq!( + compute_xmss_attestation_leaf_index(0).unwrap(), + reserved_end + ); + let bids_start = reserved_end + constants::XMSS_ATTESTATION_LEAVES; + assert!(is_builder_bid_leaf_index(bids_start)); + assert!(!is_builder_bid_leaf_index(bids_start - 1)); + let slot_leaves_start = bids_start + constants::XMSS_BUILDER_BID_LEAVES; + assert!(!is_builder_bid_leaf_index(slot_leaves_start)); + assert_eq!(compute_xmss_leaf_index(0, 0).unwrap(), slot_leaves_start); + assert_eq!( + compute_xmss_leaf_index(2, constants::LEAF_OFFSET_EXECUTION_PAYLOAD_ENVELOPE).unwrap(), + slot_leaves_start + 2 * constants::XMSS_LEAVES_PER_SLOT + 4 + ); + } + + #[test] + fn a_bls_key_is_legacy_and_an_xmss_one_is_not() { + let mut bls = BlsPubkey::default(); + bls.0[0] = 0x80; + assert!(is_legacy_pubkey(&bls)); + let xmss = XmssPubkey([7; 32]); + assert!(!is_legacy_pubkey(&xmss.to_request_key())); + assert_eq!(compute_deposit_pubkey(&xmss.to_request_key()), xmss); + assert_eq!( + compute_deposit_pubkey(&bls), + compute_legacy_xmss_pubkey(&bls) + ); + assert_ne!( + compute_legacy_xmss_pubkey(&bls).0, + compute_legacy_vrf_root(&bls).0 + ); + } + + /// A PQ state at `slot` whose block root at each earlier slot is + /// `roots[slot]`: a skipped slot repeats the root before it. + fn pq_state_with_roots(slot: Slot, roots: &[u8]) -> BeaconState { + let mut state = crate::beacon::helpers::test_state::with_validators_at(ForkName::Pq, 8); + *state.slot_mut() = slot; + for (index, &root) in roots.iter().enumerate() { + state.block_roots_mut()[index] = Root::repeat_byte(root); + } + state + } + + #[test] + fn the_latest_block_at_or_after_the_parent_is_the_parent() { + let state = pq_state_with_roots(10, &[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]); + assert_eq!(get_latest_block_slot(&state, 8, 8).unwrap(), 8); + assert_eq!(get_latest_block_slot(&state, 9, 8).unwrap(), 8); + } + + /// Blocks at 4 and 7, slots 5 and 6 skipped: an attestation to slot 6 + /// voted for the block at 4, so that is the slot whose payload counts. + #[test] + fn a_skipped_slot_resolves_to_the_block_before_it() { + let state = pq_state_with_roots(9, &[1, 2, 3, 4, 5, 5, 5, 8, 9]); + assert_eq!(get_latest_block_slot(&state, 6, 7).unwrap(), 4); + assert_eq!(get_latest_block_slot(&state, 4, 7).unwrap(), 4); + } + + /// The walk stops at the oldest root `block_roots` still holds rather + /// than reading past it. + #[test] + fn the_walk_stops_at_the_oldest_root_kept() { + let history = preset::SLOTS_PER_HISTORICAL_ROOT as u64; + let state = pq_state_with_roots(history + 2, &[]); + // Every root is the default: the walk would run to slot 0 if it could. + assert_eq!(get_latest_block_slot(&state, 3, history).unwrap(), 2); + } + + #[test] + fn only_a_pq_state_takes_the_two_slot_delay() { + let pq = crate::beacon::helpers::test_state::with_validators_at(ForkName::Pq, 8); + let gloas = crate::beacon::helpers::test_state::with_validators_at(ForkName::Gloas, 8); + assert_eq!(min_attestation_inclusion_delay(&pq), 2); + assert_eq!(min_attestation_inclusion_delay(&gloas), 1); + } +} diff --git a/crates/blockchain/state_transition/src/beacon/helpers/target_balances.rs b/crates/blockchain/state_transition/src/beacon/helpers/target_balances.rs index e2eefbb7b..feead52bf 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/target_balances.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/target_balances.rs @@ -192,6 +192,14 @@ pub fn block_has_slashings(block: &SignedBeaconBlock) -> bool { !b.message.body.proposer_slashings.is_empty() || !b.message.body.attester_slashings.is_empty() } + SignedBeaconBlock::Eip8483(b) => { + !b.message.body.proposer_slashings.is_empty() + || !b.message.body.attester_slashings.is_empty() + } + SignedBeaconBlock::Pq(b) => { + !b.message.body.proposer_slashings.is_empty() + || !b.message.body.attester_slashings.is_empty() + } // Not a Beacon Chain block: nothing here ever carries one. SignedBeaconBlock::Lean(_) => true, } diff --git a/crates/blockchain/state_transition/src/beacon/helpers/test_state.rs b/crates/blockchain/state_transition/src/beacon/helpers/test_state.rs index d939fa20d..d6371bc6c 100644 --- a/crates/blockchain/state_transition/src/beacon/helpers/test_state.rs +++ b/crates/blockchain/state_transition/src/beacon/helpers/test_state.rs @@ -29,8 +29,8 @@ use crate::beacon::containers::shared::{ Balances, EpochParticipation, InactivityScores, Validator, }; use crate::beacon::containers::{ - BeaconState, BlockRoots, RandaoMixes, Slashings, altair, bellatrix, capella, deneb, electra, - fulu, gloas, phase0, + BeaconState, BlockRoots, RandaoMixes, Slashings, altair, bellatrix, capella, deneb, eip8483, + electra, fulu, gloas, phase0, pq, }; use crate::beacon::fork::ForkName; use crate::beacon::preset; @@ -141,6 +141,8 @@ pub fn with_validators_at(fork: ForkName, count: usize) -> BeaconState { ForkName::Electra => BeaconState::Electra(electra_state(count)), ForkName::Fulu => BeaconState::Fulu(fulu_state(count)), ForkName::Gloas => BeaconState::Gloas(gloas_state(count)), + ForkName::Eip8483 => BeaconState::Eip8483(eip8483_state(count)), + ForkName::Pq => BeaconState::Pq(pq_state(count)), // A test asking for a lean state from a beacon builder, which no fixture // fork name can produce; the `fork:` form, since the argument is the // whole input. @@ -170,6 +172,8 @@ pub fn replace_balances(state: &mut BeaconState, balances: Vec) { BeaconState::Electra(state) => state.balances = bounded(balances), BeaconState::Fulu(state) => state.balances = bounded(balances), BeaconState::Gloas(state) => state.balances = balances.into(), + BeaconState::Eip8483(state) => state.balances = balances.into(), + BeaconState::Pq(state) => state.balances = balances.into(), BeaconState::Lean(_) => lean_state_unreachable("replace_balances"), } } @@ -653,6 +657,83 @@ fn gloas_state(count: usize) -> gloas::BeaconState { } } +/// [`gloas_state`] in EIP-8483's shape: no registrations yet. +fn eip8483_state(count: usize) -> eip8483::BeaconState { + let BeaconState::Eip8483(state) = crate::beacon::upgrade::upgrade_to_eip8483( + &BeaconState::Gloas(gloas_state(count)), + &crate::beacon::config::Config::active(), + ) + .expect("a gloas state upgrades to EIP-8483") else { + unreachable!("upgrade_to_eip8483 returns an EIP-8483 state") + }; + state +} + +/// [`gloas_state`] in the PQ shape: every validator keeps its all-zero key +/// (in the 48-byte request form, which the all-zero key already is) and a zero +/// VRF root, and the aggregation committee window starts empty. A test that +/// needs real keys, VRF roots or a filled window sets them afterward. +fn pq_state(count: usize) -> pq::BeaconState { + let gloas = gloas_state(count); + let validators: Vec = gloas + .validators + .iter() + .map(|validator| pq::Validator { + record: validator.clone(), + vrf_root: Root::ZERO, + }) + .collect(); + pq::BeaconState { + genesis_time: gloas.genesis_time, + genesis_validators_root: gloas.genesis_validators_root, + slot: gloas.slot, + fork: gloas.fork, + latest_block_header: gloas.latest_block_header, + block_roots: gloas.block_roots, + state_roots: gloas.state_roots, + historical_roots: gloas.historical_roots, + eth1_data: gloas.eth1_data, + eth1_data_votes: gloas.eth1_data_votes, + eth1_deposit_index: gloas.eth1_deposit_index, + validators: validators.into(), + balances: gloas.balances, + randao_mixes: gloas.randao_mixes, + slashings: gloas.slashings, + previous_epoch_participation: gloas.previous_epoch_participation, + current_epoch_participation: gloas.current_epoch_participation, + justification_bits: gloas.justification_bits, + previous_justified_checkpoint: gloas.previous_justified_checkpoint, + current_justified_checkpoint: gloas.current_justified_checkpoint, + finalized_checkpoint: gloas.finalized_checkpoint, + inactivity_scores: gloas.inactivity_scores, + latest_block_hash: gloas.latest_block_hash, + next_withdrawal_index: gloas.next_withdrawal_index, + next_withdrawal_validator_index: gloas.next_withdrawal_validator_index, + historical_summaries: gloas.historical_summaries, + deposit_requests_start_index: gloas.deposit_requests_start_index, + deposit_balance_to_consume: gloas.deposit_balance_to_consume, + exit_balance_to_consume: gloas.exit_balance_to_consume, + earliest_exit_epoch: gloas.earliest_exit_epoch, + consolidation_balance_to_consume: gloas.consolidation_balance_to_consume, + earliest_consolidation_epoch: gloas.earliest_consolidation_epoch, + pending_deposits: gloas.pending_deposits, + pending_partial_withdrawals: gloas.pending_partial_withdrawals, + pending_consolidations: gloas.pending_consolidations, + proposer_lookahead: gloas.proposer_lookahead, + builders: pq::Builders(gloas.builders), + next_withdrawal_builder_index: gloas.next_withdrawal_builder_index, + execution_payload_availability: gloas.execution_payload_availability, + builder_pending_payments: gloas.builder_pending_payments, + builder_pending_withdrawals: gloas.builder_pending_withdrawals, + latest_execution_payload_bid: gloas.latest_execution_payload_bid, + payload_expected_withdrawals: gloas.payload_expected_withdrawals, + ptc_window: gloas.ptc_window, + validator_preregistrations: Default::default(), + builder_preregistrations: Default::default(), + aggregation_committee_window: pq::empty_aggregation_committee_window(), + } +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/blockchain/state_transition/src/beacon/mod.rs b/crates/blockchain/state_transition/src/beacon/mod.rs index 770835b41..95cee760f 100644 --- a/crates/blockchain/state_transition/src/beacon/mod.rs +++ b/crates/blockchain/state_transition/src/beacon/mod.rs @@ -87,6 +87,7 @@ pub mod hash; pub mod helpers; pub mod kzg; pub mod payload_attestation_pool; +pub mod pq_block_production; pub mod precheck; pub mod stf; pub mod sync_committee_pool; diff --git a/crates/blockchain/state_transition/src/beacon/pq_block_production.rs b/crates/blockchain/state_transition/src/beacon/pq_block_production.rs new file mode 100644 index 000000000..5a9c97929 --- /dev/null +++ b/crates/blockchain/state_transition/src/beacon/pq_block_production.rs @@ -0,0 +1,672 @@ +//! PQ block production (`specs/_features/pq/validator.md`): gloas's +//! self-build with PQ's shapes. The randao reveal is the proposer's VRF proof, +//! the self-build bid carries the empty `XMSSSignature()`, attestations carry +//! no signatures and the body one merged `proof` of theirs, the execution +//! requests are PQ's, and the block has no sync aggregate, payload +//! attestations, exits or credential changes. + +use std::collections::{BTreeMap, HashSet}; +use std::sync::Arc; + +use ethlambda_crypto::stark; +use ethlambda_storage::pools::PooledPqAggregate; + +use crate::beacon::config::Config; +use crate::beacon::constants; +use crate::beacon::containers::{BeaconState, gloas, pq}; +use crate::beacon::error::{Error, Result, verify}; +use crate::beacon::gloas_block_production::GloasPayloadInputs; +use crate::beacon::helpers::accessors::{ + ActiveBalanceCache, CommitteeCache, PubkeyIndexCache, get_beacon_proposer_index, + get_block_root, get_current_epoch, get_previous_epoch, get_randao_mix, +}; +use crate::beacon::helpers::gloas::{get_attestation_participation_flag_indices, gloas_state_ref}; +use crate::beacon::helpers::misc::compute_epoch_at_slot; +use crate::beacon::helpers::pq::{ + AggregateStatement, get_attestation_statements, get_attesting_indices, get_committee_indices, +}; +use crate::beacon::preset; +use crate::beacon::primitives::{ + Bytes32, Epoch, HashTreeRoot as _, KzgCommitment, Root, ValidatorIndex, +}; +use crate::beacon::stf; + +/// `get_execution_requests` (PQ `validator.md`): the execution client's +/// EIP-7685 request list into PQ's `ExecutionRequests`, with its +/// preregistration and voluntary-exit request types. Strictly ascending +/// types and no empty element are checked, as gloas's are. +pub fn parse_pq_execution_requests(list: &[Vec]) -> Result { + let mut requests = pq::ExecutionRequests::default(); + let mut previous_type: Option = None; + for element in list { + let (&request_type, data) = element + .split_first() + .ok_or(Error::SpecAssert("an execution request carries its type"))?; + verify(!data.is_empty(), "an execution request is not empty")?; + verify( + previous_type.is_none_or(|previous| request_type > previous), + "execution request types are strictly ascending", + )?; + previous_type = Some(request_type); + let malformed = |_| Error::SpecAssert("an execution request list decodes as SSZ"); + match request_type { + constants::DEPOSIT_REQUEST_TYPE => { + requests.deposits = libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::WITHDRAWAL_REQUEST_TYPE => { + requests.withdrawals = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::CONSOLIDATION_REQUEST_TYPE => { + requests.consolidations = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::BUILDER_DEPOSIT_REQUEST_TYPE => { + requests.builder_deposits = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::BUILDER_EXIT_REQUEST_TYPE => { + requests.builder_exits = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::VALIDATOR_PREREGISTRATION_REQUEST_TYPE => { + requests.validator_preregistrations = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::BUILDER_PREREGISTRATION_REQUEST_TYPE => { + requests.builder_preregistrations = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + constants::VOLUNTARY_EXIT_REQUEST_TYPE => { + requests.voluntary_exits = + libssz::SszDecode::from_ssz_bytes(data).map_err(malformed)?; + } + _ => return Err(Error::SpecAssert("a known execution request type")), + } + } + Ok(requests) +} + +/// [`crate::beacon::gloas_block_production::gloas_payload_inputs`] on a PQ +/// state: the parent's requests, when building on its full payload, are PQ's +/// and applied by PQ's `apply_parent_execution_payload`. +pub fn pq_payload_inputs( + state: &BeaconState, + build_on_full: bool, + parent_requests: &pq::ExecutionRequests, + config: &Config, +) -> Result { + let inner = gloas_state_ref(state, "pq_payload_inputs")?; + let parent_bid = &inner.latest_execution_payload_bid; + let (withdrawals, head_block_hash) = if build_on_full { + let mut applied = state.clone(); + stf::pq::apply_parent_execution_payload( + &mut applied, + parent_requests, + config, + &ActiveBalanceCache::default(), + &PubkeyIndexCache::default(), + )?; + ( + stf::gloas::get_expected_withdrawals(&applied)?.withdrawals, + parent_bid.block_hash, + ) + } else { + ( + inner.payload_expected_withdrawals.iter().cloned().collect(), + parent_bid.parent_block_hash, + ) + }; + Ok(GloasPayloadInputs { + timestamp: stf::bellatrix::compute_timestamp_at_slot(state, state.slot(), config), + prev_randao: get_randao_mix(state, get_current_epoch(state)), + withdrawals, + head_block_hash, + parent_beacon_block_root: state.latest_block_header().hash_tree_root(), + slot_number: state.slot(), + target_gas_limit: parent_bid.gas_limit, + }) +} + +/// What a PQ block body carries beyond what this module derives from the +/// state. +#[derive(Debug, Clone)] +pub struct PqBlockInputs { + /// The proposer's VRF proof for the slot. + pub randao_reveal: pq::VrfProof, + pub graffiti: Bytes32, + pub attestations: Vec, + /// `stark.MergeProofs` of the attestations' aggregate proofs, in their + /// order: empty with no attestations. + pub proof: Vec, + /// The parent envelope's requests when building on its full payload; + /// empty otherwise. + pub parent_execution_requests: pq::ExecutionRequests, + pub execution_payload: gloas::ExecutionPayload, + pub blob_kzg_commitments: Vec, + pub execution_requests: pq::ExecutionRequests, +} + +/// A produced PQ block and the unsigned envelope that reveals its payload. +#[derive(Debug, Clone)] +pub struct PqProduced { + pub block: pq::BeaconBlock, + pub envelope: pq::ExecutionPayloadEnvelope, +} + +/// The unsigned self-built PQ block for the slot `state` has been advanced +/// to, its state root computed by running PQ's `process_block` on a copy +/// (which also refuses a body the network would), and the unsigned envelope +/// for its payload, checked against that post-state but for its signature. +pub fn assemble_pq_block( + state: &BeaconState, + inputs: PqBlockInputs, + config: &Config, + options: stf::block::BlockProcessingOptions, +) -> Result { + verify( + matches!(state, BeaconState::Pq(_)), + "PQ block production runs on a PQ state", + )?; + let payload = &inputs.execution_payload; + let bid = gloas::ExecutionPayloadBid { + parent_block_hash: payload.parent_hash, + parent_block_root: state.latest_block_header().hash_tree_root(), + block_hash: payload.block_hash, + prev_randao: payload.prev_randao, + fee_recipient: payload.fee_recipient, + gas_limit: payload.gas_limit, + builder_index: constants::BUILDER_INDEX_SELF_BUILD, + slot: state.slot(), + value: 0, + execution_payment: 0, + blob_kzg_commitments: inputs.blob_kzg_commitments.into(), + execution_requests_root: inputs.execution_requests.hash_tree_root(), + }; + let proof = pq::AttestationProof::try_from(inputs.proof) + .map_err(|_| Error::SpecAssert("the merged proof fits MAX_STARK_PROOF_SIZE"))?; + let body = pq::BeaconBlockBody { + randao_reveal: inputs.randao_reveal, + eth1_data: state.eth1_data().clone(), + graffiti: inputs.graffiti, + attestations: inputs.attestations.into(), + // Self-build: the empty `XMSSSignature()`, all zero bytes. + signed_execution_payload_bid: pq::SignedExecutionPayloadBid { + message: bid, + signature: Default::default(), + }, + parent_execution_requests: inputs.parent_execution_requests, + proof, + ..pq::BeaconBlockBody::empty() + }; + let mut block = pq::BeaconBlock { + slot: state.slot(), + proposer_index: get_beacon_proposer_index(state)?, + parent_root: state.latest_block_header().hash_tree_root(), + state_root: Root::ZERO, + body, + }; + + let mut post = state.clone(); + stf::pq::process_block( + &mut post, + &block, + config, + &CommitteeCache::default(), + &ActiveBalanceCache::default(), + &PubkeyIndexCache::default(), + options, + )?; + post.apply_pending_mutations(); + block.state_root = post.hash_tree_root(); + + let envelope = pq::ExecutionPayloadEnvelope { + payload: inputs.execution_payload, + execution_requests: inputs.execution_requests, + builder_index: constants::BUILDER_INDEX_SELF_BUILD, + beacon_block_root: block.hash_tree_root(), + parent_beacon_block_root: block.parent_root, + }; + check_envelope(&post, &block, &envelope, config)?; + Ok(PqProduced { block, envelope }) +} + +/// `verify_execution_payload_envelope`'s conditions but the signature and +/// the engine, against `post`, the state after `block`. +fn check_envelope( + post: &BeaconState, + block: &pq::BeaconBlock, + envelope: &pq::ExecutionPayloadEnvelope, + config: &Config, +) -> Result<()> { + let inner = gloas_state_ref(post, "assemble_pq_block")?; + let bid = &inner.latest_execution_payload_bid; + let payload = &envelope.payload; + verify( + envelope.beacon_block_root == block.hash_tree_root(), + "the envelope names the block", + )?; + verify( + payload.block_hash == bid.block_hash + && payload.prev_randao == bid.prev_randao + && payload.gas_limit == bid.gas_limit + && envelope.execution_requests.hash_tree_root() == bid.execution_requests_root, + "the envelope is the payload the bid commits to", + )?; + verify( + payload.slot_number == post.slot(), + "payload.slot_number == state.slot", + )?; + verify( + payload.parent_hash == *inner.latest_block_hash, + "payload.parent_hash == state.latest_block_hash", + )?; + verify( + payload.timestamp == stf::bellatrix::compute_timestamp_at_slot(post, post.slot(), config), + "payload.timestamp == compute_time_at_slot(state, state.slot)", + )?; + verify( + payload.withdrawals.hash_tree_root() == inner.payload_expected_withdrawals.hash_tree_root(), + "hash_tree_root(payload.withdrawals) == hash_tree_root(state.payload_expected_withdrawals)", + ) +} + +// --------------------------------------------------------------------------- +// Attestations +// --------------------------------------------------------------------------- + +/// What a PQ block packs from the pool: its attestations, and the network +/// aggregates behind them as the `(statements, proof)` children +/// `stark.MergeProofs` folds into the block's `proof`, in the order +/// `process_stark` reads the statements: the attestations', then each one's +/// committees ascending (PQ `validator.md`, "Proof"). +#[derive(Debug, Clone, Default)] +pub struct PqPacked { + pub attestations: Vec, + pub children: Vec<(Vec, pq::AttestationProof)>, +} + +impl PqPacked { + /// The block's `proof`: `stark.MergeProofs` of the children. Empty with + /// no attestations; one child is returned as it is, without proving. + pub fn merge(&self) -> std::result::Result, stark::ProveError> { + let children: Vec<(&[AggregateStatement], &[u8])> = self + .children + .iter() + .map(|(statements, proof)| (statements.as_slice(), &proof[..])) + .collect(); + stark::merge_proofs(&children) + } + + /// The first attestation alone, when it covers a single committee: its + /// aggregate's proof is then the block's `proof` as it is, with nothing + /// to merge. What a proposer packs when merging every child would not + /// finish in time; the first is the one the greedy pick valued most. + pub fn first_unmerged(&self) -> Option { + let first = self.attestations.first()?; + let child = self.children.first()?; + (get_committee_indices(&first.committee_bits).len() == 1).then(|| PqPacked { + attestations: vec![first.clone()], + children: vec![child.clone()], + }) + } +} + +/// A participation flag a block would newly set: `(target epoch, attester, +/// flag index)`. +type Gain = (Epoch, ValidatorIndex, usize); + +/// The network aggregates sharing one `AttestationData`, by committee, and +/// what packing them would newly credit, weighted. +#[derive(Debug, Default)] +struct DataGroup { + committees: BTreeMap>, + gains: BTreeMap, +} + +/// Pick the attestations for a PQ block at `state`'s slot from the pool's +/// `candidates`, each a validated single-committee network aggregate. +/// +/// Keeps only what `process_attestation` accepts at this slot (included at +/// least `MIN_ATTESTATION_INCLUSION_DELAY_PQ` after its own slot, targeting +/// the current or previous epoch, sourced from the justified checkpoint that +/// target implies, a payload vote of `0` or `1` and `0` for its own slot's +/// block), whose target root is this chain's, and whose committee here names +/// exactly the attesters it was validated with: a proof proves a fixed signer +/// set, so an aggregate read against another branch's committees would fail +/// `process_stark` for the whole block. +/// +/// Aggregates sharing data become one attestation over their committees +/// (`compute_on_chain_aggregate`). Up to `MAX_ATTESTATIONS_PQ` are taken +/// greedily by the weight of participation flags they newly set, so one +/// already on chain earns nothing again. +pub fn select_pq_attestations( + state: &BeaconState, + candidates: Vec>, + committees: &CommitteeCache, +) -> Result { + let current_epoch = get_current_epoch(state); + let previous_epoch = get_previous_epoch(state); + let parent_slot = state.latest_block_header().slot; + let (previous_participation, current_participation, _) = state.altair_validator_lists()?; + + let mut groups = BTreeMap::::new(); + for candidate in candidates { + let data = &candidate.attestation.data; + let target_epoch = data.target.epoch; + let expected_source = if target_epoch == current_epoch { + state.current_justified_checkpoint() + } else { + state.previous_justified_checkpoint() + }; + let includable = data.slot + preset::MIN_ATTESTATION_INCLUSION_DELAY_PQ <= state.slot() + && (target_epoch == current_epoch || target_epoch == previous_epoch) + && target_epoch == compute_epoch_at_slot(data.slot) + && data.source == expected_source + && data.index < 2 + && get_block_root(state, target_epoch).is_ok_and(|root| root == data.target.root); + if !includable { + continue; + } + let same_signers = get_attesting_indices(state, &candidate.attestation, committees) + .is_ok_and(|indices| indices == candidate.attesting_indices); + if !same_signers { + continue; + } + // Errs on a same-slot vote for the payload, which `process_attestation` + // refuses. + let Ok(flags) = get_attestation_participation_flag_indices( + state, + data, + state.slot() - data.slot, + parent_slot, + ) else { + continue; + }; + let participation = if target_epoch == current_epoch { + ¤t_participation + } else { + &previous_participation + }; + let group = groups.entry(data.hash_tree_root()).or_default(); + for &index in &candidate.attesting_indices { + let held = participation.get(index as usize).copied().unwrap_or(0); + let balance = state.validator(index)?.effective_balance; + for &flag in &flags { + if held & (1 << flag) == 0 { + let weight = constants::PARTICIPATION_FLAG_WEIGHTS[flag]; + group + .gains + .insert((target_epoch, index, flag), balance.saturating_mul(weight)); + } + } + } + group + .committees + .insert(candidate.committee_index, candidate); + } + + let mut packed = PqPacked::default(); + for group in take_greedily(groups) { + let data = match group.committees.values().next() { + Some(first) => first.attestation.data, + None => continue, + }; + let mut committee_bits = pq::CommitteeBits::default(); + let mut aggregation_bits = pq::AggregationBits::new(); + for (&committee, aggregate) in &group.committees { + committee_bits + .set(committee as usize, true) + .map_err(|_| Error::SpecAssert("committee index < MAX_COMMITTEES_PER_SLOT_PQ"))?; + let bits = &aggregate.attestation.aggregation_bits; + for position in 0..bits.len() { + aggregation_bits.push(bits.get(position).unwrap_or(false)); + } + let statements = get_attestation_statements(state, &aggregate.attestation, committees)?; + packed.children.push((statements, aggregate.proof.clone())); + } + packed.attestations.push(pq::Attestation { + aggregation_bits, + data, + committee_bits, + }); + } + Ok(packed) +} + +/// Up to `MAX_ATTESTATIONS_PQ` of `groups`, each time the one whose gains not +/// yet covered weigh the most, stopping when none adds anything. +fn take_greedily(mut groups: BTreeMap) -> Vec { + let mut covered = HashSet::::new(); + let mut taken = Vec::new(); + while (taken.len() as u64) < preset::MAX_ATTESTATIONS_PQ { + let score = |group: &DataGroup| { + group + .gains + .iter() + .filter(|(gain, _)| !covered.contains(*gain)) + .fold(0u64, |sum, (_, weight)| sum.saturating_add(*weight)) + }; + let Some((&root, best)) = groups.iter().max_by_key(|(_, group)| score(group)) else { + break; + }; + if score(best) == 0 { + break; + } + let Some(group) = groups.remove(&root) else { + break; + }; + covered.extend(group.gains.keys().copied()); + taken.push(group); + } + taken +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::beacon::containers::shared::{AttestationData, Checkpoint}; + use crate::beacon::fork::ForkName; + use crate::beacon::helpers::accessors::{CommitteeCacheExt as _, get_block_root_at_slot}; + use crate::beacon::helpers::test_state::with_validators_at; + use crate::beacon::primitives::Slot; + + /// A network aggregate of `committee` at `data`'s slot whose attesters + /// are the members at `positions`, with a stand-in proof of `marker` + /// bytes: selection never reads a proof, and the reference merge + /// concatenates them. + fn aggregate( + state: &BeaconState, + committees: &CommitteeCache, + data: AttestationData, + committee: u64, + positions: &[usize], + marker: u8, + ) -> Arc { + let members = committees + .committees(state, data.target.epoch) + .committee(data.slot, committee) + .unwrap() + .to_vec(); + let mut aggregation_bits = pq::AggregationBits::with_length(members.len()); + for &position in positions { + aggregation_bits.set(position, true).unwrap(); + } + let mut committee_bits = pq::CommitteeBits::new(); + committee_bits.set(committee as usize, true).unwrap(); + let mut attesting_indices: Vec = positions + .iter() + .map(|&position| members[position]) + .collect(); + attesting_indices.sort_unstable(); + Arc::new(PooledPqAggregate { + attestation: pq::Attestation { + aggregation_bits, + data, + committee_bits, + }, + committee_index: committee, + attesting_indices, + proof: vec![marker; 4].try_into().unwrap(), + }) + } + + /// A registry that fills exactly two committees per slot under either + /// preset (8192 validators on mainnet, 64 on minimal). + fn two_committees_per_slot() -> usize { + (2 * preset::SLOTS_PER_EPOCH * preset::TARGET_COMMITTEE_SIZE) as usize + } + + fn data_at(state: &BeaconState, slot: Slot) -> AttestationData { + let epoch = compute_epoch_at_slot(slot); + let source = if epoch == get_current_epoch(state) { + state.current_justified_checkpoint() + } else { + state.previous_justified_checkpoint() + }; + AttestationData { + slot, + index: 0, + beacon_block_root: get_block_root_at_slot(state, slot).unwrap(), + source, + target: Checkpoint { + epoch, + root: get_block_root(state, epoch).unwrap(), + }, + } + } + + /// Two committees' aggregates of one vote, offered out of order, pack as + /// one attestation: committee bits for both, the aggregation bits of + /// committee 0 then committee 1, and the proofs in that same order, which + /// is the order `process_stark` reads the statements in. An aggregate too + /// fresh to include and one whose attesters this state's committee does + /// not name are left out. + #[test] + fn aggregates_of_one_vote_pack_as_one_attestation_in_committee_order() { + let state = with_validators_at(ForkName::Pq, two_committees_per_slot()); + let committees = CommitteeCache::default(); + let data_slot = state.slot() - preset::MIN_ATTESTATION_INCLUSION_DELAY_PQ; + let data = data_at(&state, data_slot); + let epoch_committees = committees.committees(&state, data.target.epoch); + assert_eq!(epoch_committees.committees_per_slot(), 2); + let committee_len = |index| epoch_committees.committee(data_slot, index).unwrap().len(); + let (len_zero, len_one) = (committee_len(0), committee_len(1)); + + let second = aggregate(&state, &committees, data, 1, &[0, 1, 2], 2); + let first = aggregate(&state, &committees, data, 0, &[0, 2], 1); + let too_fresh = aggregate( + &state, + &committees, + data_at(&state, state.slot() - 1), + 0, + &[1], + 3, + ); + let mut foreign = (*aggregate( + &state, + &committees, + data_at(&state, data_slot - 1), + 0, + &[0], + 4, + )) + .clone(); + foreign.attesting_indices = vec![foreign.attesting_indices[0] + 1]; + + let candidates = vec![second, too_fresh, first, Arc::new(foreign)]; + let packed = select_pq_attestations(&state, candidates, &committees).unwrap(); + + assert_eq!(packed.attestations.len(), 1); + let attestation = &packed.attestations[0]; + assert_eq!(attestation.data, data); + assert_eq!( + get_committee_indices(&attestation.committee_bits), + vec![0, 1] + ); + let bits = &attestation.aggregation_bits; + assert_eq!(bits.len(), len_zero + len_one); + let set: Vec = (0..bits.len()) + .filter(|&i| bits.get(i) == Some(true)) + .collect(); + assert_eq!(set, vec![0, 2, len_zero, len_zero + 1, len_zero + 2]); + + let proofs: Vec = packed.children.iter().map(|(_, proof)| proof[0]).collect(); + assert_eq!(proofs, vec![1, 2]); + let signers: Vec = packed + .children + .iter() + .map(|(statements, _)| statements.iter().map(|s| s.pubkeys.len()).sum()) + .collect(); + assert_eq!(signers, vec![2, 3]); + // The statements are the ones `process_stark` derives from the packed + // attestation, in its order. + let expected = get_attestation_statements(&state, attestation, &committees).unwrap(); + let packed_statements: Vec<_> = packed + .children + .iter() + .flat_map(|(statements, _)| statements.iter().cloned()) + .collect(); + assert_eq!(packed_statements, expected); + // The reference merge is the children's proofs concatenated. + assert_eq!(packed.merge().unwrap(), [vec![1; 4], vec![2; 4]].concat()); + // Spanning two committees, it has no unmerged fallback. + assert!(packed.first_unmerged().is_none()); + } + + /// An aggregate whose attesters already hold every flag it would set adds + /// nothing and is not packed. + #[test] + fn an_aggregate_crediting_nobody_new_is_not_packed() { + let mut state = with_validators_at(ForkName::Pq, two_committees_per_slot()); + let committees = CommitteeCache::default(); + let data_slot = state.slot() - preset::MIN_ATTESTATION_INCLUSION_DELAY_PQ; + let data = data_at(&state, data_slot); + let candidate = aggregate(&state, &committees, data, 0, &[0], 1); + let attester = candidate.attesting_indices[0]; + let before = select_pq_attestations(&state, vec![candidate.clone()], &committees).unwrap(); + assert_eq!( + before.attestations.len(), + 1, + "packed while it credits someone" + ); + // One committee: its own proof is the block's, nothing to merge. + let single = before + .first_unmerged() + .expect("a single-committee attestation"); + assert_eq!(single.children.len(), 1); + assert_eq!(single.merge().unwrap(), vec![1; 4]); + + let current = data.target.epoch == get_current_epoch(&state); + assert!( + state + .epoch_participation_mut(current) + .unwrap() + .set(attester as usize, 0b111) + ); + state.apply_pending_mutations(); + let packed = select_pq_attestations(&state, vec![candidate], &committees).unwrap(); + assert!(packed.attestations.is_empty()); + assert!(packed.children.is_empty()); + } + + #[test] + fn pq_request_types_parse_into_their_lists() { + let exit = pq::VoluntaryExitRequest::default(); + let list = vec![ + [ + vec![constants::VOLUNTARY_EXIT_REQUEST_TYPE], + libssz::SszEncode::to_ssz(&pq::VoluntaryExitRequests::from(vec![exit.clone()])), + ] + .concat(), + ]; + let requests = parse_pq_execution_requests(&list).unwrap(); + assert_eq!(requests.voluntary_exits.len(), 1); + assert!(requests.deposits.is_empty()); + // Round trip through the list builder. + assert_eq!(stf::pq::get_execution_requests_list(&requests), list); + // Out of order is refused. + let twice = [list.clone(), list].concat(); + assert!(parse_pq_execution_requests(&twice).is_err()); + } +} diff --git a/crates/blockchain/state_transition/src/beacon/precheck.rs b/crates/blockchain/state_transition/src/beacon/precheck.rs index 5ab99dada..46309b699 100644 --- a/crates/blockchain/state_transition/src/beacon/precheck.rs +++ b/crates/blockchain/state_transition/src/beacon/precheck.rs @@ -23,11 +23,12 @@ use crate::beacon::bls; use crate::beacon::config::Config; -use crate::beacon::constants::DOMAIN_BEACON_PROPOSER; +use crate::beacon::constants::{DOMAIN_BEACON_PROPOSER, LEAF_OFFSET_BEACON_PROPOSER}; use crate::beacon::containers::{BeaconState, SignedBeaconBlock}; use crate::beacon::helpers::misc::{ compute_domain, compute_epoch_at_slot, compute_signing_root, compute_start_slot_at_epoch, }; +use crate::beacon::helpers::pq::{compute_xmss_leaf_index, xmss_verify}; use crate::beacon::primitives::{Root, Slot, ValidatorIndex}; /// Which rule a block broke. @@ -85,6 +86,10 @@ pub enum Reference<'a> { /// epoch, not from the reference state's `fork` field: the first block of a /// new fork builds on a parent whose state has not been upgraded yet, and is /// signed under the new fork's version. +/// +/// A PQ block's XMSS signature is checked against a PQ state only. Across the +/// PQ fork boundary, either way round, the reference state holds the other +/// scheme's keys, so the signature is left to the import. pub fn precheck_block( signed_block: &SignedBeaconBlock, block_root: Root, @@ -128,7 +133,31 @@ pub fn precheck_block( state.genesis_validators_root(), ); let signing_root = compute_signing_root(block_root, domain); - if !bls::verify(&pubkey, signing_root, &signed_block.signature()) { + let verified = match (signed_block, state) { + // A PQ block is signed with the proposer's XMSS key, which only a PQ + // state's registry holds. Before the fork the record names a BLS key, + // so the parent of the fork's first block cannot check it: the import + // does. + (SignedBeaconBlock::Pq(pq_block), BeaconState::Pq(_)) => { + let Ok(proposer_record) = state.pq_validator(proposer) else { + return Err(PrecheckError::UnknownProposer { proposer }); + }; + compute_xmss_leaf_index(slot, LEAF_OFFSET_BEACON_PROPOSER).is_ok_and(|leaf_index| { + xmss_verify( + &proposer_record.pubkey(), + leaf_index, + signing_root, + &pq_block.signature, + ) + }) + } + (SignedBeaconBlock::Pq(_), _) => return Ok(()), + // A PQ registry holds XMSS keys, which cannot check an earlier + // fork's BLS signature. + (_, BeaconState::Pq(_)) => return Ok(()), + _ => bls::verify(&pubkey, signing_root, &signed_block.signature()), + }; + if !verified { return Err(PrecheckError::BadSignature); } Ok(()) @@ -149,6 +178,8 @@ pub(crate) fn fixed_proposer(parent_state: &BeaconState, slot: Slot) -> Option (state.slot, &state.proposer_lookahead), BeaconState::Gloas(state) => (state.slot, &state.proposer_lookahead), + BeaconState::Eip8483(state) => (state.slot, &state.proposer_lookahead), + BeaconState::Pq(state) => (state.slot, &state.proposer_lookahead), BeaconState::Phase0(_) | BeaconState::Altair(_) | BeaconState::Bellatrix(_) @@ -164,12 +195,16 @@ pub(crate) fn fixed_proposer(parent_state: &BeaconState, slot: Slot) -> Option SignedBeaconBlock { + let message = pq::BeaconBlock { + slot, + proposer_index: proposer, + parent_root: Root::repeat_byte(0x11), + state_root: Root::ZERO, + body: pq::BeaconBlockBody::empty(), + }; + let fork_version = config.fork_version(config.fork_at_epoch(compute_epoch_at_slot(slot))); + let domain = compute_domain( + DOMAIN_BEACON_PROPOSER, + fork_version, + state.genesis_validators_root(), + ); + let signing_root = compute_signing_root(message.hash_tree_root(), domain); + let leaf = compute_xmss_leaf_index(slot, LEAF_OFFSET_BEACON_PROPOSER).unwrap(); + let signature = secret + .sign(&signing_root.0, leaf as u32) + .expect("the key covers the slot's proposer leaf") + .to_bytes(); + SignedBeaconBlock::Pq(Box::new(pq::SignedBeaconBlock { + message, + signature: XmssSignature(signature.try_into().expect("XMSS_SIGNATURE_SIZE bytes")), + })) + } + + /// A PQ state whose validator `proposer` holds `secret`'s public key, and + /// the slot after the state's. + fn pq_state_keyed(proposer: ValidatorIndex, secret: &lean_xmss::SecretKey) -> BeaconState { + let mut state = with_validators_at(ForkName::Pq, 8); + let BeaconState::Pq(inner) = &mut state else { + unreachable!("with_validators_at(Pq) builds a PQ state"); + }; + inner + .validators + .get_mut(proposer as usize) + .expect("the state has eight validators") + .record + .pubkey = XmssPubkey(secret.public_key().to_bytes()).to_request_key(); + state + } + + #[test] + fn a_pq_block_is_checked_against_its_proposers_xmss_key() { + let config = Config::mainnet(); + let slot = with_validators_at(ForkName::Pq, 8).slot() + 1; + let leaf = compute_xmss_leaf_index(slot, LEAF_OFFSET_BEACON_PROPOSER).unwrap(); + let secret = lean_xmss::range_key(&[7; 32], leaf as u32, leaf as u32 + 1); + let state = pq_state_keyed(3, &secret); + let block = signed_pq_block(&state, &config, slot, 3, &secret); + assert_eq!(check(&block, Reference::Recent(&state), &config), Ok(())); + + let SignedBeaconBlock::Pq(mut altered) = block else { + unreachable!("signed_pq_block builds a PQ block"); + }; + altered.message.state_root = Root::repeat_byte(0x22); + let altered = SignedBeaconBlock::Pq(altered); + assert_eq!( + check(&altered, Reference::Recent(&state), &config), + Err(PrecheckError::BadSignature) + ); + } + + /// Either side of the PQ fork the reference state holds the other + /// scheme's keys, so neither block can be refused on its signature there. + #[test] + fn across_the_pq_boundary_the_signature_is_left_to_the_import() { + let config = Config::mainnet(); + let eip8483 = keyed_state(ForkName::Eip8483); + let slot = eip8483.slot() + 1; + let leaf = compute_xmss_leaf_index(slot, LEAF_OFFSET_BEACON_PROPOSER).unwrap(); + let secret = lean_xmss::range_key(&[7; 32], leaf as u32, leaf as u32 + 1); + let pq_block = signed_pq_block(&eip8483, &config, slot, 3, &secret); + assert_eq!( + check(&pq_block, Reference::Recent(&eip8483), &config), + Ok(()) + ); + + let pq_state = pq_state_keyed(3, &secret); + let unsigned = SignedBeaconBlock::Phase0(phase0::SignedBeaconBlock { + message: phase0::BeaconBlock { + slot: pq_state.slot() + 1, + proposer_index: 3, + parent_root: Root::repeat_byte(0x11), + state_root: Root::ZERO, + body: phase0::BeaconBlockBody { + randao_reveal: Default::default(), + eth1_data: Default::default(), + graffiti: Root::ZERO, + proposer_slashings: Default::default(), + attester_slashings: Default::default(), + attestations: Default::default(), + deposits: Default::default(), + voluntary_exits: Default::default(), + }, + }, + signature: Default::default(), + }); + assert_eq!( + check(&unsigned, Reference::Recent(&pq_state), &config), + Ok(()) + ); + } } diff --git a/crates/blockchain/state_transition/src/beacon/stf/block.rs b/crates/blockchain/state_transition/src/beacon/stf/block.rs index e17f63592..bf31dc0fd 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/block.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/block.rs @@ -182,6 +182,26 @@ fn process_block_with_signatures( options, signatures, ), + containers::SignedBeaconBlock::Eip8483(signed) => super::eip8483::process_block( + state, + &signed.message, + config, + committees, + active_balances, + pubkey_indices, + options, + signatures, + ), + // Signed with leanXMSS: nothing goes into the BLS batch. + containers::SignedBeaconBlock::Pq(signed) => super::pq::process_block( + state, + &signed.message, + config, + committees, + active_balances, + pubkey_indices, + options, + ), containers::SignedBeaconBlock::Lean(_) => { crate::beacon::lean_block_unreachable("process_block") } diff --git a/crates/blockchain/state_transition/src/beacon/stf/eip8483.rs b/crates/blockchain/state_transition/src/beacon/stf/eip8483.rs new file mode 100644 index 000000000..12c34394e --- /dev/null +++ b/crates/blockchain/state_transition/src/beacon/stf/eip8483.rs @@ -0,0 +1,361 @@ +//! EIP-8483's block processing (consensus-specs +//! `specs/_features/eip8483/beacon-chain.md`, on the `pq` branch at +//! `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`). +//! +//! Gloas's block processing, with two more execution-layer request lists in +//! the parent payload: validators and builders register post-quantum material +//! the PQ fork adopts. Everything but [`apply_parent_execution_payload`] and the +//! two request processors is gloas's own code, reached through the +//! gloas-family view (`containers::GloasFamily`). +//! +//! # A newer gloas underneath +//! +//! The `pq` branch builds on a newer gloas than this crate's pin. Its +//! `apply_parent_execution_payload` settles the parent's builder payment +//! *before* processing the parent's requests (so a builder exit request is +//! rejected while the payment is pending), its builders sweep pays a balance +//! net of the sweep's earlier withdrawals, and its `slash_validator` clears the +//! slashed validator's pending builder payments. All three apply from EIP-8483 +//! on and are recorded in `docs/spec_deviations.md`; gloas itself keeps its +//! pinned rules until its own fixtures move. + +use crate::beacon::config::Config; +use crate::beacon::constants; +use crate::beacon::containers::{BeaconState, eip8483}; +use crate::beacon::error::{Error, Result, verify}; +use crate::beacon::helpers::accessors::{ActiveBalanceCache, CommitteeCache, PubkeyIndexCache}; +use crate::beacon::helpers::electra::has_execution_withdrawal_credential; +use crate::beacon::preset; +use crate::beacon::primitives::{HashTreeRoot as _, ValidatorIndex}; +use crate::beacon::stf::signatures::SignatureBatch; +use crate::metrics::{StfPhase, timed}; + +use super::gloas::{ + GloasRequests, mark_parent_payload_available, parent_payload_context, + process_gloas_execution_requests, settle_parent_builder_payment, +}; + +/// `process_block`: gloas's, over EIP-8483's block. +#[allow(clippy::too_many_arguments)] +pub fn process_block( + state: &mut BeaconState, + block: &eip8483::BeaconBlock, + config: &Config, + committees: &CommitteeCache, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, + options: super::block::BlockProcessingOptions, + signatures: &mut SignatureBatch, +) -> Result<()> { + let parent_slot = state.latest_block_header().slot; + timed(StfPhase::ExecutionPayload, || { + process_parent_execution_payload(state, block, config, active_balances, pubkey_indices) + })?; + timed(StfPhase::BlockHeader, || { + super::block::process_block_header( + state, + block.slot, + block.proposer_index, + block.parent_root, + block.body.hash_tree_root(), + ) + })?; + timed(StfPhase::ExecutionPayload, || -> Result<()> { + super::gloas::process_withdrawals(state)?; + super::gloas::process_execution_payload_bid( + state, + &block.body.signed_execution_payload_bid, + config, + signatures, + ) + })?; + super::block::process_randao(state, &block.body.randao_reveal, options.randao, signatures)?; + super::block::process_eth1_data(state, &block.body.eth1_data)?; + super::gloas::process_operations( + state, + &block.body, + parent_slot, + config, + committees, + active_balances, + signatures, + options.target_delta, + )?; + super::altair::process_sync_aggregate( + state, + &block.body.sync_aggregate, + active_balances, + pubkey_indices, + signatures, + )?; + Ok(()) +} + +/// `process_parent_execution_payload`: gloas's, over EIP-8483's requests. +pub fn process_parent_execution_payload( + state: &mut BeaconState, + block: &eip8483::BeaconBlock, + config: &Config, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, +) -> Result<()> { + let bid = &block.body.signed_execution_payload_bid.message; + let (parent_bid, _, _) = parent_payload_context(state)?; + let requests = &block.body.parent_execution_requests; + + if bid.parent_block_hash != parent_bid.block_hash { + // Parent was EMPTY: no execution requests expected. + verify( + *requests == eip8483::ExecutionRequests::default(), + "process_parent_execution_payload: requests == ExecutionRequests.empty()", + )?; + return Ok(()); + } + + verify( + requests.hash_tree_root() == parent_bid.execution_requests_root, + "process_parent_execution_payload: hash_tree_root(requests) == parent_bid.execution_requests_root", + )?; + apply_parent_execution_payload(state, requests, config, active_balances, pubkey_indices) +} + +/// `apply_parent_execution_payload` (modified in EIP-8483): the two new +/// request lists' bounds and processing, on the newer gloas's settle-first +/// order (see the module doc). +pub fn apply_parent_execution_payload( + state: &mut BeaconState, + requests: &eip8483::ExecutionRequests, + config: &Config, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, +) -> Result<()> { + let (parent_bid, parent_slot, parent_epoch) = parent_payload_context(state)?; + + verify( + requests.withdrawals.len() <= preset::MAX_WITHDRAWAL_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.withdrawals) <= MAX_WITHDRAWAL_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.consolidations.len() <= preset::MAX_CONSOLIDATION_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.consolidations) <= MAX_CONSOLIDATION_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.builder_deposits.len() as u64 <= preset::MAX_BUILDER_DEPOSIT_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.builder_deposits) <= MAX_BUILDER_DEPOSIT_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.builder_exits.len() as u64 <= preset::MAX_BUILDER_EXIT_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.builder_exits) <= MAX_BUILDER_EXIT_REQUESTS_PER_PAYLOAD", + )?; + // [New in EIP8483] + verify( + requests.validator_pq_registrations.len() as u64 + <= preset::MAX_VALIDATOR_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.validator_pq_registrations) <= MAX_VALIDATOR_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD", + )?; + // [New in EIP8483] + verify( + requests.builder_pq_registrations.len() as u64 + <= preset::MAX_BUILDER_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.builder_pq_registrations) <= MAX_BUILDER_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD", + )?; + + // Settled before the requests, so a builder exit request is rejected while + // the payment is pending. + settle_parent_builder_payment(state, &parent_bid, parent_slot, parent_epoch)?; + + let gloas_requests = GloasRequests { + deposits: &requests.deposits, + withdrawals: &requests.withdrawals, + consolidations: &requests.consolidations, + builder_deposits: &requests.builder_deposits, + builder_exits: &requests.builder_exits, + }; + process_gloas_execution_requests( + state, + gloas_requests, + config, + active_balances, + pubkey_indices, + )?; + // [New in EIP8483] + for request in requests.validator_pq_registrations.iter() { + process_validator_pq_registration_request(state, request)?; + } + // [New in EIP8483] + for request in requests.builder_pq_registrations.iter() { + process_builder_pq_registration_request(state, request)?; + } + + mark_parent_payload_available(state, &parent_bid, parent_slot) +} + +fn eip8483_state<'a>( + state: &'a mut BeaconState, + function: &'static str, +) -> Result<&'a mut eip8483::BeaconState> { + match state { + BeaconState::Eip8483(inner) => Ok(inner), + other => Err(Error::UnsupportedForFork { + function, + fork: other.fork_name(), + }), + } +} + +/// `process_validator_pq_registration_request`. Invalid requests are ignored. +/// No status is checked, so pending, active, exited and slashed validators can +/// all register, as long as the request comes from their withdrawal address +/// and neither half of its PQ data is registered already, to anyone, +/// themselves included. +pub fn process_validator_pq_registration_request( + state: &mut BeaconState, + request: &eip8483::ValidatorPqRegistrationRequest, +) -> Result<()> { + let Some(validator_index) = state + .iter_validators() + .position(|validator| validator.pubkey == request.validator_pubkey) + else { + return Ok(()); + }; + let validator_index = validator_index as ValidatorIndex; + let validator = state.validator(validator_index)?; + + // Verify the validator has execution withdrawal credentials. + if !has_execution_withdrawal_credential(validator) { + return Ok(()); + } + // Verify the request comes from the validator's withdrawal address. + if validator.withdrawal_credentials.0[12..] != request.source_address.0 { + return Ok(()); + } + + let inner = eip8483_state(state, "process_validator_pq_registration_request")?; + let (key_half, vrf_half) = request.pq_data.split_at(32); + // Verify the PQ data is not already registered. + if inner.validator_pq_registrations.iter().any(|registration| { + registration.pq_data[..32] == *key_half || registration.pq_data[32..] == *vrf_half + }) { + return Ok(()); + } + + // Overwrite an existing registration for this validator. + if let Some(registration) = inner + .validator_pq_registrations + .iter_mut() + .find(|registration| registration.validator_index == validator_index) + { + registration.pq_data = request.pq_data; + return Ok(()); + } + + inner + .validator_pq_registrations + .push(eip8483::ValidatorPqRegistration { + validator_index, + pq_data: request.pq_data, + }); + Ok(()) +} + +/// `process_builder_pq_registration_request`. Invalid requests are ignored. +/// Active and exited builders can both register, as long as the request comes +/// from their execution address and its PQ data is not registered already. +pub fn process_builder_pq_registration_request( + state: &mut BeaconState, + request: &eip8483::BuilderPqRegistrationRequest, +) -> Result<()> { + let inner = eip8483_state(state, "process_builder_pq_registration_request")?; + let Some(builder) = inner + .builders + .iter() + .find(|builder| builder.pubkey == request.builder_pubkey) + else { + return Ok(()); + }; + + // Verify the request comes from the builder's execution address. + if builder.execution_address != request.source_address { + return Ok(()); + } + + // Verify the PQ data is not already registered. + if inner + .builder_pq_registrations + .iter() + .any(|registration| registration.pq_data == request.pq_data) + { + return Ok(()); + } + + // Overwrite an existing registration for this builder. + if let Some(registration) = inner + .builder_pq_registrations + .iter_mut() + .find(|registration| registration.builder_pubkey == request.builder_pubkey) + { + registration.pq_data = request.pq_data; + return Ok(()); + } + + inner + .builder_pq_registrations + .push(eip8483::BuilderPqRegistration { + builder_pubkey: request.builder_pubkey, + pq_data: request.pq_data, + }); + Ok(()) +} + +/// `get_execution_requests_list` (modified in EIP-8483): the EIP-7685 +/// encoding of the payload's requests, the two registration lists included. +pub fn get_execution_requests_list(requests: &eip8483::ExecutionRequests) -> Vec> { + use libssz::SszEncode as _; + let typed: [(u8, Vec, bool); 7] = [ + ( + constants::DEPOSIT_REQUEST_TYPE, + requests.deposits.to_ssz(), + requests.deposits.is_empty(), + ), + ( + constants::WITHDRAWAL_REQUEST_TYPE, + requests.withdrawals.to_ssz(), + requests.withdrawals.is_empty(), + ), + ( + constants::CONSOLIDATION_REQUEST_TYPE, + requests.consolidations.to_ssz(), + requests.consolidations.is_empty(), + ), + ( + constants::BUILDER_DEPOSIT_REQUEST_TYPE, + requests.builder_deposits.to_ssz(), + requests.builder_deposits.is_empty(), + ), + ( + constants::BUILDER_EXIT_REQUEST_TYPE, + requests.builder_exits.to_ssz(), + requests.builder_exits.is_empty(), + ), + ( + constants::VALIDATOR_PQ_REGISTRATION_REQUEST_TYPE, + requests.validator_pq_registrations.to_ssz(), + requests.validator_pq_registrations.is_empty(), + ), + ( + constants::BUILDER_PQ_REGISTRATION_REQUEST_TYPE, + requests.builder_pq_registrations.to_ssz(), + requests.builder_pq_registrations.is_empty(), + ), + ]; + typed + .into_iter() + .filter(|(_, _, empty)| !empty) + .map(|(request_type, data, _)| { + let mut entry = Vec::with_capacity(1 + data.len()); + entry.push(request_type); + entry.extend(data); + entry + }) + .collect() +} diff --git a/crates/blockchain/state_transition/src/beacon/stf/epoch/capella.rs b/crates/blockchain/state_transition/src/beacon/stf/epoch/capella.rs index 5260efe51..fdb5a3e7d 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/epoch/capella.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/epoch/capella.rs @@ -96,6 +96,8 @@ fn historical_summaries_mut<'a>( BeaconState::Electra(state) => Ok(&mut state.historical_summaries), BeaconState::Fulu(state) => Ok(&mut state.historical_summaries), BeaconState::Gloas(state) => Ok(&mut state.historical_summaries), + BeaconState::Eip8483(state) => Ok(&mut state.historical_summaries), + BeaconState::Pq(state) => Ok(&mut state.historical_summaries), other => Err(Error::UnsupportedForFork { function, fork: other.fork_name(), diff --git a/crates/blockchain/state_transition/src/beacon/stf/epoch/electra.rs b/crates/blockchain/state_transition/src/beacon/stf/epoch/electra.rs index b0f4fc679..cbe03b9d1 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/epoch/electra.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/epoch/electra.rs @@ -122,7 +122,9 @@ pub(super) fn process_unfused_steps( | ForkName::Bellatrix | ForkName::Capella | ForkName::Deneb - | ForkName::Gloas) => { + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq) => { return Err(Error::UnsupportedForFork { function: "process_unfused_steps", fork, diff --git a/crates/blockchain/state_transition/src/beacon/stf/epoch/fulu.rs b/crates/blockchain/state_transition/src/beacon/stf/epoch/fulu.rs index 9b57e4975..b5cb47945 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/epoch/fulu.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/epoch/fulu.rs @@ -150,7 +150,7 @@ pub fn process_proposer_lookahead(state: &mut BeaconState) -> Result<()> { // sampling; see `crate::beacon::helpers::gloas::get_beacon_proposer_indices`'s // own doc. This is the one part of the step gloas cannot share // unchanged; see this module's own module doc. - ForkName::Gloas => { + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => { crate::beacon::helpers::gloas::get_beacon_proposer_indices(state, new_epoch)? } fork @ (ForkName::Phase0 diff --git a/crates/blockchain/state_transition/src/beacon/stf/epoch/gloas.rs b/crates/blockchain/state_transition/src/beacon/stf/epoch/gloas.rs index 778be7ab9..70d5b3e0a 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/epoch/gloas.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/epoch/gloas.rs @@ -119,9 +119,9 @@ pub fn process_participation_flag_updates(state: &mut BeaconState) -> Result<()> let validator_count = state.validator_count(); let inner = gloas_state(state, "process_participation_flag_updates")?; - inner.previous_epoch_participation = core::mem::take(&mut inner.current_epoch_participation); + *inner.previous_epoch_participation = core::mem::take(inner.current_epoch_participation); // `repeat` rather than a zeroed `Vec`, for the reason given in altair's. - inner.current_epoch_participation = gloas::EpochParticipation::repeat(0, validator_count); + *inner.current_epoch_participation = gloas::EpochParticipation::repeat(0, validator_count); Ok(()) } @@ -225,7 +225,7 @@ pub fn process_ptc_window(state: &mut BeaconState) -> Result<()> { let mut window = Vec::with_capacity(preset::PTC_WINDOW_LENGTH); window.extend_from_slice(&inner.ptc_window[slots_per_epoch..]); window.extend(new_slice); - inner.ptc_window = window.try_into().expect( + *inner.ptc_window = window.try_into().expect( "dropping SLOTS_PER_EPOCH entries and appending SLOTS_PER_EPOCH more preserves \ PTC_WINDOW_LENGTH", ); @@ -239,7 +239,7 @@ mod tests { use crate::beacon::containers::electra; use crate::beacon::fork::ForkName; use crate::beacon::helpers::accessors::get_total_active_balance; - use crate::beacon::helpers::gloas::{get_ptc, gloas_state_ref}; + use crate::beacon::helpers::gloas::get_ptc; use crate::beacon::primitives::ValidatorIndex; /// A gloas state with `count` fully active, full-balance validators, @@ -251,11 +251,17 @@ mod tests { } fn gloas_inner(state: &BeaconState) -> &gloas::BeaconState { - gloas_state_ref(state, "test").unwrap() + match state { + BeaconState::Gloas(inner) => inner, + _ => panic!("a gloas test state"), + } } fn gloas_inner_mut(state: &mut BeaconState) -> &mut gloas::BeaconState { - gloas_state(state, "test").unwrap() + match state { + BeaconState::Gloas(inner) => inner, + _ => panic!("a gloas test state"), + } } // ----------------------------------------------------------------------- @@ -597,7 +603,7 @@ mod tests { // `> 0` regardless of weight. let mut settle_state = gloas_state_with_validators(64); gloas_inner_mut(&mut settle_state).builder_pending_payments[0] = payment; - settle_builder_payment(gloas_inner_mut(&mut settle_state), 0).unwrap(); + settle_builder_payment(&mut gloas_state(&mut settle_state, "test").unwrap(), 0).unwrap(); assert!( gloas_inner(&settle_state) .builder_pending_withdrawals diff --git a/crates/blockchain/state_transition/src/beacon/stf/epoch/mod.rs b/crates/blockchain/state_transition/src/beacon/stf/epoch/mod.rs index 6d59a5394..e7fc4877e 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/epoch/mod.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/epoch/mod.rs @@ -24,6 +24,7 @@ pub mod gloas; pub mod justification; #[cfg(test)] mod participation_equivalence; +pub mod pq; pub mod registry; pub mod rewards; mod single_pass; @@ -105,7 +106,11 @@ pub fn process_epoch(state: &mut BeaconState, config: &Config) -> Result<()> { // EIP-7688's progressive participation, inactivity, and pending-queue // lists need their own copies of several more (see `gloas`'s own // module doc for which), so gloas gets its own driver too. - ForkName::Gloas => gloas::process_epoch(state, config), + // EIP-8483 changes no epoch processing. + ForkName::Gloas | ForkName::Eip8483 => gloas::process_epoch(state, config), + // Without sync committees, and with the aggregation committee window + // and preregistration expiry appended. + ForkName::Pq => pq::process_epoch(state, config), ForkName::Lean => lean_state_unreachable("process_epoch"), } } @@ -140,7 +145,9 @@ pub fn process_justification_and_finalization( | ForkName::Deneb | ForkName::Electra | ForkName::Fulu - | ForkName::Gloas => altair::process_justification_and_finalization(state), + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => altair::process_justification_and_finalization(state), ForkName::Lean => lean_state_unreachable("process_justification_and_finalization"), } } diff --git a/crates/blockchain/state_transition/src/beacon/stf/epoch/pq.rs b/crates/blockchain/state_transition/src/beacon/stf/epoch/pq.rs new file mode 100644 index 000000000..28f34f3d3 --- /dev/null +++ b/crates/blockchain/state_transition/src/beacon/stf/epoch/pq.rs @@ -0,0 +1,237 @@ +//! The PQ fork's epoch processing (consensus-specs +//! `specs/_features/pq/beacon-chain.md`, on the `pq` branch at +//! `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`). +//! +//! Gloas's driver without `process_sync_committee_updates` (sync committees are +//! gone), with [`process_aggregation_committee_window`] and +//! [`process_preregistration_expiry`] appended, and its own +//! [`process_pending_deposits`]: a deposit names an XMSS key after 16 zero +//! bytes or a legacy BLS key, carries a VRF root where its signature was, and +//! is never signature-checked. + +use crate::beacon::config::Config; +use crate::beacon::constants::FAR_FUTURE_EPOCH; +use crate::beacon::containers::{BeaconState, electra, pq, shared::Validator}; +use crate::beacon::error::{Error, Result}; +use crate::beacon::helpers::accessors::{CommitteeCache, get_current_epoch}; +use crate::beacon::helpers::electra::{get_max_effective_balance, pending_queue_fields}; +use crate::beacon::helpers::gloas::get_activation_churn_limit; +use crate::beacon::helpers::misc::compute_start_slot_at_epoch; +use crate::beacon::helpers::mutators::increase_balance; +use crate::beacon::helpers::pq::{ + compute_aggregation_committees, compute_deposit_pubkey, compute_deposit_vrf_root, + is_active_builder_preregistration, is_active_validator_preregistration, is_legacy_pubkey, + pq_state, +}; +use crate::beacon::preset; +use crate::beacon::primitives::{Gwei, ValidatorIndex}; + +/// `process_epoch` (modified in PQ). +pub fn process_epoch(state: &mut BeaconState, config: &Config) -> Result<()> { + super::altair::process_justification_and_finalization(state)?; + super::altair::process_inactivity_updates(state, config)?; + super::altair::process_rewards_and_penalties(state, config)?; + super::electra::process_registry_updates(state, config)?; + super::electra::process_slashings(state, config)?; + super::process_eth1_data_reset(state)?; + // [Modified in PQ] + process_pending_deposits(state, config)?; + super::electra::process_pending_consolidations(state, config)?; + super::gloas::process_builder_pending_payments(state)?; + super::electra::process_effective_balance_updates(state)?; + super::process_slashings_reset(state)?; + super::process_randao_mixes_reset(state)?; + super::capella::process_historical_summaries_update(state)?; + super::gloas::process_participation_flag_updates(state)?; + // [Modified in PQ] Removed `process_sync_committee_updates`. + super::fulu::process_proposer_lookahead(state)?; + super::gloas::process_ptc_window(state)?; + // [New in PQ] + process_aggregation_committee_window(state)?; + // [New in PQ:EIP8205] + process_preregistration_expiry(state)?; + Ok(()) +} + +/// `process_aggregation_committee_window`: shifts the window down one epoch +/// and fills the newly visible epoch's aggregation committees. +pub fn process_aggregation_committee_window(state: &mut BeaconState) -> Result<()> { + let next_epoch = get_current_epoch(state) + preset::MIN_SEED_LOOKAHEAD + 1; + let start_slot = compute_start_slot_at_epoch(next_epoch); + let committees = CommitteeCache::default(); + let mut new_slice = Vec::with_capacity(preset::SLOTS_PER_EPOCH as usize); + for slot in start_slot..start_slot + preset::SLOTS_PER_EPOCH { + new_slice.push(compute_aggregation_committees(state, slot, &committees)?); + } + + let inner = pq_state(state, "process_aggregation_committee_window")?; + let slots_per_epoch = preset::SLOTS_PER_EPOCH as usize; + let mut window = Vec::with_capacity(preset::AGGREGATION_COMMITTEE_WINDOW_LENGTH); + window.extend_from_slice(&inner.aggregation_committee_window[slots_per_epoch..]); + window.extend(new_slice); + inner.aggregation_committee_window = window.try_into().expect( + "dropping SLOTS_PER_EPOCH entries and appending SLOTS_PER_EPOCH more preserves the length", + ); + Ok(()) +} + +/// `process_preregistration_expiry`: garbage collection only. Bindings are +/// enforced through `is_active_*_preregistration`, so when the sweep runs has +/// no effect on whether a binding is enforced or a request admitted. +pub fn process_preregistration_expiry(state: &mut BeaconState) -> Result<()> { + let inner = pq_state(state, "process_preregistration_expiry")?; + let validators: Vec = inner + .validator_preregistrations + .iter() + .filter(|record| is_active_validator_preregistration(inner, record)) + .cloned() + .collect(); + let builders: Vec = inner + .builder_preregistrations + .iter() + .filter(|record| is_active_builder_preregistration(inner, record)) + .cloned() + .collect(); + inner.validator_preregistrations = validators.into(); + inner.builder_preregistrations = builders.into(); + Ok(()) +} + +/// `process_pending_deposits` (modified in PQ): gloas's activation-only churn +/// budget, each deposit's validator found by [`compute_deposit_pubkey`], and a +/// legacy deposit applied without consuming churn, since its validator never +/// becomes active. +pub fn process_pending_deposits(state: &mut BeaconState, config: &Config) -> Result<()> { + let next_epoch = get_current_epoch(state) + 1; + let churn_limit = get_activation_churn_limit(state, config)?; + let finalized_slot = compute_start_slot_at_epoch(state.finalized_checkpoint().epoch); + + let (available_for_processing, deposits) = { + let mut fields = pending_queue_fields(state, "process_pending_deposits")?; + let available = fields + .deposit_balance_to_consume() + .checked_add(churn_limit) + .ok_or(Error::ArithmeticOverflow( + "deposit_balance_to_consume + get_activation_churn_limit(state)", + ))?; + (available, fields.take_pending_deposits()) + }; + + let mut processed_amount: Gwei = 0; + let mut next_deposit_index = 0usize; + let mut deposits_to_postpone: Vec = Vec::new(); + let mut is_churn_limit_reached = false; + + for deposit in &deposits { + if deposit.slot > finalized_slot { + break; + } + if next_deposit_index >= preset::MAX_PENDING_DEPOSITS_PER_EPOCH as usize { + break; + } + + let record_key = compute_deposit_pubkey(&deposit.pubkey).to_request_key(); + let (is_validator_exited, is_validator_withdrawn) = state + .iter_validators() + .find(|validator| validator.pubkey == record_key) + .map(|validator| { + ( + validator.exit_epoch < FAR_FUTURE_EPOCH, + validator.withdrawable_epoch < next_epoch, + ) + }) + .unwrap_or((false, false)); + + if is_validator_withdrawn || is_legacy_pubkey(&deposit.pubkey) { + // Deposited balance will never become active: credited without + // consuming churn. + apply_pending_deposit(state, deposit, config)?; + } else if is_validator_exited { + deposits_to_postpone.push(deposit.clone()); + } else { + match processed_amount.checked_add(deposit.amount) { + Some(sum) if sum <= available_for_processing => { + processed_amount = sum; + apply_pending_deposit(state, deposit, config)?; + } + _ => { + is_churn_limit_reached = true; + break; + } + } + } + next_deposit_index += 1; + } + + let remaining: Vec = deposits + .into_iter() + .skip(next_deposit_index) + .chain(deposits_to_postpone) + .collect(); + let deposit_balance_to_consume = if is_churn_limit_reached { + available_for_processing + .checked_sub(processed_amount) + .ok_or(Error::ArithmeticOverflow( + "available_for_processing - processed_amount", + ))? + } else { + 0 + }; + + let mut fields = pending_queue_fields(state, "process_pending_deposits")?; + fields.set_pending_deposits(remaining)?; + *fields.deposit_balance_to_consume_mut() = deposit_balance_to_consume; + Ok(()) +} + +/// `apply_pending_deposit` (modified in PQ): a deposit for a new validator is +/// not verified, its validator takes the deposit's VRF root, and a legacy +/// deposit's validator is exited at once, without consuming exit churn, since +/// it was never active. +pub fn apply_pending_deposit( + state: &mut BeaconState, + deposit: &electra::PendingDeposit, + config: &Config, +) -> Result<()> { + let record_key = compute_deposit_pubkey(&deposit.pubkey).to_request_key(); + let existing = state + .iter_validators() + .position(|validator| validator.pubkey == record_key); + if let Some(index) = existing { + return increase_balance(state, index as ValidatorIndex, deposit.amount); + } + + let index = state.validator_count() as ValidatorIndex; + let mut validator = Validator { + pubkey: record_key, + withdrawal_credentials: deposit.withdrawal_credentials, + activation_eligibility_epoch: FAR_FUTURE_EPOCH, + activation_epoch: FAR_FUTURE_EPOCH, + exit_epoch: FAR_FUTURE_EPOCH, + withdrawable_epoch: FAR_FUTURE_EPOCH, + ..Default::default() + }; + let max_effective_balance = get_max_effective_balance(&validator); + validator.effective_balance = (deposit.amount + - deposit.amount % preset::EFFECTIVE_BALANCE_INCREMENT) + .min(max_effective_balance); + let validator = pq::Validator { + record: validator, + vrf_root: compute_deposit_vrf_root(deposit), + }; + state.push_pq_validator(validator, deposit.amount)?; + pending_queue_fields(state, "apply_pending_deposit")? + .push_empty_participation_and_inactivity()?; + + if is_legacy_pubkey(&deposit.pubkey) { + let epoch = get_current_epoch(state); + let validator = state.validator_mut(index)?; + validator.exit_epoch = epoch; + validator.withdrawable_epoch = epoch + .checked_add(config.min_validator_withdrawability_delay) + .ok_or(Error::ArithmeticOverflow( + "epoch + MIN_VALIDATOR_WITHDRAWABILITY_DELAY", + ))?; + } + Ok(()) +} diff --git a/crates/blockchain/state_transition/src/beacon/stf/epoch/registry.rs b/crates/blockchain/state_transition/src/beacon/stf/epoch/registry.rs index 387a28284..1233d7641 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/epoch/registry.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/epoch/registry.rs @@ -53,12 +53,14 @@ fn activation_churn_limit(state: &BeaconState, config: &Config) -> Result { Ok(churn_limit) } ForkName::Deneb => Ok(config.max_per_epoch_activation_churn_limit.min(churn_limit)), - fork @ (ForkName::Electra | ForkName::Fulu | ForkName::Gloas) => { - Err(Error::UnsupportedForFork { - function: "process_registry_updates", - fork, - }) - } + fork @ (ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq) => Err(Error::UnsupportedForFork { + function: "process_registry_updates", + fork, + }), ForkName::Lean => lean_state_unreachable("activation_churn_limit"), } } diff --git a/crates/blockchain/state_transition/src/beacon/stf/gloas.rs b/crates/blockchain/state_transition/src/beacon/stf/gloas.rs index 8a9d4ad8c..9f02faf11 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/gloas.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/gloas.rs @@ -51,8 +51,9 @@ use crate::beacon::bls::{self, SignatureSet}; use crate::beacon::config::Config; use crate::beacon::constants; use crate::beacon::containers::shared::{DepositMessage, ProposerSlashing}; -use crate::beacon::containers::{BeaconState, capella, gloas}; +use crate::beacon::containers::{BeaconState, SignedExecutionPayloadEnvelope, capella, gloas}; use crate::beacon::error::{Error, Result, verify}; +use crate::beacon::fork::ForkName; use crate::beacon::helpers::accessors::{ ActiveBalanceCache, ActiveBalanceCacheExt, CommitteeCache, CommitteeCacheExt, PubkeyIndexCache, get_beacon_proposer_index, get_block_root_at_slot, get_current_epoch, get_domain, @@ -281,7 +282,13 @@ pub fn get_builders_sweep_withdrawals( let mut processed_count: u64 = 0; let mut withdrawals = Vec::new(); - let mut builder_index = inner.next_withdrawal_builder_index; + let mut builder_index = *inner.next_withdrawal_builder_index; + // The specification's `pq` branch builds on a newer gloas than this + // crate's pin, whose builders sweep pays out a builder's balance net of + // the withdrawals already taken from it in this same sweep + // (`get_builder_balance_after_withdrawals`). It applies from EIP-8483 on; + // see `docs/spec_deviations.md`. + let net_of_prior = inner.fork >= ForkName::Eip8483; for _ in 0..builders_limit { if prior_withdrawals.len() + withdrawals.len() >= withdrawals_limit { break; @@ -295,12 +302,24 @@ pub fn get_builders_sweep_withdrawals( index: builder_index as usize, len: inner.builders.len(), })?; - if builder.withdrawable_epoch <= epoch && builder.balance > 0 { + let balance = if net_of_prior { + let validator_index = convert_builder_index_to_validator_index(builder_index); + let withdrawn: Gwei = prior_withdrawals + .iter() + .chain(&withdrawals) + .filter(|withdrawal| withdrawal.validator_index == validator_index) + .map(|withdrawal| withdrawal.amount) + .sum(); + builder.balance.saturating_sub(withdrawn) + } else { + builder.balance + }; + if builder.withdrawable_epoch <= epoch && balance > 0 { withdrawals.push(capella::Withdrawal { index: withdrawal_index, validator_index: convert_builder_index_to_validator_index(builder_index), address: builder.execution_address, - amount: builder.balance, + amount: balance, }); withdrawal_index = withdrawal_index .checked_add(1) @@ -415,7 +434,7 @@ pub fn update_payload_expected_withdrawals( state: &mut BeaconState, withdrawals: &[capella::Withdrawal], ) -> Result<()> { - gloas_state(state, "update_payload_expected_withdrawals")?.payload_expected_withdrawals = + *gloas_state(state, "update_payload_expected_withdrawals")?.payload_expected_withdrawals = gloas::Withdrawals::from(withdrawals.to_vec()); Ok(()) } @@ -438,9 +457,9 @@ pub fn update_builder_pending_withdrawals( return Ok(()); } let inner = gloas_state(state, "update_builder_pending_withdrawals")?; - let mut owned = core::mem::take(&mut inner.builder_pending_withdrawals).into_inner(); + let mut owned = core::mem::take(inner.builder_pending_withdrawals).into_inner(); let remaining = owned.split_off(processed_builder_withdrawals_count as usize); - inner.builder_pending_withdrawals = gloas::BuilderPendingWithdrawals::from(remaining); + *inner.builder_pending_withdrawals = gloas::BuilderPendingWithdrawals::from(remaining); Ok(()) } @@ -457,9 +476,9 @@ pub fn update_pending_partial_withdrawals( return Ok(()); } let inner = gloas_state(state, "update_pending_partial_withdrawals")?; - let mut owned = core::mem::take(&mut inner.pending_partial_withdrawals).into_inner(); + let mut owned = core::mem::take(inner.pending_partial_withdrawals).into_inner(); let remaining = owned.split_off(processed_partial_withdrawals_count as usize); - inner.pending_partial_withdrawals = gloas::PendingPartialWithdrawals::from(remaining); + *inner.pending_partial_withdrawals = gloas::PendingPartialWithdrawals::from(remaining); Ok(()) } @@ -483,7 +502,7 @@ pub fn update_next_withdrawal_builder_index( .ok_or(Error::ArithmeticOverflow( "update_next_withdrawal_builder_index: next_withdrawal_builder_index + processed_builders_sweep_count", ))?; - inner.next_withdrawal_builder_index = next_index % builders_len; + *inner.next_withdrawal_builder_index = next_index % builders_len; } Ok(()) } @@ -553,7 +572,7 @@ fn update_next_withdrawal_validator_index( /// correctly names what the *next* revealed payload owes. pub fn process_withdrawals(state: &mut BeaconState) -> Result<()> { let inner = gloas_state_ref(state, "process_withdrawals")?; - if inner.latest_block_hash != inner.latest_execution_payload_bid.block_hash { + if *inner.latest_block_hash != inner.latest_execution_payload_bid.block_hash { return Ok(()); } @@ -648,7 +667,43 @@ pub fn verify_execution_payload_envelope_signature( state: &BeaconState, signed_envelope: &gloas::SignedExecutionPayloadEnvelope, ) -> Result { - let builder_index = signed_envelope.message.builder_index; + verify_bls_envelope_signature( + state, + signed_envelope.message.builder_index, + signed_envelope.message.hash_tree_root(), + &signed_envelope.signature, + ) +} + +/// The envelope signature check of whichever fork `signed_envelope` is: +/// gloas's and EIP-8483's BLS one over their own message roots, PQ's +/// leanXMSS one. +pub fn verify_any_execution_payload_envelope_signature( + state: &BeaconState, + signed_envelope: &SignedExecutionPayloadEnvelope, +) -> Result { + match signed_envelope { + SignedExecutionPayloadEnvelope::Gloas(envelope) => { + verify_execution_payload_envelope_signature(state, envelope) + } + SignedExecutionPayloadEnvelope::Eip8483(envelope) => verify_bls_envelope_signature( + state, + envelope.message.builder_index, + envelope.message.hash_tree_root(), + &envelope.signature, + ), + SignedExecutionPayloadEnvelope::Pq(envelope) => { + super::pq::verify_execution_payload_envelope_signature(state, envelope) + } + } +} + +fn verify_bls_envelope_signature( + state: &BeaconState, + builder_index: gloas::BuilderIndex, + message_root: Root, + signature: &crate::beacon::primitives::BlsSignature, +) -> Result { let pubkey = if builder_index == constants::BUILDER_INDEX_SELF_BUILD { let validator_index = state.latest_block_header().proposer_index; state.validator(validator_index)?.pubkey @@ -665,12 +720,8 @@ pub fn verify_execution_payload_envelope_signature( }; let domain = get_domain(state, constants::DOMAIN_BEACON_BUILDER, None); - let signing_root = compute_signing_root(signed_envelope.message.hash_tree_root(), domain); - Ok(bls::verify( - &pubkey, - signing_root, - &signed_envelope.signature, - )) + let signing_root = compute_signing_root(message_root, domain); + Ok(bls::verify(&pubkey, signing_root, signature)) } /// `verify_execution_payload_envelope` (gloas `fork-choice.md`). @@ -688,18 +739,21 @@ pub fn verify_execution_payload_envelope_signature( /// [`ExecutionEngine`]'s own doc for why the versioned hashes below are /// computed but never themselves checked against anything beyond the /// engine's own opaque verdict. +/// +/// Takes any fork's envelope: EIP-8483's and PQ's are gloas's with their own +/// execution requests (which only the bid's `execution_requests_root` check +/// reads) and, for PQ, a leanXMSS signature. pub fn verify_execution_payload_envelope( state: &BeaconState, - signed_envelope: &gloas::SignedExecutionPayloadEnvelope, + signed_envelope: &SignedExecutionPayloadEnvelope, config: &Config, engine: &ExecutionEngine, ) -> Result<()> { - let envelope = &signed_envelope.message; - let payload = &envelope.payload; + let payload = signed_envelope.payload(); // Verify signature. verify( - verify_execution_payload_envelope_signature(state, signed_envelope)?, + verify_any_execution_payload_envelope_signature(state, signed_envelope)?, "verify_execution_payload_envelope: verify_execution_payload_envelope_signature(state, signed_envelope)", )?; @@ -723,11 +777,11 @@ pub fn verify_execution_payload_envelope( let mut header = state.latest_block_header().clone(); header.state_root = state.compute_state_root(); verify( - envelope.beacon_block_root == header.hash_tree_root(), + signed_envelope.beacon_block_root() == header.hash_tree_root(), "verify_execution_payload_envelope: envelope.beacon_block_root == hash_tree_root(header)", )?; verify( - envelope.parent_beacon_block_root == state.latest_block_header().parent_root, + signed_envelope.parent_beacon_block_root() == state.latest_block_header().parent_root, "verify_execution_payload_envelope: envelope.parent_beacon_block_root == state.latest_block_header.parent_root", )?; @@ -735,7 +789,7 @@ pub fn verify_execution_payload_envelope( let inner = gloas_state_ref(state, "verify_execution_payload_envelope")?; let bid = &inner.latest_execution_payload_bid; verify( - envelope.builder_index == bid.builder_index, + signed_envelope.builder_index() == bid.builder_index, "verify_execution_payload_envelope: envelope.builder_index == bid.builder_index", )?; verify( @@ -751,7 +805,7 @@ pub fn verify_execution_payload_envelope( "verify_execution_payload_envelope: payload.block_hash == bid.block_hash", )?; verify( - envelope.execution_requests.hash_tree_root() == bid.execution_requests_root, + signed_envelope.execution_requests_root() == bid.execution_requests_root, "verify_execution_payload_envelope: hash_tree_root(envelope.execution_requests) == bid.execution_requests_root", )?; @@ -761,7 +815,7 @@ pub fn verify_execution_payload_envelope( "verify_execution_payload_envelope: payload.slot_number == state.slot", )?; verify( - payload.parent_hash == inner.latest_block_hash, + payload.parent_hash == *inner.latest_block_hash, "verify_execution_payload_envelope: payload.parent_hash == state.latest_block_hash", )?; verify( @@ -844,7 +898,32 @@ pub fn process_execution_payload_bid( config: &Config, signatures: &mut SignatureBatch, ) -> Result<()> { - let bid = signed_bid.message.clone(); + process_execution_payload_bid_with( + state, + &signed_bid.message, + config, + signed_bid.signature == g2_point_at_infinity(), + |state| { + signatures.push( + execution_payload_bid_signature_set(state, signed_bid)?, + "process_execution_payload_bid: verify_execution_payload_bid_signature(state, signed_bid)", + ) + }, + ) +} + +/// The body of `process_execution_payload_bid`, which the PQ fork keeps +/// except for its signature: `self_build_signature_empty` is whether the +/// signature is the scheme's empty one (BLS's point at infinity, an all-zero +/// XMSS signature), and `verify_signature` checks a builder's. +pub(crate) fn process_execution_payload_bid_with( + state: &mut BeaconState, + bid: &gloas::ExecutionPayloadBid, + config: &Config, + self_build_signature_empty: bool, + verify_signature: impl FnOnce(&BeaconState) -> Result<()>, +) -> Result<()> { + let bid = bid.clone(); let builder_index = bid.builder_index; let amount = bid.value; @@ -853,14 +932,14 @@ pub fn process_execution_payload_bid( // credential prefix. verify(amount == 0, "process_execution_payload_bid: amount == 0")?; verify( - signed_bid.signature == g2_point_at_infinity(), - "process_execution_payload_bid: signature == G2_POINT_AT_INFINITY", + self_build_signature_empty, + "process_execution_payload_bid: signature is the empty signature", )?; } else { // Verify that the builder is active. verify( is_active_builder( - gloas_state_ref(state, "process_execution_payload_bid")?, + &gloas_state_ref(state, "process_execution_payload_bid")?, builder_index, )?, "process_execution_payload_bid: is_active_builder(state, builder_index)", @@ -884,17 +963,14 @@ pub fn process_execution_payload_bid( // Verify that the builder has funds to cover the bid. verify( can_builder_cover_bid( - gloas_state_ref(state, "process_execution_payload_bid")?, + &gloas_state_ref(state, "process_execution_payload_bid")?, builder_index, amount, )?, "process_execution_payload_bid: can_builder_cover_bid(state, builder_index, amount)", )?; // Verify that the bid signature is valid. - signatures.push( - execution_payload_bid_signature_set(state, signed_bid)?, - "process_execution_payload_bid: verify_execution_payload_bid_signature(state, signed_bid)", - )?; + verify_signature(state)?; } // Verify commitments are under limit. @@ -916,7 +992,7 @@ pub fn process_execution_payload_bid( // Verify that the bid is for the right parent block. verify( bid.parent_block_hash - == gloas_state_ref(state, "process_execution_payload_bid")?.latest_block_hash, + == *gloas_state_ref(state, "process_execution_payload_bid")?.latest_block_hash, "process_execution_payload_bid: bid.parent_block_hash == state.latest_block_hash", )?; // Verify that the bid's block hash differs from its parent block hash. @@ -959,7 +1035,7 @@ pub fn process_execution_payload_bid( } // Cache the signed execution payload bid. - gloas_state(state, "process_execution_payload_bid")?.latest_execution_payload_bid = bid; + *gloas_state(state, "process_execution_payload_bid")?.latest_execution_payload_bid = bid; Ok(()) } @@ -982,15 +1058,7 @@ pub fn apply_parent_execution_payload( active_balances: &ActiveBalanceCache, pubkey_indices: &PubkeyIndexCache, ) -> Result<()> { - let (parent_bid, parent_slot, parent_epoch) = { - let inner = gloas_state_ref(state, "apply_parent_execution_payload")?; - let parent_slot = inner.latest_block_header.slot; - ( - inner.latest_execution_payload_bid.clone(), - parent_slot, - compute_epoch_at_slot(parent_slot), - ) - }; + let (parent_bid, parent_slot, parent_epoch) = parent_payload_context(state)?; verify( requests.withdrawals.len() <= preset::MAX_WITHDRAWAL_REQUESTS_PER_PAYLOAD, @@ -1009,13 +1077,53 @@ pub fn apply_parent_execution_payload( "apply_parent_execution_payload: len(requests.builder_exits) <= MAX_BUILDER_EXIT_REQUESTS_PER_PAYLOAD", )?; - // Process execution requests from the parent's payload. The execution - // requests are processed at state.slot (child's slot), not the parent's - // slot. - // - // Deposits, withdrawals, and consolidations are unmodified since electra - // (fulu, for deposits): see this module's own doc for why these three - // are shared calls into `stf::fulu`/`stf::electra` rather than copies. + process_gloas_execution_requests( + state, + requests.into(), + config, + active_balances, + pubkey_indices, + )?; + settle_parent_builder_payment(state, &parent_bid, parent_slot, parent_epoch)?; + mark_parent_payload_available(state, &parent_bid, parent_slot) +} + +/// Gloas's five execution-layer request lists, borrowed from whichever +/// fork's `ExecutionRequests` carries them: gloas's own, or EIP-8483's wider +/// one. +pub struct GloasRequests<'a> { + pub deposits: &'a gloas::DepositRequests, + pub withdrawals: &'a gloas::WithdrawalRequests, + pub consolidations: &'a gloas::ConsolidationRequests, + pub builder_deposits: &'a gloas::BuilderDepositRequests, + pub builder_exits: &'a gloas::BuilderExitRequests, +} + +impl<'a> From<&'a gloas::ExecutionRequests> for GloasRequests<'a> { + fn from(requests: &'a gloas::ExecutionRequests) -> Self { + GloasRequests { + deposits: &requests.deposits, + withdrawals: &requests.withdrawals, + consolidations: &requests.consolidations, + builder_deposits: &requests.builder_deposits, + builder_exits: &requests.builder_exits, + } + } +} + +/// Gloas's five execution-layer request lists, in the specification's order, +/// processed at `state.slot` (the child's slot), not the parent's. +/// +/// Deposits, withdrawals, and consolidations are unmodified since electra +/// (fulu, for deposits): see this module's own doc for why these three are +/// shared calls into `stf::fulu`/`stf::electra` rather than copies. +pub(crate) fn process_gloas_execution_requests( + state: &mut BeaconState, + requests: GloasRequests<'_>, + config: &Config, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, +) -> Result<()> { for deposit in requests.deposits.iter() { crate::beacon::stf::fulu::process_deposit_request(state, deposit)?; } @@ -1043,8 +1151,31 @@ pub fn apply_parent_execution_payload( for builder_exit in requests.builder_exits.iter() { process_builder_exit_request(state, builder_exit, config)?; } + Ok(()) +} + +/// The parent's bid, slot and epoch, as `apply_parent_execution_payload` +/// reads them before touching anything. +pub(crate) fn parent_payload_context( + state: &BeaconState, +) -> Result<(gloas::ExecutionPayloadBid, Slot, u64)> { + let inner = gloas_state_ref(state, "apply_parent_execution_payload")?; + let parent_slot = inner.latest_block_header.slot; + Ok(( + inner.latest_execution_payload_bid.clone(), + parent_slot, + compute_epoch_at_slot(parent_slot), + )) +} - // Settle the builder payment. +/// Settles the parent's builder payment, or queues its withdrawal directly +/// when the payment has already left the two-epoch window. +pub(crate) fn settle_parent_builder_payment( + state: &mut BeaconState, + parent_bid: &gloas::ExecutionPayloadBid, + parent_slot: Slot, + parent_epoch: u64, +) -> Result<()> { let current_epoch = get_current_epoch(state); if parent_epoch == current_epoch { let payment_index = preset::SLOTS_PER_EPOCH @@ -1053,13 +1184,13 @@ pub fn apply_parent_execution_payload( "apply_parent_execution_payload: SLOTS_PER_EPOCH + parent_slot % SLOTS_PER_EPOCH", ))?; settle_builder_payment( - gloas_state(state, "apply_parent_execution_payload")?, + &mut gloas_state(state, "apply_parent_execution_payload")?, payment_index, )?; } else if parent_epoch == get_previous_epoch(state) { let payment_index = parent_slot % preset::SLOTS_PER_EPOCH; settle_builder_payment( - gloas_state(state, "apply_parent_execution_payload")?, + &mut gloas_state(state, "apply_parent_execution_payload")?, payment_index, )?; } else if parent_bid.value > 0 { @@ -1074,8 +1205,16 @@ pub fn apply_parent_execution_payload( builder_index: parent_bid.builder_index, }); } + Ok(()) +} - // Update parent payload availability and latest block hash. +/// Marks the parent's payload available and makes its hash the latest block +/// hash: `apply_parent_execution_payload`'s last step at every fork. +pub(crate) fn mark_parent_payload_available( + state: &mut BeaconState, + parent_bid: &gloas::ExecutionPayloadBid, + parent_slot: Slot, +) -> Result<()> { let slot_index = (parent_slot as usize) % preset::SLOTS_PER_HISTORICAL_ROOT; let inner = gloas_state(state, "apply_parent_execution_payload")?; inner @@ -1085,7 +1224,7 @@ pub fn apply_parent_execution_payload( index: err.index, len: err.len, })?; - inner.latest_block_hash = parent_bid.block_hash; + *inner.latest_block_hash = parent_bid.block_hash; Ok(()) } @@ -1179,7 +1318,7 @@ pub fn process_builder_deposit_request( return Ok(()); } - let inner = gloas_state(state, "process_builder_deposit_request")?; + let mut inner = gloas_state(state, "process_builder_deposit_request")?; let existing = inner .builders .iter() @@ -1188,9 +1327,9 @@ pub fn process_builder_deposit_request( match existing { None => { if is_valid_builder_deposit_signature(request, config) { - let slot = inner.slot; + let slot = *inner.slot; add_builder_to_registry( - inner, + &mut inner, request.pubkey, constants::PAYLOAD_BUILDER_VERSION, ExecutionAddress::from_slice(&request.withdrawal_credentials.0[12..]), @@ -1200,7 +1339,7 @@ pub fn process_builder_deposit_request( } } Some(builder_index) => { - let epoch = compute_epoch_at_slot(inner.slot); + let epoch = compute_epoch_at_slot(*inner.slot); let builder = &mut inner.builders[builder_index]; // If exited and swept, reset the withdrawable epoch. @@ -1232,7 +1371,7 @@ pub fn process_builder_exit_request( request: &gloas::BuilderExitRequest, config: &Config, ) -> Result<()> { - let inner = gloas_state(state, "process_builder_exit_request")?; + let mut inner = gloas_state(state, "process_builder_exit_request")?; let Some(builder_index) = inner .builders .iter() @@ -1242,21 +1381,21 @@ pub fn process_builder_exit_request( }; let builder_index = builder_index as gloas::BuilderIndex; - if !is_active_builder(inner, builder_index)? { + if !is_active_builder(&inner.as_ref(), builder_index)? { return Ok(()); } if inner.builders[builder_index as usize].execution_address != request.source_address { return Ok(()); } if crate::beacon::helpers::gloas::get_pending_balance_to_withdraw_for_builder( - inner, + &inner.as_ref(), builder_index, )? != 0 { return Ok(()); } - initiate_builder_exit(inner, builder_index, config) + initiate_builder_exit(&mut inner, builder_index, config) } // --------------------------------------------------------------------------- @@ -1402,7 +1541,41 @@ pub fn process_attestation( signatures: &mut SignatureBatch, target_delta: Option<&TargetBalanceDelta>, ) -> Result<()> { - let data = attestation.data; + process_attestation_with( + state, + attestation.data, + &get_committee_indices(&attestation.committee_bits), + &attestation.aggregation_bits, + parent_slot, + committees, + active_balances, + target_delta, + |state, _| { + let indexed_attestation = get_indexed_attestation(state, attestation, committees)?; + signatures.push( + gloas_attestation_signature_set(state, attestation, &indexed_attestation, committees), + "process_attestation: is_valid_indexed_attestation(state, get_indexed_attestation(state, attestation))", + ) + }, + ) +} + +/// The body of gloas's `process_attestation`, which the PQ fork keeps but for +/// its signature: an attestation there carries none, the block's `proof` +/// covering every attestation at once (`process_stark`). `verify_signature` +/// is handed the attesting indices, sorted, after every structural check. +#[allow(clippy::too_many_arguments)] +pub(crate) fn process_attestation_with( + state: &mut BeaconState, + data: crate::beacon::containers::shared::AttestationData, + committee_indices: &[crate::beacon::primitives::CommitteeIndex], + aggregation_bits: &gloas::AggregationBits, + parent_slot: Slot, + committees: &CommitteeCache, + active_balances: &ActiveBalanceCache, + target_delta: Option<&TargetBalanceDelta>, + verify_signature: impl FnOnce(&BeaconState, &[ValidatorIndex]) -> Result<()>, +) -> Result<()> { let current_epoch = get_current_epoch(state); let previous_epoch = get_previous_epoch(state); @@ -1414,9 +1587,12 @@ pub fn process_attestation( data.target.epoch == compute_epoch_at_slot(data.slot), "process_attestation: data.target.epoch == compute_epoch_at_slot(data.slot)", )?; + // [Modified in PQ] `MIN_ATTESTATION_INCLUSION_DELAY_PQ` on a PQ state. let min_slot = data .slot - .checked_add(preset::MIN_ATTESTATION_INCLUSION_DELAY) + .checked_add(crate::beacon::helpers::pq::min_attestation_inclusion_delay( + state, + )) .ok_or(Error::ArithmeticOverflow( "process_attestation: data.slot + MIN_ATTESTATION_INCLUSION_DELAY", ))?; @@ -1428,31 +1604,35 @@ pub fn process_attestation( // [Modified in Gloas:EIP7732] `data.index` is now a payload-availability // bit (0 or 1), not always zero. verify(data.index < 2, "process_attestation: data.index < 2")?; - let committee_indices = get_committee_indices(&attestation.committee_bits); let epoch_committees = committees.committees(state, data.target.epoch); let mut committee_offset = 0usize; - for committee_index in committee_indices { + let mut attesting_indices: Vec = Vec::new(); + for &committee_index in committee_indices { verify( committee_index < epoch_committees.committees_per_slot(), "process_attestation: committee_index < get_committee_count_per_slot(state, data.target.epoch)", )?; let committee = epoch_committees.committee(data.slot, committee_index)?; - let committee_has_an_attester = (0..committee.len()).any(|position| { - attestation - .aggregation_bits + let before = attesting_indices.len(); + for (position, &attester) in committee.iter().enumerate() { + if aggregation_bits .get(committee_offset + position) .unwrap_or(false) - }); + { + attesting_indices.push(attester); + } + } verify( - committee_has_an_attester, + attesting_indices.len() > before, "process_attestation: len(committee_attesters) > 0", )?; committee_offset += committee.len(); } verify( - attestation.aggregation_bits.len() == committee_offset, + aggregation_bits.len() == committee_offset, "process_attestation: len(attestation.aggregation_bits) == committee_offset", )?; + attesting_indices.sort_unstable(); // Safe: `min_slot <= state.slot()` above and `min_slot >= data.slot` (the // inclusion delay is non-negative), so `data.slot <= state.slot()`. @@ -1460,11 +1640,7 @@ pub fn process_attestation( let participation_flag_indices = get_attestation_participation_flag_indices(state, &data, inclusion_delay, parent_slot)?; - let indexed_attestation = get_indexed_attestation(state, attestation, committees)?; - signatures.push( - gloas_attestation_signature_set(state, attestation, &indexed_attestation, committees), - "process_attestation: is_valid_indexed_attestation(state, get_indexed_attestation(state, attestation))", - )?; + verify_signature(state, &attesting_indices)?; let current_epoch_target = data.target.epoch == current_epoch; let payment_index = if current_epoch_target { @@ -1489,7 +1665,6 @@ pub fn process_attestation( })? }; - let attesting_indices: Vec = indexed_attestation.attesting_indices.to_vec(); // The specification calls `is_attestation_same_slot(state, data)` inside // the per-attester short-circuit below, so hoisting it out to a single // call here changes nothing: `get_attestation_participation_flag_indices` @@ -1702,10 +1877,44 @@ pub fn process_payload_attestation( /// `assert len(body.deposits) == 0` unconditionally, with no `process_deposit` /// call at all: like fulu, gloas has no eth1-bridge deposit path left to /// drain (`crate::beacon::stf::fulu::process_operations`'s own doc explains why). +/// The operation lists of a gloas-shaped block body. EIP-8483's body carries +/// the same lists (only its `parent_execution_requests` is wider), so +/// [`process_operations`] reads them through this rather than one body type. +pub struct Operations<'a> { + pub deposits: &'a gloas::Deposits, + pub proposer_slashings: &'a gloas::ProposerSlashings, + pub attester_slashings: &'a gloas::AttesterSlashings, + pub attestations: &'a gloas::Attestations, + pub voluntary_exits: &'a gloas::VoluntaryExits, + pub bls_to_execution_changes: &'a gloas::BlsToExecutionChanges, + pub payload_attestations: &'a gloas::PayloadAttestations, +} + +macro_rules! operations_from_body { + ($body:ty) => { + impl<'a> From<&'a $body> for Operations<'a> { + fn from(body: &'a $body) -> Self { + Operations { + deposits: &body.deposits, + proposer_slashings: &body.proposer_slashings, + attester_slashings: &body.attester_slashings, + attestations: &body.attestations, + voluntary_exits: &body.voluntary_exits, + bls_to_execution_changes: &body.bls_to_execution_changes, + payload_attestations: &body.payload_attestations, + } + } + } + }; +} + +operations_from_body!(gloas::BeaconBlockBody); +operations_from_body!(crate::beacon::containers::eip8483::BeaconBlockBody); + #[allow(clippy::too_many_arguments)] -pub fn process_operations( +pub fn process_operations<'a>( state: &mut BeaconState, - body: &gloas::BeaconBlockBody, + body: impl Into>, parent_slot: Slot, config: &Config, committees: &CommitteeCache, @@ -1713,6 +1922,7 @@ pub fn process_operations( signatures: &mut SignatureBatch, target_delta: Option<&TargetBalanceDelta>, ) -> Result<()> { + let body: Operations<'a> = body.into(); verify( body.deposits.is_empty(), "process_operations: len(body.deposits) == 0", @@ -1831,7 +2041,7 @@ mod tests { builder_index: constants::BUILDER_INDEX_SELF_BUILD, slot: state.slot(), value: 0, - parent_block_hash: gloas_state_ref(&state, "test").unwrap().latest_block_hash, + parent_block_hash: *gloas_state_ref(&state, "test").unwrap().latest_block_hash, parent_block_root: get_block_root_at_slot(&state, state.slot() - 1).unwrap(), prev_randao: get_randao_mix(&state, get_current_epoch(&state)), block_hash: crate::beacon::primitives::ExecutionBlockHash::repeat_byte(1), @@ -1863,7 +2073,7 @@ mod tests { builder_index: constants::BUILDER_INDEX_SELF_BUILD, slot: state.slot(), value: 1, - parent_block_hash: gloas_state_ref(&state, "test").unwrap().latest_block_hash, + parent_block_hash: *gloas_state_ref(&state, "test").unwrap().latest_block_hash, parent_block_root: get_block_root_at_slot(&state, state.slot() - 1).unwrap(), prev_randao: get_randao_mix(&state, get_current_epoch(&state)), block_hash: crate::beacon::primitives::ExecutionBlockHash::repeat_byte(1), @@ -2202,9 +2412,9 @@ mod tests { { let inner = gloas_state(&mut state, "test").unwrap(); - inner.latest_block_hash = parent_hash; - inner.latest_execution_payload_bid = bid.clone(); - inner.payload_expected_withdrawals = gloas::Withdrawals::default(); + *inner.latest_block_hash = parent_hash; + *inner.latest_execution_payload_bid = bid.clone(); + *inner.payload_expected_withdrawals = gloas::Withdrawals::default(); } // `validators` is the tree-backed field `validator_mut` above // buffered a write against; flush it before `state.compute_state_root()` @@ -2288,7 +2498,7 @@ mod tests { matches!( verify_execution_payload_envelope( &state, - &signed, + &signed.clone().into(), &config, &ExecutionEngine::invalid() ), @@ -2301,8 +2511,13 @@ mod tests { ), "every check but the engine's should have already passed" ); - verify_execution_payload_envelope(&state, &signed, &config, &ExecutionEngine::valid()) - .expect("a consistent envelope passes every check, including a valid engine's"); + verify_execution_payload_envelope( + &state, + &signed.clone().into(), + &config, + &ExecutionEngine::valid(), + ) + .expect("a consistent envelope passes every check, including a valid engine's"); } #[test] @@ -2325,8 +2540,13 @@ mod tests { let cached_root = state.compute_state_root(); state.latest_block_header_mut().state_root = cached_root; - verify_execution_payload_envelope(&state, &signed, &config, &ExecutionEngine::valid()) - .expect("a stored state's cached header.state_root must still match the envelope"); + verify_execution_payload_envelope( + &state, + &signed.clone().into(), + &config, + &ExecutionEngine::valid(), + ) + .expect("a stored state's cached header.state_root must still match the envelope"); } #[test] @@ -2340,8 +2560,12 @@ mod tests { // the signature. let signed = sign_envelope(&state, &secret, message); - let result = - verify_execution_payload_envelope(&state, &signed, &config, &ExecutionEngine::valid()); + let result = verify_execution_payload_envelope( + &state, + &signed.clone().into(), + &config, + &ExecutionEngine::valid(), + ); assert!( matches!( result, @@ -2365,8 +2589,12 @@ mod tests { consistent_envelope_message(|bid| bid.builder_index = 0, |_| {}); let signed = sign_envelope(&state, &secret, message); - let result = - verify_execution_payload_envelope(&state, &signed, &config, &ExecutionEngine::valid()); + let result = verify_execution_payload_envelope( + &state, + &signed.clone().into(), + &config, + &ExecutionEngine::valid(), + ); assert!( matches!( result, diff --git a/crates/blockchain/state_transition/src/beacon/stf/mod.rs b/crates/blockchain/state_transition/src/beacon/stf/mod.rs index c43beb066..4735370e2 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/mod.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/mod.rs @@ -58,11 +58,13 @@ pub mod bellatrix; pub mod block; pub mod capella; pub mod deneb; +pub mod eip8483; pub mod electra; pub mod epoch; pub mod fulu; pub mod gloas; pub mod operations; +pub mod pq; pub mod signatures; use crate::beacon::containers; @@ -292,16 +294,26 @@ pub fn verify_block_signature( state: &BeaconState, signed_block: &containers::SignedBeaconBlock, ) -> bool { + // [Modified in PQ] An XMSS signature at the slot's proposer leaf. + if let containers::SignedBeaconBlock::Pq(signed) = signed_block { + return pq::verify_block_signature(state, signed); + } block_signature_set(state, signed_block).is_some_and(|set| set.verify()) } /// The proposer's signature check [`verify_block_signature`] makes, resolved /// but not yet run. `None` is a proposer index outside the registry, or a key /// or signature that does not validate. +/// +/// `None` for a PQ block too: it carries an XMSS signature, which +/// [`verify_block_signature`] checks on its own. pub fn block_signature_set( state: &BeaconState, signed_block: &containers::SignedBeaconBlock, ) -> Option { + if matches!(signed_block, containers::SignedBeaconBlock::Pq(_)) { + return None; + } let index = signed_block.proposer_index(); let proposer = state.validator(index).ok()?; let domain = get_domain(state, constants::DOMAIN_BEACON_PROPOSER, None); @@ -461,7 +473,10 @@ pub fn process_slot(state: &mut BeaconState) -> Result<()> { | BeaconState::Deneb(_) | BeaconState::Electra(_) | BeaconState::Fulu(_) => {} - BeaconState::Gloas(inner) => { + BeaconState::Gloas(_) | BeaconState::Eip8483(_) | BeaconState::Pq(_) => { + let inner = state + .gloas_family_mut() + .expect("a gloas-family state has the gloas-family fields"); let next_slot = inner .slot .checked_add(1) @@ -582,6 +597,45 @@ mod tests { assert_eq!(inner.execution_payload_availability.get(0), Some(false)); } + /// `process_slots` crosses gloas's successors one fork at a time, in + /// order: with both boundaries at one epoch the loop upgrades twice, PQ + /// from the EIP-8483 state, and with them an epoch apart the state spends + /// that epoch as EIP-8483's. + #[test] + fn process_slots_upgrades_gloas_through_eip8483_to_pq() { + // Enough validators for every slot's committee to have members, which + // the PTC and aggregation committees sample from. + let gloas = test_state::with_validators_at(ForkName::Gloas, 64); + // The test state starts in epoch 1, so the boundaries come after it. + let next = gloas.slot() / preset::SLOTS_PER_EPOCH + 1; + let config = |eip8483, pq| { + Config::mainnet() + .with_fork_epoch(ForkName::Gloas, 0) + .with_fork_epoch(ForkName::Eip8483, eip8483) + .with_fork_epoch(ForkName::Pq, pq) + }; + let epoch_start = |epoch| epoch * preset::SLOTS_PER_EPOCH; + + let together = config(next, next); + let mut state = gloas.clone(); + process_slots(&mut state, epoch_start(next) - 1, &together).unwrap(); + assert_eq!(state.fork_name(), ForkName::Gloas, "not before the epoch"); + process_slots(&mut state, epoch_start(next), &together).unwrap(); + assert_eq!(state.fork_name(), ForkName::Pq); + assert_eq!( + state.fork().previous_version, + together.fork_version(ForkName::Eip8483), + "PQ upgraded from the EIP-8483 state, not from gloas's" + ); + + let apart = config(next, next + 1); + let mut state = gloas; + process_slots(&mut state, epoch_start(next), &apart).unwrap(); + assert_eq!(state.fork_name(), ForkName::Eip8483); + process_slots(&mut state, epoch_start(next + 1), &apart).unwrap(); + assert_eq!(state.fork_name(), ForkName::Pq); + } + #[test] fn process_slot_leaves_no_buffered_registry_writes() { let mut state = test_state::with_validators(4); @@ -660,16 +714,8 @@ mod tests { } } - const BEACON_FORKS: [ForkName; 8] = [ - ForkName::Phase0, - ForkName::Altair, - ForkName::Bellatrix, - ForkName::Capella, - ForkName::Deneb, - ForkName::Electra, - ForkName::Fulu, - ForkName::Gloas, - ]; + /// Every beacon fork, the two built on gloas included. + const BEACON_FORKS: [ForkName; 10] = ForkName::ALL; /// Writes one element of the field a case names, or answers `false` if the /// fork does not have it. @@ -683,6 +729,8 @@ mod tests { BeaconState::Electra(s) => s.historical_summaries.push(summary).unwrap(), BeaconState::Fulu(s) => s.historical_summaries.push(summary).unwrap(), BeaconState::Gloas(s) => s.historical_summaries.push(summary).unwrap(), + BeaconState::Eip8483(s) => s.historical_summaries.push(summary).unwrap(), + BeaconState::Pq(s) => s.historical_summaries.push(summary).unwrap(), _ => return false, } true @@ -726,10 +774,11 @@ mod tests { s.slashings_mut()[0] += 1; true }), - // Gloas keeps its scores in a flat progressive list that buffers - // nothing, so a write there leaves nothing pending. + // Gloas, and the forks built on it, keep their scores in a flat + // progressive list that buffers nothing, so a write there leaves + // nothing pending. ("inactivity_scores", |s| { - let buffered = !matches!(s, BeaconState::Gloas(_)); + let buffered = s.gloas_family().is_none(); match s.inactivity_score_mut(0) { Ok(score) => { *score += 1; diff --git a/crates/blockchain/state_transition/src/beacon/stf/operations.rs b/crates/blockchain/state_transition/src/beacon/stf/operations.rs index f31fe8b40..1f57a55e5 100644 --- a/crates/blockchain/state_transition/src/beacon/stf/operations.rs +++ b/crates/blockchain/state_transition/src/beacon/stf/operations.rs @@ -98,7 +98,12 @@ pub fn process_operations( let altair_attestations = match state.fork_name() { ForkName::Phase0 => false, ForkName::Altair | ForkName::Bellatrix | ForkName::Capella => true, - fork @ (ForkName::Deneb | ForkName::Electra | ForkName::Fulu | ForkName::Gloas) => { + fork @ (ForkName::Deneb + | ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq) => { return Err(Error::UnsupportedForFork { function: "process_operations", fork, @@ -521,7 +526,11 @@ pub fn add_validator_to_registry( // are progressive rather than `SszList` (see `BeaconState::altair_validator_lists`'s // own documentation), so it is refused for both reasons rather than // folded into either branch above. - fork @ (ForkName::Electra | ForkName::Fulu | ForkName::Gloas) => { + fork @ (ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq) => { return Err(Error::UnsupportedForFork { function: "add_validator_to_registry", fork, diff --git a/crates/blockchain/state_transition/src/beacon/stf/pq.rs b/crates/blockchain/state_transition/src/beacon/stf/pq.rs new file mode 100644 index 000000000..d52fb20a0 --- /dev/null +++ b/crates/blockchain/state_transition/src/beacon/stf/pq.rs @@ -0,0 +1,1413 @@ +//! The PQ fork's block processing (consensus-specs +//! `specs/_features/pq/beacon-chain.md`, on the `pq` branch at +//! `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`). +//! +//! Gloas's block processing with leanXMSS in place of BLS: +//! +//! - the block, a bid, an envelope and a proposer slashing's headers are each +//! signed at a leaf the message's slot and type determine +//! (`compute_xmss_leaf_index`), a bid at any of the bid leaves; +//! - attestations carry no signature: the block's `proof` covers all of them, +//! checked once in [`process_stark`] after every attestation is processed; +//! - the RANDAO reveal is a VRF proof, the leaf of the proposer's committed +//! tree for the slot ([`process_randao`]); +//! - voluntary exits are execution-layer requests, and the body loses +//! `voluntary_exits`, `sync_aggregate`, `bls_to_execution_changes` and +//! `payload_attestations`; +//! - deposits carry no signature: a 48-byte key field names an XMSS key after +//! 16 zero bytes, a legacy BLS key otherwise, and preregistrations +//! (EIP-8205) bind a key to its credentials before anyone can front-run its +//! first deposit. +//! +//! What is not new is gloas's code, reached through the gloas-family view: +//! withdrawals, the bid's body, attestation participation and builder-payment +//! weights, and the payment settlement. Every XMSS check goes through +//! [`crate::beacon::helpers::pq::xmss_verify`] directly, not the BLS +//! [`SignatureBatch`] the earlier forks defer their checks into. + +use crate::beacon::config::Config; +use crate::beacon::constants; +use crate::beacon::containers::shared::BeaconBlockHeader; +use crate::beacon::containers::{BeaconState, electra, gloas, pq}; +use crate::beacon::error::{Error, Result, verify}; +use crate::beacon::helpers::accessors::{ + ActiveBalanceCache, CommitteeCache, PubkeyIndexCache, get_beacon_proposer_index, + get_current_epoch, get_domain, get_randao_mix, +}; +use crate::beacon::helpers::electra::get_pending_balance_to_withdraw; +use crate::beacon::helpers::gloas::{ + get_index_for_new_builder, gloas_state, initiate_builder_exit, is_builder_withdrawal_credential, +}; +use crate::beacon::helpers::misc::{compute_epoch_at_slot, compute_signing_root}; +use crate::beacon::helpers::mutators::slash_validator; +use crate::beacon::helpers::pq::{ + compute_deposit_pubkey, compute_xmss_leaf_index, get_active_builder_preregistration, + get_active_validator_preregistration, get_attestation_statements, get_committee_indices, + get_leaf_index, is_builder_bid_leaf_index, is_legacy_pubkey, is_valid_indexed_attestation, + is_valid_vrf_proof, pq_state, verify_aggregates, xmss_verify, +}; +use crate::beacon::helpers::predicates::{ + is_active_validator, is_slashable_attestation_data, is_slashable_validator, +}; +use crate::beacon::helpers::target_balances::TargetBalanceDelta; +use crate::beacon::preset; +use crate::beacon::primitives::{ + ExecutionAddress, HashTreeRoot as _, Root, Slot, ValidatorIndex, XmssPubkey, XmssSignature, +}; +use crate::metrics::{StfPhase, timed, try_for_each_timed}; + +use super::block::RandaoVerification; +use super::gloas::{ + mark_parent_payload_available, parent_payload_context, settle_parent_builder_payment, +}; + +// --------------------------------------------------------------------------- +// Block processing +// --------------------------------------------------------------------------- + +/// `process_block` (modified in PQ): gloas's without `process_sync_aggregate`. +pub fn process_block( + state: &mut BeaconState, + block: &pq::BeaconBlock, + config: &Config, + committees: &CommitteeCache, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, + options: super::block::BlockProcessingOptions, +) -> Result<()> { + let parent_slot = state.latest_block_header().slot; + timed(StfPhase::ExecutionPayload, || { + process_parent_execution_payload(state, block, config, active_balances, pubkey_indices) + })?; + timed(StfPhase::BlockHeader, || { + super::block::process_block_header( + state, + block.slot, + block.proposer_index, + block.parent_root, + block.body.hash_tree_root(), + ) + })?; + timed(StfPhase::ExecutionPayload, || -> Result<()> { + super::gloas::process_withdrawals(state)?; + process_execution_payload_bid(state, &block.body.signed_execution_payload_bid, config) + })?; + timed(StfPhase::Randao, || { + process_randao(state, &block.body.randao_reveal, options.randao) + })?; + super::block::process_eth1_data(state, &block.body.eth1_data)?; + process_operations( + state, + &block.body, + parent_slot, + config, + committees, + active_balances, + options.target_delta, + ) +} + +/// `verify_block_signature` (modified in PQ): the proposer's XMSS signature at +/// the block's slot's proposer leaf. +pub fn verify_block_signature(state: &BeaconState, signed_block: &pq::SignedBeaconBlock) -> bool { + let block = &signed_block.message; + let Ok(proposer) = state.pq_validator(block.proposer_index) else { + return false; + }; + let domain = get_domain(state, constants::DOMAIN_BEACON_PROPOSER, None); + let signing_root = compute_signing_root(block.hash_tree_root(), domain); + let Ok(leaf_index) = + compute_xmss_leaf_index(block.slot, constants::LEAF_OFFSET_BEACON_PROPOSER) + else { + return false; + }; + xmss_verify( + &proposer.pubkey(), + leaf_index, + signing_root, + &signed_block.signature, + ) +} + +/// `process_randao` (modified in PQ): the reveal is the proposer's VRF proof +/// for the slot, committed to with its VRF root before the fork, and the leaf +/// it opens is mixed in. +pub fn process_randao( + state: &mut BeaconState, + randao_reveal: &pq::VrfProof, + verification: RandaoVerification, +) -> Result<()> { + let epoch = get_current_epoch(state); + if verification == RandaoVerification::Verify { + let proposer_index = get_beacon_proposer_index(state)?; + verify( + is_valid_vrf_proof(state, proposer_index, state.slot(), randao_reveal)?, + "process_randao: is_valid_vrf_proof(state, get_beacon_proposer_index(state), state.slot, body.randao_reveal)", + )?; + } + let mut input = Vec::with_capacity(32 + 32 + 4); + input.extend_from_slice(&get_randao_mix(state, epoch).0); + input.extend_from_slice(&randao_reveal[0].0); + input.extend_from_slice(&constants::DOMAIN_RANDAO); + let mix = crate::beacon::hash::hash(&input); + let position = epoch as usize % preset::EPOCHS_PER_HISTORICAL_VECTOR; + state.randao_mixes_mut()[position] = mix; + Ok(()) +} + +// --------------------------------------------------------------------------- +// Execution payload bid and envelope +// --------------------------------------------------------------------------- + +/// `verify_execution_payload_bid_signature` (modified in PQ): the builder's +/// XMSS signature, at one of the bid leaves. +pub fn verify_execution_payload_bid_signature( + state: &BeaconState, + signed_bid: &pq::SignedExecutionPayloadBid, +) -> Result { + let builder_index = signed_bid.message.builder_index; + let inner = state.gloas_family().ok_or(Error::UnsupportedForFork { + function: "verify_execution_payload_bid_signature", + fork: state.fork_name(), + })?; + let builder = inner + .builders + .get(builder_index as usize) + .ok_or(Error::IndexOutOfBounds { + index: builder_index as usize, + len: inner.builders.len(), + })?; + let domain = get_domain(state, constants::DOMAIN_BEACON_BUILDER, None); + let signing_root = compute_signing_root(signed_bid.message.hash_tree_root(), domain); + let leaf_index = get_leaf_index(&signed_bid.signature); + if !is_builder_bid_leaf_index(leaf_index) { + return Ok(false); + } + Ok(xmss_verify( + &XmssPubkey::from_request_key(&builder.pubkey), + leaf_index, + signing_root, + &signed_bid.signature, + )) +} + +/// `process_execution_payload_bid` (modified in PQ): gloas's, a self-build's +/// signature being the empty (all-zero) XMSS signature. +pub fn process_execution_payload_bid( + state: &mut BeaconState, + signed_bid: &pq::SignedExecutionPayloadBid, + config: &Config, +) -> Result<()> { + super::gloas::process_execution_payload_bid_with( + state, + &signed_bid.message, + config, + signed_bid.signature == XmssSignature::default(), + |state| { + verify( + verify_execution_payload_bid_signature(state, signed_bid)?, + "process_execution_payload_bid: verify_execution_payload_bid_signature(state, signed_bid)", + ) + }, + ) +} + +/// `verify_execution_payload_envelope_signature` (modified in PQ): the +/// builder's (or, for a self-build, the proposer's) XMSS signature at the +/// envelope leaf of the block's slot. +pub fn verify_execution_payload_envelope_signature( + state: &BeaconState, + signed_envelope: &pq::SignedExecutionPayloadEnvelope, +) -> Result { + let builder_index = signed_envelope.message.builder_index; + let header = state.latest_block_header(); + let pubkey = if builder_index == constants::BUILDER_INDEX_SELF_BUILD { + state.pq_validator(header.proposer_index)?.pubkey() + } else { + let inner = state.gloas_family().ok_or(Error::UnsupportedForFork { + function: "verify_execution_payload_envelope_signature", + fork: state.fork_name(), + })?; + let builder = + inner + .builders + .get(builder_index as usize) + .ok_or(Error::IndexOutOfBounds { + index: builder_index as usize, + len: inner.builders.len(), + })?; + XmssPubkey::from_request_key(&builder.pubkey) + }; + let domain = get_domain(state, constants::DOMAIN_BEACON_BUILDER, None); + let signing_root = compute_signing_root(signed_envelope.message.hash_tree_root(), domain); + let leaf_index = compute_xmss_leaf_index( + header.slot, + constants::LEAF_OFFSET_EXECUTION_PAYLOAD_ENVELOPE, + )?; + Ok(xmss_verify( + &pubkey, + leaf_index, + signing_root, + &signed_envelope.signature, + )) +} + +// --------------------------------------------------------------------------- +// Operations +// --------------------------------------------------------------------------- + +/// `process_operations` (modified in PQ): slashings and attestations, then +/// [`process_stark`] over every attestation of the block. +pub fn process_operations( + state: &mut BeaconState, + body: &pq::BeaconBlockBody, + parent_slot: Slot, + config: &Config, + committees: &CommitteeCache, + active_balances: &ActiveBalanceCache, + target_delta: Option<&TargetBalanceDelta>, +) -> Result<()> { + verify( + body.deposits.is_empty(), + "process_operations: len(body.deposits) == 0", + )?; + verify( + body.proposer_slashings.len() <= preset::MAX_PROPOSER_SLASHINGS, + "process_operations: len(body.proposer_slashings) <= MAX_PROPOSER_SLASHINGS", + )?; + verify( + body.attester_slashings.len() <= preset::MAX_ATTESTER_SLASHINGS_ELECTRA, + "process_operations: len(body.attester_slashings) <= MAX_ATTESTER_SLASHINGS_ELECTRA", + )?; + verify( + body.attestations.len() as u64 <= preset::MAX_ATTESTATIONS_PQ, + "process_operations: len(body.attestations) <= MAX_ATTESTATIONS_PQ", + )?; + + try_for_each_timed( + StfPhase::ProposerSlashings, + body.proposer_slashings.iter(), + |slashing| process_proposer_slashing(state, slashing, config), + )?; + try_for_each_timed( + StfPhase::AttesterSlashings, + body.attester_slashings.iter(), + |slashing| process_attester_slashing(state, slashing, config), + )?; + try_for_each_timed( + StfPhase::Attestations, + body.attestations.iter(), + |attestation| { + process_attestation( + state, + attestation, + parent_slot, + committees, + active_balances, + target_delta, + ) + }, + )?; + process_stark(state, body, committees) +} + +/// `process_stark`: the block's `proof` proves, for each attestation and each +/// of its committees in order, its attesters' signatures. +pub fn process_stark( + state: &BeaconState, + body: &pq::BeaconBlockBody, + committees: &CommitteeCache, +) -> Result<()> { + let mut statements = Vec::new(); + for attestation in body.attestations.iter() { + statements.extend(get_attestation_statements(state, attestation, committees)?); + } + verify( + verify_aggregates(&statements, &body.proof), + "process_stark: stark.VerifyAggregates(statements, body.proof)", + ) +} + +/// `process_proposer_slashing` (modified in PQ): both headers are signed at +/// the same leaf, which exposes that leaf of the proposer's key; the leaf +/// belongs to a past slot, and the proposer is slashed. The slashed +/// proposer's pending builder payments are cleared by `slash_validator`. +pub fn process_proposer_slashing( + state: &mut BeaconState, + proposer_slashing: &pq::ProposerSlashing, + config: &Config, +) -> Result<()> { + let header_1 = &proposer_slashing.signed_header_1.message; + let header_2 = &proposer_slashing.signed_header_2.message; + + verify( + header_1.slot == header_2.slot, + "process_proposer_slashing: header_1.slot == header_2.slot", + )?; + verify( + header_1.proposer_index == header_2.proposer_index, + "process_proposer_slashing: header_1.proposer_index == header_2.proposer_index", + )?; + verify( + header_1 != header_2, + "process_proposer_slashing: header_1 != header_2", + )?; + let proposer_index = header_1.proposer_index; + verify( + is_slashable_validator(state.validator(proposer_index)?, get_current_epoch(state)), + "process_proposer_slashing: is_slashable_validator(proposer, get_current_epoch(state))", + )?; + let pubkey = state.pq_validator(proposer_index)?.pubkey(); + for signed_header in [ + &proposer_slashing.signed_header_1, + &proposer_slashing.signed_header_2, + ] { + verify( + header_signature_valid( + state, + &pubkey, + &signed_header.message, + &signed_header.signature, + )?, + "process_proposer_slashing: xmss.Verify(proposer.pubkey, leaf_index, signing_root, signed_header.signature)", + )?; + } + + slash_validator(state, proposer_index, None, config) +} + +pub(crate) fn header_signature_valid( + state: &BeaconState, + pubkey: &XmssPubkey, + header: &BeaconBlockHeader, + signature: &XmssSignature, +) -> Result { + let domain = get_domain( + state, + constants::DOMAIN_BEACON_PROPOSER, + Some(compute_epoch_at_slot(header.slot)), + ); + let signing_root = compute_signing_root(header.hash_tree_root(), domain); + let leaf_index = compute_xmss_leaf_index(header.slot, constants::LEAF_OFFSET_BEACON_PROPOSER)?; + Ok(xmss_verify(pubkey, leaf_index, signing_root, signature)) +} + +/// `process_attester_slashing`: unchanged in shape, each indexed attestation +/// now carrying a proof of its attesters' signatures. +pub fn process_attester_slashing( + state: &mut BeaconState, + attester_slashing: &pq::AttesterSlashing, + config: &Config, +) -> Result<()> { + let attestation_1 = &attester_slashing.attestation_1; + let attestation_2 = &attester_slashing.attestation_2; + verify( + is_slashable_attestation_data(&attestation_1.data, &attestation_2.data), + "process_attester_slashing: is_slashable_attestation_data(attestation_1.data, attestation_2.data)", + )?; + verify( + is_valid_indexed_attestation(state, attestation_1)?, + "process_attester_slashing: is_valid_indexed_attestation(state, attestation_1)", + )?; + verify( + is_valid_indexed_attestation(state, attestation_2)?, + "process_attester_slashing: is_valid_indexed_attestation(state, attestation_2)", + )?; + crate::beacon::stf::operations::slash_attesting_index_intersection( + state, + &attestation_1.attesting_indices, + &attestation_2.attesting_indices, + config, + ) +} + +/// `process_attestation` (modified in PQ): gloas's, with no signature to +/// check here ([`process_stark`] covers the block's attestations at once) and +/// the PQ fork's committee count. +pub fn process_attestation( + state: &mut BeaconState, + attestation: &pq::Attestation, + parent_slot: Slot, + committees: &CommitteeCache, + active_balances: &ActiveBalanceCache, + target_delta: Option<&TargetBalanceDelta>, +) -> Result<()> { + super::gloas::process_attestation_with( + state, + attestation.data, + &get_committee_indices(&attestation.committee_bits), + &attestation.aggregation_bits, + parent_slot, + committees, + active_balances, + target_delta, + |_, _| Ok(()), + ) +} + +// --------------------------------------------------------------------------- +// Parent execution payload +// --------------------------------------------------------------------------- + +/// `process_parent_execution_payload`: gloas's, over the PQ fork's requests. +pub fn process_parent_execution_payload( + state: &mut BeaconState, + block: &pq::BeaconBlock, + config: &Config, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, +) -> Result<()> { + let bid = &block.body.signed_execution_payload_bid.message; + let (parent_bid, _, _) = parent_payload_context(state)?; + let requests = &block.body.parent_execution_requests; + + if bid.parent_block_hash != parent_bid.block_hash { + verify( + *requests == pq::ExecutionRequests::default(), + "process_parent_execution_payload: requests == ExecutionRequests.empty()", + )?; + return Ok(()); + } + + verify( + requests.hash_tree_root() == parent_bid.execution_requests_root, + "process_parent_execution_payload: hash_tree_root(requests) == parent_bid.execution_requests_root", + )?; + apply_parent_execution_payload(state, requests, config, active_balances, pubkey_indices) +} + +/// `apply_parent_execution_payload` (modified in PQ): settles the parent's +/// builder payment, then processes its requests, every one of them in the PQ +/// fork's own version, then marks its payload available. +pub fn apply_parent_execution_payload( + state: &mut BeaconState, + requests: &pq::ExecutionRequests, + config: &Config, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, +) -> Result<()> { + let (parent_bid, parent_slot, parent_epoch) = parent_payload_context(state)?; + + verify( + requests.withdrawals.len() <= preset::MAX_WITHDRAWAL_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.withdrawals) <= MAX_WITHDRAWAL_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.consolidations.len() <= preset::MAX_CONSOLIDATION_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.consolidations) <= MAX_CONSOLIDATION_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.builder_deposits.len() as u64 <= preset::MAX_BUILDER_DEPOSIT_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.builder_deposits) <= MAX_BUILDER_DEPOSIT_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.builder_exits.len() as u64 <= preset::MAX_BUILDER_EXIT_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.builder_exits) <= MAX_BUILDER_EXIT_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.validator_preregistrations.len() as u64 + <= preset::MAX_VALIDATOR_PREREGISTRATION_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.validator_preregistrations) <= MAX_VALIDATOR_PREREGISTRATION_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.builder_preregistrations.len() as u64 + <= preset::MAX_BUILDER_PREREGISTRATION_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.builder_preregistrations) <= MAX_BUILDER_PREREGISTRATION_REQUESTS_PER_PAYLOAD", + )?; + verify( + requests.voluntary_exits.len() as u64 <= preset::MAX_VOLUNTARY_EXIT_REQUESTS_PER_PAYLOAD, + "apply_parent_execution_payload: len(requests.voluntary_exits) <= MAX_VOLUNTARY_EXIT_REQUESTS_PER_PAYLOAD", + )?; + + // Settled before the requests, so a builder exit request is rejected while + // the payment is pending. + settle_parent_builder_payment(state, &parent_bid, parent_slot, parent_epoch)?; + + for request in requests.deposits.iter() { + process_deposit_request(state, request)?; + } + for request in requests.withdrawals.iter() { + process_withdrawal_request(state, request, config, active_balances, pubkey_indices)?; + } + for request in requests.consolidations.iter() { + process_consolidation_request(state, request, config, active_balances, pubkey_indices)?; + } + for request in requests.builder_deposits.iter() { + process_builder_deposit_request(state, request, config)?; + } + for request in requests.builder_exits.iter() { + process_builder_exit_request(state, request, config)?; + } + for request in requests.validator_preregistrations.iter() { + process_validator_preregistration_request(state, request)?; + } + for request in requests.builder_preregistrations.iter() { + process_builder_preregistration_request(state, request)?; + } + for request in requests.voluntary_exits.iter() { + process_voluntary_exit_request(state, request, config, active_balances)?; + } + + mark_parent_payload_available(state, &parent_bid, parent_slot) +} + +/// `get_execution_requests_list` (modified in PQ): the EIP-7685 encoding of a +/// payload's requests, the preregistrations and voluntary exits in place of +/// EIP-8483's registrations. +pub fn get_execution_requests_list(requests: &pq::ExecutionRequests) -> Vec> { + use libssz::SszEncode as _; + let typed: [(u8, Vec, bool); 8] = [ + ( + constants::DEPOSIT_REQUEST_TYPE, + requests.deposits.to_ssz(), + requests.deposits.is_empty(), + ), + ( + constants::WITHDRAWAL_REQUEST_TYPE, + requests.withdrawals.to_ssz(), + requests.withdrawals.is_empty(), + ), + ( + constants::CONSOLIDATION_REQUEST_TYPE, + requests.consolidations.to_ssz(), + requests.consolidations.is_empty(), + ), + ( + constants::BUILDER_DEPOSIT_REQUEST_TYPE, + requests.builder_deposits.to_ssz(), + requests.builder_deposits.is_empty(), + ), + ( + constants::BUILDER_EXIT_REQUEST_TYPE, + requests.builder_exits.to_ssz(), + requests.builder_exits.is_empty(), + ), + ( + constants::VALIDATOR_PREREGISTRATION_REQUEST_TYPE, + requests.validator_preregistrations.to_ssz(), + requests.validator_preregistrations.is_empty(), + ), + ( + constants::BUILDER_PREREGISTRATION_REQUEST_TYPE, + requests.builder_preregistrations.to_ssz(), + requests.builder_preregistrations.is_empty(), + ), + ( + constants::VOLUNTARY_EXIT_REQUEST_TYPE, + requests.voluntary_exits.to_ssz(), + requests.voluntary_exits.is_empty(), + ), + ]; + typed + .into_iter() + .filter(|(_, _, empty)| !empty) + .map(|(request_type, data, _)| { + let mut entry = Vec::with_capacity(1 + data.len()); + entry.push(request_type); + entry.extend(data); + entry + }) + .collect() +} + +// --------------------------------------------------------------------------- +// Execution-layer requests +// --------------------------------------------------------------------------- + +/// `process_deposit_request` (modified in PQ): an active preregistration of +/// the key discards a deposit that does not match its binding, and is used up +/// by one that does. +pub fn process_deposit_request( + state: &mut BeaconState, + request: &electra::DepositRequest, +) -> Result<()> { + let pubkey = compute_deposit_pubkey(&request.pubkey); + let inner = pq_state(state, "process_deposit_request")?; + if let Some(preregistration) = get_active_validator_preregistration(inner, &pubkey) { + if request.withdrawal_credentials != preregistration.withdrawal_credentials { + return Ok(()); + } + if Root::from_slice(&request.signature.0[..32]) != preregistration.vrf_root { + return Ok(()); + } + remove_validator_preregistration(inner, &pubkey); + } + inner.pending_deposits.push(electra::PendingDeposit { + pubkey: request.pubkey, + withdrawal_credentials: request.withdrawal_credentials, + amount: request.amount, + signature: request.signature, + slot: inner.slot, + })?; + Ok(()) +} + +/// `process_withdrawal_request` (modified in PQ): a request naming a BLS key is +/// ignored; any other names its validator by XMSS key, which is how the +/// registry holds it (see `containers::pq`), so the rest is electra's. +pub fn process_withdrawal_request( + state: &mut BeaconState, + request: &electra::WithdrawalRequest, + config: &Config, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, +) -> Result<()> { + if is_legacy_pubkey(&request.validator_pubkey) { + return Ok(()); + } + super::electra::process_withdrawal_request( + state, + request, + config, + active_balances, + pubkey_indices, + ) +} + +/// `process_consolidation_request` (modified in PQ): a request naming a BLS +/// key on either side is ignored; the rest is electra's, for the reason +/// [`process_withdrawal_request`] gives. +pub fn process_consolidation_request( + state: &mut BeaconState, + request: &electra::ConsolidationRequest, + config: &Config, + active_balances: &ActiveBalanceCache, + pubkey_indices: &PubkeyIndexCache, +) -> Result<()> { + if is_legacy_pubkey(&request.source_pubkey) || is_legacy_pubkey(&request.target_pubkey) { + return Ok(()); + } + super::electra::process_consolidation_request( + state, + request, + config, + active_balances, + pubkey_indices, + ) +} + +/// `process_builder_deposit_request` (modified in PQ): no signature to check, +/// an active preregistration binds the key's credentials, and a legacy +/// deposit's new builder initiates its exit at once. +pub fn process_builder_deposit_request( + state: &mut BeaconState, + request: &gloas::BuilderDepositRequest, + config: &Config, +) -> Result<()> { + // Ignore deposits with unexpected withdrawal credential prefixes. + if !is_builder_withdrawal_credential(request.withdrawal_credentials) { + return Ok(()); + } + + let pubkey = compute_deposit_pubkey(&request.pubkey); + { + let inner = pq_state(state, "process_builder_deposit_request")?; + if let Some(preregistration) = get_active_builder_preregistration(inner, &pubkey) { + if request.withdrawal_credentials != preregistration.withdrawal_credentials { + return Ok(()); + } + remove_builder_preregistration(inner, &pubkey); + } + } + + let record_key = pubkey.to_request_key(); + let mut inner = gloas_state(state, "process_builder_deposit_request")?; + let existing = inner + .builders + .iter() + .position(|builder| builder.pubkey == record_key); + match existing { + None => { + let builder_index = get_index_for_new_builder(&inner.as_ref()); + let slot = *inner.slot; + crate::beacon::helpers::gloas::add_builder_to_registry( + &mut inner, + record_key, + constants::PAYLOAD_BUILDER_VERSION, + ExecutionAddress::from_slice(&request.withdrawal_credentials.0[12..]), + request.amount, + slot, + ); + if is_legacy_pubkey(&request.pubkey) { + initiate_builder_exit(&mut inner, builder_index, config)?; + } + } + Some(builder_index) => { + let epoch = compute_epoch_at_slot(*inner.slot); + let builder = &mut inner.builders[builder_index]; + // If exited and swept, reset the withdrawable epoch. + if builder.withdrawable_epoch != constants::FAR_FUTURE_EPOCH && builder.balance == 0 { + builder.withdrawable_epoch = epoch + .checked_add(config.min_builder_withdrawability_delay) + .ok_or(Error::ArithmeticOverflow( + "process_builder_deposit_request: epoch + MIN_BUILDER_WITHDRAWABILITY_DELAY", + ))?; + } + builder.balance = + builder + .balance + .checked_add(request.amount) + .ok_or(Error::ArithmeticOverflow( + "process_builder_deposit_request: builder.balance + request.amount", + ))?; + } + } + Ok(()) +} + +/// `process_builder_exit_request` (modified in PQ): the builder is named by +/// the XMSS key in the request's last 32 bytes; the rest is gloas's. +pub fn process_builder_exit_request( + state: &mut BeaconState, + request: &gloas::BuilderExitRequest, + config: &Config, +) -> Result<()> { + let named = gloas::BuilderExitRequest { + source_address: request.source_address, + pubkey: XmssPubkey::from_request_key(&request.pubkey).to_request_key(), + }; + super::gloas::process_builder_exit_request(state, &named, config) +} + +/// `process_validator_preregistration_request` (EIP-8205): binds a key that +/// has no validator and no pending deposit to its withdrawal credentials and +/// VRF root, until `PREREGISTRATION_EXPIRY_SLOTS` from now. An active binding +/// for the key makes a new request a no-op. +pub fn process_validator_preregistration_request( + state: &mut BeaconState, + request: &pq::ValidatorPreregistrationRequest, +) -> Result<()> { + let pubkey = request.pubkey; + let record_key = pubkey.to_request_key(); + let registered = state + .iter_validators() + .any(|validator| validator.pubkey == record_key); + let inner = pq_state(state, "process_validator_preregistration_request")?; + + if get_active_validator_preregistration(inner, &pubkey).is_some() { + return Ok(()); + } + // A pubkey with an existing validator cannot be preregistered. + if registered { + return Ok(()); + } + // A pubkey with a pending deposit cannot be preregistered. + if inner + .pending_deposits + .iter() + .any(|deposit| compute_deposit_pubkey(&deposit.pubkey) == pubkey) + { + return Ok(()); + } + // Only active records count towards the limit, so the timing of the + // garbage-collection sweep does not affect admission. + let active = inner + .validator_preregistrations + .iter() + .filter(|record| { + crate::beacon::helpers::pq::is_active_validator_preregistration(inner, record) + }) + .count() as u64; + if active >= preset::PREREGISTRATIONS_LIMIT { + return Ok(()); + } + + let preregistration = pq::ValidatorPreregistration { + pubkey, + withdrawal_credentials: request.withdrawal_credentials, + vrf_root: request.vrf_root, + expiry_slot: expiry_slot(inner.slot)?, + }; + match inner + .validator_preregistrations + .iter() + .position(|record| record.pubkey == pubkey) + { + // Replace the expired record in place. + Some(index) => inner.validator_preregistrations[index] = preregistration, + None => inner.validator_preregistrations.push(preregistration), + } + Ok(()) +} + +/// `process_builder_preregistration_request` (EIP-8205): the builder +/// counterpart of [`process_validator_preregistration_request`], with no VRF +/// root and no pending-deposit check. +pub fn process_builder_preregistration_request( + state: &mut BeaconState, + request: &pq::BuilderPreregistrationRequest, +) -> Result<()> { + let pubkey = request.pubkey; + let record_key = pubkey.to_request_key(); + let inner = pq_state(state, "process_builder_preregistration_request")?; + + if get_active_builder_preregistration(inner, &pubkey).is_some() { + return Ok(()); + } + // A pubkey with an existing builder cannot be preregistered. + if inner + .builders + .iter() + .any(|builder| builder.pubkey == record_key) + { + return Ok(()); + } + let active = inner + .builder_preregistrations + .iter() + .filter(|record| { + crate::beacon::helpers::pq::is_active_builder_preregistration(inner, record) + }) + .count() as u64; + if active >= preset::PREREGISTRATIONS_LIMIT { + return Ok(()); + } + + let preregistration = pq::BuilderPreregistration { + pubkey, + withdrawal_credentials: request.withdrawal_credentials, + expiry_slot: expiry_slot(inner.slot)?, + }; + match inner + .builder_preregistrations + .iter() + .position(|record| record.pubkey == pubkey) + { + Some(index) => inner.builder_preregistrations[index] = preregistration, + None => inner.builder_preregistrations.push(preregistration), + } + Ok(()) +} + +fn expiry_slot(slot: Slot) -> Result { + slot.checked_add(preset::PREREGISTRATION_EXPIRY_SLOTS) + .ok_or(Error::ArithmeticOverflow( + "state.slot + PREREGISTRATION_EXPIRY_SLOTS", + )) +} + +/// `remove_validator_preregistration`: removal keeps the remaining records' +/// order, which the state root commits to. +fn remove_validator_preregistration(state: &mut pq::BeaconState, pubkey: &XmssPubkey) { + let kept: Vec = state + .validator_preregistrations + .iter() + .filter(|record| record.pubkey != *pubkey) + .cloned() + .collect(); + state.validator_preregistrations = kept.into(); +} + +/// `remove_builder_preregistration`. +fn remove_builder_preregistration(state: &mut pq::BeaconState, pubkey: &XmssPubkey) { + let kept: Vec = state + .builder_preregistrations + .iter() + .filter(|record| record.pubkey != *pubkey) + .cloned() + .collect(); + state.builder_preregistrations = kept.into(); +} + +/// `process_voluntary_exit_request`: the system contract verified the +/// validator's exit signature at a reserved leaf before emitting the request, +/// so anyone may have submitted it. Ignored unless the validator could exit +/// voluntarily. +pub fn process_voluntary_exit_request( + state: &mut BeaconState, + request: &pq::VoluntaryExitRequest, + config: &Config, + active_balances: &ActiveBalanceCache, +) -> Result<()> { + let record_key = request.pubkey.to_request_key(); + let Some(index) = state + .iter_validators() + .position(|validator| validator.pubkey == record_key) + else { + return Ok(()); + }; + let index = index as ValidatorIndex; + let validator = state.validator(index)?; + let current_epoch = get_current_epoch(state); + + if !is_active_validator(validator, current_epoch) { + return Ok(()); + } + if validator.exit_epoch != constants::FAR_FUTURE_EPOCH { + return Ok(()); + } + let eligible_epoch = validator + .activation_epoch + .checked_add(config.shard_committee_period) + .ok_or(Error::ArithmeticOverflow( + "validator.activation_epoch + SHARD_COMMITTEE_PERIOD", + ))?; + if current_epoch < eligible_epoch { + return Ok(()); + } + if get_pending_balance_to_withdraw(state, index)? != 0 { + return Ok(()); + } + crate::beacon::helpers::electra::initiate_validator_exit( + state, + index, + config, + Some(active_balances), + ) +} + +#[cfg(test)] +mod tests { + use ethlambda_crypto::lean_xmss; + + use super::*; + use crate::beacon::fork::ForkName; + use crate::beacon::hash::hash; + use crate::beacon::helpers::accessors::{CommitteeCacheExt, get_block_root_at_slot}; + use crate::beacon::helpers::pq::{ + attestation_message, compute_xmss_attestation_leaf_index, + initialize_aggregation_committee_window, + }; + use crate::beacon::helpers::test_state::with_validators_at; + use crate::beacon::primitives::Bytes32; + + /// A key real over `leaves`, for `seed`: a range key, so building one costs + /// a few hashes rather than a whole top tree. + fn key(seed: u8, first: u64, last: u64) -> lean_xmss::SecretKey { + lean_xmss::range_key(&[seed; 32], first as u32, last as u32) + } + + fn xmss_signature(signature: &lean_xmss::Signature) -> XmssSignature { + XmssSignature( + signature + .to_bytes() + .try_into() + .expect("a leanXMSS signature is XMSS_SIGNATURE_SIZE bytes"), + ) + } + + /// A VRF branch for `slot` under a root it determines: the leaf and every + /// sibling chosen, the root folded up from them, as `is_valid_merkle_branch` + /// folds. + fn vrf_proof_and_root(slot: Slot, leaf: Bytes32) -> (pq::VrfProof, Root) { + let siblings: Vec = (0..constants::VRF_TREE_DEPTH) + // Distinct from any leaf a test opens, so no level's order is moot. + .map(|level| Bytes32::repeat_byte(level as u8 + 0x40)) + .collect(); + let mut value = leaf; + for (level, sibling) in siblings.iter().enumerate() { + value = if (slot >> level) & 1 == 1 { + hash(&[sibling.0, value.0].concat()) + } else { + hash(&[value.0, sibling.0].concat()) + }; + } + let mut proof = vec![leaf]; + proof.extend(siblings); + ( + pq::VrfProof::try_from(proof).expect("VRF_TREE_DEPTH + 1 roots"), + value, + ) + } + + fn set_key(state: &mut BeaconState, index: ValidatorIndex, pubkey: XmssPubkey, vrf_root: Root) { + let BeaconState::Pq(inner) = state else { + panic!("a PQ test state") + }; + let validator = inner + .validators + .get_mut(index as usize) + .expect("index is in the registry"); + validator.record.pubkey = pubkey.to_request_key(); + validator.vrf_root = vrf_root; + } + + #[test] + fn a_valid_vrf_proof_mixes_its_leaf_in() { + let mut state = with_validators_at(ForkName::Pq, 64); + let slot = state.slot(); + let proposer = get_beacon_proposer_index(&state).unwrap(); + let leaf = Bytes32::repeat_byte(0xab); + let (proof, root) = vrf_proof_and_root(slot, leaf); + set_key(&mut state, proposer, XmssPubkey::default(), root); + + let epoch = get_current_epoch(&state); + let previous = get_randao_mix(&state, epoch); + process_randao(&mut state, &proof, RandaoVerification::Verify).unwrap(); + let expected = hash(&[&previous.0[..], &leaf.0, &constants::DOMAIN_RANDAO].concat()); + assert_eq!(get_randao_mix(&state, epoch), expected); + } + + #[test] + fn a_vrf_proof_for_another_slot_or_leaf_is_refused() { + let mut state = with_validators_at(ForkName::Pq, 64); + let slot = state.slot(); + let proposer = get_beacon_proposer_index(&state).unwrap(); + // A root committing to the leaf at the next slot: the same leaf and + // branch opened at this slot fold to a different root. + let (proof, next_slot_root) = vrf_proof_and_root(slot + 1, Bytes32::repeat_byte(1)); + set_key(&mut state, proposer, XmssPubkey::default(), next_slot_root); + assert!(process_randao(&mut state.clone(), &proof, RandaoVerification::Verify).is_err()); + + let (_, root) = vrf_proof_and_root(slot, Bytes32::repeat_byte(1)); + set_key(&mut state, proposer, XmssPubkey::default(), root); + let (wrong_leaf, _) = vrf_proof_and_root(slot, Bytes32::repeat_byte(2)); + assert!( + process_randao(&mut state.clone(), &wrong_leaf, RandaoVerification::Verify).is_err() + ); + // Block production's skip still mixes the reveal in, unchecked. + assert!(process_randao(&mut state, &wrong_leaf, RandaoVerification::Skip).is_ok()); + } + + #[test] + fn the_block_signature_is_checked_at_the_slots_proposer_leaf() { + let mut state = with_validators_at(ForkName::Pq, 64); + let slot = state.slot() + 1; + let proposer = 3; + let leaf = compute_xmss_leaf_index(slot, constants::LEAF_OFFSET_BEACON_PROPOSER).unwrap(); + let secret = key(9, leaf, leaf + 1); + let pubkey = XmssPubkey(secret.public_key().to_bytes()); + set_key(&mut state, proposer, pubkey, Root::ZERO); + + let block = pq::BeaconBlock { + slot, + proposer_index: proposer, + parent_root: Root::repeat_byte(4), + state_root: Root::repeat_byte(5), + body: pq::BeaconBlockBody::empty(), + }; + let domain = get_domain(&state, constants::DOMAIN_BEACON_PROPOSER, None); + let signing_root = compute_signing_root(block.hash_tree_root(), domain); + let sign = |leaf: u64| xmss_signature(&secret.sign(&signing_root.0, leaf as u32).unwrap()); + + let signed = pq::SignedBeaconBlock { + message: block.clone(), + signature: sign(leaf), + }; + assert!(verify_block_signature(&state, &signed)); + // The next leaf belongs to another message type of the same slot. + let wrong_leaf = pq::SignedBeaconBlock { + message: block, + signature: sign(leaf + 1), + }; + assert!(!verify_block_signature(&state, &wrong_leaf)); + } + + /// One attestation from a committee whose members signed at the target + /// epoch's leaf: the block's proof is their signatures in committee order, + /// and a proof missing one, or carrying them out of order, is refused. + #[test] + fn process_stark_checks_the_attesters_signatures() { + let mut state = with_validators_at(ForkName::Pq, 64); + // Attest to the previous slot, which the state has a root for. + let data_slot = state.slot() - 1; + let committees = CommitteeCache::default(); + let epoch = compute_epoch_at_slot(data_slot); + let committee: Vec = committees + .committees(&state, epoch) + .committee(data_slot, 0) + .unwrap() + .to_vec(); + assert!(committee.len() >= 2, "the committee has members to sign"); + + let data = crate::beacon::containers::shared::AttestationData { + slot: data_slot, + index: 0, + beacon_block_root: get_block_root_at_slot(&state, data_slot).unwrap(), + source: state.current_justified_checkpoint(), + target: crate::beacon::containers::shared::Checkpoint { + epoch, + root: Root::repeat_byte(6), + }, + }; + let leaf = compute_xmss_attestation_leaf_index(epoch).unwrap(); + let keys: Vec = committee + .iter() + .map(|&index| key(index as u8 + 20, leaf, leaf)) + .collect(); + for (&index, secret) in committee.iter().zip(&keys) { + set_key( + &mut state, + index, + XmssPubkey(secret.public_key().to_bytes()), + Root::ZERO, + ); + } + let (message, statement_leaf) = attestation_message(&state, &data).unwrap(); + assert_eq!(statement_leaf, leaf); + let signatures: Vec> = keys + .iter() + .map(|secret| secret.sign(&message.0, leaf as u32).unwrap().to_bytes()) + .collect(); + + let mut aggregation_bits = pq::AggregationBits::with_length(committee.len()); + for position in 0..committee.len() { + aggregation_bits.set(position, true).unwrap(); + } + let mut committee_bits = pq::CommitteeBits::new(); + committee_bits.set(0, true).unwrap(); + let attestation = pq::Attestation { + aggregation_bits, + data, + committee_bits, + }; + + let body_with = |proof: Vec| pq::BeaconBlockBody { + attestations: vec![attestation.clone()].into(), + proof: proof.try_into().expect("far below MAX_STARK_PROOF_SIZE"), + ..pq::BeaconBlockBody::empty() + }; + // The reference encoding: the signatures, in statement order. + let proof = signatures.concat(); + assert!(process_stark(&state, &body_with(proof.clone()), &committees).is_ok()); + + let short = signatures[1..].concat(); + assert!(process_stark(&state, &body_with(short), &committees).is_err()); + let mut reversed = signatures.clone(); + reversed.reverse(); + let reversed = reversed.concat(); + assert!(process_stark(&state, &body_with(reversed), &committees).is_err()); + } + + /// A full-committee attestation to `data_slot` voting for the head and + /// target the state holds there. + fn attestation_at(state: &BeaconState, data_slot: Slot) -> pq::Attestation { + let epoch = compute_epoch_at_slot(data_slot); + let committee_len = CommitteeCache::default() + .committees(state, epoch) + .committee(data_slot, 0) + .unwrap() + .len(); + let mut aggregation_bits = pq::AggregationBits::with_length(committee_len); + for position in 0..committee_len { + aggregation_bits.set(position, true).unwrap(); + } + let mut committee_bits = pq::CommitteeBits::new(); + committee_bits.set(0, true).unwrap(); + let source = if epoch == get_current_epoch(state) { + state.current_justified_checkpoint() + } else { + state.previous_justified_checkpoint() + }; + pq::Attestation { + aggregation_bits, + data: crate::beacon::containers::shared::AttestationData { + slot: data_slot, + index: 0, + beacon_block_root: get_block_root_at_slot(state, data_slot).unwrap(), + source, + target: crate::beacon::containers::shared::Checkpoint { + epoch, + root: crate::beacon::helpers::accessors::get_block_root(state, epoch).unwrap(), + }, + }, + committee_bits, + } + } + + /// `MIN_ATTESTATION_INCLUSION_DELAY_PQ`: an attestation from the previous + /// slot is too recent, and one from two slots back is timely for the head. + #[test] + fn attestations_wait_two_slots_and_two_is_timely_for_the_head() { + let state = with_validators_at(ForkName::Pq, 64); + let parent_slot = state.slot() - 1; + let process = |data_slot: Slot| { + process_attestation( + &mut state.clone(), + &attestation_at(&state, data_slot), + parent_slot, + &CommitteeCache::default(), + &ActiveBalanceCache::default(), + None, + ) + }; + assert!(process(state.slot() - 1).is_err()); + assert!(process(state.slot() - 2).is_ok()); + + let flags = |data_slot: Slot| { + let data = attestation_at(&state, data_slot).data; + crate::beacon::helpers::gloas::get_attestation_participation_flag_indices( + &state, + &data, + state.slot() - data_slot, + parent_slot, + ) + .unwrap() + }; + assert!(flags(state.slot() - 2).contains(&constants::TIMELY_HEAD_FLAG_INDEX)); + assert!(!flags(state.slot() - 3).contains(&constants::TIMELY_HEAD_FLAG_INDEX)); + } + + #[test] + fn a_self_built_bid_must_carry_the_empty_signature() { + let state = with_validators_at(ForkName::Pq, 64); + let bid = gloas::ExecutionPayloadBid { + builder_index: constants::BUILDER_INDEX_SELF_BUILD, + ..Default::default() + }; + let mut signed = pq::SignedExecutionPayloadBid { + message: bid, + signature: XmssSignature::default(), + }; + let config = Config::active(); + // An all-zero signature passes the self-build check and fails later, + // on the bid's slot; a nonzero one fails the self-build check itself. + let err = process_execution_payload_bid(&mut state.clone(), &signed, &config).unwrap_err(); + assert!(!format!("{err:?}").contains("empty signature")); + signed.signature.0[0] = 1; + let err = process_execution_payload_bid(&mut state.clone(), &signed, &config).unwrap_err(); + assert!(format!("{err:?}").contains("empty signature"), "{err:?}"); + } + + #[test] + fn a_bid_signature_off_the_bid_leaves_is_refused() { + let mut state = with_validators_at(ForkName::Pq, 64); + let bid_leaf = constants::XMSS_RESERVED_LEAVES + constants::XMSS_ATTESTATION_LEAVES; + let secret = key(40, bid_leaf - 1, bid_leaf); + { + let inner = state.gloas_family_mut().unwrap(); + inner.builders.push(gloas::Builder { + pubkey: XmssPubkey(secret.public_key().to_bytes()).to_request_key(), + ..Default::default() + }); + } + let bid = gloas::ExecutionPayloadBid { + builder_index: 0, + ..Default::default() + }; + let domain = get_domain(&state, constants::DOMAIN_BEACON_BUILDER, None); + let signing_root = compute_signing_root(bid.hash_tree_root(), domain); + let signed_at = |leaf: u64| pq::SignedExecutionPayloadBid { + message: bid.clone(), + signature: xmss_signature(&secret.sign(&signing_root.0, leaf as u32).unwrap()), + }; + assert!(verify_execution_payload_bid_signature(&state, &signed_at(bid_leaf)).unwrap()); + assert!(!verify_execution_payload_bid_signature(&state, &signed_at(bid_leaf - 1)).unwrap()); + } + + #[test] + fn preregistrations_bind_deposits_and_expire() { + let mut state = with_validators_at(ForkName::Pq, 64); + let pubkey = XmssPubkey([0x55; 32]); + let credentials = Bytes32::repeat_byte(0x01); + let vrf_root = Root::repeat_byte(0x77); + let request = pq::ValidatorPreregistrationRequest { + pubkey, + withdrawal_credentials: credentials, + vrf_root, + }; + process_validator_preregistration_request(&mut state, &request).unwrap(); + let deposit = |credentials: Bytes32, vrf_root: Root| { + let mut signature = [0u8; 96]; + signature[..32].copy_from_slice(&vrf_root.0); + electra::DepositRequest { + pubkey: pubkey.to_request_key(), + withdrawal_credentials: credentials, + amount: 32_000_000_000, + signature: crate::beacon::primitives::BlsSignature(signature), + index: 0, + } + }; + let pending = |state: &BeaconState| match state { + BeaconState::Pq(inner) => inner.pending_deposits.len(), + _ => unreachable!(), + }; + // A deposit naming other credentials, or another VRF root, is discarded. + process_deposit_request(&mut state, &deposit(Bytes32::repeat_byte(2), vrf_root)).unwrap(); + process_deposit_request(&mut state, &deposit(credentials, Root::repeat_byte(3))).unwrap(); + assert_eq!(pending(&state), 0); + // The matching one is queued and uses the binding up. + process_deposit_request(&mut state, &deposit(credentials, vrf_root)).unwrap(); + assert_eq!(pending(&state), 1); + let BeaconState::Pq(inner) = &state else { + unreachable!() + }; + assert!(inner.validator_preregistrations.is_empty()); + // With a pending deposit for the key, it cannot be preregistered again. + process_validator_preregistration_request(&mut state, &request).unwrap(); + let BeaconState::Pq(inner) = &state else { + unreachable!() + }; + assert!(inner.validator_preregistrations.is_empty()); + } + + #[test] + fn a_voluntary_exit_request_exits_an_eligible_validator() { + let mut state = with_validators_at(ForkName::Pq, 64); + let mut config = Config::active(); + config.shard_committee_period = 0; + let pubkey = XmssPubkey([0x31; 32]); + set_key(&mut state, 7, pubkey, Root::ZERO); + let request = pq::VoluntaryExitRequest { pubkey }; + process_voluntary_exit_request( + &mut state, + &request, + &config, + &ActiveBalanceCache::default(), + ) + .unwrap(); + assert_ne!( + state.validator(7).unwrap().exit_epoch, + constants::FAR_FUTURE_EPOCH + ); + // A key nobody holds changes nothing. + let before = state.clone(); + let unknown = pq::VoluntaryExitRequest { + pubkey: XmssPubkey([0x32; 32]), + }; + process_voluntary_exit_request( + &mut state, + &unknown, + &config, + &ActiveBalanceCache::default(), + ) + .unwrap(); + assert_eq!(state, before); + } + + #[test] + fn a_legacy_withdrawal_request_is_ignored() { + let mut state = with_validators_at(ForkName::Pq, 64); + let before = state.clone(); + let mut legacy = crate::beacon::primitives::BlsPubkey::default(); + legacy.0[0] = 0x80; + let request = electra::WithdrawalRequest { + source_address: Default::default(), + validator_pubkey: legacy, + amount: 0, + }; + process_withdrawal_request( + &mut state, + &request, + &Config::active(), + &ActiveBalanceCache::default(), + &PubkeyIndexCache::default(), + ) + .unwrap(); + assert_eq!(state, before); + } + + #[test] + fn the_aggregation_committee_window_holds_a_sample_of_each_committee() { + let state = with_validators_at(ForkName::Pq, 64); + let window = initialize_aggregation_committee_window(&state).unwrap(); + let committees = CommitteeCache::default(); + let epoch = get_current_epoch(&state); + let slot = crate::beacon::helpers::misc::compute_start_slot_at_epoch(epoch); + let first = &window[preset::SLOTS_PER_EPOCH as usize][0]; + let committee = committees + .committees(&state, epoch) + .committee(slot, 0) + .unwrap() + .to_vec(); + assert!(!first.is_empty()); + assert!(first.len() as u64 <= constants::AGGREGATORS_PER_COMMITTEE); + assert!(first.iter().all(|member| committee.contains(member))); + let mut unique = first.to_vec(); + unique.sort_unstable(); + unique.dedup(); + assert_eq!(unique.len(), first.len(), "sampled without duplicates"); + // The previous epoch starts empty. + assert!(window[0].is_empty()); + } +} diff --git a/crates/blockchain/state_transition/src/beacon/upgrade.rs b/crates/blockchain/state_transition/src/beacon/upgrade.rs index e17eadafb..50fa67314 100644 --- a/crates/blockchain/state_transition/src/beacon/upgrade.rs +++ b/crates/blockchain/state_transition/src/beacon/upgrade.rs @@ -21,8 +21,8 @@ use std::collections::{HashMap, HashSet}; use crate::beacon::config::Config; use crate::beacon::constants; use crate::beacon::containers::{ - BeaconState, EpochParticipation, Fork, InactivityScores, altair, bellatrix, capella, deneb, - electra, fulu, gloas, phase0, + BeaconState, EpochParticipation, Fork, GloasFamilyMut, InactivityScores, altair, bellatrix, + capella, deneb, eip8483, electra, fulu, gloas, phase0, pq, }; use crate::beacon::error::{Error, Result}; use crate::beacon::fork::ForkName; @@ -31,11 +31,12 @@ use crate::beacon::helpers::attestation::get_attesting_indices; use crate::beacon::helpers::misc::{ compute_activation_exit_epoch, compute_epoch_at_slot, compute_start_slot_at_epoch, }; +use crate::beacon::helpers::pq::{compute_legacy_vrf_root, compute_legacy_xmss_pubkey}; use crate::beacon::lean_fork_unreachable; use crate::beacon::preset; use crate::beacon::primitives::{ - BlsPubkey, BlsSignature, Bytes32, ExecutionAddress, ExecutionBlockHash, HashTreeRoot as _, - Root, Uint256, ValidatorIndex, + BlsPubkey, BlsSignature, Bytes32, Epoch, ExecutionAddress, ExecutionBlockHash, Gwei, + HashTreeRoot as _, Root, Uint256, ValidatorIndex, XmssPubkey, }; /// Replays `pending_attestations` into `post`'s `previous_epoch_participation`. @@ -1049,7 +1050,7 @@ fn onboard_builders_from_pending_deposits( } let index = crate::beacon::helpers::gloas::add_builder_to_registry( - state, + &mut GloasFamilyMut::from(&mut *state), deposit.pubkey, constants::PAYLOAD_BUILDER_VERSION, ExecutionAddress::from_slice(&deposit.withdrawal_credentials.0[12..]), @@ -1237,15 +1238,405 @@ pub fn upgrade_to_gloas(pre: &BeaconState, config: &Config) -> Result Result { + let BeaconState::Gloas(pre) = pre else { + return Err(Error::UnsupportedForFork { + function: "upgrade_to_eip8483", + fork: pre.fork_name(), + }); + }; + let pre = pre.clone(); + let fork = Fork { + previous_version: pre.fork.current_version, + current_version: config.eip8483_fork_version, + epoch: compute_epoch_at_slot(pre.slot), + }; + Ok(BeaconState::Eip8483(eip8483::BeaconState { + genesis_time: pre.genesis_time, + genesis_validators_root: pre.genesis_validators_root, + slot: pre.slot, + fork, + latest_block_header: pre.latest_block_header, + block_roots: pre.block_roots, + state_roots: pre.state_roots, + historical_roots: pre.historical_roots, + eth1_data: pre.eth1_data, + eth1_data_votes: pre.eth1_data_votes, + eth1_deposit_index: pre.eth1_deposit_index, + validators: pre.validators, + balances: pre.balances, + randao_mixes: pre.randao_mixes, + slashings: pre.slashings, + previous_epoch_participation: pre.previous_epoch_participation, + current_epoch_participation: pre.current_epoch_participation, + justification_bits: pre.justification_bits, + previous_justified_checkpoint: pre.previous_justified_checkpoint, + current_justified_checkpoint: pre.current_justified_checkpoint, + finalized_checkpoint: pre.finalized_checkpoint, + inactivity_scores: pre.inactivity_scores, + current_sync_committee: pre.current_sync_committee, + next_sync_committee: pre.next_sync_committee, + latest_block_hash: pre.latest_block_hash, + next_withdrawal_index: pre.next_withdrawal_index, + next_withdrawal_validator_index: pre.next_withdrawal_validator_index, + historical_summaries: pre.historical_summaries, + deposit_requests_start_index: pre.deposit_requests_start_index, + deposit_balance_to_consume: pre.deposit_balance_to_consume, + exit_balance_to_consume: pre.exit_balance_to_consume, + earliest_exit_epoch: pre.earliest_exit_epoch, + consolidation_balance_to_consume: pre.consolidation_balance_to_consume, + earliest_consolidation_epoch: pre.earliest_consolidation_epoch, + pending_deposits: pre.pending_deposits, + pending_partial_withdrawals: pre.pending_partial_withdrawals, + pending_consolidations: pre.pending_consolidations, + proposer_lookahead: pre.proposer_lookahead, + builders: pre.builders, + next_withdrawal_builder_index: pre.next_withdrawal_builder_index, + execution_payload_availability: pre.execution_payload_availability, + builder_pending_payments: pre.builder_pending_payments, + builder_pending_withdrawals: pre.builder_pending_withdrawals, + latest_execution_payload_bid: pre.latest_execution_payload_bid, + payload_expected_withdrawals: pre.payload_expected_withdrawals, + ptc_window: pre.ptc_window, + // [New in EIP8483] + validator_pq_registrations: Default::default(), + // [New in EIP8483] + builder_pq_registrations: Default::default(), + })) +} + +/// `get_validator_pq_registrations` (PQ `fork.md`): the validator +/// registrations, except those whose XMSS key is the legacy key of some +/// validator's BLS key, which anyone can compute and so must not be claimable. +fn get_validator_pq_registrations( + pre: &eip8483::BeaconState, +) -> HashMap { + let legacy: HashSet = pre + .validators + .iter() + .map(|validator| compute_legacy_xmss_pubkey(&validator.pubkey)) + .collect(); + // Later entries win, as the specification's dict comprehension does; the + // state holds at most one entry per validator anyway. + pre.validator_pq_registrations + .iter() + .filter(|registration| { + !legacy.contains(&XmssPubkey( + registration.pq_data[..32] + .try_into() + .expect("PQ data's first half is 32 bytes"), + )) + }) + .map(|registration| (registration.validator_index, registration.pq_data)) + .collect() +} + +/// `get_builder_pq_registrations` (PQ `fork.md`): the builder registrations, +/// except those whose key is the legacy key of some builder's BLS key. +fn get_builder_pq_registrations(pre: &eip8483::BeaconState) -> HashMap { + let legacy: HashSet = pre + .builders + .iter() + .map(|builder| compute_legacy_xmss_pubkey(&builder.pubkey)) + .collect(); + pre.builder_pq_registrations + .iter() + .filter(|registration| !legacy.contains(&XmssPubkey(registration.pq_data.0))) + .map(|registration| { + ( + registration.builder_pubkey, + XmssPubkey(registration.pq_data.0), + ) + }) + .collect() +} + +/// `compute_validators` (PQ `fork.md`): a registered validator takes its XMSS +/// key and VRF root; one that did not takes the legacy derivations of its BLS +/// key and is exited at the fork, withdrawable +/// `UNREGISTERED_VALIDATOR_WITHDRAWABILITY_DELAY` epochs later unless it was +/// due sooner (a slashed one keeps its withdrawable epoch, so +/// `process_slashings` still applies its penalty). +fn compute_validators( + pre: &eip8483::BeaconState, + epoch: Epoch, + config: &Config, +) -> Result { + let registered = get_validator_pq_registrations(pre); + let unregistered_withdrawable = epoch + .checked_add(config.unregistered_validator_withdrawability_delay) + .ok_or(Error::ArithmeticOverflow( + "epoch + UNREGISTERED_VALIDATOR_WITHDRAWABILITY_DELAY", + ))?; + let validators: Vec = pre + .validators + .iter() + .enumerate() + .map(|(index, validator)| { + let mut record = validator.clone(); + let (pubkey, vrf_root) = match registered.get(&(index as ValidatorIndex)) { + Some(pq_data) => ( + XmssPubkey(pq_data[..32].try_into().expect("32 bytes")), + Root::from_slice(&pq_data[32..]), + ), + None => { + record.exit_epoch = record.exit_epoch.min(epoch); + if !record.slashed { + record.withdrawable_epoch = + record.withdrawable_epoch.min(unregistered_withdrawable); + } + ( + compute_legacy_xmss_pubkey(&validator.pubkey), + compute_legacy_vrf_root(&validator.pubkey), + ) + } + }; + record.pubkey = pubkey.to_request_key(); + pq::Validator { record, vrf_root } + }) + .collect(); + Ok(validators.into()) +} + +/// `compute_builders` (PQ `fork.md`): a registered builder takes its XMSS key; +/// one that did not takes the legacy derivation of its BLS key and becomes +/// withdrawable at the fork. +fn compute_builders(pre: &eip8483::BeaconState, epoch: Epoch) -> pq::Builders { + let registered = get_builder_pq_registrations(pre); + let builders: Vec = pre + .builders + .iter() + .map(|builder| { + let mut builder = builder.clone(); + let pubkey = match registered.get(&builder.pubkey) { + Some(pubkey) => *pubkey, + None => { + builder.withdrawable_epoch = builder.withdrawable_epoch.min(epoch); + compute_legacy_xmss_pubkey(&builder.pubkey) + } + }; + builder.pubkey = pubkey.to_request_key(); + builder + }) + .collect(); + pq::Builders(builders.into()) +} + +/// `compute_pending_deposits` (PQ `fork.md`): a pending deposit for an +/// existing validator is renamed to that validator's XMSS key, so it still +/// tops it up; one for a new validator keeps its BLS key, a legacy deposit. +fn compute_pending_deposits( + pre: &eip8483::BeaconState, + validators: &pq::Validators, +) -> gloas::PendingDeposits { + // The specification's `validator_pubkeys.index`: the first index holding + // the key (a registry never holds a key twice, so there is only one). + let mut index_of: HashMap = HashMap::new(); + for (index, validator) in pre.validators.iter().enumerate() { + index_of.entry(validator.pubkey).or_insert(index); + } + let deposits: Vec = pre + .pending_deposits + .iter() + .map(|deposit| { + let mut deposit = deposit.clone(); + if let Some(&index) = index_of.get(&deposit.pubkey) { + deposit.pubkey = validators + .get(index) + .expect("validators has an entry for every pre-state index") + .record + .pubkey; + } + deposit + }) + .collect(); + deposits.into() +} + +/// The keys of the builders that registered a PQ key, read from the +/// registrations the fork honors. +fn registered_builders(pre: &eip8483::BeaconState) -> HashSet { + get_builder_pq_registrations(pre).into_keys().collect() +} + +/// `compute_builder_pending_withdrawals` (PQ `fork.md`): the pending +/// withdrawals, plus each unregistered builder's pending payment that has +/// reached the payment quorum, settled at the fork. +fn compute_builder_pending_withdrawals( + pre: &eip8483::BeaconState, + quorum: Gwei, +) -> Result { + let registered = registered_builders(pre); + let mut withdrawals: Vec = + pre.builder_pending_withdrawals.to_vec(); + for payment in pre.builder_pending_payments.iter() { + if payment.withdrawal.amount > 0 + && !registered.contains(&builder_of(pre, payment)?.pubkey) + && payment.weight >= quorum + { + withdrawals.push(payment.withdrawal.clone()); + } + } + Ok(withdrawals.into()) +} + +/// `compute_builder_pending_payments` (PQ `fork.md`): an unregistered +/// builder's pending payment is dropped (settled into the withdrawals above if +/// it reached the quorum). +fn compute_builder_pending_payments( + pre: &eip8483::BeaconState, +) -> Result { + let registered = registered_builders(pre); + let mut payments = Vec::with_capacity(pre.builder_pending_payments.len()); + for payment in pre.builder_pending_payments.iter() { + if payment.withdrawal.amount > 0 && !registered.contains(&builder_of(pre, payment)?.pubkey) + { + payments.push(gloas::BuilderPendingPayment::default()); + } else { + payments.push(payment.clone()); + } + } + Ok(payments + .try_into() + .expect("one entry per BUILDER_PENDING_PAYMENTS_LENGTH slot")) +} +/// The builder a pending payment names. +/// +/// Read only for a payment with a nonzero amount, the one case the result is +/// used in. The specification reads `pre.builders[payment.withdrawal.builder_index]` +/// for every payment, an empty one (builder index zero) included, which raises +/// on a registry without builders: the fork could not happen on such a chain. +/// Reading it lazily differs only there, where the specification has no +/// outcome to match (`docs/spec_deviations.md`). +fn builder_of<'a>( + pre: &'a eip8483::BeaconState, + payment: &gloas::BuilderPendingPayment, +) -> Result<&'a gloas::Builder> { + let index = payment.withdrawal.builder_index as usize; + pre.builders.get(index).ok_or(Error::IndexOutOfBounds { + index, + len: pre.builders.len(), + }) +} + +/// `upgrade_to_pq` (consensus-specs `_features/pq/fork.md`). +/// +/// Validators and builders keep their indices and take the XMSS keys (and, for +/// validators, the VRF roots) they registered through EIP-8483; those that did +/// not are exited (validators) or made withdrawable (builders) at the fork. +/// Pending deposits for existing validators are renamed to their XMSS keys, +/// unregistered builders' pending payments are settled or dropped, the sync +/// committees and EIP-8483's registrations leave the state, and the aggregation +/// committee window is initialized from the upgraded state. +pub fn upgrade_to_pq(pre: &BeaconState, config: &Config) -> Result { + let quorum = crate::beacon::helpers::gloas::get_builder_payment_quorum_threshold(pre)?; + let BeaconState::Eip8483(pre) = pre else { + return Err(Error::UnsupportedForFork { + function: "upgrade_to_pq", + fork: pre.fork_name(), + }); + }; + let epoch = compute_epoch_at_slot(pre.slot); + let validators = compute_validators(pre, epoch, config)?; + let pending_deposits = compute_pending_deposits(pre, &validators); + let builders = compute_builders(pre, epoch); + let builder_pending_payments = compute_builder_pending_payments(pre)?; + let builder_pending_withdrawals = compute_builder_pending_withdrawals(pre, quorum)?; + let pre = pre.clone(); + + let post = pq::BeaconState { + genesis_time: pre.genesis_time, + genesis_validators_root: pre.genesis_validators_root, + slot: pre.slot, + fork: Fork { + previous_version: pre.fork.current_version, + current_version: config.pq_fork_version, + epoch, + }, + latest_block_header: pre.latest_block_header, + block_roots: pre.block_roots, + state_roots: pre.state_roots, + historical_roots: pre.historical_roots, + eth1_data: pre.eth1_data, + eth1_data_votes: pre.eth1_data_votes, + eth1_deposit_index: pre.eth1_deposit_index, + // [Modified in PQ] + validators, + balances: pre.balances, + randao_mixes: pre.randao_mixes, + slashings: pre.slashings, + previous_epoch_participation: pre.previous_epoch_participation, + current_epoch_participation: pre.current_epoch_participation, + justification_bits: pre.justification_bits, + previous_justified_checkpoint: pre.previous_justified_checkpoint, + current_justified_checkpoint: pre.current_justified_checkpoint, + finalized_checkpoint: pre.finalized_checkpoint, + inactivity_scores: pre.inactivity_scores, + // [Modified in PQ] Removed `current_sync_committee` and + // `next_sync_committee`. + latest_block_hash: pre.latest_block_hash, + next_withdrawal_index: pre.next_withdrawal_index, + next_withdrawal_validator_index: pre.next_withdrawal_validator_index, + historical_summaries: pre.historical_summaries, + deposit_requests_start_index: pre.deposit_requests_start_index, + deposit_balance_to_consume: pre.deposit_balance_to_consume, + exit_balance_to_consume: pre.exit_balance_to_consume, + earliest_exit_epoch: pre.earliest_exit_epoch, + consolidation_balance_to_consume: pre.consolidation_balance_to_consume, + earliest_consolidation_epoch: pre.earliest_consolidation_epoch, + // [Modified in PQ] + pending_deposits, + pending_partial_withdrawals: pre.pending_partial_withdrawals, + pending_consolidations: pre.pending_consolidations, + proposer_lookahead: pre.proposer_lookahead, + // [Modified in PQ] + builders, + next_withdrawal_builder_index: pre.next_withdrawal_builder_index, + execution_payload_availability: pre.execution_payload_availability, + // [Modified in PQ] + builder_pending_payments, + // [Modified in PQ] + builder_pending_withdrawals, + latest_execution_payload_bid: pre.latest_execution_payload_bid, + payload_expected_withdrawals: pre.payload_expected_withdrawals, + ptc_window: pre.ptc_window, + // [Modified in PQ] Removed `validator_pq_registrations` and + // `builder_pq_registrations`. + // [New in PQ:EIP8205] + validator_preregistrations: Default::default(), + // [New in PQ:EIP8205] + builder_preregistrations: Default::default(), + // [New in PQ] Overwritten below, once `post` is a whole state. + aggregation_committee_window: pq::empty_aggregation_committee_window(), + }; + let mut post = BeaconState::Pq(post); + let window = crate::beacon::helpers::pq::initialize_aggregation_committee_window(&post)?; + let BeaconState::Pq(inner) = &mut post else { + unreachable!("`post` was built as a PQ state just above") + }; + inner.aggregation_committee_window = window; Ok(post) } @@ -1280,6 +1671,8 @@ pub fn upgrade_state(state: &BeaconState, to: ForkName, config: &Config) -> Resu ForkName::Electra => upgrade_to_electra(state, config), ForkName::Fulu => upgrade_to_fulu(state, config), ForkName::Gloas => upgrade_to_gloas(state, config), + ForkName::Eip8483 => upgrade_to_eip8483(state, config), + ForkName::Pq => upgrade_to_pq(state, config), // The `fork:` form, not `state:`: this dispatches on the requested // target, so it is the argument that is wrong, not what `state` holds. ForkName::Lean => lean_fork_unreachable("upgrade_state"), @@ -1610,9 +2003,231 @@ mod tests { fn upgrade_state_walks_every_fork_boundary_end_to_end() { // A cheap integration check on top of the per-function tests above: // `advance_to` already exercises every arm of `upgrade_state`, so - // reaching fulu without an error proves the whole dispatch chain - // (not just each function in isolation) is wired correctly. - let (fulu_state, _config) = advance_to(ForkName::Fulu, 4); - assert_eq!(fulu_state.fork_name(), ForkName::Fulu); + // reaching the last fork without an error proves the whole dispatch + // chain (not just each function in isolation) is wired correctly, + // through gloas and the two forks built on it. Gloas samples its PTC + // from every slot's committee, so every slot needs a member. + let last = *ForkName::ALL.last().unwrap(); + assert_eq!(last, ForkName::Pq); + let (state, _config) = advance_to(last, 64); + assert_eq!(state.fork_name(), last); + } + + /// EIP-8483 builds on gloas and PQ on EIP-8483: each upgrade takes its + /// predecessor's state and no other, so a schedule cannot reach PQ + /// without passing through both. + #[test] + fn the_forks_after_gloas_upgrade_from_their_predecessor_only() { + let config = Config::active(); + let (fulu, _) = advance_to(ForkName::Fulu, 64); + let (gloas, _) = advance_to(ForkName::Gloas, 64); + let (eip8483, _) = advance_to(ForkName::Eip8483, 64); + let (pq, _) = advance_to(ForkName::Pq, 64); + + assert!(upgrade_to_eip8483(&fulu, &config).is_err()); + assert!(upgrade_to_eip8483(&eip8483, &config).is_err()); + assert!(upgrade_to_eip8483(&gloas, &config).is_ok()); + + assert!( + upgrade_to_pq(&gloas, &config).is_err(), + "PQ skipping EIP-8483" + ); + assert!(upgrade_to_pq(&pq, &config).is_err()); + assert!(upgrade_to_pq(&eip8483, &config).is_ok()); + } + + /// An EIP-8483 state with distinct BLS keys, two builders, and the given + /// validator and builder registrations. + fn eip8483_with_registrations() -> BeaconState { + let mut state = + crate::beacon::helpers::test_state::with_validators_at(ForkName::Eip8483, 64); + for index in 0..64u64 { + let mut key = [0u8; 48]; + key[0] = 0x80; + key[1] = index as u8 + 1; + state.validator_mut(index).unwrap().pubkey = BlsPubkey(key); + } + let BeaconState::Eip8483(inner) = &mut state else { + unreachable!() + }; + for (index, byte) in [(1u64, 0x11u8), (2, 0x22)] { + let mut pq_data = [byte; 64]; + pq_data[32..].fill(byte + 1); + inner + .validator_pq_registrations + .push(eip8483::ValidatorPqRegistration { + validator_index: index, + pq_data, + }); + } + for byte in [0xa1u8, 0xb2] { + let mut key = [0u8; 48]; + key[0] = 0x80; + key[1] = byte; + inner.builders.push(gloas::Builder { + pubkey: BlsPubkey(key), + withdrawable_epoch: constants::FAR_FUTURE_EPOCH, + balance: 1, + ..Default::default() + }); + } + let first_builder = inner.builders[0].pubkey; + inner + .builder_pq_registrations + .push(eip8483::BuilderPqRegistration { + builder_pubkey: first_builder, + pq_data: Bytes32::repeat_byte(0x33), + }); + // A pending payment to the unregistered builder, short of the quorum. + inner.builder_pending_payments[0] = gloas::BuilderPendingPayment { + weight: 0, + withdrawal: gloas::BuilderPendingWithdrawal { + amount: 5, + builder_index: 1, + ..Default::default() + }, + proposer_index: 0, + }; + // A deposit topping up validator 2, and one for a new validator. + let top_up = inner.validators.get(2).unwrap().pubkey; + let mut new_key = [0u8; 48]; + new_key[0] = 0x80; + new_key[1] = 0xee; + for pubkey in [top_up, BlsPubkey(new_key)] { + inner + .pending_deposits + .push(electra::PendingDeposit { + pubkey, + withdrawal_credentials: Bytes32::ZERO, + amount: 1, + signature: BlsSignature::default(), + slot: 0, + }) + .unwrap(); + } + state + } + + #[test] + fn upgrade_to_pq_adopts_registrations_and_exits_the_rest() { + let config = Config::active(); + let pre = eip8483_with_registrations(); + let BeaconState::Eip8483(pre_inner) = &pre else { + unreachable!() + }; + let epoch = compute_epoch_at_slot(pre_inner.slot); + let post = upgrade_to_pq(&pre, &config).unwrap(); + let BeaconState::Pq(inner) = &post else { + panic!("upgrade_to_pq returns a PQ state") + }; + assert_eq!(inner.fork.current_version, config.pq_fork_version); + assert_eq!(inner.fork.previous_version, pre_inner.fork.current_version); + + // Registered: the key and VRF root of its registration, still active. + let registered = inner.validators.get(1).unwrap(); + assert_eq!(registered.pubkey(), XmssPubkey([0x11; 32])); + assert_eq!(registered.vrf_root, Root::repeat_byte(0x12)); + assert_eq!(registered.record.exit_epoch, constants::FAR_FUTURE_EPOCH); + + // Unregistered: the legacy derivations, exited now, withdrawable after + // the delay. + let bls = pre_inner.validators.get(0).unwrap().pubkey; + let unregistered = inner.validators.get(0).unwrap(); + assert_eq!(unregistered.pubkey(), compute_legacy_xmss_pubkey(&bls)); + assert_eq!(unregistered.vrf_root, compute_legacy_vrf_root(&bls)); + assert_eq!(unregistered.record.exit_epoch, epoch); + assert_eq!( + unregistered.record.withdrawable_epoch, + epoch + config.unregistered_validator_withdrawability_delay + ); + + // A top-up is renamed to its validator's XMSS key; a new validator's + // deposit keeps its BLS key. + let deposits: Vec<_> = inner.pending_deposits.iter().cloned().collect(); + assert_eq!(deposits[0].pubkey, XmssPubkey([0x22; 32]).to_request_key()); + assert_eq!(deposits[1].pubkey.0[1], 0xee); + + // The registered builder takes its key; the other is withdrawable now, + // and its payment short of the quorum is dropped. + assert_eq!( + inner.builders[0].pubkey, + XmssPubkey([0x33; 32]).to_request_key() + ); + assert_eq!( + inner.builders[0].withdrawable_epoch, + constants::FAR_FUTURE_EPOCH + ); + assert_eq!(inner.builders[1].withdrawable_epoch, epoch); + assert_eq!( + inner.builder_pending_payments[0], + gloas::BuilderPendingPayment::default() + ); + assert!(inner.builder_pending_withdrawals.is_empty()); + + // The window's current and lookahead epochs are filled, the previous + // one empty. + let slots = preset::SLOTS_PER_EPOCH as usize; + assert!( + inner.aggregation_committee_window[..slots] + .iter() + .all(|slot| slot.is_empty()) + ); + assert!( + inner.aggregation_committee_window[slots..] + .iter() + .all(|slot| !slot.is_empty()) + ); + + // The upgraded state encodes and decodes back to itself. + let decoded = BeaconState::from_ssz(ForkName::Pq, &post.to_ssz()).unwrap(); + assert_eq!(decoded, post); + assert_eq!(decoded.hash_tree_root(), post.hash_tree_root()); + } + + #[test] + fn a_legacy_key_cannot_be_registered_for_another_validator() { + let config = Config::active(); + let mut pre = eip8483_with_registrations(); + let BeaconState::Eip8483(inner) = &mut pre else { + unreachable!() + }; + // Validator 3 claims the key validator 0 takes for not registering. + let legacy = compute_legacy_xmss_pubkey(&inner.validators.get(0).unwrap().pubkey); + let mut pq_data = [0x44u8; 64]; + pq_data[..32].copy_from_slice(&legacy.0); + inner + .validator_pq_registrations + .push(eip8483::ValidatorPqRegistration { + validator_index: 3, + pq_data, + }); + let post = upgrade_to_pq(&pre, &config).unwrap(); + let BeaconState::Pq(post) = &post else { + unreachable!() + }; + let BeaconState::Eip8483(pre) = &pre else { + unreachable!() + }; + let validator = post.validators.get(3).unwrap(); + assert_eq!( + validator.pubkey(), + compute_legacy_xmss_pubkey(&pre.validators.get(3).unwrap().pubkey), + "the claim is ignored and validator 3 takes its own legacy key" + ); + } + + #[test] + fn gloas_upgrades_to_eip8483_with_empty_registrations() { + let config = Config::active(); + let pre = crate::beacon::helpers::test_state::with_validators_at(ForkName::Gloas, 8); + let post = upgrade_to_eip8483(&pre, &config).unwrap(); + let BeaconState::Eip8483(inner) = &post else { + panic!("an EIP-8483 state") + }; + assert!(inner.validator_pq_registrations.is_empty()); + assert!(inner.builder_pq_registrations.is_empty()); + assert_eq!(inner.fork.current_version, config.eip8483_fork_version); + let decoded = BeaconState::from_ssz(ForkName::Eip8483, &post.to_ssz()).unwrap(); + assert_eq!(decoded.hash_tree_root(), post.hash_tree_root()); } } diff --git a/crates/blockchain/state_transition/tests/beacon_spec/epoch_processing.rs b/crates/blockchain/state_transition/tests/beacon_spec/epoch_processing.rs index cc92102d9..226d87166 100644 --- a/crates/blockchain/state_transition/tests/beacon_spec/epoch_processing.rs +++ b/crates/blockchain/state_transition/tests/beacon_spec/epoch_processing.rs @@ -92,7 +92,9 @@ fn apply( | ForkName::Deneb | ForkName::Electra | ForkName::Fulu - | ForkName::Gloas => epoch::altair::process_rewards_and_penalties(state, config), + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => epoch::altair::process_rewards_and_penalties(state, config), ForkName::Lean => lean_is_not_a_fixture_fork("rewards_and_penalties"), }, // Deneb's own change (EIP-7514's activation-churn cap) is selected @@ -111,9 +113,11 @@ fn apply( // `BeaconState` already reaches through element accessors generic // over the bounded/progressive split, so a gloas state runs the // identical function unchanged. - ForkName::Electra | ForkName::Fulu | ForkName::Gloas => { - epoch::electra::process_registry_updates(state, config) - } + ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => epoch::electra::process_registry_updates(state, config), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -134,9 +138,11 @@ fn apply( // Unmodified in gloas too, and, like `registry_updates` above, // only touches `validators`/`balances` and `state.slashings()` // (a fork-invariant field), so it runs unchanged on a gloas state. - ForkName::Electra | ForkName::Fulu | ForkName::Gloas => { - epoch::electra::process_slashings(state, config) - } + ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => epoch::electra::process_slashings(state, config), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -153,9 +159,11 @@ fn apply( "effective_balance_updates" => match fork { // Same reasoning as `registry_updates`/`slashings` above: unmodified // in gloas, and reaches only `validators`/`balances`. - ForkName::Electra | ForkName::Fulu | ForkName::Gloas => { - epoch::electra::process_effective_balance_updates(state) - } + ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => epoch::electra::process_effective_balance_updates(state), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -203,7 +211,9 @@ fn apply( | ForkName::Deneb | ForkName::Electra | ForkName::Fulu - | ForkName::Gloas => epoch::altair::process_inactivity_updates(state, config), + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => epoch::altair::process_inactivity_updates(state, config), ForkName::Phase0 => Err( ethlambda_state_transition::beacon::Error::UnsupportedForFork { function: "inactivity_updates", @@ -213,7 +223,9 @@ fn apply( ForkName::Lean => lean_is_not_a_fixture_fork("inactivity_updates"), }, "participation_flag_updates" => match fork { - ForkName::Gloas => epoch::gloas::process_participation_flag_updates(state), + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => { + epoch::gloas::process_participation_flag_updates(state) + } ForkName::Altair | ForkName::Bellatrix | ForkName::Capella @@ -244,7 +256,11 @@ fn apply( // gloas state for; see `epoch::gloas`'s own module doc. "pending_deposits" => match fork { ForkName::Fulu => epoch::fulu::process_pending_deposits(state, config), - ForkName::Gloas => epoch::gloas::process_pending_deposits(state, config), + ForkName::Gloas | ForkName::Eip8483 => { + epoch::gloas::process_pending_deposits(state, config) + } + // Legacy deposits apply without churn, and deposits carry VRF roots. + ForkName::Pq => epoch::pq::process_pending_deposits(state, config), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -259,16 +275,21 @@ fn apply( // (`max_per_epoch_activation_churn_limit_gloas`) in isolation. No // `process_pending_deposits_churn` exists in `beacon-chain.md`; this // is not a new sub-transition. - "pending_deposits_churn" => epoch::gloas::process_pending_deposits(state, config), + "pending_deposits_churn" => match fork { + ForkName::Pq => epoch::pq::process_pending_deposits(state, config), + _ => epoch::gloas::process_pending_deposits(state, config), + }, // Unmodified in gloas too (`beacon-chain.md` does not touch this // function), and `PendingQueueFields::take_pending_consolidations`/ // `set_pending_consolidations` reach a gloas state's queue at the // `Vec` level, so this one copy serves it as well; see // `epoch::electra::process_pending_consolidations`'s own doc. "pending_consolidations" => match fork { - ForkName::Electra | ForkName::Fulu | ForkName::Gloas => { - epoch::electra::process_pending_consolidations(state, config) - } + ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => epoch::electra::process_pending_consolidations(state, config), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -287,7 +308,9 @@ fn apply( // excludes slashed validators for gloas), so this one copy serves // both; see that function's own doc. "proposer_lookahead" => match fork { - ForkName::Fulu | ForkName::Gloas => epoch::fulu::process_proposer_lookahead(state), + ForkName::Fulu | ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => { + epoch::fulu::process_proposer_lookahead(state) + } ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -304,6 +327,9 @@ fn apply( // New in gloas (EIP-7732); no earlier fork has either handler at all. "builder_pending_payments" => epoch::gloas::process_builder_pending_payments(state), "ptc_window" => epoch::gloas::process_ptc_window(state), + // New in PQ. + "aggregation_committee_window" => epoch::pq::process_aggregation_committee_window(state), + "preregistration_expiry" => epoch::pq::process_preregistration_expiry(state), other => return Err(format!("unhandled epoch step `{other}`")), }; diff --git a/crates/blockchain/state_transition/tests/beacon_spec/fork_choice.rs b/crates/blockchain/state_transition/tests/beacon_spec/fork_choice.rs index 1d942825a..2fec38d24 100644 --- a/crates/blockchain/state_transition/tests/beacon_spec/fork_choice.rs +++ b/crates/blockchain/state_transition/tests/beacon_spec/fork_choice.rs @@ -123,8 +123,8 @@ use std::sync::Arc; use ethlambda_state_transition::beacon::ForkName; use ethlambda_state_transition::beacon::config::Config; use ethlambda_state_transition::beacon::containers::{ - BeaconState, Checkpoint, SignedBeaconBlock, altair, bellatrix, capella, deneb, electra, fulu, - gloas, phase0, + BeaconState, Checkpoint, SignedBeaconBlock, altair, bellatrix, capella, deneb, eip8483, + electra, fulu, gloas, phase0, pq, }; use ethlambda_state_transition::beacon::fork_choice::{ self, DataAvailability, ForkChoiceNode, ForkRules, PayloadStatus, Store, @@ -345,6 +345,14 @@ pub(super) fn decode_anchor_block(case: &Case) -> Result Ok(SignedBeaconBlock::Eip8483(eip8483::SignedBeaconBlock { + message: decode(case, "anchor_block")?, + signature: Default::default(), + })), + ForkName::Pq => Ok(SignedBeaconBlock::Pq(Box::new(pq::SignedBeaconBlock { + message: decode(case, "anchor_block")?, + signature: Default::default(), + }))), ForkName::Lean => lean_is_not_a_fixture_fork("fork_choice"), } } @@ -372,7 +380,10 @@ fn decode_attestation(case: &Case, name: &str) -> Result Ok(fork_choice::Attestation::Phase0(decode(case, name)?)), - ForkName::Gloas => Ok(fork_choice::Attestation::Gloas(decode(case, name)?)), + ForkName::Gloas | ForkName::Eip8483 => { + Ok(fork_choice::Attestation::Gloas(decode(case, name)?)) + } + ForkName::Pq => Ok(fork_choice::Attestation::Pq(decode(case, name)?)), ForkName::Lean => lean_is_not_a_fixture_fork("fork_choice"), } } @@ -392,7 +403,10 @@ fn decode_attester_slashing( | ForkName::Bellatrix | ForkName::Capella | ForkName::Deneb => Ok(fork_choice::AttesterSlashing::Phase0(decode(case, name)?)), - ForkName::Gloas => Ok(fork_choice::AttesterSlashing::Gloas(decode(case, name)?)), + ForkName::Gloas | ForkName::Eip8483 => { + Ok(fork_choice::AttesterSlashing::Gloas(decode(case, name)?)) + } + ForkName::Pq => Ok(fork_choice::AttesterSlashing::Pq(decode(case, name)?)), ForkName::Lean => lean_is_not_a_fixture_fork("fork_choice"), } } @@ -605,7 +619,12 @@ fn apply_execution_step( } if let Some(name) = &step.execution_payload { - let signed_envelope: gloas::SignedExecutionPayloadEnvelope = decode(case, name)?; + // The case's own fork's envelope: gloas's, EIP-8483's or PQ's. + let signed_envelope = ethlambda_state_transition::beacon::containers::SignedExecutionPayloadEnvelope::from_ssz( + case.fork, + &case.ssz_bytes(name), + ) + .map_err(|err| format!("decoding {name}: {err:?}"))?; // The format names no sampled columns for this step, so the evidence // is the empty retrieval, which reads as available. return expect_outcome( diff --git a/crates/blockchain/state_transition/tests/beacon_spec/gossip.rs b/crates/blockchain/state_transition/tests/beacon_spec/gossip.rs index 5cd5b9f51..161eaeb9f 100644 --- a/crates/blockchain/state_transition/tests/beacon_spec/gossip.rs +++ b/crates/blockchain/state_transition/tests/beacon_spec/gossip.rs @@ -280,11 +280,15 @@ fn decode_signed_aggregate(case: &Case, name: &str) -> Result Ok(SignedAggregateAndProof::Gloas( + ForkName::Gloas | ForkName::Eip8483 => Ok(SignedAggregateAndProof::Gloas( gloas::SignedAggregateAndProof::from_ssz_bytes(&bytes) .map_err(|err| format!("decoding {name}: {err:?}"))?, )), - ForkName::Lean => Err(format!("no aggregate shape for fork {:?}", case.fork)), + // The PQ fork's aggregate carries a proof and an XMSS signature, which + // the gossip rules here do not model. + ForkName::Pq | ForkName::Lean => { + Err(format!("no aggregate shape for fork {:?}", case.fork)) + } } } @@ -323,7 +327,11 @@ fn deliver_payload( let Some(name) = &entry.payload else { return Ok(()); }; - let envelope = gloas::SignedExecutionPayloadEnvelope::from_ssz_bytes(&case.ssz_bytes(name)) + let envelope = + ethlambda_state_transition::beacon::containers::SignedExecutionPayloadEnvelope::from_ssz( + case.fork, + &case.ssz_bytes(name), + ) .map_err(|err| format!("decoding {name}: {err:?}"))?; if trusted { fork_choice::accept_execution_payload_envelope(store, &envelope); @@ -574,7 +582,7 @@ fn run_case(case: &Case) -> Result<(), String> { fork_choice::get_head(&mut store, &config) .map_err(|err| format!("computing the head: {err:?}"))?; } - let market = BuilderMarket::default(); + let market: BuilderMarket = BuilderMarket::default(); let capacity = NonZeroUsize::new(SEEN_CAPACITY).expect("non-zero"); let mut seen_blocks = SeenBlocks::new(capacity); let mut seen_columns = SeenColumns::new(capacity); diff --git a/crates/blockchain/state_transition/tests/beacon_spec/harness.rs b/crates/blockchain/state_transition/tests/beacon_spec/harness.rs index 33386e176..766d8e34d 100644 --- a/crates/blockchain/state_transition/tests/beacon_spec/harness.rs +++ b/crates/blockchain/state_transition/tests/beacon_spec/harness.rs @@ -69,7 +69,14 @@ pub fn trials() -> Vec { }), Trial::test("harness/discovery_covers_every_supported_fork", || { let cases = collect(PRESET, "ssz_static", "Checkpoint"); - for fork in ForkName::ALL { + // The consensus-specs feature forks no release ships fixtures for: + // theirs are generated from the specification's `pq` branch, so + // a tree without them is the pinned release, not a broken one. + let generated_only = [ForkName::Eip8483, ForkName::Pq]; + for fork in ForkName::ALL + .into_iter() + .filter(|fork| !generated_only.contains(fork)) + { assert!( cases.iter().any(|case| case.fork == fork), "discovery found no {fork} cases; the fixture release is expected to \ diff --git a/crates/blockchain/state_transition/tests/beacon_spec/operations.rs b/crates/blockchain/state_transition/tests/beacon_spec/operations.rs index 082384143..5d090d9ea 100644 --- a/crates/blockchain/state_transition/tests/beacon_spec/operations.rs +++ b/crates/blockchain/state_transition/tests/beacon_spec/operations.rs @@ -129,7 +129,7 @@ use ethlambda_state_transition::beacon::ForkName; use ethlambda_state_transition::beacon::config::Config; use ethlambda_state_transition::beacon::containers::{ - BeaconState, altair, bellatrix, capella, deneb, electra, gloas, phase0, shared, + BeaconState, altair, bellatrix, capella, deneb, eip8483, electra, gloas, phase0, pq, shared, }; use ethlambda_state_transition::beacon::helpers::accessors::{ ActiveBalanceCache, CommitteeCache, PubkeyIndexCache, @@ -139,9 +139,11 @@ use ethlambda_state_transition::beacon::stf::altair as altair_stf; use ethlambda_state_transition::beacon::stf::bellatrix as bellatrix_stf; use ethlambda_state_transition::beacon::stf::capella as capella_stf; use ethlambda_state_transition::beacon::stf::deneb as deneb_stf; +use ethlambda_state_transition::beacon::stf::eip8483 as eip8483_stf; use ethlambda_state_transition::beacon::stf::electra as electra_stf; use ethlambda_state_transition::beacon::stf::fulu as fulu_stf; use ethlambda_state_transition::beacon::stf::gloas as gloas_stf; +use ethlambda_state_transition::beacon::stf::pq as pq_stf; use ethlambda_state_transition::beacon::stf::signatures::SignatureBatch; use ethlambda_state_transition::beacon::stf::{ExecutionEngine, block, operations}; use libtest_mimic::{Failed, Trial}; @@ -295,6 +297,26 @@ fn run( block.body.hash_tree_root(), ) } + ForkName::Eip8483 => { + let block: eip8483::BeaconBlock = case.ssz("block"); + block::process_block_header( + state, + block.slot, + block.proposer_index, + block.parent_root, + block.body.hash_tree_root(), + ) + } + ForkName::Pq => { + let block: pq::BeaconBlock = case.ssz("block"); + block::process_block_header( + state, + block.slot, + block.proposer_index, + block.parent_root, + block.body.hash_tree_root(), + ) + } // No `other` arm: the patterns above already cover every `ForkName` // there is, so a catch-all here would be dead code rather than a // safety net. `Lean` is covered by name for the same reason, rather @@ -354,7 +376,7 @@ fn run( // accounting, EIP-7732), so this reads the case's own `meta.yaml` // for `parent_slot` rather than only the operation file every // other fork's arm reads. - ForkName::Gloas => { + ForkName::Gloas | ForkName::Eip8483 => { let attestation: gloas::Attestation = case.ssz("attestation"); let meta: AttestationMeta = case.yaml("meta"); gloas_stf::process_attestation( @@ -367,6 +389,20 @@ fn run( None, ) } + // No signature: the block's `proof` covers its attestations, in + // `process_stark`, which the operation alone does not run. + ForkName::Pq => { + let attestation: pq::Attestation = case.ssz("attestation"); + let meta: AttestationMeta = case.yaml("meta"); + pq_stf::process_attestation( + state, + &attestation, + meta.parent_slot, + committees, + &active_balances, + None, + ) + } ForkName::Lean => lean_is_not_a_fixture_fork("attestation"), }, "attester_slashing" => match case.fork { @@ -390,10 +426,14 @@ fn run( // Gloas's container widens again (EIP-7688's unbounded // `AttestingIndices`); the function is still not behaviorally // modified, only transcribed against gloas's own type. - ForkName::Gloas => { + ForkName::Gloas | ForkName::Eip8483 => { let slashing: gloas::AttesterSlashing = case.ssz("attester_slashing"); gloas_stf::process_attester_slashing(state, &slashing, config, signatures) } + ForkName::Pq => { + let slashing: pq::AttesterSlashing = case.ssz("attester_slashing"); + pq_stf::process_attester_slashing(state, &slashing, config) + } ForkName::Lean => lean_is_not_a_fixture_fork("attester_slashing"), }, // `ProposerSlashing` never changes shape, container included, gloas's @@ -402,24 +442,28 @@ fn run( // which every earlier fork's `process_proposer_slashing` has no // notion of, so this alone needs the routing every other fork's // `proposer_slashing` handler above already has. - "proposer_slashing" => { - let slashing: shared::ProposerSlashing = case.ssz("proposer_slashing"); - match case.fork { - ForkName::Phase0 - | ForkName::Altair - | ForkName::Bellatrix - | ForkName::Capella - | ForkName::Deneb - | ForkName::Electra - | ForkName::Fulu => { - operations::process_proposer_slashing(state, &slashing, config, signatures) - } - ForkName::Gloas => { - gloas_stf::process_proposer_slashing(state, &slashing, config, signatures) - } - ForkName::Lean => lean_is_not_a_fixture_fork("proposer_slashing"), + "proposer_slashing" => match case.fork { + ForkName::Phase0 + | ForkName::Altair + | ForkName::Bellatrix + | ForkName::Capella + | ForkName::Deneb + | ForkName::Electra + | ForkName::Fulu => { + let slashing: shared::ProposerSlashing = case.ssz("proposer_slashing"); + operations::process_proposer_slashing(state, &slashing, config, signatures) } - } + ForkName::Gloas | ForkName::Eip8483 => { + let slashing: shared::ProposerSlashing = case.ssz("proposer_slashing"); + gloas_stf::process_proposer_slashing(state, &slashing, config, signatures) + } + // XMSS-signed headers, a container of the PQ fork's own. + ForkName::Pq => { + let slashing: pq::ProposerSlashing = case.ssz("proposer_slashing"); + pq_stf::process_proposer_slashing(state, &slashing, config) + } + ForkName::Lean => lean_is_not_a_fixture_fork("proposer_slashing"), + }, "deposit" => match case.fork { // The `Deposit` container never changes shape; through deneb a // deposit's amount is credited the moment it is processed. @@ -446,10 +490,10 @@ fn run( let deposit: shared::Deposit = case.ssz("deposit"); electra_stf::process_deposit(state, &deposit, config) } - ForkName::Gloas => Err( + fork @ (ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq) => Err( ethlambda_state_transition::beacon::Error::UnsupportedForFork { function: "deposit", - fork: ForkName::Gloas, + fork, }, ), ForkName::Lean => lean_is_not_a_fixture_fork("deposit"), @@ -478,7 +522,7 @@ fn run( // limits are read by `crate::beacon::helpers::electra::compute_exit_epoch_and_update_churn` // dispatching on the state's own fork, not by a gloas copy of // `process_voluntary_exit`; see that function's own doc). - ForkName::Electra | ForkName::Fulu | ForkName::Gloas => { + ForkName::Electra | ForkName::Fulu | ForkName::Gloas | ForkName::Eip8483 => { let exit: shared::SignedVoluntaryExit = case.ssz("voluntary_exit"); electra_stf::process_voluntary_exit( state, @@ -488,6 +532,11 @@ fn run( signatures, ) } + // The PQ fork's exits are execution-layer requests + // (`voluntary_exit_request` below). + ForkName::Pq => { + return Err("voluntary_exit has no handler for fork `pq`".to_string()); + } ForkName::Lean => lean_is_not_a_fixture_fork("voluntary_exit"), }, // New in altair, and never listed as modified again, so this needs no @@ -569,10 +618,10 @@ fn run( "execution_payload has no handler for fork `{fork}`" )); } - ForkName::Gloas => Err( + fork @ (ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq) => Err( ethlambda_state_transition::beacon::Error::UnsupportedForFork { function: "execution_payload", - fork: ForkName::Gloas, + fork, }, ), ForkName::Lean => lean_is_not_a_fixture_fork("execution_payload"), @@ -619,28 +668,59 @@ fn run( // its `payload` parameter entirely, since withdrawals are now // deterministic from the state alone. This is the one fork whose // arm reads no operation file at all. - ForkName::Gloas => gloas_stf::process_withdrawals(state), + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => { + gloas_stf::process_withdrawals(state) + } ForkName::Lean => lean_is_not_a_fixture_fork("withdrawals"), }, // New in gloas (EIP-7732): the block's binding commitment to a // builder's payload. - "execution_payload_bid" => { - let signed_bid: gloas::SignedExecutionPayloadBid = case.ssz("execution_payload_bid"); - gloas_stf::process_execution_payload_bid(state, &signed_bid, config, signatures) - } + "execution_payload_bid" => match case.fork { + ForkName::Pq => { + let signed_bid: pq::SignedExecutionPayloadBid = case.ssz("execution_payload_bid"); + pq_stf::process_execution_payload_bid(state, &signed_bid, config) + } + _ => { + let signed_bid: gloas::SignedExecutionPayloadBid = + case.ssz("execution_payload_bid"); + gloas_stf::process_execution_payload_bid(state, &signed_bid, config, signatures) + } + }, // New in gloas (EIP-7732): the *next* block's processing of the // previous slot's payload, so the fixture's operation file is a whole // `BeaconBlock`, named `block` like `block_header`'s. - "parent_execution_payload" => { - let block: gloas::BeaconBlock = case.ssz("block"); - gloas_stf::process_parent_execution_payload( - state, - &block, - config, - &active_balances, - &pubkey_indices, - ) - } + "parent_execution_payload" => match case.fork { + ForkName::Eip8483 => { + let block: eip8483::BeaconBlock = case.ssz("block"); + eip8483_stf::process_parent_execution_payload( + state, + &block, + config, + &active_balances, + &pubkey_indices, + ) + } + ForkName::Pq => { + let block: pq::BeaconBlock = case.ssz("block"); + pq_stf::process_parent_execution_payload( + state, + &block, + config, + &active_balances, + &pubkey_indices, + ) + } + _ => { + let block: gloas::BeaconBlock = case.ssz("block"); + gloas_stf::process_parent_execution_payload( + state, + &block, + config, + &active_balances, + &pubkey_indices, + ) + } + }, // `consolidation_request` and `withdrawal_request` are new in // electra, alongside `withdrawals`; neither fulu's nor gloas's own // specification lists either as modified (gloas's own @@ -652,13 +732,23 @@ fn run( // match is needed for either the type or the call. "consolidation_request" => { let request: electra::ConsolidationRequest = case.ssz("consolidation_request"); - electra_stf::process_consolidation_request( - state, - &request, - config, - &active_balances, - &pubkey_indices, - ) + if case.fork == ForkName::Pq { + pq_stf::process_consolidation_request( + state, + &request, + config, + &active_balances, + &pubkey_indices, + ) + } else { + electra_stf::process_consolidation_request( + state, + &request, + config, + &active_balances, + &pubkey_indices, + ) + } } // `deposit_request` is new in electra too, but fulu's specification // does list a modified version (`beacon-chain.md`'s "Modified @@ -674,10 +764,14 @@ fn run( // so far, since pyspec's own genesis for both already sets that // field, but it is not the function either fork's specification names. "deposit_request" => match case.fork { - ForkName::Fulu | ForkName::Gloas => { + ForkName::Fulu | ForkName::Gloas | ForkName::Eip8483 => { let request: electra::DepositRequest = case.ssz("deposit_request"); fulu_stf::process_deposit_request(state, &request) } + ForkName::Pq => { + let request: electra::DepositRequest = case.ssz("deposit_request"); + pq_stf::process_deposit_request(state, &request) + } ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -691,13 +785,23 @@ fn run( }, "withdrawal_request" => { let request: electra::WithdrawalRequest = case.ssz("withdrawal_request"); - electra_stf::process_withdrawal_request( - state, - &request, - config, - &active_balances, - &pubkey_indices, - ) + if case.fork == ForkName::Pq { + pq_stf::process_withdrawal_request( + state, + &request, + config, + &active_balances, + &pubkey_indices, + ) + } else { + electra_stf::process_withdrawal_request( + state, + &request, + config, + &active_balances, + &pubkey_indices, + ) + } } // New in gloas (EIP-8282): the builder registry's own onboarding and // exit requests, applied by `apply_parent_execution_payload` the same @@ -705,11 +809,54 @@ fn run( // requests are. "builder_deposit_request" => { let request: gloas::BuilderDepositRequest = case.ssz("builder_deposit_request"); - gloas_stf::process_builder_deposit_request(state, &request, config) + if case.fork == ForkName::Pq { + pq_stf::process_builder_deposit_request(state, &request, config) + } else { + gloas_stf::process_builder_deposit_request(state, &request, config) + } } "builder_exit_request" => { let request: gloas::BuilderExitRequest = case.ssz("builder_exit_request"); - gloas_stf::process_builder_exit_request(state, &request, config) + if case.fork == ForkName::Pq { + pq_stf::process_builder_exit_request(state, &request, config) + } else { + gloas_stf::process_builder_exit_request(state, &request, config) + } + } + // New in EIP-8483: the post-quantum material the PQ fork adopts. + "validator_pq_registration_request" => { + let request: eip8483::ValidatorPqRegistrationRequest = + case.ssz("validator_pq_registration_request"); + eip8483_stf::process_validator_pq_registration_request(state, &request) + } + "builder_pq_registration_request" => { + let request: eip8483::BuilderPqRegistrationRequest = + case.ssz("builder_pq_registration_request"); + eip8483_stf::process_builder_pq_registration_request(state, &request) + } + // New in PQ (preregistrations from EIP-8205, exits moved to the + // execution layer). + "validator_preregistration_request" => { + let request: pq::ValidatorPreregistrationRequest = + case.ssz("validator_preregistration_request"); + pq_stf::process_validator_preregistration_request(state, &request) + } + "builder_preregistration_request" => { + let request: pq::BuilderPreregistrationRequest = + case.ssz("builder_preregistration_request"); + pq_stf::process_builder_preregistration_request(state, &request) + } + "voluntary_exit_request" => { + let request: pq::VoluntaryExitRequest = case.ssz("voluntary_exit_request"); + pq_stf::process_voluntary_exit_request(state, &request, config, &active_balances) + } + "randao" if case.fork == ForkName::Pq => { + let body: pq::BeaconBlockBody = case.ssz("body"); + pq_stf::process_randao( + state, + &body.randao_reveal, + ethlambda_state_transition::beacon::stf::block::RandaoVerification::Verify, + ) } // New in gloas (EIP-7732): the payload timeliness committee's vote on // whether the *previous* slot's payload showed up on time. diff --git a/crates/blockchain/state_transition/tests/beacon_spec/rewards.rs b/crates/blockchain/state_transition/tests/beacon_spec/rewards.rs index b2eecb833..cee843fc9 100644 --- a/crates/blockchain/state_transition/tests/beacon_spec/rewards.rs +++ b/crates/blockchain/state_transition/tests/beacon_spec/rewards.rs @@ -142,7 +142,9 @@ fn components( | ForkName::Deneb | ForkName::Electra | ForkName::Fulu - | ForkName::Gloas => vec![ + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => vec![ ( "source_deltas", altair_helpers::get_flag_index_deltas(state, TIMELY_SOURCE_FLAG_INDEX), diff --git a/crates/blockchain/state_transition/tests/beacon_spec_tests.rs b/crates/blockchain/state_transition/tests/beacon_spec_tests.rs index 40f3a0331..5e544a38f 100644 --- a/crates/blockchain/state_transition/tests/beacon_spec_tests.rs +++ b/crates/blockchain/state_transition/tests/beacon_spec_tests.rs @@ -30,6 +30,12 @@ mod beacon_spec; fn main() { let args = libtest_mimic::Arguments::from_args(); + // consensus-specs generates the EIP-8483 and PQ fixtures with XMSS + // inactive (its reference key generation takes minutes), so their + // signatures are stubs and their proofs empty. Only the PQ fork verifies + // XMSS, so no earlier fork's case is affected. + ethlambda_state_transition::beacon::helpers::pq::disable_xmss_verification(); + let mut trials = Vec::new(); trials.extend(beacon_spec::harness::trials()); trials.extend(beacon_spec::fixture_fork_trials()); diff --git a/crates/common/crypto/Cargo.toml b/crates/common/crypto/Cargo.toml index c324d55e1..964a42241 100644 --- a/crates/common/crypto/Cargo.toml +++ b/crates/common/crypto/Cargo.toml @@ -15,8 +15,13 @@ ethlambda-metrics.workspace = true ethlambda-types.workspace = true rand.workspace = true rayon.workspace = true +sha2.workspace = true +sha3.workspace = true leanvm.workspace = true +leanvm_claims.workspace = true +leanvm_lean_xmss.workspace = true +leanvm_sidecar.workspace = true postcard.workspace = true thiserror.workspace = true diff --git a/crates/common/crypto/src/lean_xmss.rs b/crates/common/crypto/src/lean_xmss.rs new file mode 100644 index 000000000..662dab789 --- /dev/null +++ b/crates/common/crypto/src/lean_xmss.rs @@ -0,0 +1,1005 @@ +//! leanXMSS, the hash-based signature scheme the Beacon Chain's PQ fork signs +//! with (the specification's `xmss` module: `Sign`, `Verify`, `GetLeafIndex`). +//! +//! **Not** the XMSS that lean consensus signs with ([`crate::signature`]). That +//! one is leanVM's own scheme over BLAKE2s, with 1208-byte signatures. This one +//! is the note "leanXMSS for Ethereum PQ-Consensus" over Keccak-256, with 32-byte +//! public keys and 2404-byte signatures, which is what consensus-specs' `pq` +//! feature names, through EthLib's `EthLib/Cryptography/Xmss`. +//! +//! Ported from leanVM's Rust implementation of that note, +//! `crates/signatures/src/lean_xmss` and `crates/signatures/src/tweak.rs` at +//! `4af2cdddd7cd41b9df2cc7e123e314a91b0b4462` (MIT), the commit EthLib's README +//! cites as the note's source. Ported rather than depended on: this workspace's +//! `leanvm` pin predates that crate, and a second leanVM revision in the graph +//! would bring a second copy of its proving stack along for ~300 lines of hashing. +//! The known-answer tests below pin the bytes against both leanVM's and +//! consensus-specs' own vectors, so a drift in either copy shows up here. +//! +//! # The scheme +//! +//! A key is two layers of Merkle trees of height 16 whose leaves are WOTS+C +//! one-time keys. The top layer is one tree, whose root is the public key; the +//! bottom layer is `2^16` trees, and top key `b` signs the root of bottom tree +//! `b`. A message is signed by a key of a bottom tree, so a key signs at `2^32` +//! leaf indices, each at most once, ever. A signature carries its leaf index in +//! its first four bytes, little endian. +//! +//! Every hash is the tweakable hash `Th(P, A, M) = Keccak256(P | A | M)` +//! truncated to 16 bytes, `P` the key's 16-byte public parameter and `A` a +//! 16-byte address naming where in the key the call happens. +//! +//! # Signing twice at one leaf breaks the key +//! +//! Nothing here records which leaf indices a key has signed at. The PQ fork +//! assigns every message type its own leaf (`compute_xmss_leaf_index`), and the +//! caller must never sign two different messages at one leaf. + +use std::sync::{Arc, Mutex}; + +use rayon::prelude::*; +use sha3::{Digest as _, Keccak256}; + +// --------------------------------------------------------------------------- +// Parameters +// --------------------------------------------------------------------------- + +/// `n`: hash value and Merkle node length, in bytes. +const N: usize = 16; +type Digest = [u8; N]; + +const PUBLIC_PARAM_LEN: usize = 16; +type PublicParam = [u8; PUBLIC_PARAM_LEN]; + +const ADDRESS_LEN: usize = 16; + +/// The seed every secret is derived from: the whole secret key. +pub const SEED_LEN: usize = 32; +pub type Seed = [u8; SEED_LEN]; + +/// Hashed before the seed, so no other scheme derives from the same bytes. +const SEED_TAG: &[u8] = b"leanXMSS"; + +/// The randomizer a layer's message digest is computed under. +const RANDOMIZER_LEN: usize = 16; +type Randomizer = [u8; RANDOMIZER_LEN]; + +/// The message a key signs: a 32-byte signing root. +pub const MESSAGE_LEN: usize = 32; +pub type Message = [u8; MESSAGE_LEN]; + +/// The leaf a signature uses: its top 16 bits name the bottom tree, its low 16 +/// bits the leaf in it. +pub type LeafIndex = u32; + +/// `w`: digit size in bits. +const W: usize = 2; +/// `2^w`: one more than the steps of a hash chain. +const CHAIN_LEN: usize = 1 << W; +/// The digest's digits: 128 bits of base `2^w`. +const DIGITS: usize = 8 * N / W; +/// The last digits, which the signer grinds to zero, so they have no chain. +const ZERO_DIGITS: usize = 6; +/// `v`: the chains of a one-time key. +const V: usize = DIGITS - ZERO_DIGITS; +/// `T`: the sum of every codeword's `v` digits. +const TARGET_SUM: usize = 106; +/// Randomizers tried per layer signature before signing gives up. +const MAX_RANDOMIZER_TRIALS: u64 = 1 << 32; +/// Randomizer trials searched per parallel dispatch: the first hit is expected +/// after about `2^20`, and a batch has to be small for the search to stop near it. +const GRIND_BATCH: u64 = 1 << 14; + +/// Height of each layer's tree. +const LAYER_HEIGHT: usize = 16; + +/// The encoded public key: the top tree's root, then the public parameter. +pub const PUBKEY_SIZE: usize = N + PUBLIC_PARAM_LEN; +/// The encoded leaf index a signature starts with. +pub const LEAF_INDEX_SIZE: usize = size_of::(); +/// One layer's signature: the randomizer, the chain values and the path. +const LAYER_SIG_SIZE: usize = RANDOMIZER_LEN + V * N + LAYER_HEIGHT * N; +/// The leaf index, then the bottom layer's signature, then the top one's. +pub const SIGNATURE_SIZE: usize = LEAF_INDEX_SIZE + 2 * LAYER_SIG_SIZE; + +const _: () = assert!(V == 58 && TARGET_SUM < V * (CHAIN_LEN - 1)); +const _: () = assert!(PUBKEY_SIZE == 32 && LAYER_SIG_SIZE == 1200 && SIGNATURE_SIZE == 2404); +const _: () = assert!(2 * LAYER_HEIGHT == 8 * LEAF_INDEX_SIZE); + +// --------------------------------------------------------------------------- +// The tweakable hash +// --------------------------------------------------------------------------- + +// Hash types: the low five bits of an address's eighth byte. +const TWEAK_PRF: u8 = 0; +const TWEAK_CHAIN: u8 = 1; +const TWEAK_LEAF: u8 = 2; +const TWEAK_NODE: u8 = 3; +const TWEAK_PARAMETER: u8 = 5; +const TWEAK_RANDOMIZER: u8 = 7; +const TWEAK_MSG: u8 = 12; +/// A node outside a range key's leaves (see [`range_key`]). +const TWEAK_FILLER: u8 = 13; + +/// The top layer's flag in an address's `hi` field: the two layers share one +/// public parameter, so the address is what separates them. +const TOP: u32 = 1 << 23; + +/// The address of a hash call: its type, a chain step, and two fields, +/// `hi < 2^24` and `lo`. +#[derive(Clone, Copy)] +struct Tweak { + t: u8, + step: u8, + hi: u32, + lo: u32, +} + +fn tweak(t: u8, hi: u32, lo: u32) -> Tweak { + debug_assert!(t < 32 && hi < 1 << 24); + Tweak { t, step: 0, hi, lo } +} + +impl Tweak { + /// The same address at chain step `step < 8`. + fn step(self, step: usize) -> Self { + debug_assert!(step < 8); + Self { + step: step as u8, + ..self + } + } + + /// `[lo:4 | hi:3 | t + 32 step:1 | zero:8]`, the fields little endian. + fn address(self) -> [u8; ADDRESS_LEN] { + let mut out = [0u8; ADDRESS_LEN]; + out[..4].copy_from_slice(&self.lo.to_le_bytes()); + out[4..7].copy_from_slice(&self.hi.to_le_bytes()[..3]); + out[7] = self.t | (self.step << 5); + out + } +} + +/// `hi` for `field` on `layer` (0 the bottom, 1 the top). +fn layer_hi(layer: usize, field: u32) -> u32 { + field | (layer as u32 * TOP) +} + +/// A Keccak-256 hasher that has absorbed `P | A`. +fn hasher(pp: &PublicParam, tw: Tweak) -> Keccak256 { + let mut hasher = Keccak256::new(); + hasher.update(pp); + hasher.update(tw.address()); + hasher +} + +fn truncate(output: &[u8]) -> Digest { + output[..N] + .try_into() + .expect("a Keccak-256 output is 32 bytes") +} + +/// `Th` over a byte payload. +fn th(pp: &PublicParam, tw: Tweak, payload: &[u8]) -> Digest { + let mut hasher = hasher(pp, tw); + hasher.update(payload); + truncate(&hasher.finalize()) +} + +/// `Th` over a concatenation of digests: a Merkle node or a leaf. +fn th_digests(pp: &PublicParam, tw: Tweak, values: &[Digest]) -> Digest { + th(pp, tw, values.as_flattened()) +} + +/// Two secrets for one hash: the two halves of the untruncated hash of the seed. +fn th_pair(pp: &PublicParam, tw: Tweak, seed: &Seed) -> [Digest; 2] { + let mut hasher = hasher(pp, tw); + hasher.update(seed); + let output = hasher.finalize(); + [truncate(&output[..N]), truncate(&output[N..])] +} + +/// The seed every secret derives from: `H("leanXMSS" | S_0)`. +fn scheme_seed(seed: &Seed) -> Seed { + let mut hasher = Keccak256::new(); + hasher.update(SEED_TAG); + hasher.update(seed); + hasher.finalize().into() +} + +/// The public parameter, derived from the scheme seed under `P = 0`. +fn public_param(seed: &Seed) -> PublicParam { + th(&[0; PUBLIC_PARAM_LEN], tweak(TWEAK_PARAMETER, 0, 0), seed) +} + +// --------------------------------------------------------------------------- +// WOTS+C +// --------------------------------------------------------------------------- + +/// The index of `leaf_index`'s one-time key on `layer`: the leaf index itself at +/// the bottom, the bottom tree's index at the top. +fn layer_index(leaf_index: LeafIndex, layer: usize) -> u32 { + (u64::from(leaf_index) >> (layer * LAYER_HEIGHT)) as u32 +} + +/// `leaf_index`'s leaf within its bottom tree. +fn tree_leaf(leaf_index: LeafIndex) -> u32 { + leaf_index & ((1 << LAYER_HEIGHT) - 1) +} + +/// The starts of the `v` chains of key `e` on `layer`. One hash of the seed gives +/// two of them: chains `2t` and `2t + 1` are its two halves. +fn wots_secrets(pp: &PublicParam, seed: &Seed, layer: usize, e: u32) -> [Digest; V] { + let mut secrets = [[0u8; N]; V]; + for (t, pair) in secrets.as_chunks_mut::<2>().0.iter_mut().enumerate() { + *pair = th_pair(pp, tweak(TWEAK_PRF, layer_hi(layer, t as u32), e), seed); + } + secrets +} + +/// `steps` steps of chain `i` from position `start`. A step's address carries the +/// position it leaves, and the chain index rides after the value, so the address +/// is the same for every chain of a key. +fn chain( + pp: &PublicParam, + layer: usize, + e: u32, + i: usize, + start: usize, + steps: usize, + value: Digest, +) -> Digest { + debug_assert!(start + steps < CHAIN_LEN); + let address = tweak(TWEAK_CHAIN, layer_hi(layer, 0), e); + (start..start + steps).fold(value, |current, from| { + th_digests( + pp, + address.step(from), + &[current, (i as u128).to_le_bytes()], + ) + }) +} + +/// The Merkle leaf of a one-time key: `Th` over its `v` chain ends. +fn wots_leaf_hash(pp: &PublicParam, layer: usize, e: u32, ends: &[Digest; V]) -> Digest { + th_digests(pp, tweak(TWEAK_LEAF, layer_hi(layer, 0), e), ends) +} + +/// `P | A | M`, the hasher a randomizer trial continues. +fn digest_prefix(pp: &PublicParam, layer: usize, e: u32, message: &[u8]) -> Keccak256 { + let mut prefix = hasher(pp, tweak(TWEAK_MSG, layer_hi(layer, 0), e)); + prefix.update(message); + prefix +} + +/// The digest key `e` on `layer` signs: `Th(P, A, M | R)`, `M` the message at the +/// bottom and the bottom root at the top. +fn message_digest( + pp: &PublicParam, + layer: usize, + e: u32, + randomizer: &Randomizer, + message: &[u8], +) -> Digest { + let mut hasher = digest_prefix(pp, layer, e, message); + hasher.update(randomizer); + truncate(&hasher.finalize()) +} + +/// The digest's first `v` digits, if it is a codeword: its last `ZERO_DIGITS` +/// digits zero and the other `v` summing to `T`. +fn codeword(digest: &Digest) -> Option<[u8; V]> { + let d = u128::from_le_bytes(*digest); + let x: [u8; V] = std::array::from_fn(|i| ((d >> (W * i)) & (CHAIN_LEN as u128 - 1)) as u8); + let sum: usize = x.iter().map(|&digit| usize::from(digit)).sum(); + (d >> (W * V) == 0 && sum == TARGET_SUM).then_some(x) +} + +/// `Ots.sign`: the randomizer of the least admissible trial `R_0 + c`, `R_0` a +/// hash of the seed and the message, and the chain value each digit opens. +/// Deterministic in its inputs, so signing one message again gives the same +/// signature and a key never opens two codewords. +fn wots_sign( + pp: &PublicParam, + seed: &Seed, + layer: usize, + e: u32, + message: &[u8], +) -> Option<(Randomizer, [Digest; V])> { + let base = { + let mut hasher = hasher(pp, tweak(TWEAK_RANDOMIZER, layer_hi(layer, 0), e)); + hasher.update(seed); + hasher.update(message); + u128::from_le_bytes(truncate(&hasher.finalize())) + }; + let trial = |c: u64| base.wrapping_add(u128::from(c)).to_le_bytes(); + let prefix = digest_prefix(pp, layer, e, message); + let admits = |c: u64| { + let mut hasher = prefix.clone(); + hasher.update(trial(c)); + codeword(&truncate(&hasher.finalize())).is_some() + }; + let c = (0..MAX_RANDOMIZER_TRIALS) + .step_by(GRIND_BATCH as usize) + .find_map(|start| { + (start..start + GRIND_BATCH) + .into_par_iter() + .find_first(|&c| admits(c)) + })?; + let randomizer = trial(c); + let x = codeword(&message_digest(pp, layer, e, &randomizer, message))?; + let secrets = wots_secrets(pp, seed, layer, e); + let values = std::array::from_fn(|i| chain(pp, layer, e, i, 0, usize::from(x[i]), secrets[i])); + Some((randomizer, values)) +} + +/// `Ots.leaf`: the leaf a claimed signature recovers, or `None` if the digest is +/// not a codeword. +fn wots_leaf( + pp: &PublicParam, + layer: usize, + e: u32, + message: &[u8], + randomizer: &Randomizer, + values: &[Digest; V], +) -> Option { + let x = codeword(&message_digest(pp, layer, e, randomizer, message))?; + let ends = std::array::from_fn(|i| { + let start = usize::from(x[i]); + chain(pp, layer, e, i, start, CHAIN_LEN - 1 - start, values[i]) + }); + Some(wots_leaf_hash(pp, layer, e, &ends)) +} + +/// The leaf of key `e` on `layer`, from the seed: what key generation spends its +/// hashes on. +fn wots_public_leaf(pp: &PublicParam, seed: &Seed, layer: usize, e: u32) -> Digest { + let secrets = wots_secrets(pp, seed, layer, e); + let ends = std::array::from_fn(|i| chain(pp, layer, e, i, 0, CHAIN_LEN - 1, secrets[i])); + wots_leaf_hash(pp, layer, e, &ends) +} + +// --------------------------------------------------------------------------- +// Trees +// --------------------------------------------------------------------------- + +/// The Merkle parent at `level` and index `j` of `layer`. +fn node(pp: &PublicParam, layer: usize, level: usize, j: u32, children: &[Digest; 2]) -> Digest { + th_digests( + pp, + tweak(TWEAK_NODE, layer_hi(layer, level as u32), j), + children, + ) +} + +fn filler(pp: &PublicParam, seed: &Seed, layer: usize, level: usize, j: u32) -> Digest { + th( + pp, + tweak(TWEAK_FILLER, layer_hi(layer, level as u32), j), + seed, + ) +} + +/// One layer's Merkle tree, over the leaves `first..=last` of its `2^16`. A full +/// key's trees hold every leaf; a range key's hold its range, every node outside +/// it a pseudorandom filler. +struct LayerTree { + layer: usize, + /// The index of leaf 0 on its layer: the bottom tree's first leaf index, or 0. + base: u32, + first: u32, + last: u32, + /// Level `l` holds the nodes `first >> l ..= last >> l`. + levels: Vec>, +} + +impl LayerTree { + fn build( + pp: &PublicParam, + seed: &Seed, + layer: usize, + base: u32, + first: u32, + last: u32, + ) -> Self { + debug_assert!(first <= last && last >> LAYER_HEIGHT == 0); + let leaves = (0..=(last - first)) + .into_par_iter() + .map(|j| wots_public_leaf(pp, seed, layer, base + first + j)) + .collect(); + let mut tree = Self { + layer, + base, + first, + last, + levels: vec![leaves], + }; + for level in 1..=LAYER_HEIGHT { + let start = first >> level; + let up = (start..=(last >> level)) + .into_par_iter() + .map(|j| { + let children = [ + tree.at(pp, seed, level - 1, 2 * j), + tree.at(pp, seed, level - 1, 2 * j + 1), + ]; + node(pp, layer, level, (base >> level) + j, &children) + }) + .collect(); + tree.levels.push(up); + } + tree + } + + /// Node `j` of `level`, stored if it is over a leaf in range, else a filler. + fn at(&self, pp: &PublicParam, seed: &Seed, level: usize, j: u32) -> Digest { + if (self.first >> level..=self.last >> level).contains(&j) { + self.levels[level][(j - (self.first >> level)) as usize] + } else { + filler(pp, seed, self.layer, level, (self.base >> level) + j) + } + } + + fn root(&self) -> Digest { + self.levels[LAYER_HEIGHT][0] + } + + fn path(&self, pp: &PublicParam, seed: &Seed, leaf: u32) -> [Digest; LAYER_HEIGHT] { + debug_assert!((self.first..=self.last).contains(&leaf)); + std::array::from_fn(|level| self.at(pp, seed, level, (leaf >> level) ^ 1)) + } +} + +// --------------------------------------------------------------------------- +// Keys and signatures +// --------------------------------------------------------------------------- + +/// A public key: the top tree's root followed by the public parameter. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)] +pub struct PublicKey { + merkle_root: Digest, + public_param: PublicParam, +} + +impl PublicKey { + pub fn from_bytes(bytes: &[u8; PUBKEY_SIZE]) -> Self { + let (merkle_root, public_param) = bytes.split_at(N); + Self { + merkle_root: merkle_root.try_into().expect("split at N"), + public_param: public_param.try_into().expect("the rest is N bytes"), + } + } + + pub fn to_bytes(&self) -> [u8; PUBKEY_SIZE] { + let mut out = [0u8; PUBKEY_SIZE]; + out[..N].copy_from_slice(&self.merkle_root); + out[N..].copy_from_slice(&self.public_param); + out + } +} + +/// One layer's signature: the randomizer, the value each digit opens, and the +/// authentication path. +#[derive(Debug, Clone, PartialEq, Eq)] +struct LayerSignature { + randomizer: Randomizer, + chain_values: [Digest; V], + path: [Digest; LAYER_HEIGHT], +} + +impl LayerSignature { + fn append(&self, buf: &mut Vec) { + buf.extend_from_slice(&self.randomizer); + for value in self.chain_values.iter().chain(&self.path) { + buf.extend_from_slice(value); + } + } +} + +/// A signature. It carries the leaf index it was made at, so a verifier needs +/// only the key and the message. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct Signature { + leaf_index: LeafIndex, + bottom: LayerSignature, + top: LayerSignature, +} + +/// Peels fixed-size fields off a buffer whose length was checked up front. +struct Reader<'a>(&'a [u8]); + +impl Reader<'_> { + fn take(&mut self) -> [u8; L] { + let (head, tail) = self.0.split_at(L); + self.0 = tail; + head.try_into().expect("split at L") + } + + fn layer(&mut self) -> LayerSignature { + LayerSignature { + randomizer: self.take(), + chain_values: std::array::from_fn(|_| self.take()), + path: std::array::from_fn(|_| self.take()), + } + } +} + +impl Signature { + /// Decodes a signature. Every string of exactly [`SIGNATURE_SIZE`] bytes is + /// one, since all its fields are fixed-size; any other length is `None`. + pub fn from_bytes(bytes: &[u8]) -> Option { + if bytes.len() != SIGNATURE_SIZE { + return None; + } + let mut reader = Reader(bytes); + Some(Self { + leaf_index: LeafIndex::from_le_bytes(reader.take()), + bottom: reader.layer(), + top: reader.layer(), + }) + } + + pub fn to_bytes(&self) -> Vec { + let mut buf = Vec::with_capacity(SIGNATURE_SIZE); + buf.extend_from_slice(&self.leaf_index.to_le_bytes()); + self.bottom.append(&mut buf); + self.top.append(&mut buf); + buf + } + + pub fn leaf_index(&self) -> LeafIndex { + self.leaf_index + } +} + +/// The root `signature`'s layer reaches from key `e`, or `None` if its digest is +/// not a codeword. +fn layer_root( + pp: &PublicParam, + layer: usize, + e: u32, + message: &[u8], + signature: &LayerSignature, +) -> Option { + let leaf = wots_leaf( + pp, + layer, + e, + message, + &signature.randomizer, + &signature.chain_values, + )?; + let root = signature + .path + .iter() + .enumerate() + .fold(leaf, |current, (level, sibling)| { + let children = if (e >> level) & 1 == 0 { + [current, *sibling] + } else { + [*sibling, current] + }; + node(pp, layer, level + 1, e >> (level + 1), &children) + }); + Some(root) +} + +/// Whether `signature` signs `message` under `pubkey`, at the leaf index the +/// signature itself carries. +pub fn verify_signature(pubkey: &PublicKey, message: &Message, signature: &Signature) -> bool { + let pp = &pubkey.public_param; + let leaf_index = signature.leaf_index; + let Some(bottom_root) = layer_root( + pp, + 0, + layer_index(leaf_index, 0), + message, + &signature.bottom, + ) else { + return false; + }; + layer_root( + pp, + 1, + layer_index(leaf_index, 1), + &bottom_root, + &signature.top, + ) + .is_some_and(|root| root == pubkey.merkle_root) +} + +/// The specification's `xmss.Verify(pubkey, leaf_index, message, signature)`: +/// `true` only if `signature` signs `message` under `pubkey` and was made at +/// `leaf_index`. +/// +/// A leaf index past `u32::MAX` never verifies: a signature carries four bytes +/// of it. Neither does a signature of the wrong length, the way the reference +/// implementation's decoding failure reads as `False`. +pub fn verify( + pubkey: &[u8; PUBKEY_SIZE], + leaf_index: u64, + message: &Message, + signature: &[u8], +) -> bool { + let Some(signature) = Signature::from_bytes(signature) else { + return false; + }; + if u64::from(signature.leaf_index) != leaf_index { + return false; + } + verify_signature(&PublicKey::from_bytes(pubkey), message, &signature) +} + +/// The specification's `xmss.GetLeafIndex(signature)`: the leaf index a +/// signature carries, its first four bytes as a little-endian integer, read +/// without verifying anything else. +/// +/// `None` for fewer than four bytes; the fork's `XMSSSignature` is a fixed +/// 2404-byte vector, so a consensus caller never sees it. +pub fn signature_index(signature: &[u8]) -> Option { + let bytes = signature.get(..LEAF_INDEX_SIZE)?; + let bytes: [u8; LEAF_INDEX_SIZE] = bytes.try_into().expect("sliced to LEAF_INDEX_SIZE"); + Some(u64::from(LeafIndex::from_le_bytes(bytes))) +} + +/// Why signing failed. +#[derive(Debug, Clone, Copy, PartialEq, Eq, thiserror::Error)] +pub enum SignError { + /// No randomizer within the trial limit made a layer's digest a codeword. + /// The note bounds the probability of this below `exp(-4000)`. + #[error("no admissible encoding within the randomizer trial limit")] + NoAdmissibleEncoding, + /// The leaf index is outside the leaves a range key covers. + #[error("leaf index {0} is outside the key's range")] + OutOfRange(LeafIndex), +} + +struct BottomTree { + index: u32, + tree: LayerTree, + top_signature: LayerSignature, +} + +/// How many bottom trees a key keeps built. Two: a validator on the PQ fork +/// signs its attestations in one bottom tree (leaves `2^15 + epoch`) and its +/// per-slot messages (proposals, aggregates, payload attestations) in another, +/// so one cached tree would be rebuilt, about a second and a half of hashing, +/// on nearly every signature. +const BOTTOM_TREES_CACHED: usize = 2; + +/// A secret key: the seed, the top tree, and the bottom trees it last signed in. +/// +/// Holding the top tree is what makes signing cost one bottom tree rather than +/// both layers: key generation builds `2^16` one-time keys, about a second of +/// hashing, so a key is generated once per process and kept. +pub struct SecretKey { + /// [`scheme_seed`] of the key's seed, which every secret derives from. + seed: Seed, + public_param: PublicParam, + /// The leaf indices the key's trees hold: every one for a full key. + leaves: std::ops::RangeInclusive, + top: LayerTree, + /// Most recently used last, at most [`BOTTOM_TREES_CACHED`]. + bottom: Mutex>>, +} + +impl std::fmt::Debug for SecretKey { + fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + f.debug_struct("SecretKey") + .field("public_key", &self.public_key()) + .finish_non_exhaustive() + } +} + +/// The key over the leaf indices `leaves`; nodes over no leaf in it are fillers. +fn generate(key_seed: &Seed, leaves: std::ops::RangeInclusive) -> SecretKey { + let seed = scheme_seed(key_seed); + let public_param = public_param(&seed); + let first = layer_index(*leaves.start(), 1); + let last = layer_index(*leaves.end(), 1); + let top = LayerTree::build(&public_param, &seed, 1, 0, first, last); + SecretKey { + seed, + public_param, + leaves, + top, + bottom: Mutex::new(Vec::new()), + } +} + +/// The key a seed grows into: what EthLib's `xmss.key_gen` returns. Generates the +/// whole top tree. +pub fn key_gen(seed: &Seed) -> SecretKey { + generate(seed, 0..=LeafIndex::MAX) +} + +/// A key that signs like the full key only over the leaf indices +/// `first..=last`, every node outside them a pseudorandom filler, for a fraction +/// of [`key_gen`]'s cost. For tests: its public key is not the full key's. +/// +/// A seed must back **one** key: never a range key and the full key, nor two +/// range keys over different ranges. They share every one-time key but not +/// their bottom roots, so one top one-time key would sign two roots, and its +/// signatures would forge others. +pub fn range_key(seed: &Seed, first: LeafIndex, last: LeafIndex) -> SecretKey { + assert!(first <= last, "an empty range"); + generate(seed, first..=last) +} + +impl SecretKey { + pub fn public_key(&self) -> PublicKey { + PublicKey { + merkle_root: self.top.root(), + public_param: self.public_param, + } + } + + /// Signs `message` at `leaf_index`. Deterministic. + /// + /// Never sign two different messages at one leaf index. The first signature + /// in a bottom tree builds that tree with rayon, so never sign from inside a + /// rayon task. + pub fn sign(&self, message: &Message, leaf_index: LeafIndex) -> Result { + if !self.leaves.contains(&leaf_index) { + return Err(SignError::OutOfRange(leaf_index)); + } + let bottom = self.bottom_tree(leaf_index)?; + let (randomizer, chain_values) = + wots_sign(&self.public_param, &self.seed, 0, leaf_index, message) + .ok_or(SignError::NoAdmissibleEncoding)?; + Ok(Signature { + leaf_index, + bottom: LayerSignature { + randomizer, + chain_values, + path: bottom + .tree + .path(&self.public_param, &self.seed, tree_leaf(leaf_index)), + }, + top: bottom.top_signature.clone(), + }) + } + + /// Builds and caches the bottom tree `leaf_index` is in, without signing: + /// a signer that knows which leaves it signs at next pays the tree's + /// cost ahead of the deadline rather than at it. Signing anything to the + /// same end would use the leaf up. + pub fn prepare(&self, leaf_index: LeafIndex) -> Result<(), SignError> { + if !self.leaves.contains(&leaf_index) { + return Err(SignError::OutOfRange(leaf_index)); + } + self.bottom_tree(leaf_index).map(|_| ()) + } + + /// The bottom tree `leaf_index` is in, with the top layer's signature of its + /// root: the cached one, or built and cached. + fn bottom_tree(&self, leaf_index: LeafIndex) -> Result, SignError> { + let index = layer_index(leaf_index, 1); + { + let mut cache = self + .bottom + .lock() + .expect("the bottom-tree cache is never poisoned"); + if let Some(position) = cache.iter().position(|bottom| bottom.index == index) { + let bottom = cache.remove(position); + cache.push(Arc::clone(&bottom)); + return Ok(bottom); + } + } + let (pp, seed) = (&self.public_param, &self.seed); + let base = index << LAYER_HEIGHT; + let first = (*self.leaves.start()).max(base) - base; + let last = (*self.leaves.end()).min(base | tree_leaf(LeafIndex::MAX)) - base; + let tree = LayerTree::build(pp, seed, 0, base, first, last); + let (randomizer, chain_values) = + wots_sign(pp, seed, 1, index, &tree.root()).ok_or(SignError::NoAdmissibleEncoding)?; + let bottom = Arc::new(BottomTree { + index, + top_signature: LayerSignature { + randomizer, + chain_values, + path: self.top.path(pp, seed, index), + }, + tree, + }); + let mut cache = self + .bottom + .lock() + .expect("the bottom-tree cache is never poisoned"); + if cache.len() >= BOTTOM_TREES_CACHED { + cache.remove(0); + } + cache.push(Arc::clone(&bottom)); + Ok(bottom) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn hex_bytes(text: &str) -> Vec { + hex::decode(text).expect("valid hex in a test vector") + } + + /// consensus-specs' `tests/core/pyspec/eth_consensus_specs/utils/xmss_kat.json` + /// at the `pq` pin `ee867e912a7c4c47229e08332db7c5c9c4f4b129`: the vector its + /// `test_xmss.py` checks EthLib's `xmss.verify` against. + const KAT_LEAF_INDEX: u64 = 1; + const KAT_MESSAGE: &str = "202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f"; + const KAT_PUBKEY: &str = "3ae358d01b96d7ca6dd9dd5bc7683b860111dafedf7a9be6717c2100d622acf0"; + const KAT_SIGNATURE: &str = include_str!("../tests/fixtures/xmss_kat_signature.hex"); + + fn kat() -> ([u8; PUBKEY_SIZE], Message, Vec) { + let pubkey = hex_bytes(KAT_PUBKEY).try_into().unwrap(); + let message = hex_bytes(KAT_MESSAGE).try_into().unwrap(); + (pubkey, message, hex_bytes(KAT_SIGNATURE.trim())) + } + + #[test] + fn consensus_specs_known_answer_verifies() { + let (pubkey, message, signature) = kat(); + assert_eq!(signature.len(), SIGNATURE_SIZE); + assert!(verify(&pubkey, KAT_LEAF_INDEX, &message, &signature)); + assert_eq!(signature_index(&signature), Some(KAT_LEAF_INDEX)); + } + + /// The four negative cases of consensus-specs' `test_xmss.py`. + #[test] + fn consensus_specs_known_answer_rejections() { + let (pubkey, message, signature) = kat(); + assert!(!verify(&pubkey, KAT_LEAF_INDEX + 1, &message, &signature)); + assert!(!verify(&pubkey, KAT_LEAF_INDEX, &[0; 32], &signature)); + let other: [u8; PUBKEY_SIZE] = [7; PUBKEY_SIZE]; + assert!(!verify(&other, KAT_LEAF_INDEX, &message, &signature)); + assert!(!verify( + &pubkey, + KAT_LEAF_INDEX, + &message, + &signature[..SIGNATURE_SIZE - 1] + )); + } + + /// leanVM's own known answer (`tests/signatures/lean_xmss.rs`, + /// `known_answer`), the vector EthLib's README pins too: the seed + /// `37 i + 11`, the message `7 i + 3`, the leaf `0xA5C37E19`. Generates the + /// full key (2^16 one-time keys) and a bottom tree: under a second and a half + /// of hashing across cores in the `release-fast` profile `make test` uses. + #[test] + fn lean_vm_known_answer() { + let seed: Seed = std::array::from_fn(|i| (i as u8).wrapping_mul(37).wrapping_add(11)); + let key = key_gen(&seed); + let message: Message = std::array::from_fn(|i| (i * 7 + 3) as u8); + let signature = key.sign(&message, 0xA5C3_7E19).unwrap(); + assert_eq!( + hex::encode(key.public_key().to_bytes()), + "4066673001949b6ff8b9299fd4ea17d28787ad501aafe4b64e2488107b5e3ec3" + ); + let digest = Keccak256::digest(signature.to_bytes()); + assert_eq!( + hex::encode(digest), + "f10d62beec95e64e1477028252c2c8683695eab9d8787ad8fa722f47b5fd5234" + ); + assert!(verify_signature(&key.public_key(), &message, &signature)); + } + + /// consensus-specs' known answer from the signing side: the seed + /// `00 01 .. 1f` grows into the vector's public key, and signs its message + /// at leaf 1 into its signature byte for byte. + #[test] + fn consensus_specs_known_answer_signs() { + let (pubkey, message, signature) = kat(); + let seed: Seed = std::array::from_fn(|i| i as u8); + let key = key_gen(&seed); + assert_eq!(key.public_key().to_bytes(), pubkey); + let signed = key.sign(&message, KAT_LEAF_INDEX as LeafIndex).unwrap(); + assert_eq!(signed.to_bytes(), signature); + } + + /// The key ethpandaops' genesis tooling and Lighthouse give validator 0 of + /// ethereum-package's default mnemonic: the seed is + /// `sha256("leanxmss-seed" || sk)` of its EIP-2333 signing key + /// (`m/12381/3600/0/0/0`), and the public key is the first 32 bytes of + /// ethereum-package's `static_files/xmss-keys/e383db4eb96b6f02.bin`. + #[test] + fn ethereum_package_validator_zero() { + let seed: Seed = + hex_bytes("6d78e7072bfcbde1a75261f6e16772ea352de90462819cdc53e05f10d4700ad1") + .try_into() + .unwrap(); + let expected = + hex_bytes("d7792e263b94cf0b7a9f70d8374c6d34a600b4247057d4079f810caff711c632"); + assert_eq!(key_gen(&seed).public_key().to_bytes().to_vec(), expected); + } + + /// Alternating between two bottom trees keeps both built, and a third + /// evicts the least recently used. + #[test] + fn two_bottom_trees_stay_built() { + let key = range_key(&[3; SEED_LEN], 0, LeafIndex::MAX); + let message: Message = [9; 32]; + let attestation_leaf: LeafIndex = (1 << 15) + 7; + let slot_leaf: LeafIndex = (1 << 30) + (1 << 16) + 8 * 100; + let cached = |key: &SecretKey| -> Vec { + key.bottom + .lock() + .unwrap() + .iter() + .map(|tree| tree.index) + .collect() + }; + key.sign(&message, attestation_leaf).unwrap(); + key.sign(&message, slot_leaf).unwrap(); + for offset in 0..4 { + key.sign(&message, attestation_leaf + offset).unwrap(); + key.sign(&message, slot_leaf + offset).unwrap(); + } + let both = [layer_index(attestation_leaf, 1), layer_index(slot_leaf, 1)]; + assert_eq!(cached(&key), both); + key.prepare(5 << LAYER_HEIGHT).unwrap(); + assert_eq!(cached(&key), [both[1], 5]); + } + + #[test] + fn range_keys_sign_in_range() { + let message: Message = std::array::from_fn(|i| (i * 3 + 7) as u8); + // A range across two bottom trees. + let start = (3 << LAYER_HEIGHT) - 2; + let key = range_key(&[2; SEED_LEN], start, start + 5); + let pubkey = key.public_key().to_bytes(); + for leaf_index in start..=start + 5 { + let signature = key.sign(&message, leaf_index).unwrap().to_bytes(); + assert!(verify(&pubkey, u64::from(leaf_index), &message, &signature)); + } + assert_eq!( + key.sign(&message, start + 6), + Err(SignError::OutOfRange(start + 6)) + ); + assert_eq!( + range_key(&[2; SEED_LEN], start, start + 5).public_key(), + key.public_key() + ); + } + + /// The leaf index rides the signature, so the verifier hashes at it: moved + /// to a neighbour, or the message changed, the signature no longer verifies. + #[test] + fn signature_is_bound_to_its_leaf_index_and_message() { + let key = range_key(&[4; SEED_LEN], 0, 7); + let pubkey = key.public_key(); + let message: Message = std::array::from_fn(|i| i as u8); + let signature = key.sign(&message, 5).unwrap(); + let mut moved = signature.clone(); + moved.leaf_index = 4; + assert!(!verify_signature(&pubkey, &message, &moved)); + let mut other = message; + other[0] ^= 1; + assert!(!verify_signature(&pubkey, &other, &signature)); + assert!(verify_signature(&pubkey, &message, &signature)); + } + + #[test] + fn encoding_round_trips_and_is_exact() { + let key = range_key(&[7; SEED_LEN], 290, 310); + let message: Message = [9; MESSAGE_LEN]; + let signature = key.sign(&message, 300).unwrap(); + let bytes = signature.to_bytes(); + assert_eq!(bytes.len(), SIGNATURE_SIZE); + assert_eq!(bytes[..LEAF_INDEX_SIZE], 300u32.to_le_bytes()); + assert_eq!(Signature::from_bytes(&bytes), Some(signature)); + assert_eq!(Signature::from_bytes(&bytes[1..]), None); + let pubkey = key.public_key(); + assert_eq!(PublicKey::from_bytes(&pubkey.to_bytes()), pubkey); + } + + #[test] + fn a_leaf_index_past_32_bits_never_verifies() { + let (pubkey, message, signature) = kat(); + assert!(!verify( + &pubkey, + KAT_LEAF_INDEX + (1 << 32), + &message, + &signature + )); + } +} diff --git a/crates/common/crypto/src/lib.rs b/crates/common/crypto/src/lib.rs index f2e827328..842fb5f24 100644 --- a/crates/common/crypto/src/lib.rs +++ b/crates/common/crypto/src/lib.rs @@ -45,8 +45,17 @@ use tracing::error; /// Beacon Chain BLS12-381 signatures (the specification's `bls` module). pub mod bls; +/// The Beacon Chain PQ fork's leanXMSS signatures (the specification's `xmss` +/// module), over Keccak-256. Not the XMSS of [`signature`]. +pub mod lean_xmss; mod metrics; +/// A validator's PQ-fork leanXMSS key and VRF tree, derived from its BLS +/// signing key as the PQ devnet's genesis tooling derives them. +pub mod pq_keys; pub mod signature; +/// The Beacon Chain PQ fork's proofs of many leanXMSS signatures (the +/// specification's `stark` module), in its reference concatenation encoding. +pub mod stark; #[cfg(feature = "shadow-integration")] pub mod shadow_cost; diff --git a/crates/common/crypto/src/pq_keys.rs b/crates/common/crypto/src/pq_keys.rs new file mode 100644 index 000000000..c8c283db2 --- /dev/null +++ b/crates/common/crypto/src/pq_keys.rs @@ -0,0 +1,233 @@ +//! A validator's PQ-fork key material, derived from its BLS signing key the +//! way ethPandaOps' PQ genesis tooling (eth-beacon-genesis `284025a0`) and +//! Lighthouse's PQ branch derive it, so a validator's keystore is all it +//! needs on a network whose genesis that tooling built: +//! +//! - the leanXMSS key grows from `sha256("leanxmss-seed" || sk)`; +//! - the VRF tree's seed is `sha256("vrf-seed" || sk)`. +//! +//! `sk` is the 32-byte big-endian BLS secret key a keystore holds. The +//! specification leaves the derivation to the validator (`pq/validator.md`, +//! "XMSS public key" and "VRF tree"); this is the devnet's convention, not the +//! specification's. +//! +//! The VRF tree is the specification's: depth `VRF_TREE_DEPTH` (31), its leaf +//! at index `slot` the proposer's RANDAO reveal for that slot. The tooling +//! populates the first 2^20 leaves, each `sha256(seed || le64(index))`, and +//! leaves the rest zero. Holding the whole populated tree costs 64 MiB a key, +//! so [`VrfTree`] keeps the roots of its 2^10-leaf subtrees (32 KiB) and +//! rebuilds one subtree per proof, as Lighthouse does. + +use sha2::{Digest, Sha256}; + +use crate::lean_xmss; + +/// The depth of the VRF tree (the specification's `VRF_TREE_DEPTH`). +pub const VRF_TREE_DEPTH: usize = 31; +const POPULATED_DEPTH: usize = 20; +const POPULATED_LEAVES: u64 = 1 << POPULATED_DEPTH; +const SUBTREE_DEPTH: usize = 10; +const SUBTREE_LEAVES: u64 = 1 << SUBTREE_DEPTH; + +type Node = [u8; 32]; + +fn sha256_concat(prefix: &[u8], body: &[u8]) -> Node { + let mut hasher = Sha256::new(); + hasher.update(prefix); + hasher.update(body); + hasher.finalize().into() +} + +/// The seed a validator's leanXMSS key grows from. +pub fn xmss_seed(bls_secret_key: &[u8; 32]) -> lean_xmss::Seed { + sha256_concat(b"leanxmss-seed", bls_secret_key) +} + +/// The seed of a validator's VRF tree. +pub fn vrf_seed(bls_secret_key: &[u8; 32]) -> Node { + sha256_concat(b"vrf-seed", bls_secret_key) +} + +/// A validator's leanXMSS key. Generating it builds the key's whole top tree: +/// about a second and a half across cores. +pub fn xmss_key(bls_secret_key: &[u8; 32]) -> lean_xmss::SecretKey { + lean_xmss::key_gen(&xmss_seed(bls_secret_key)) +} + +/// A validator's VRF tree, holding the roots of its populated subtrees. +pub struct VrfTree { + seed: Node, + subtree_roots: Vec, +} + +impl VrfTree { + pub fn from_bls(bls_secret_key: &[u8; 32]) -> Self { + Self::from_seed(vrf_seed(bls_secret_key)) + } + + /// Hashes every populated leaf once: about two million SHA-256s. + pub fn from_seed(seed: Node) -> Self { + let subtree_roots = (0..POPULATED_LEAVES) + .step_by(SUBTREE_LEAVES as usize) + .map(|start| { + let mut nodes = populated_subtree(&seed, start); + reduce(&mut nodes, 0, &mut Vec::new()); + nodes[0] + }) + .collect(); + Self { + seed, + subtree_roots, + } + } + + /// The tree's root, which a validator registers as its `vrf_root`. + pub fn root(&self) -> Node { + let mut nodes = self.subtree_roots.clone(); + reduce(&mut nodes, 0, &mut Vec::new()); + let zeros = zero_hashes(); + (POPULATED_DEPTH..VRF_TREE_DEPTH) + .fold(nodes[0], |node, level| hash_pair(&node, &zeros[level])) + } + + /// The proof for `slot`: the leaf at index `slot`, then its branch, + /// siblings bottom-up (`VRF_TREE_DEPTH + 1` nodes), the specification's + /// `VRFProof`. + pub fn proof(&self, slot: u64) -> Vec { + let index = slot & ((1 << VRF_TREE_DEPTH) - 1); + let zeros = zero_hashes(); + let mut proof = Vec::with_capacity(VRF_TREE_DEPTH + 1); + if index < POPULATED_LEAVES { + let start = index & !(SUBTREE_LEAVES - 1); + let mut nodes = populated_subtree(&self.seed, start); + proof.push(vrf_leaf(&self.seed, index)); + reduce(&mut nodes, (index % SUBTREE_LEAVES) as usize, &mut proof); + let mut roots = self.subtree_roots.clone(); + reduce(&mut roots, (index / SUBTREE_LEAVES) as usize, &mut proof); + proof.extend_from_slice(&zeros[POPULATED_DEPTH..VRF_TREE_DEPTH]); + } else { + // A zero leaf in the empty part of the tree: the populated + // subtree is the one non-zero sibling on the way up. + let joined_depth = index.ilog2() as usize; + let mut roots = self.subtree_roots.clone(); + reduce(&mut roots, 0, &mut Vec::new()); + let mut populated = roots[0]; + proof.push([0; 32]); + for (level, zero) in zeros.iter().enumerate() { + proof.push(if level == joined_depth { + populated + } else { + *zero + }); + if (POPULATED_DEPTH..joined_depth).contains(&level) { + populated = hash_pair(&populated, zero); + } + } + } + proof + } +} + +fn populated_subtree(seed: &Node, start: u64) -> Vec { + (0..SUBTREE_LEAVES) + .map(|offset| vrf_leaf(seed, start + offset)) + .collect() +} + +fn vrf_leaf(seed: &Node, index: u64) -> Node { + sha256_concat(seed, &index.to_le_bytes()) +} + +fn hash_pair(left: &Node, right: &Node) -> Node { + sha256_concat(left, right) +} + +fn zero_hashes() -> [Node; VRF_TREE_DEPTH] { + let mut previous = [0; 32]; + std::array::from_fn(|_| { + let zero = previous; + previous = hash_pair(&zero, &zero); + zero + }) +} + +/// Folds `nodes` (a power-of-two layer) to its root in `nodes[0]`, pushing +/// the sibling of the node at `index` at every level onto `branch`. +fn reduce(nodes: &mut Vec, mut index: usize, branch: &mut Vec) { + while nodes.len() > 1 { + branch.push(nodes[index ^ 1]); + let next: Vec = nodes + .chunks_exact(2) + .map(|pair| hash_pair(&pair[0], &pair[1])) + .collect(); + *nodes = next; + index /= 2; + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn hex32(text: &str) -> [u8; 32] { + hex::decode(text.trim_start_matches("0x")) + .unwrap() + .try_into() + .unwrap() + } + + /// The specification's `is_valid_merkle_branch` for a VRF proof. + fn verifies(proof: &[Node], slot: u64, root: &Node) -> bool { + let mut node = proof[0]; + for (level, sibling) in proof[1..].iter().enumerate() { + node = if (slot >> level) & 1 == 1 { + hash_pair(sibling, &node) + } else { + hash_pair(&node, sibling) + }; + } + &node == root + } + + /// Validator 0 of ethereum-package's default mnemonic: the second 32 bytes + /// of `static_files/xmss-keys/e383db4eb96b6f02.bin` are its VRF root. + #[test] + fn ethereum_package_validator_zero_vrf_root() { + let sk = hex32("0dce41fa73ae9f6bdfd51df4d422d75eee174553dba5fd450c4437e4ed3fc903"); + let tree = VrfTree::from_bls(&sk); + assert_eq!( + tree.root(), + hex32("b1e3a6168d285a532e0dd3c6e1b5588b3bc68c27b45af71ee60ad5243600d281") + ); + } + + /// Lighthouse's `pq_ethpandaops_keys_and_vrf_tree_boundaries`: its key and + /// VRF root, and proofs at the boundaries of the populated part. + #[test] + fn lighthouse_known_answer_and_boundary_proofs() { + let sk = hex32("2892f3ba475c3640a67fe730798ad9c4eed3a6d934db85f91c41cfc837ce8834"); + let tree = VrfTree::from_bls(&sk); + let root = tree.root(); + assert_eq!( + root, + hex32("0fc00857fdae6d29c18f54563c64e166b1fff1c6d943bb2c43e8b096bb60585e") + ); + assert_eq!( + xmss_key(&sk).public_key().to_bytes(), + hex32("155a29b971da2b771fdf52c768af63fa03f78f25487e3e0ed03d012b7039d88b") + ); + for slot in [ + 0, + 5, + POPULATED_LEAVES - 1, + POPULATED_LEAVES, + (1 << VRF_TREE_DEPTH) - 1, + ] { + let proof = tree.proof(slot); + assert_eq!(proof.len(), VRF_TREE_DEPTH + 1); + assert_eq!(proof[0] == [0; 32], slot >= POPULATED_LEAVES, "slot {slot}"); + assert!(verifies(&proof, slot, &root), "slot {slot}"); + assert!(!verifies(&proof, slot, &[0; 32]), "slot {slot}"); + } + } +} diff --git a/crates/common/crypto/src/stark.rs b/crates/common/crypto/src/stark.rs new file mode 100644 index 000000000..d78e3218e --- /dev/null +++ b/crates/common/crypto/src/stark.rs @@ -0,0 +1,281 @@ +//! The PQ fork's proofs of many leanXMSS signatures at once (the +//! specification's `stark` module: `ProveAggregates`, `MergeProofs`, +//! `VerifyAggregates`), behind one of two backends, chosen once per process: +//! +//! - **Reference** (the default): the specification's stand-in for a +//! recursive STARK, which concatenates signatures. A proof is the signatures +//! of every statement's public keys, in statement order and within a +//! statement in public-key order, and verification checks each in turn. It +//! is what the specification's fixtures use. +//! - **Sidecar** ([`install_sidecar`]): real leanVM proofs, made and checked by +//! a `leanvm-sidecar` process over a Unix socket, as the PQ devnet's other +//! clients make them. See [`sidecar`]. +//! +//! The two encodings are not interchangeable: a node verifies only the proofs +//! its own backend makes, so every node of a network must run the same one. +//! The functions below take the same arguments whichever is installed. +//! +//! Under the reference, the proof is bounded by `MAX_STARK_PROOF_SIZE` (1 MiB), +//! so a block carries at most `MAX_STARK_PROOF_SIZE / SIGNATURE_SIZE` (436) +//! attester signatures. That is a property of the placeholder, not of the fork. + +use std::sync::OnceLock; + +use rayon::prelude::*; + +use crate::lean_xmss::{self, Message, PUBKEY_SIZE, SIGNATURE_SIZE}; + +pub mod sidecar; + +/// The backend every proof of this process goes through. +enum Backend { + Reference, + Sidecar(sidecar::SidecarBackend), +} + +static BACKEND: OnceLock = OnceLock::new(); + +fn backend() -> &'static Backend { + BACKEND.get_or_init(|| Backend::Reference) +} + +/// Makes every proof of this process a leanVM proof from the sidecar at +/// `config.socket`, after checking it answers with this protocol and +/// verifier. Call once, at startup, before anything proves or verifies: +/// whichever backend is first used stays for the life of the process. +pub fn install_sidecar(config: sidecar::SidecarConfig) -> Result<(), sidecar::Error> { + let backend = sidecar::SidecarBackend::connect(config)?; + BACKEND + .set(Backend::Sidecar(backend)) + .map_err(|_| sidecar::Error::AlreadyInstalled) +} + +/// The installed backend's name, for logs: `reference` or `sidecar`. +pub fn backend_name() -> &'static str { + match backend() { + Backend::Reference => "reference", + Backend::Sidecar(_) => "sidecar", + } +} + +/// The specification's `XMSSAggregateStatement`: every public key in `pubkeys` +/// signed `message` at `leaf_index`. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct AggregateStatement { + pub pubkeys: Vec<[u8; PUBKEY_SIZE]>, + pub leaf_index: u64, + pub message: Message, +} + +/// Why [`prove_aggregates`] or [`merge_proofs`] refused its input. +#[derive(Debug, Clone, PartialEq, Eq, thiserror::Error)] +pub enum ProveError { + #[error("{statements} statements but {signature_lists} signature lists")] + StatementCount { + statements: usize, + signature_lists: usize, + }, + #[error("statement {statement} has {pubkeys} public keys but {signatures} signatures")] + SignatureCount { + statement: usize, + pubkeys: usize, + signatures: usize, + }, + #[error("a signature is {0} bytes, not {SIGNATURE_SIZE}")] + SignatureLength(usize), + #[error("signature {signature} of statement {statement} is at leaf {found}, not {expected}")] + LeafIndex { + statement: usize, + signature: usize, + expected: u64, + found: u64, + }, + #[error("the prover sidecar failed: {0}")] + Sidecar(String), +} + +/// `stark.ProveAggregates(statements, signatures)`: `signatures[i][j]` is the +/// signature of `statements[i].pubkeys[j]`. The reference verifies nothing +/// and only asserts the shapes line up; the sidecar refuses a signature that +/// does not verify. Blocks the calling thread for as long as the proof takes, +/// which under the sidecar is seconds. +pub fn prove_aggregates>( + statements: &[AggregateStatement], + signatures: &[Vec], +) -> Result, ProveError> { + check_shape(statements, signatures)?; + match backend() { + Backend::Reference => Ok(signatures + .iter() + .flatten() + .flat_map(|signature| signature.as_ref()) + .copied() + .collect()), + Backend::Sidecar(sidecar) => sidecar.prove(statements, signatures), + } +} + +/// `stark.MergeProofs(proofs)`: one proof of the concatenation, in order, of +/// the statement lists the children prove. Each child is its statements and +/// its proof. Children are not verified: verify untrusted ones first. +pub fn merge_proofs, P: AsRef<[u8]>>( + children: &[(S, P)], +) -> Result, ProveError> { + match backend() { + Backend::Reference => Ok(children + .iter() + .flat_map(|(_, proof)| proof.as_ref()) + .copied() + .collect()), + Backend::Sidecar(sidecar) => sidecar.merge(children), + } +} + +/// `stark.VerifyAggregates(statements, proof)`. Under the reference, `true` +/// only if `proof` holds, for each statement in turn and each of its public +/// keys in order, a signature `xmss.Verify` accepts at the statement's leaf +/// index and message, and nothing else. Under the sidecar, whether the leanVM +/// proof proves every `(public key, message)` claim; it cannot check the leaf +/// indices, which leanVM proofs do not attest. A sidecar that cannot answer +/// reads as `false`, and is logged. +pub fn verify_aggregates(statements: &[AggregateStatement], proof: &[u8]) -> bool { + match backend() { + Backend::Reference => reference_verify(statements, proof), + Backend::Sidecar(sidecar) => sidecar.verify(statements, proof).unwrap_or_else(|err| { + tracing::error!(%err, "The prover sidecar could not verify an attestation proof"); + false + }), + } +} + +/// Checks `signatures` has one list per statement and one signature of +/// [`SIGNATURE_SIZE`] bytes per public key. +fn check_shape>( + statements: &[AggregateStatement], + signatures: &[Vec], +) -> Result<(), ProveError> { + if statements.len() != signatures.len() { + return Err(ProveError::StatementCount { + statements: statements.len(), + signature_lists: signatures.len(), + }); + } + for (statement, (index, list)) in statements.iter().zip(signatures.iter().enumerate()) { + if statement.pubkeys.len() != list.len() { + return Err(ProveError::SignatureCount { + statement: index, + pubkeys: statement.pubkeys.len(), + signatures: list.len(), + }); + } + } + for signature in signatures.iter().flatten() { + let bytes = signature.as_ref(); + if bytes.len() != SIGNATURE_SIZE { + return Err(ProveError::SignatureLength(bytes.len())); + } + } + Ok(()) +} + +/// The reference's `VerifyAggregates`. The signatures are checked in parallel: +/// they are independent, and a block's proof holds hundreds. +fn reference_verify(statements: &[AggregateStatement], proof: &[u8]) -> bool { + let claims: Vec<(&[u8; PUBKEY_SIZE], u64, &Message)> = statements + .iter() + .flat_map(|statement| { + statement + .pubkeys + .iter() + .map(move |pubkey| (pubkey, statement.leaf_index, &statement.message)) + }) + .collect(); + if proof.len() != claims.len() * SIGNATURE_SIZE { + return false; + } + claims + .par_iter() + .zip(proof.par_chunks_exact(SIGNATURE_SIZE)) + .all(|((pubkey, leaf_index, message), signature)| { + lean_xmss::verify(pubkey, *leaf_index, message, signature) + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::lean_xmss::range_key; + + fn signed(seed: u8, leaf: u32, message: &Message) -> ([u8; PUBKEY_SIZE], Vec) { + let key = range_key(&[seed; 32], leaf, leaf); + let signature = key.sign(message, leaf).unwrap().to_bytes(); + (key.public_key().to_bytes(), signature) + } + + #[test] + fn a_merged_proof_verifies_against_the_concatenated_statements() { + let first_message: Message = [1; 32]; + let second_message: Message = [2; 32]; + let (a, sig_a) = signed(10, 40, &first_message); + let (b, sig_b) = signed(11, 40, &first_message); + let (c, sig_c) = signed(12, 41, &second_message); + let first = AggregateStatement { + pubkeys: vec![a, b], + leaf_index: 40, + message: first_message, + }; + let second = AggregateStatement { + pubkeys: vec![c], + leaf_index: 41, + message: second_message, + }; + let proof_one = + prove_aggregates(std::slice::from_ref(&first), &[vec![&sig_a, &sig_b]]).unwrap(); + let proof_two = prove_aggregates(std::slice::from_ref(&second), &[vec![&sig_c]]).unwrap(); + assert!(verify_aggregates(std::slice::from_ref(&first), &proof_one)); + let one = [first.clone()]; + let two = [second.clone()]; + let merged = + merge_proofs(&[(&one[..], &proof_one[..]), (&two[..], &proof_two[..])]).unwrap(); + let both = [first.clone(), second.clone()]; + assert!(verify_aggregates(&both, &merged)); + // Out of order, short, or with a trailing byte, the same signatures fail. + assert!(!verify_aggregates(&[second, first], &merged)); + assert!(!verify_aggregates(&both, &merged[..merged.len() - 1])); + let mut long = merged.clone(); + long.push(0); + assert!(!verify_aggregates(&both, &long)); + } + + #[test] + fn no_statements_take_the_empty_proof_only() { + assert!(verify_aggregates(&[], &[])); + assert!(!verify_aggregates(&[], &[0; SIGNATURE_SIZE])); + let empty = AggregateStatement { + pubkeys: vec![], + leaf_index: 0, + message: [0; 32], + }; + assert!(verify_aggregates(&[empty], &[])); + } + + #[test] + fn mismatched_shapes_are_refused() { + let statement = AggregateStatement { + pubkeys: vec![[0; PUBKEY_SIZE]], + leaf_index: 0, + message: [0; 32], + }; + assert_eq!( + prove_aggregates::>(std::slice::from_ref(&statement), &[]), + Err(ProveError::StatementCount { + statements: 1, + signature_lists: 0 + }) + ); + assert_eq!( + prove_aggregates(std::slice::from_ref(&statement), &[vec![vec![0u8; 3]]]), + Err(ProveError::SignatureLength(3)) + ); + } +} diff --git a/crates/common/crypto/src/stark/sidecar.rs b/crates/common/crypto/src/stark/sidecar.rs new file mode 100644 index 000000000..8e10f48cb --- /dev/null +++ b/crates/common/crypto/src/stark/sidecar.rs @@ -0,0 +1,410 @@ +//! Attestation proofs from a `leanvm-sidecar` process, over its Unix socket +//! (protocol version 1, `bins/leanvm-sidecar/README.md` in the leanVM +//! repository). +//! +//! The PQ devnet does not run the specification's reference concatenation: +//! its clients prove with leanVM, through this sidecar. The wire protocol is +//! leanVM's own client crate at the revision the workspace pins, which must be +//! the revision the sidecar binary is built from: a proof only verifies with a +//! compatible verifier, and nothing on the wire detects a mismatch beyond the +//! hello's verifier tag. +//! +//! The mapping from the specification's statements is Lighthouse's, so the +//! two clients' proofs verify against each other: +//! +//! - A leanVM proof attests `(public key, message)` claims. A statement list +//! flattens to the sorted, deduplicated list of its pairs (the verifier only +//! accepts a claim list strictly sorted by the 64 bytes `pubkey || message`). +//! - **The leaf index is not part of a claim** (leanEthereum/leanVM#787), so +//! [`SidecarBackend::verify`] shows each key signed the message at *some* +//! leaf, not at the statement's. [`SidecarBackend::prove`] refuses +//! signatures made at another leaf, since it sees them. +//! - The empty proof stands for "nothing to prove" (the sidecar refuses to +//! prove zero claims), and verifies only against no claims. +//! - A merge folds the children [`MAX_CHILDREN_PER_AGGREGATE`] at a time, +//! round after round, until one proof remains. +//! +//! Two connections are kept, opened on first use and reopened after a +//! failure: one for proving, where requests queue in the sidecar, and one for +//! verifying, which the sidecar answers without waiting for proofs in +//! progress. + +use std::path::PathBuf; +use std::sync::Mutex; +use std::time::{Duration, Instant}; + +use leanvm_claims::{AggregateClaims, OutputClaims}; +use leanvm_lean_xmss::{LeanXmssPublicKey, LeanXmssSignature}; +use leanvm_sidecar::{Client, SidecarError}; + +use super::{AggregateStatement, ProveError}; +use crate::lean_xmss::{self, Message, PUBKEY_SIZE, SIGNATURE_SIZE}; + +/// The most children one aggregate request takes. +pub const MAX_CHILDREN_PER_AGGREGATE: usize = 16; + +/// The leanVM revision the workspace's sidecar client is pinned to, which the +/// `leanvm-sidecar` binary must be built from. +pub const LEANVM_REV: &str = "283686575c4e81790591d2c6e3ed21c69c0e3bbe"; + +#[derive(Debug, thiserror::Error)] +pub enum Error { + #[error("sidecar connection: {0}")] + Io(#[from] std::io::Error), + #[error("sidecar: {0}")] + Sidecar(String), + #[error("a proof backend is already installed")] + AlreadyInstalled, + #[error("the sidecar proved {found} claims, {expected} were asked")] + ClaimsMismatch { expected: usize, found: usize }, + #[error("child {0} has claims and an empty proof, or a proof and no claims")] + MalformedChild(usize), +} + +impl From for Error { + fn from(error: SidecarError) -> Self { + match error { + SidecarError::Io(error) => Error::Io(error), + other => Error::Sidecar(format!("{other:?}")), + } + } +} + +impl From for ProveError { + fn from(error: Error) -> Self { + ProveError::Sidecar(error.to_string()) + } +} + +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct SidecarConfig { + /// The Unix socket `leanvm-sidecar --socket` listens on. + pub socket: PathBuf, + /// `log_inv_rate` of the proofs requested, 1 to 4: a higher rate gives + /// smaller proofs that take longer to make. A proof verifies whatever rate + /// it was made at. + pub log_inv_rate: u8, + /// How long one prove or merge request may wait on the sidecar, queueing + /// behind others included. + pub prove_timeout: Duration, + /// How long one verify request may wait on the sidecar. + pub verify_timeout: Duration, +} + +impl SidecarConfig { + /// Lighthouse's defaults. + pub fn new(socket: impl Into) -> Self { + Self { + socket: socket.into(), + log_inv_rate: 2, + prove_timeout: Duration::from_secs(300), + verify_timeout: Duration::from_secs(30), + } + } +} + +pub struct SidecarBackend { + config: SidecarConfig, + prove_connection: Mutex>, + verify_connection: Mutex>, +} + +impl SidecarBackend { + /// Opens both connections now, checking each hello (magic, protocol + /// version, verifier tag), so a missing or incompatible sidecar is + /// reported at startup. + pub fn connect(config: SidecarConfig) -> Result { + let prove_connection = open(&config, config.prove_timeout)?; + let verify_connection = open(&config, config.verify_timeout)?; + tracing::info!( + socket = %config.socket.display(), + prover = prove_connection.prover(), + leanvm_rev = LEANVM_REV, + "Connected to the leanVM prover sidecar" + ); + Ok(Self { + config, + prove_connection: Mutex::new(Some(prove_connection)), + verify_connection: Mutex::new(Some(verify_connection)), + }) + } + + pub(super) fn prove>( + &self, + statements: &[AggregateStatement], + signatures: &[Vec], + ) -> Result, ProveError> { + for (statement_index, (statement, group)) in statements.iter().zip(signatures).enumerate() { + for (signature_index, signature) in group.iter().enumerate() { + let found = lean_xmss::signature_index(signature.as_ref()).unwrap_or(u64::MAX); + if found != statement.leaf_index { + return Err(ProveError::LeafIndex { + statement: statement_index, + signature: signature_index, + expected: statement.leaf_index, + found, + }); + } + } + } + let expected = statements_to_claims(statements); + if expected.lean_xmss.is_empty() { + return Ok(Vec::new()); + } + let raw_signatures: Vec<_> = statements + .iter() + .zip(signatures) + .flat_map(|(statement, group)| { + statement + .pubkeys + .iter() + .zip(group) + .map(|(pubkey, signature)| { + let signature: &[u8; SIGNATURE_SIZE] = signature + .as_ref() + .try_into() + .expect("check_shape bounds every signature's length"); + ( + LeanXmssPublicKey::from_bytes(pubkey), + statement.message, + LeanXmssSignature::from_bytes(signature), + ) + }) + }) + .collect(); + Ok(self.aggregate(&[], &raw_signatures, &expected)?) + } + + pub(super) fn merge, P: AsRef<[u8]>>( + &self, + children: &[(S, P)], + ) -> Result, ProveError> { + let mut level: Vec<(AggregateClaims, Vec)> = Vec::with_capacity(children.len()); + for (index, (statements, proof)) in children.iter().enumerate() { + let claims = statements_to_claims(statements.as_ref()); + match (claims.lean_xmss.is_empty(), proof.as_ref().is_empty()) { + (true, true) => {} + (false, false) => level.push((claims, proof.as_ref().to_vec())), + _ => return Err(Error::MalformedChild(index).into()), + } + } + while level.len() > 1 { + let mut next = Vec::with_capacity(level.len().div_ceil(MAX_CHILDREN_PER_AGGREGATE)); + for chunk in level.chunks(MAX_CHILDREN_PER_AGGREGATE) { + if let [only] = chunk { + next.push(only.clone()); + continue; + } + let expected = union(chunk); + let proof = self.aggregate(chunk, &[], &expected)?; + next.push((expected, proof)); + } + level = next; + } + Ok(level.pop().map(|(_, proof)| proof).unwrap_or_default()) + } + + pub(super) fn verify( + &self, + statements: &[AggregateStatement], + proof: &[u8], + ) -> Result { + let claims = statements_to_claims(statements); + if claims.lean_xmss.is_empty() || proof.is_empty() { + return Ok(claims.lean_xmss.is_empty() && proof.is_empty()); + } + with_connection( + &self.config, + &self.verify_connection, + self.config.verify_timeout, + |client| client.verify(&claims, proof), + ) + } + + /// One aggregate request on the proving connection. Checks the sidecar + /// proved exactly `expected`. + fn aggregate( + &self, + children: &[(AggregateClaims, Vec)], + raw_signatures: &[(LeanXmssPublicKey, Message, LeanXmssSignature)], + expected: &AggregateClaims, + ) -> Result, Error> { + let started = Instant::now(); + let (claims, proof) = with_connection( + &self.config, + &self.prove_connection, + self.config.prove_timeout, + |client| { + client.aggregate( + children, + raw_signatures, + &[], + &[], + OutputClaims::All, + usize::from(self.config.log_inv_rate), + ) + }, + )?; + if claims != *expected { + return Err(Error::ClaimsMismatch { + expected: expected.num_signature_claims(), + found: claims.num_signature_claims(), + }); + } + tracing::debug!( + children = children.len(), + signatures = raw_signatures.len(), + claims = claims.num_signature_claims(), + proof_bytes = proof.len(), + elapsed_ms = started.elapsed().as_millis() as u64, + "Proved an attestation aggregate" + ); + Ok(proof) + } +} + +/// The claims a proof of `statements` attests: every `(pubkey, message)` +/// pair, strictly sorted by its bytes, without duplicates. No leanSPHINCS +/// claims and no DA commitment. +pub fn statements_to_claims(statements: &[AggregateStatement]) -> AggregateClaims { + let mut pairs: Vec<([u8; PUBKEY_SIZE], Message)> = statements + .iter() + .flat_map(|statement| { + statement + .pubkeys + .iter() + .map(|pubkey| (*pubkey, statement.message)) + }) + .collect(); + // Tuples of byte arrays order as their concatenation does. + pairs.sort_unstable(); + pairs.dedup(); + AggregateClaims { + lean_xmss: pairs + .into_iter() + .map(|(pubkey, message)| (LeanXmssPublicKey::from_bytes(&pubkey), message)) + .collect(), + lean_sphincs: Vec::new(), + da_commitment: None, + } +} + +/// The claims a proof of all of `children` attests: their union, strictly +/// sorted. +fn union(children: &[(AggregateClaims, Vec)]) -> AggregateClaims { + let mut lean_xmss: Vec<_> = children + .iter() + .flat_map(|(claims, _)| claims.lean_xmss.iter().copied()) + .collect(); + lean_xmss.sort_unstable(); + lean_xmss.dedup(); + AggregateClaims { + lean_xmss, + lean_sphincs: Vec::new(), + da_commitment: None, + } +} + +fn open(config: &SidecarConfig, timeout: Duration) -> Result { + let client = Client::connect(&config.socket)?; + client.set_timeout(Some(timeout))?; + Ok(client) +} + +/// Runs `call` on the connection in `slot`, opening it first if needed. A +/// connection that fails is dropped, as the protocol requires after anything +/// but a rejection; if a reused one failed other than by timing out, the +/// sidecar may have restarted, so this reconnects and tries once more. +fn with_connection( + config: &SidecarConfig, + slot: &Mutex>, + timeout: Duration, + mut call: impl FnMut(&mut Client) -> Result, +) -> Result { + let mut slot = slot + .lock() + .unwrap_or_else(std::sync::PoisonError::into_inner); + let mut reused = slot.is_some(); + loop { + let client = match slot.as_mut() { + Some(client) => client, + None => slot.insert(open(config, timeout)?), + }; + match call(client) { + Ok(value) => return Ok(value), + Err(SidecarError::Rejected(rejection, message)) => { + return Err(Error::Sidecar(format!("{rejection:?}: {message}"))); + } + Err(SidecarError::Io(error)) + if reused + && !matches!( + error.kind(), + std::io::ErrorKind::TimedOut | std::io::ErrorKind::WouldBlock + ) => + { + *slot = None; + reused = false; + } + Err(error) => { + *slot = None; + return Err(error.into()); + } + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn statement( + pubkeys: &[[u8; PUBKEY_SIZE]], + leaf_index: u64, + message: Message, + ) -> AggregateStatement { + AggregateStatement { + pubkeys: pubkeys.to_vec(), + leaf_index, + message, + } + } + + #[test] + fn no_claims_encode_as_empty_lists() { + let expected = [0u8; 8 + 8 + 1]; + assert_eq!(statements_to_claims(&[]).to_bytes(), expected); + assert_eq!( + statements_to_claims(&[statement(&[], 1, [7; 32])]).to_bytes(), + expected + ); + } + + /// Lighthouse's `claims_bytes_match_the_sidecar_format`: out of order within + /// and across statements, one repeated claim, one key signing two + /// messages, and no leaf index on the wire. + #[test] + fn claims_are_sorted_deduplicated_pairs() { + let mut low_key = [3; 32]; + low_key[31] = 2; + let statements = [ + statement(&[[9; 32], [3; 32], low_key], 40_000, [0xaa; 32]), + statement(&[[3; 32], [1; 32]], 50_000, [0x11; 32]), + statement(&[[9; 32]], 60_000, [0xaa; 32]), + ]; + let mut expected = Vec::new(); + expected.extend_from_slice(&5u64.to_le_bytes()); + for (key, message) in [ + ([1; 32], [0x11; 32]), + (low_key, [0xaa; 32]), + ([3; 32], [0x11; 32]), + ([3; 32], [0xaa; 32]), + ([9; 32], [0xaa; 32]), + ] { + expected.extend_from_slice(&key); + expected.extend_from_slice(&message); + } + expected.extend_from_slice(&0u64.to_le_bytes()); + expected.push(0); + assert_eq!(statements_to_claims(&statements).to_bytes(), expected); + } +} diff --git a/crates/common/crypto/tests/fixtures/xmss_kat_signature.hex b/crates/common/crypto/tests/fixtures/xmss_kat_signature.hex new file mode 100644 index 000000000..659bf5c5a --- /dev/null +++ b/crates/common/crypto/tests/fixtures/xmss_kat_signature.hex @@ -0,0 +1 @@ +010000007006dc0f7aad4d5b6a6852e13b15ef784e172e47456a50f1358dce9bf22cf38b4087388923e36800b329f1c40b92b1120727e7f8ebb0d7f7f7161d83d3fb84bfb07b4352893c0e9f4d023bab2a12b57ba9452ffae122e67369045c3643fe9d05fcbd4633228146897451cc5af84188721ffc44650701bd79fa6154850dcc24a4a4fc2c97f868f7029a2c3f5dd47f38a492c93b3ac880f0210cf6eefc8f12362671317a081f46b1fd62b830024718b157df6b6b602f4f00abc9be562122afdcbc2a5dcbc6e9d8f0a3a9c549a744df093cc9a24e52f446861cac7e701ba598ed4de503332fcf3f166aeeb5efa03ceeec8cffd613b071a967bbf55006d5479d66e7b160782f136899e7d07e5a21bef3419cf15b4c1ef11619546a5aa2b353798f236c8c6b882a70010c0c1ae76ac8beb803f24cb1a597452f31dd2e971b65b625708abb2fa940b10228fe10b2fd3f22124c3753a70e697bb83444843eb7e235650f409f962a0d7bc8290658d882695bdb5fcd7028a2bc92ee000ba71593e434899698e7db0db088ecf1250f0a6dc8595ac76b4364fd11e79c5ee7b7c8c90b70f3777b58d72bb2ff6117eafd7d59e518f6716d1f5e1dd4ff38804e4c38296a2023b08d90c943a3aae5e4af9f2dca5e9e781514ef443c3e0a998af05ca4569498a848118879f98b14ed5738e24a1498396a8ee40fb86b5dfc9b0f1713683946d13bee59abc169d69480ca126911ccbe8dff83c4201630566c0c7544466a46ea63bd2a8757423a4a16c97a8839d2ab6c23900fb952d5250a91d5bd685543878f036d1f30bf228082cc97fe340b455210f94fb0d2c6e3c741261f2d0957320f9cdc4f9dbd11658d60f7ff640d800ab55e3103d7c97bd8288063deac099729cbb14950289044bed516bc5b61f16c4fe287c843159b67404b3c71a31df4b311e86ef9f1067bfff2fbbeaa2892453291d8e5e08ca029ab922575bdd699174681f3c902c0111d00a1db657324d4b7b1a8ec5354b1409c0a3fcd7a280759822ee42aabbd1df9b7d449f50b6c88dd55586446c400d934424adbcaea21d8bcdee73b3554a16a85d25164805fefdb6567759a7b082ad7915bbcc63b7e00691d19245faa5b39f684006c4657a842f2bcad8dc05cfde2f741d6369800c96c83c17eef0b8eb6cfb9115a969f4a73d5d31a7c3af68553a348e8854b187c4de2a28a2816f23ed2d6501370e53a142b81702e7e839b1f9d0c4b6f5a4ba9e44e5c3bea4a73d04e32ce51a78cf0e4da7d589fbd07c65b34d99c634db23d8fe5207f7f96b902512dc6c350b3c99815080f126c53c23d76e06e795e7b5e69dda19e3be8d936209b1aea7f628038add040a1e09f05f403cd438e801d5999025b9a2bf6a119c667b45e54e8ef70282390b662bee399f2e789cf4d84b28a14e59a548234c5d4167fd6b54cfa782c08fac91a365c62d9a7c63505dbe5a420a95ecabdf5961e620b5a0888a9fe5c88d1f5aa251ce212c27c1194da589e27175eaaf37900920158efae1763622a72ae2c5a045ed2d02b78ee4c9928219562dcb4403cd32164e85d5306aa8e2e24b08de6c51921823cacac4d44719c1dba3f244d498903954b9de0cc6842e5539a2cb1fa50a9977f463b77e569874d3112f9dec161dfe1e623d049718ddf33c50395aa3bf509ef5134277121a20b80bf977e5db9f11aad2c08fb5fad7b564c4ce681107f968f0c5313def23b40ce8829199e7f2c282b5c57ca7fba30ab01283284a34d64271da9e5203e01df35fa0749a929d64d511a0ca4f98fb54f75c47ffbf126781ba441b08740378dc64fe7d52f7b95601ca65181972f343360fee20a92d11bca01a3c7a4e5a0e12fcd4b5cb258515bbf9d92f0285648fe31d233e2285c03aca126b5926cfee335b1eedd6fbe25d6fadc863f4586af1aa28620fc1c1838a925d6bd8b0bf2704e77e16dc3e469ecca20581c16a9756c66435600838a09d39855a1007913dc9f2a3b47d0f53117a15803a70993e5512e048781bd5bfa57bbe49ea22f1e2f06c3a5e8cbe9f0bc93c3e21f23b4a8138413e06ebc229d5d892b7fccf4b82f1b2289146cf9d758001d3f2300fb96e2bd3d42429f03a92951528ee3a114b53a1d890bf2d7262b56a6c8143a1d430bb59e06ede93472675a152cf3e7bd49038d5d6da9914ad79b19f4c2ee97c4b4829a044849e4f7f87ceebe7479acb7c85f3d32c5035d8e8db43ab2f18a2be19cadc2a992bc653a3259821855cd9c4af2b0d43494a020c53dfc99c6ff6bbd8b8e03c80707d81f0751bfc129b86c0834b1865dd9d73a2cfd1a4354876ea2eefdd4de257fddbdf358270f11fe7faada24a7edc1fb2946dfb3f43d40796035e1575f6b896460bee3ac524722667664a6dcb5c68ea4abd2138355258105b6ee85d8d3fc602bddd93803c224ac7499948c1a4d9e7f13692e0b0e37bb492ff98f0c95c408edf1cc8a8a8ed3a8c99850ac099f8bb56a625bbb05112aa89705c3c3ae4876f6583f12433f6c81d9d94fa53fee506046097a6098ccc267e5d33f8bf96196e22aac7607ac86229f23163466fc98093909abd474f3a69f91d582124906df35d2a76fdc0e1e8ea8a79098e52fe97dea0dd2760f6d0ec43598980e1cf7ee4f039c52ef4e47e1f32f479ce94967079c9af4a39aafe399257d71d1928ed612e2711e671a128f1e643af9e77b92d8431e80051b03cf6a5424742585ff819a0c1cdac87f79a1370bd8d475c86c38c2d6a2fc9557fdf87fbba282ac556d76e00bfade455051f30e50bee240eafaec31087dc4752e1d116d6e35586f9f3a15c2745403917cdab88d95499a062111c3c23a236ee561cd4d65a3b4f2881fdaf2cdc94bd167aec5f60912e2070066d20d13936ab362469f81ea37e5a7c02a49d8d8658bbf829df92fa6033818239cba4fedc5c92fa305d0ecf83463c8a2aaab1e438d68983ccefc37b45791c8f25992d72429e0918dcd590954f9fbc5b24219919b870f365992e43e56a2b6050b62bb4c6eb60c6c0b3467e12b63a4cdf2536f026283f10a7b9644b39efdf526501eb903cd9b8ec06b3679344f1503c2b7983b150251f61bea31505a272bede614b9cfce5efd31bac3a05f7d1054f3617dffefbd5c6659f17afc8021f84e35717652e2c62d8235e7a9aa4ee99241354fecd19ee2e3c8b411a3a1c8a6bd5f2a64f1b9e6bcecdfc79eb3ec920322f141d86b89c4dbe39c8f9b05fd5bef84280847d25e13fdf7efc9893c945463f4a7773a67dec875789bd61061676f0d1b3401bad44113b3e0af1df60c94920342119b46ba71b7d4b4b3fd30cb3e53705496842f2a1af86fc9282a4078391d1cde6b810f6e5a1fbff8cbd36440d7d8fc4a2ffbb63d228abe548e1d8a78 diff --git a/crates/common/types/src/beacon/config.rs b/crates/common/types/src/beacon/config.rs index cd3b56e48..982739f64 100644 --- a/crates/common/types/src/beacon/config.rs +++ b/crates/common/types/src/beacon/config.rs @@ -324,6 +324,22 @@ pub struct Config { /// is not scheduled. #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] pub gloas_fork_epoch: Epoch, + /// The `Fork.current_version` an EIP-8483 block signs under. + #[serde(with = "crate::beacon::serde_helpers::hex_array")] + pub eip8483_fork_version: Version, + /// The epoch EIP-8483 activates at, or [`constants::FAR_FUTURE_EPOCH`] if + /// it is not scheduled. A consensus-specs feature fork, never scheduled + /// on a built-in network. + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub eip8483_fork_epoch: Epoch, + /// The `Fork.current_version` a PQ block signs under. + #[serde(with = "crate::beacon::serde_helpers::hex_array")] + pub pq_fork_version: Version, + /// The epoch PQ activates at, or [`constants::FAR_FUTURE_EPOCH`] if it is + /// not scheduled. A consensus-specs feature fork, never scheduled on a + /// built-in network. + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub pq_fork_epoch: Epoch, // -- Time parameters --------------------------------------------------- /// Wall-clock seconds per slot. Deprecated in favor of @@ -411,6 +427,11 @@ pub struct Config { /// to [`Self::min_validator_withdrawability_delay`]. #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] pub min_builder_withdrawability_delay: Epoch, + /// PQ: epochs after the fork at which a validator that registered no PQ + /// key before it becomes withdrawable (`upgrade_to_pq`'s + /// `compute_validators`). + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub unregistered_validator_withdrawability_delay: Epoch, // -- Validator cycle ----------------------------------------------------- /// Score points added to a validator's inactivity score for each epoch it @@ -549,6 +570,12 @@ pub struct Config { pub attestation_subnet_count: u64, #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] pub attestation_subnet_extra_bits: u64, + /// PQ: how many attestation subnets the PQ fork's digests carry + /// (`ATTESTATION_SUBNET_COUNT_PQ`, `specs/_features/pq/p2p-interface.md`). + /// `attnets` keeps [`Self::attestation_subnet_count`] bits; only the first + /// this many may be set under a PQ digest. + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub attestation_subnet_count_pq: u64, #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] pub blob_sidecar_subnet_count: u64, #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] @@ -721,6 +748,10 @@ impl Config { fulu_fork_epoch: 411_392, gloas_fork_version: [0x07, 0x00, 0x00, 0x00], gloas_fork_epoch: constants::FAR_FUTURE_EPOCH, + eip8483_fork_version: [0xe8, 0x48, 0x30, 0x00], + eip8483_fork_epoch: constants::FAR_FUTURE_EPOCH, + pq_fork_version: [0xe0, 0x00, 0x00, 0x00], + pq_fork_epoch: constants::FAR_FUTURE_EPOCH, seconds_per_slot: 12, slot_duration_ms: 12_000, @@ -740,6 +771,7 @@ impl Config { payload_due_bps: 5_000, payload_attestation_due_bps: 7_500, min_builder_withdrawability_delay: 64, + unregistered_validator_withdrawability_delay: 16_384, inactivity_score_bias: 4, inactivity_score_recovery_rate: 16, @@ -783,6 +815,7 @@ impl Config { attestation_propagation_slot_range: 32, attestation_subnet_count: 64, attestation_subnet_extra_bits: 0, + attestation_subnet_count_pq: 16, blob_sidecar_subnet_count: 6, blob_sidecar_subnet_count_electra: 9, data_column_sidecar_subnet_count: 128, @@ -855,6 +888,10 @@ impl Config { fulu_fork_epoch: constants::FAR_FUTURE_EPOCH, gloas_fork_version: [0x07, 0x00, 0x00, 0x01], gloas_fork_epoch: constants::FAR_FUTURE_EPOCH, + eip8483_fork_version: [0xe8, 0x48, 0x30, 0x01], + eip8483_fork_epoch: constants::FAR_FUTURE_EPOCH, + pq_fork_version: [0xe0, 0x00, 0x00, 0x01], + pq_fork_epoch: constants::FAR_FUTURE_EPOCH, seconds_per_slot: 6, slot_duration_ms: 6_000, @@ -874,6 +911,7 @@ impl Config { payload_due_bps: 5_000, payload_attestation_due_bps: 7_500, min_builder_withdrawability_delay: 2, + unregistered_validator_withdrawability_delay: 16_384, inactivity_score_bias: 4, inactivity_score_recovery_rate: 16, @@ -904,6 +942,7 @@ impl Config { attestation_propagation_slot_range: 32, attestation_subnet_count: 64, attestation_subnet_extra_bits: 0, + attestation_subnet_count_pq: 16, blob_sidecar_subnet_count: 6, blob_sidecar_subnet_count_electra: 9, data_column_sidecar_subnet_count: 128, @@ -977,6 +1016,8 @@ impl Config { electra_fork_epoch: constants::FAR_FUTURE_EPOCH, fulu_fork_epoch: constants::FAR_FUTURE_EPOCH, gloas_fork_epoch: constants::FAR_FUTURE_EPOCH, + eip8483_fork_epoch: constants::FAR_FUTURE_EPOCH, + pq_fork_epoch: constants::FAR_FUTURE_EPOCH, ..Config::mainnet() } } @@ -1060,6 +1101,8 @@ impl Config { ForkName::Electra => self.electra_fork_version, ForkName::Fulu => self.fulu_fork_version, ForkName::Gloas => self.gloas_fork_version, + ForkName::Eip8483 => self.eip8483_fork_version, + ForkName::Pq => self.pq_fork_version, ForkName::Lean => lean_fork_unreachable("Config::fork_version"), } } @@ -1078,6 +1121,8 @@ impl Config { ForkName::Electra => self.electra_fork_epoch, ForkName::Fulu => self.fulu_fork_epoch, ForkName::Gloas => self.gloas_fork_epoch, + ForkName::Eip8483 => self.eip8483_fork_epoch, + ForkName::Pq => self.pq_fork_epoch, ForkName::Lean => lean_fork_unreachable("Config::fork_epoch"), } } @@ -1103,6 +1148,8 @@ impl Config { ForkName::Electra => self.electra_fork_epoch = epoch, ForkName::Fulu => self.fulu_fork_epoch = epoch, ForkName::Gloas => self.gloas_fork_epoch = epoch, + ForkName::Eip8483 => self.eip8483_fork_epoch = epoch, + ForkName::Pq => self.pq_fork_epoch = epoch, ForkName::Lean => lean_fork_unreachable("Config::with_fork_epoch"), } self @@ -1120,6 +1167,8 @@ impl Config { ForkName::Electra => self.electra_fork_version = version, ForkName::Fulu => self.fulu_fork_version = version, ForkName::Gloas => self.gloas_fork_version = version, + ForkName::Eip8483 => self.eip8483_fork_version = version, + ForkName::Pq => self.pq_fork_version = version, // Matches `fork_version`'s own arm: reaching this means a caller // dispatched on the wrong chain, which is a bug in the caller. ForkName::Lean => lean_fork_unreachable("Config::with_fork_version"), diff --git a/crates/common/types/src/beacon/constants.rs b/crates/common/types/src/beacon/constants.rs index 2b285c1b3..b105b696a 100644 --- a/crates/common/types/src/beacon/constants.rs +++ b/crates/common/types/src/beacon/constants.rs @@ -427,6 +427,79 @@ pub const PTC_TIMELINESS_INDEX: usize = 1; /// length every entry has, one slot per index above. pub const NUM_BLOCK_TIMELINESS_DEADLINES: usize = 2; +// --------------------------------------------------------------------------- +// EIP-8483 (consensus-specs `_features/eip8483`) +// --------------------------------------------------------------------------- + +/// The first byte identifying an execution layer request as a validator's PQ +/// registration (EIP-8483). +pub const VALIDATOR_PQ_REGISTRATION_REQUEST_TYPE: u8 = 0x05; +/// The first byte identifying an execution layer request as a builder's PQ +/// registration (EIP-8483). +pub const BUILDER_PQ_REGISTRATION_REQUEST_TYPE: u8 = 0x06; + +// --------------------------------------------------------------------------- +// PQ (consensus-specs `_features/pq`) +// --------------------------------------------------------------------------- + +/// How many validators a beacon committee's aggregation committee samples. +pub const AGGREGATORS_PER_COMMITTEE: u64 = 4; + +/// The encoded length of a leanXMSS public key: the top tree's root, then the +/// public parameter. +pub const XMSS_PUBKEY_SIZE: usize = 32; +/// The encoded length of a leanXMSS signature: the leaf index, then one +/// signature per layer. +pub const XMSS_SIGNATURE_SIZE: usize = 2404; + +/// The first leaves of every key, which a message without a leaf of its own (a +/// voluntary exit, a preregistration) may be signed at. +pub const XMSS_RESERVED_LEAVES: u64 = 1 << 15; +/// The leaves after the reserved ones: one attestation per target epoch. +pub const XMSS_ATTESTATION_LEAVES: u64 = 1 << 29; +/// The leaves after the attestation ones, which a builder signs its bids at, +/// any of them, each once. +pub const XMSS_BUILDER_BID_LEAVES: u64 = 1 << 30; +/// The leaves each slot owns past the bid leaves, one per `LEAF_OFFSET_*`. +pub const XMSS_LEAVES_PER_SLOT: u64 = 1 << 3; + +/// The depth of the tree under a validator's VRF root: a leaf per slot. +pub const VRF_TREE_DEPTH: usize = 31; + +/// The slot leaf a beacon block (and so a proposer slashing's headers) is +/// signed at. +pub const LEAF_OFFSET_BEACON_PROPOSER: u64 = 0; +/// The slot leaf a `SignedAggregateAndProof` is signed at. +pub const LEAF_OFFSET_AGGREGATE_AND_PROOF: u64 = 1; +/// The slot leaf a payload timeliness committee vote is signed at. +pub const LEAF_OFFSET_PTC_ATTESTER: u64 = 2; +/// The slot leaf `SignedProposerPreferences` is signed at. +pub const LEAF_OFFSET_PROPOSER_PREFERENCES: u64 = 3; +/// The slot leaf a `SignedExecutionPayloadEnvelope` is signed at. +pub const LEAF_OFFSET_EXECUTION_PAYLOAD_ENVELOPE: u64 = 4; + +/// The 16 zero bytes that precede an XMSS public key in a request's or pending +/// deposit's 48-byte key field. A BLS key never starts with them: its first +/// byte has the compression flag set. +pub const XMSS_DEPOSIT_PREFIX: [u8; 16] = [0; 16]; + +/// Appended to a BLS key to derive the XMSS key a validator or builder that +/// did not register takes at the fork. +pub const LEGACY_XMSS_PUBKEY_DOMAIN: u8 = 0x00; +/// Appended to a BLS key to derive the VRF root a validator that did not +/// register takes at the fork. +pub const LEGACY_VRF_ROOT_DOMAIN: u8 = 0x01; + +/// The first byte identifying an execution layer request as a validator +/// preregistration (EIP-8205, adopted by PQ). +pub const VALIDATOR_PREREGISTRATION_REQUEST_TYPE: u8 = 0x07; +/// The first byte identifying an execution layer request as a builder +/// preregistration. +pub const BUILDER_PREREGISTRATION_REQUEST_TYPE: u8 = 0x08; +/// The first byte identifying an execution layer request as a voluntary exit, +/// which PQ moves to the execution layer. +pub const VOLUNTARY_EXIT_REQUEST_TYPE: u8 = 0x09; + #[cfg(test)] mod tests { use super::*; diff --git a/crates/common/types/src/beacon/containers/eip8483.rs b/crates/common/types/src/beacon/containers/eip8483.rs new file mode 100644 index 000000000..9e77464fc --- /dev/null +++ b/crates/common/types/src/beacon/containers/eip8483.rs @@ -0,0 +1,400 @@ +//! Containers whose shape is specific to EIP-8483 (consensus-specs +//! `specs/_features/eip8483`, on the `pq` branch at +//! `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`). +//! +//! EIP-8483 adds two execution-layer requests with which validators and +//! builders register opaque post-quantum material: 64 bytes for a validator +//! (an XMSS public key and a VRF root, as the PQ fork reads them) and 32 for a +//! builder (an XMSS public key). The state keeps at most one entry per +//! validator, keyed by index, and one per builder, keyed by public key, since a +//! builder index is reused once its builder has withdrawn. This fork gives the +//! material no meaning beyond requiring each 32-byte part to be unique; the PQ +//! fork ([`super::pq`]) is what adopts it. +//! +//! Every other container is gloas's. [`ExecutionRequests`] gains the two +//! request lists, which reshapes every container holding one: +//! [`BeaconBlockBody`] (`parent_execution_requests`) and so +//! [`BeaconBlock`]/[`SignedBeaconBlock`], and [`ExecutionPayloadEnvelope`] and +//! so [`SignedExecutionPayloadEnvelope`]. [`BeaconState`] is gloas's with the +//! two registration lists appended. + +use libssz_derive::{HashTreeRoot, SszDecode, SszEncode}; +use libssz_types::ProgressiveList; + +use super::altair::{SyncAggregate, SyncCommittee}; +use super::fulu::ProposerLookahead; +use super::gloas::{ + Attestations, AttesterSlashings, BlsToExecutionChanges, BuilderDepositRequests, + BuilderExitRequests, BuilderIndex, BuilderPendingPayments, BuilderPendingWithdrawals, Builders, + ConsolidationRequests, DepositRequests, Deposits, EpochParticipation, ExecutionPayload, + ExecutionPayloadAvailability, ExecutionPayloadBid, InactivityScores, PayloadAttestations, + PayloadTimelinessCommitteeWindow, PendingConsolidations, PendingDeposits, + PendingPartialWithdrawals, ProposerSlashings, SignedExecutionPayloadBid, VoluntaryExits, + WithdrawalRequests, Withdrawals, +}; +use super::shared::{ + BeaconBlockHeader, BlockRoots, Checkpoint, Eth1Data, Eth1DataVotes, Fork, HistoricalRoots, + HistoricalSummaries, JustificationBits, ProgressiveBalances, ProgressiveValidators, + RandaoMixes, Slashings, StateRoots, +}; +use crate::beacon::primitives::{ + BlsPubkey, BlsSignature, Bytes32, Epoch, ExecutionAddress, ExecutionBlockHash, Gwei, Root, + Slot, ValidatorIndex, WithdrawalIndex, +}; + +/// A validator's registered PQ material: an XMSS public key, then a VRF root, +/// as the PQ fork reads them. +pub type PqData = [u8; 64]; + +/// The validator PQ registration requests of one execution payload. +pub type ValidatorPqRegistrationRequests = ProgressiveList; +/// The registered validator PQ material, at most one entry per validator. +pub type ValidatorPqRegistrations = ProgressiveList; +/// The builder PQ registration requests of one execution payload. +pub type BuilderPqRegistrationRequests = ProgressiveList; +/// The registered builder PQ material, at most one entry per builder. +pub type BuilderPqRegistrations = ProgressiveList; + +/// A validator's request to register PQ material. Its SSZ encoding is the +/// 132-byte record the validator predeploy contract returns. +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct ValidatorPqRegistrationRequest { + pub source_address: ExecutionAddress, + pub validator_pubkey: BlsPubkey, + #[serde(with = "crate::beacon::serde_helpers::hex_array")] + pub pq_data: PqData, +} + +/// One validator's registered PQ material. +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct ValidatorPqRegistration { + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub validator_index: ValidatorIndex, + #[serde(with = "crate::beacon::serde_helpers::hex_array")] + pub pq_data: PqData, +} + +/// A builder's request to register PQ material. Its SSZ encoding is the +/// 100-byte record the builder predeploy contract returns. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct BuilderPqRegistrationRequest { + pub source_address: ExecutionAddress, + pub builder_pubkey: BlsPubkey, + pub pq_data: Bytes32, +} + +/// One builder's registered PQ material, keyed by the builder's BLS key. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct BuilderPqRegistration { + pub builder_pubkey: BlsPubkey, + pub pq_data: Bytes32, +} + +/// Gloas's execution requests plus EIP-8483's two registration lists. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +#[ssz(progressive_container)] +pub struct ExecutionRequests { + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub deposits: DepositRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub withdrawals: WithdrawalRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub consolidations: ConsolidationRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_deposits: BuilderDepositRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_exits: BuilderExitRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub validator_pq_registrations: ValidatorPqRegistrationRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_pq_registrations: BuilderPqRegistrationRequests, +} + +/// Gloas's envelope, carrying EIP-8483's wider [`ExecutionRequests`]. +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +#[ssz(progressive_container)] +pub struct ExecutionPayloadEnvelope { + pub payload: ExecutionPayload, + pub execution_requests: ExecutionRequests, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub builder_index: BuilderIndex, + pub beacon_block_root: Root, + pub parent_beacon_block_root: Root, +} + +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SignedExecutionPayloadEnvelope { + pub message: ExecutionPayloadEnvelope, + pub signature: BlsSignature, +} + +/// Gloas's block body, with `parent_execution_requests` now EIP-8483's +/// [`ExecutionRequests`]. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +#[ssz(progressive_container)] +pub struct BeaconBlockBody { + pub randao_reveal: BlsSignature, + pub eth1_data: Eth1Data, + pub graffiti: Bytes32, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub proposer_slashings: ProposerSlashings, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub attester_slashings: AttesterSlashings, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub attestations: Attestations, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub deposits: Deposits, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub voluntary_exits: VoluntaryExits, + pub sync_aggregate: SyncAggregate, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub bls_to_execution_changes: BlsToExecutionChanges, + pub signed_execution_payload_bid: SignedExecutionPayloadBid, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub payload_attestations: PayloadAttestations, + pub parent_execution_requests: ExecutionRequests, +} + +impl BeaconBlockBody { + /// An empty body, for parity with every earlier fork's `empty()`. + pub fn empty() -> Self { + Self::default() + } +} + +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct BeaconBlock { + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub slot: Slot, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub proposer_index: ValidatorIndex, + pub parent_root: Root, + pub state_root: Root, + pub body: BeaconBlockBody, +} + +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SignedBeaconBlock { + pub message: BeaconBlock, + pub signature: BlsSignature, +} + +/// The EIP-8483 beacon state: gloas's 46 fields, then the two registration +/// lists, 48 in all, in the specification's order. +#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, SszEncode, SszDecode, HashTreeRoot)] +#[ssz(progressive_container)] +pub struct BeaconState { + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub genesis_time: u64, + pub genesis_validators_root: Root, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub slot: Slot, + pub fork: Fork, + pub latest_block_header: BeaconBlockHeader, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub block_roots: BlockRoots, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub state_roots: StateRoots, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub historical_roots: HistoricalRoots, + pub eth1_data: Eth1Data, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub eth1_data_votes: Eth1DataVotes, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub eth1_deposit_index: u64, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub validators: ProgressiveValidators, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub balances: ProgressiveBalances, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub randao_mixes: RandaoMixes, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub slashings: Slashings, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub previous_epoch_participation: EpochParticipation, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub current_epoch_participation: EpochParticipation, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub justification_bits: JustificationBits, + pub previous_justified_checkpoint: Checkpoint, + pub current_justified_checkpoint: Checkpoint, + pub finalized_checkpoint: Checkpoint, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub inactivity_scores: InactivityScores, + pub current_sync_committee: SyncCommittee, + pub next_sync_committee: SyncCommittee, + pub latest_block_hash: ExecutionBlockHash, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub next_withdrawal_index: WithdrawalIndex, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub next_withdrawal_validator_index: ValidatorIndex, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub historical_summaries: HistoricalSummaries, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub deposit_requests_start_index: u64, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub deposit_balance_to_consume: Gwei, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub exit_balance_to_consume: Gwei, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub earliest_exit_epoch: Epoch, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub consolidation_balance_to_consume: Gwei, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub earliest_consolidation_epoch: Epoch, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub pending_deposits: PendingDeposits, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub pending_partial_withdrawals: PendingPartialWithdrawals, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub pending_consolidations: PendingConsolidations, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub proposer_lookahead: ProposerLookahead, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builders: Builders, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub next_withdrawal_builder_index: BuilderIndex, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub execution_payload_availability: ExecutionPayloadAvailability, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_pending_payments: BuilderPendingPayments, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_pending_withdrawals: BuilderPendingWithdrawals, + pub latest_execution_payload_bid: ExecutionPayloadBid, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub payload_expected_withdrawals: Withdrawals, + #[serde(serialize_with = "crate::beacon::serde_helpers::nested_quoted_u64_seq::serialize")] + pub ptc_window: PayloadTimelinessCommitteeWindow, + // -- EIP-8483 -- + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub validator_pq_registrations: ValidatorPqRegistrations, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_pq_registrations: BuilderPqRegistrations, +} + +#[cfg(test)] +mod tests { + use libssz::{SszDecode as _, SszEncode as _}; + + use super::*; + + /// The predeploy contracts' record sizes the specification's notes give. + #[test] + fn request_records_have_the_predeploys_sizes() { + let validator = ValidatorPqRegistrationRequest { + source_address: Default::default(), + validator_pubkey: Default::default(), + pq_data: [7; 64], + }; + assert_eq!(validator.to_ssz().len(), 132); + assert_eq!(BuilderPqRegistrationRequest::default().to_ssz().len(), 100); + let decoded = ValidatorPqRegistrationRequest::from_ssz_bytes(&validator.to_ssz()).unwrap(); + assert_eq!(decoded, validator); + } +} diff --git a/crates/common/types/src/beacon/containers/gloas_family.rs b/crates/common/types/src/beacon/containers/gloas_family.rs new file mode 100644 index 000000000..284df7922 --- /dev/null +++ b/crates/common/types/src/beacon/containers/gloas_family.rs @@ -0,0 +1,193 @@ +//! Borrowed views of the gloas-family states: gloas, EIP-8483 and PQ. +//! +//! The three forks' states are separate structs, since each derives its own SSZ +//! shape, but they share every field the ePBS (EIP-7732) machinery reads: the +//! builder registry, the payment queues, the latest bid, the payload timeliness +//! committee window, the progressive pending queues. [`GloasFamily`] and +//! [`GloasFamilyMut`] borrow exactly those fields, at one type whichever of the +//! three states they come from, so the gloas state transition runs on all three +//! instead of existing once per fork. `fork` says which one it was, for the +//! few places the later forks change a rule. +//! +//! What is *not* here: `validators` and `balances` (the registry accessors on +//! [`BeaconState`](super::BeaconState) already cover every fork, and the PQ +//! element differs), the sync committees (the PQ fork removes them), and each +//! later fork's own tail (EIP-8483's registrations, PQ's preregistrations and +//! aggregation committee window), which only that fork's code reads. +//! +//! The PQ state's builders are a [`super::pq::Builders`], which keeps gloas's +//! records in memory (see that type's doc); the view borrows the gloas list +//! inside it. + +use super::fulu::ProposerLookahead; +use super::gloas::{ + BuilderIndex, BuilderPendingPayments, BuilderPendingWithdrawals, Builders, EpochParticipation, + ExecutionPayloadAvailability, ExecutionPayloadBid, InactivityScores, + PayloadTimelinessCommitteeWindow, PendingConsolidations, PendingDeposits, + PendingPartialWithdrawals, Withdrawals, +}; +use super::shared::{ + BeaconBlockHeader, BlockRoots, Checkpoint, Eth1Data, Eth1DataVotes, HistoricalRoots, + HistoricalSummaries, JustificationBits, RandaoMixes, Slashings, StateRoots, +}; +use super::{BeaconState, eip8483, gloas, pq}; +use crate::beacon::fork::ForkName; +use crate::beacon::primitives::{ + Epoch, ExecutionBlockHash, Gwei, Root, Slot, ValidatorIndex, WithdrawalIndex, +}; + +/// Declares both views over one field list, and their constructors from each +/// of the three states, so a field cannot be in one view and not the other. +macro_rules! gloas_family { + ($($field:ident: $ty:ty),* $(,)?) => { + /// The gloas-family fields of a gloas, EIP-8483 or PQ state. See the + /// module doc. + pub struct GloasFamily<'a> { + /// Which of the three forks the state is. + pub fork: ForkName, + /// The builder registry, gloas's records whichever fork. + pub builders: &'a Builders, + $(pub $field: &'a $ty,)* + } + + /// [`GloasFamily`], mutably. + pub struct GloasFamilyMut<'a> { + /// Which of the three forks the state is. + pub fork: ForkName, + /// The builder registry, gloas's records whichever fork. + pub builders: &'a mut Builders, + $(pub $field: &'a mut $ty,)* + } + + impl<'a> GloasFamily<'a> { + fn of_gloas(state: &'a gloas::BeaconState) -> Self { + Self { fork: ForkName::Gloas, builders: &state.builders, $($field: &state.$field,)* } + } + + fn of_eip8483(state: &'a eip8483::BeaconState) -> Self { + Self { fork: ForkName::Eip8483, builders: &state.builders, $($field: &state.$field,)* } + } + + fn of_pq(state: &'a pq::BeaconState) -> Self { + Self { fork: ForkName::Pq, builders: &state.builders.0, $($field: &state.$field,)* } + } + } + + impl<'a> GloasFamilyMut<'a> { + fn of_gloas(state: &'a mut gloas::BeaconState) -> Self { + Self { + fork: ForkName::Gloas, + builders: &mut state.builders, + $($field: &mut state.$field,)* + } + } + + fn of_eip8483(state: &'a mut eip8483::BeaconState) -> Self { + Self { + fork: ForkName::Eip8483, + builders: &mut state.builders, + $($field: &mut state.$field,)* + } + } + + fn of_pq(state: &'a mut pq::BeaconState) -> Self { + Self { + fork: ForkName::Pq, + builders: &mut state.builders.0, + $($field: &mut state.$field,)* + } + } + + /// The same view, read-only, for a helper that takes a + /// [`GloasFamily`] while this one is held. + pub fn as_ref(&self) -> GloasFamily<'_> { + GloasFamily { fork: self.fork, builders: &*self.builders, $($field: &*self.$field,)* } + } + } + }; +} + +gloas_family!( + genesis_time: u64, + genesis_validators_root: Root, + slot: Slot, + latest_block_header: BeaconBlockHeader, + block_roots: BlockRoots, + state_roots: StateRoots, + historical_roots: HistoricalRoots, + eth1_data: Eth1Data, + eth1_data_votes: Eth1DataVotes, + eth1_deposit_index: u64, + randao_mixes: RandaoMixes, + slashings: Slashings, + previous_epoch_participation: EpochParticipation, + current_epoch_participation: EpochParticipation, + justification_bits: JustificationBits, + previous_justified_checkpoint: Checkpoint, + current_justified_checkpoint: Checkpoint, + finalized_checkpoint: Checkpoint, + inactivity_scores: InactivityScores, + latest_block_hash: ExecutionBlockHash, + next_withdrawal_index: WithdrawalIndex, + next_withdrawal_validator_index: ValidatorIndex, + historical_summaries: HistoricalSummaries, + deposit_requests_start_index: u64, + deposit_balance_to_consume: Gwei, + exit_balance_to_consume: Gwei, + earliest_exit_epoch: Epoch, + consolidation_balance_to_consume: Gwei, + earliest_consolidation_epoch: Epoch, + pending_deposits: PendingDeposits, + pending_partial_withdrawals: PendingPartialWithdrawals, + pending_consolidations: PendingConsolidations, + proposer_lookahead: ProposerLookahead, + next_withdrawal_builder_index: BuilderIndex, + execution_payload_availability: ExecutionPayloadAvailability, + builder_pending_payments: BuilderPendingPayments, + builder_pending_withdrawals: BuilderPendingWithdrawals, + latest_execution_payload_bid: ExecutionPayloadBid, + payload_expected_withdrawals: Withdrawals, + ptc_window: PayloadTimelinessCommitteeWindow, +); + +macro_rules! from_state { + ($state:ty, $ctor:ident) => { + impl<'a> From<&'a $state> for GloasFamily<'a> { + fn from(state: &'a $state) -> Self { + GloasFamily::$ctor(state) + } + } + + impl<'a> From<&'a mut $state> for GloasFamilyMut<'a> { + fn from(state: &'a mut $state) -> Self { + GloasFamilyMut::$ctor(state) + } + } + }; +} + +from_state!(gloas::BeaconState, of_gloas); +from_state!(eip8483::BeaconState, of_eip8483); +from_state!(pq::BeaconState, of_pq); + +impl BeaconState { + /// The gloas-family fields, or `None` before gloas (and for lean). + pub fn gloas_family(&self) -> Option> { + match self { + BeaconState::Gloas(state) => Some(GloasFamily::of_gloas(state)), + BeaconState::Eip8483(state) => Some(GloasFamily::of_eip8483(state)), + BeaconState::Pq(state) => Some(GloasFamily::of_pq(state)), + _ => None, + } + } + + /// [`Self::gloas_family`], mutably. + pub fn gloas_family_mut(&mut self) -> Option> { + match self { + BeaconState::Gloas(state) => Some(GloasFamilyMut::of_gloas(state)), + BeaconState::Eip8483(state) => Some(GloasFamilyMut::of_eip8483(state)), + BeaconState::Pq(state) => Some(GloasFamilyMut::of_pq(state)), + _ => None, + } + } +} diff --git a/crates/common/types/src/beacon/containers/mod.rs b/crates/common/types/src/beacon/containers/mod.rs index f236c28c2..dbbb65cfa 100644 --- a/crates/common/types/src/beacon/containers/mod.rs +++ b/crates/common/types/src/beacon/containers/mod.rs @@ -43,12 +43,16 @@ pub mod altair; pub mod bellatrix; pub mod capella; pub mod deneb; +pub mod eip8483; pub mod electra; pub mod fulu; pub mod gloas; +pub mod gloas_family; pub mod phase0; +pub mod pq; pub mod shared; +pub use gloas_family::{GloasFamily, GloasFamilyMut}; pub use shared::*; use libssz::{SszDecode as _, SszEncode as _}; @@ -57,7 +61,7 @@ use crate::beacon::error::{Error, Result}; use crate::beacon::fork::ForkName; use crate::beacon::primitives::{ BlsSignature, Bytes32, CommitteeIndex, Epoch, ExecutionBlockHash, Gwei, HashTreeRoot as _, - ParticipationFlags, Root, Slot, ValidatorIndex, WithdrawalIndex, + ParticipationFlags, Root, Slot, ValidatorIndex, WithdrawalIndex, XmssSignature, }; use crate::beacon::{beacon_value_unreachable, lean_block_unreachable, lean_state_unreachable}; @@ -78,6 +82,8 @@ macro_rules! dispatch_state { BeaconState::Electra($state) => $body, BeaconState::Fulu($state) => $body, BeaconState::Gloas($state) => $body, + BeaconState::Eip8483($state) => $body, + BeaconState::Pq($state) => $body, BeaconState::Lean(_) => lean_state_unreachable($function), } }; @@ -108,6 +114,8 @@ macro_rules! dispatch_state_including_lean { BeaconState::Electra($state) => $body, BeaconState::Fulu($state) => $body, BeaconState::Gloas($state) => $body, + BeaconState::Eip8483($state) => $body, + BeaconState::Pq($state) => $body, BeaconState::Lean($state) => $body, } }; @@ -153,6 +161,8 @@ macro_rules! dispatch_block { SignedBeaconBlock::Electra($block) => $body, SignedBeaconBlock::Fulu($block) => $body, SignedBeaconBlock::Gloas($block) => $body, + SignedBeaconBlock::Eip8483($block) => $body, + SignedBeaconBlock::Pq($block) => $body, SignedBeaconBlock::Lean(_) => lean_block_unreachable($function), } }; @@ -178,6 +188,8 @@ macro_rules! dispatch_block_including_lean { SignedBeaconBlock::Electra($block) => $body, SignedBeaconBlock::Fulu($block) => $body, SignedBeaconBlock::Gloas($block) => $body, + SignedBeaconBlock::Eip8483($block) => $body, + SignedBeaconBlock::Pq($block) => $body, SignedBeaconBlock::Lean($block) => $body, } }; @@ -204,6 +216,12 @@ pub enum BeaconState { Electra(electra::BeaconState), Fulu(fulu::BeaconState), Gloas(gloas::BeaconState), + /// EIP-8483, a consensus-specs feature fork: gloas plus PQ registrations. + Eip8483(eip8483::BeaconState), + /// The post-quantum fork. Its registry element is [`pq::Validator`], which + /// the registry accessors hand out as the [`Validator`] record every other + /// fork stores (see [`pq`]'s module doc). + Pq(pq::BeaconState), Lean(crate::state::State), } @@ -248,6 +266,8 @@ impl serde::Serialize for BeaconState { BeaconState::Electra(state) => state.serialize(serializer), BeaconState::Fulu(state) => state.serialize(serializer), BeaconState::Gloas(state) => state.serialize(serializer), + BeaconState::Eip8483(state) => state.serialize(serializer), + BeaconState::Pq(state) => state.serialize(serializer), BeaconState::Lean(_) => Err(serde::ser::Error::custom( "a lean state has no JSON encoding: /lean/v0/states/finalized serves SSZ only", )), @@ -289,6 +309,8 @@ impl BeaconState { BeaconState::Electra(_) => ForkName::Electra, BeaconState::Fulu(_) => ForkName::Fulu, BeaconState::Gloas(_) => ForkName::Gloas, + BeaconState::Eip8483(_) => ForkName::Eip8483, + BeaconState::Pq(_) => ForkName::Pq, BeaconState::Lean(_) => ForkName::Lean, } } @@ -354,6 +376,10 @@ impl BeaconState { ForkName::Gloas => Ok(BeaconState::Gloas(gloas::BeaconState::from_ssz_bytes( bytes, )?)), + ForkName::Eip8483 => Ok(BeaconState::Eip8483(eip8483::BeaconState::from_ssz_bytes( + bytes, + )?)), + ForkName::Pq => Ok(BeaconState::Pq(pq::BeaconState::from_ssz_bytes(bytes)?)), ForkName::Lean => Ok(BeaconState::Lean(crate::state::State::from_ssz_bytes( bytes, )?)), @@ -419,6 +445,7 @@ impl BeaconState { ::fixed_size(), ::fixed_size(), ::fixed_size(), + ::fixed_size(), ]), } } @@ -531,6 +558,14 @@ tree_fields!( gloas::BeaconState => validators, balances, block_roots, state_roots, historical_roots, eth1_data_votes, randao_mixes, slashings, historical_summaries, previous_epoch_participation, current_epoch_participation, pending_deposits ); +tree_fields!( + eip8483::BeaconState => validators, balances, block_roots, state_roots, historical_roots, eth1_data_votes, randao_mixes, slashings, + historical_summaries, previous_epoch_participation, current_epoch_participation, pending_deposits +); +tree_fields!( + pq::BeaconState => validators, balances, block_roots, state_roots, historical_roots, eth1_data_votes, randao_mixes, slashings, + historical_summaries, previous_epoch_participation, current_epoch_participation, pending_deposits +); /// Generates read and write accessors for state fields that every fork shares. /// @@ -670,6 +705,9 @@ impl BeaconState { enum Registry<'a> { Bounded(&'a Validators, &'a Balances), Progressive(&'a ProgressiveValidators, &'a ProgressiveBalances), + /// The PQ fork's registry, whose element wraps the shared record with the + /// validator's VRF root (see [`pq::Validator`]). + Pq(&'a pq::Validators, &'a ProgressiveBalances), } /// [`Registry`], mutably. A separate type rather than a lifetime trick on the @@ -679,6 +717,7 @@ enum Registry<'a> { enum RegistryMut<'a> { Bounded(&'a mut Validators, &'a mut Balances), Progressive(&'a mut ProgressiveValidators, &'a mut ProgressiveBalances), + Pq(&'a mut pq::Validators, &'a mut ProgressiveBalances), } /// One of a state's participation lists, whichever list kind its fork uses. @@ -1009,6 +1048,70 @@ impl<'a> Iterator for ScoresIter<'a> { impl ExactSizeIterator for ScoresIter<'_> {} +/// The iterator behind [`BeaconState::iter_validators`]: either list kind of +/// shared records, or the PQ registry read as its elements' shared records. +enum ValidatorIter<'a> { + Shared(RegistryIter<'a, Validator, std::collections::BTreeMap>), + Pq( + ethlambda_ssz_tree::ProgressiveIter< + 'a, + pq::Validator, + std::collections::BTreeMap, + >, + ), +} + +impl<'a> Iterator for ValidatorIter<'a> { + type Item = &'a Validator; + + fn next(&mut self) -> Option<&'a Validator> { + match self { + Self::Shared(it) => it.next(), + Self::Pq(it) => it.next().map(|validator| &validator.record), + } + } + + fn size_hint(&self) -> (usize, Option) { + match self { + Self::Shared(it) => it.size_hint(), + Self::Pq(it) => it.size_hint(), + } + } +} + +impl ExactSizeIterator for ValidatorIter<'_> {} + +/// One validator in [`BeaconState::try_update_validators_with_balances`]'s +/// write cursor, whichever registry kind the state keeps: reads go through +/// `Deref`, writes through [`ValidatorCow::make_mut`], and both see the shared +/// record, the PQ element's included. +pub enum ValidatorCow<'c> { + Shared(ethlambda_ssz_tree::ElemCow<'c, Validator>), + Pq(ethlambda_ssz_tree::ElemCow<'c, pq::Validator>), +} + +impl core::ops::Deref for ValidatorCow<'_> { + type Target = Validator; + + fn deref(&self) -> &Validator { + match self { + Self::Shared(cow) => cow, + Self::Pq(cow) => &cow.record, + } + } +} + +impl ValidatorCow<'_> { + /// The record, to be written. See `ElemCow::make_mut` for what the first + /// write costs. + pub fn make_mut(&mut self) -> &mut Validator { + match self { + Self::Shared(cow) => cow.make_mut(), + Self::Pq(cow) => &mut cow.make_mut().record, + } + } +} + /// An iterator over either registry list kind, so [`BeaconState::iter_validators`] /// and [`BeaconState::iter_balances`] can promise one return type across every /// fork despite [`Registry`]'s two underlying list types having different @@ -1062,8 +1165,11 @@ impl BeaconState { /// descent, where this walks the registry once, sequentially. pub fn iter_validators(&self) -> impl ExactSizeIterator { match self.registry() { - Some(Registry::Bounded(v, _)) => RegistryIter::Bounded(v.iter()), - Some(Registry::Progressive(v, _)) => RegistryIter::Progressive(v.iter()), + Some(Registry::Bounded(v, _)) => ValidatorIter::Shared(RegistryIter::Bounded(v.iter())), + Some(Registry::Progressive(v, _)) => { + ValidatorIter::Shared(RegistryIter::Progressive(v.iter())) + } + Some(Registry::Pq(v, _)) => ValidatorIter::Pq(v.iter()), None => lean_state_unreachable("iter_validators"), } } @@ -1073,8 +1179,13 @@ impl BeaconState { /// registry's appended tail that must not walk the part before it. pub fn iter_validators_from(&self, start: usize) -> impl ExactSizeIterator { match self.registry() { - Some(Registry::Bounded(v, _)) => RegistryIter::Bounded(v.iter_from(start)), - Some(Registry::Progressive(v, _)) => RegistryIter::Progressive(v.iter_from(start)), + Some(Registry::Bounded(v, _)) => { + ValidatorIter::Shared(RegistryIter::Bounded(v.iter_from(start))) + } + Some(Registry::Progressive(v, _)) => { + ValidatorIter::Shared(RegistryIter::Progressive(v.iter_from(start))) + } + Some(Registry::Pq(v, _)) => ValidatorIter::Pq(v.iter_from(start)), None => lean_state_unreachable("iter_validators_from"), } } @@ -1084,7 +1195,9 @@ impl BeaconState { pub fn iter_balances(&self) -> impl ExactSizeIterator { match self.registry() { Some(Registry::Bounded(_, b)) => RegistryIter::Bounded(b.iter()), - Some(Registry::Progressive(_, b)) => RegistryIter::Progressive(b.iter()), + Some(Registry::Progressive(_, b) | Registry::Pq(_, b)) => { + RegistryIter::Progressive(b.iter()) + } None => lean_state_unreachable("iter_balances"), } .copied() @@ -1095,9 +1208,12 @@ impl BeaconState { /// A named error rather than an `Option`, since the specification indexes the /// registry in many places and an out-of-range index is always a fault. pub fn validator(&self, index: ValidatorIndex) -> Result<&Validator> { - dispatch_state!(self, "validator", |state| state - .validators - .get(index as usize)) + match self.registry() { + Some(Registry::Bounded(v, _)) => v.get(index as usize), + Some(Registry::Progressive(v, _)) => v.get(index as usize), + Some(Registry::Pq(v, _)) => v.get(index as usize).map(|validator| &validator.record), + None => lean_state_unreachable("validator"), + } .ok_or(Error::UnknownValidator(index)) } @@ -1105,12 +1221,33 @@ impl BeaconState { /// the underlying list clones the element into its update buffer on this /// call alone, whether or not the caller goes on to change it. pub fn validator_mut(&mut self, index: ValidatorIndex) -> Result<&mut Validator> { - dispatch_state!(self, "validator_mut", |state| state - .validators - .get_mut(index as usize)) + match self.registry_mut() { + Some(RegistryMut::Bounded(v, _)) => v.get_mut(index as usize), + Some(RegistryMut::Progressive(v, _)) => v.get_mut(index as usize), + Some(RegistryMut::Pq(v, _)) => v + .get_mut(index as usize) + .map(|validator| &mut validator.record), + None => lean_state_unreachable("validator_mut"), + } .ok_or(Error::UnknownValidator(index)) } + /// The PQ registry element at `index`: the shared record and the + /// validator's VRF root. [`Error::UnsupportedForFork`] before PQ. + pub fn pq_validator(&self, index: ValidatorIndex) -> Result<&pq::Validator> { + match self { + BeaconState::Pq(state) => state + .validators + .get(index as usize) + .ok_or(Error::UnknownValidator(index)), + BeaconState::Lean(_) => lean_state_unreachable("pq_validator"), + other => Err(Error::UnsupportedForFork { + function: "pq_validator", + fork: other.fork_name(), + }), + } + } + /// The balance of the validator at `index`. pub fn balance(&self, index: ValidatorIndex) -> Result { dispatch_state!(self, "balance", |state| state @@ -1136,12 +1273,45 @@ impl BeaconState { /// forks still push to them separately, through /// [`Self::altair_validator_lists_mut`], since this function only owns /// the two lists every fork has. + /// + /// [`Error::UnsupportedForFork`] on a PQ state, whose element also carries + /// a VRF root: [`Self::push_pq_validator`] appends there. pub fn push_validator(&mut self, validator: Validator, balance: Gwei) -> Result<()> { - dispatch_state!(self, "push_validator", |state| { - state.validators.push(validator)?; - state.balances.push(balance)?; - Ok(()) - }) + match self.registry_mut() { + Some(RegistryMut::Bounded(v, b)) => { + v.push(validator)?; + b.push(balance)?; + } + Some(RegistryMut::Progressive(v, b)) => { + v.push(validator)?; + b.push(balance)?; + } + Some(RegistryMut::Pq(..)) => { + return Err(Error::UnsupportedForFork { + function: "push_validator", + fork: ForkName::Pq, + }); + } + None => lean_state_unreachable("push_validator"), + } + Ok(()) + } + + /// [`Self::push_validator`] for the PQ registry, whose element carries the + /// validator's VRF root beside the shared record. + pub fn push_pq_validator(&mut self, validator: pq::Validator, balance: Gwei) -> Result<()> { + match self { + BeaconState::Pq(state) => { + state.validators.push(validator)?; + state.balances.push(balance)?; + Ok(()) + } + BeaconState::Lean(_) => lean_state_unreachable("push_pq_validator"), + other => Err(Error::UnsupportedForFork { + function: "push_pq_validator", + fork: other.fork_name(), + }), + } } /// The registry's own root: the value genesis records as @@ -1163,6 +1333,7 @@ impl BeaconState { match (self.registry(), other.registry()) { (Some(Registry::Bounded(v, _)), Some(Registry::Bounded(ov, _))) => v.ptr_eq(ov), (Some(Registry::Progressive(v, _)), Some(Registry::Progressive(ov, _))) => v.ptr_eq(ov), + (Some(Registry::Pq(v, _)), Some(Registry::Pq(ov, _))) => v.ptr_eq(ov), _ => false, } } @@ -1185,6 +1356,8 @@ impl BeaconState { BeaconState::Electra(s) => Some(Registry::Bounded(&s.validators, &s.balances)), BeaconState::Fulu(s) => Some(Registry::Bounded(&s.validators, &s.balances)), BeaconState::Gloas(s) => Some(Registry::Progressive(&s.validators, &s.balances)), + BeaconState::Eip8483(s) => Some(Registry::Progressive(&s.validators, &s.balances)), + BeaconState::Pq(s) => Some(Registry::Pq(&s.validators, &s.balances)), BeaconState::Lean(_) => None, } } @@ -1214,6 +1387,10 @@ impl BeaconState { BeaconState::Gloas(s) => { Some(RegistryMut::Progressive(&mut s.validators, &mut s.balances)) } + BeaconState::Eip8483(s) => { + Some(RegistryMut::Progressive(&mut s.validators, &mut s.balances)) + } + BeaconState::Pq(s) => Some(RegistryMut::Pq(&mut s.validators, &mut s.balances)), BeaconState::Lean(_) => None, } } @@ -1281,12 +1458,17 @@ impl BeaconState { v.rebase_on(bv); b.rebase_on(bb); } + (Some(RegistryMut::Pq(v, b)), Registry::Pq(bv, bb)) => { + v.rebase_on(bv); + b.rebase_on(bb); + } // Self is lean (no registry at all), or the two states disagree // on list kind: exactly the fork-boundary case this function's // own doc names as a no-op. (None, _) - | (Some(RegistryMut::Bounded(..)), Registry::Progressive(..)) - | (Some(RegistryMut::Progressive(..)), Registry::Bounded(..)) => {} + | (Some(RegistryMut::Bounded(..)), _) + | (Some(RegistryMut::Progressive(..)), _) + | (Some(RegistryMut::Pq(..)), _) => {} } self.block_roots_mut().rebase_on(base.block_roots()); self.state_roots_mut().rebase_on(base.state_roots()); @@ -1313,6 +1495,7 @@ impl BeaconState { match self.registry() { Some(Registry::Bounded(v, b)) => v.has_pending_updates() || b.has_pending_updates(), Some(Registry::Progressive(v, b)) => v.has_pending_updates() || b.has_pending_updates(), + Some(Registry::Pq(v, b)) => v.has_pending_updates() || b.has_pending_updates(), None => false, } } @@ -1347,28 +1530,26 @@ impl BeaconState { previous.rebase_on(base_previous); current.rebase_on(base_current); } - } else if let (BeaconState::Gloas(state), BeaconState::Gloas(base)) = (self, base) { + } else if let (Some(state), Some(base)) = (self.gloas_family_mut(), base.gloas_family()) { state .previous_epoch_participation - .rebase_on(&base.previous_epoch_participation); + .rebase_on(base.previous_epoch_participation); state .current_epoch_participation - .rebase_on(&base.current_epoch_participation); + .rebase_on(base.current_epoch_participation); } } /// Shares `pending_deposits` with `base`'s where both keep the same list /// kind (electra and fulu, or both gloas). fn rebase_pending_deposits_on(&mut self, base: &BeaconState) { + if let (Some(state), Some(base)) = (self.gloas_family_mut(), base.gloas_family()) { + state.pending_deposits.rebase_on(base.pending_deposits); + return; + } let base_deposits = match base { BeaconState::Electra(state) => &state.pending_deposits, BeaconState::Fulu(state) => &state.pending_deposits, - BeaconState::Gloas(base) => { - if let BeaconState::Gloas(state) = self { - state.pending_deposits.rebase_on(&base.pending_deposits); - } - return; - } _ => return, }; match self { @@ -1388,6 +1569,8 @@ impl BeaconState { BeaconState::Electra(state) => &state.historical_summaries, BeaconState::Fulu(state) => &state.historical_summaries, BeaconState::Gloas(state) => &state.historical_summaries, + BeaconState::Eip8483(state) => &state.historical_summaries, + BeaconState::Pq(state) => &state.historical_summaries, BeaconState::Phase0(_) | BeaconState::Altair(_) | BeaconState::Bellatrix(_) @@ -1399,6 +1582,8 @@ impl BeaconState { BeaconState::Electra(state) => &mut state.historical_summaries, BeaconState::Fulu(state) => &mut state.historical_summaries, BeaconState::Gloas(state) => &mut state.historical_summaries, + BeaconState::Eip8483(state) => &mut state.historical_summaries, + BeaconState::Pq(state) => &mut state.historical_summaries, BeaconState::Phase0(_) | BeaconState::Altair(_) | BeaconState::Bellatrix(_) @@ -1435,26 +1620,29 @@ impl BeaconState { /// list is an error. pub fn try_update_validators_with_balances( &mut self, - mut f: impl FnMut( - &mut ethlambda_ssz_tree::ElemCow<'_, Validator>, - Gwei, - ) -> core::result::Result<(), E>, + mut f: impl FnMut(&mut ValidatorCow<'_>, Gwei) -> core::result::Result<(), E>, ) -> core::result::Result<(), E> { - dispatch_state!( - self, - "BeaconState::try_update_validators_with_balances", - |state| { - let mut balances = state.balances.iter(); - let mut pass = state.validators.iter_cow(); - while let Some(mut validator) = pass.next_cow() { + // One loop per registry kind: the cursors' element types differ, and + // `ValidatorCow` is what makes the closure see one type. + macro_rules! pass { + ($validators:expr, $balances:expr, $wrap:path) => {{ + let mut balances = $balances.iter(); + let mut pass = $validators.iter_cow(); + while let Some(validator) = pass.next_cow() { let Some(&balance) = balances.next() else { break; }; - f(&mut validator, balance)?; + f(&mut $wrap(validator), balance)?; } Ok(()) - } - ) + }}; + } + match self.registry_mut() { + Some(RegistryMut::Bounded(v, b)) => pass!(v, b, ValidatorCow::Shared), + Some(RegistryMut::Progressive(v, b)) => pass!(v, b, ValidatorCow::Shared), + Some(RegistryMut::Pq(v, b)) => pass!(v, b, ValidatorCow::Pq), + None => lean_state_unreachable("BeaconState::try_update_validators_with_balances"), + } } /// The randao mix for `epoch`, which the specification indexes modulo the @@ -1486,7 +1674,7 @@ impl BeaconState { state.next_withdrawal_index, state.next_withdrawal_validator_index, ), - carried_by: [Capella, Deneb, Electra, Fulu, Gloas], + carried_by: [Capella, Deneb, Electra, Fulu, Gloas, Eip8483, Pq], absent_from: [Phase0, Altair, Bellatrix], ) } @@ -1500,7 +1688,7 @@ impl BeaconState { &mut state.next_withdrawal_index, &mut state.next_withdrawal_validator_index, ), - carried_by: [Capella, Deneb, Electra, Fulu, Gloas], + carried_by: [Capella, Deneb, Electra, Fulu, Gloas, Eip8483, Pq], absent_from: [Phase0, Altair, Bellatrix], ) } @@ -1550,7 +1738,7 @@ impl BeaconState { ParticipationRef::from(&state.current_epoch_participation), InactivityScoresRef::from(&state.inactivity_scores), ), - carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas], + carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas, Eip8483, Pq], absent_from: [Phase0], ) } @@ -1604,7 +1792,7 @@ impl BeaconState { &mut state.inactivity_scores, ), carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu], - absent_from: [Phase0, Gloas], + absent_from: [Phase0, Gloas, Eip8483, Pq], ) } @@ -1636,7 +1824,7 @@ impl BeaconState { .get_mut(index) .ok_or(Error::IndexOutOfBounds { index, len })? }, - carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas], + carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas, Eip8483, Pq], absent_from: [Phase0], ) } @@ -1663,7 +1851,7 @@ impl BeaconState { } else { &mut state.previous_epoch_participation }), - carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas], + carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas, Eip8483, Pq], absent_from: [Phase0], ) } @@ -1684,8 +1872,8 @@ impl BeaconState { self, "BeaconState::sync_committees", |state| (&state.current_sync_committee, &state.next_sync_committee), - carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas], - absent_from: [Phase0], + carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas, Eip8483], + absent_from: [Phase0, Pq], ) } @@ -1707,8 +1895,8 @@ impl BeaconState { &mut state.current_sync_committee, &mut state.next_sync_committee, ), - carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas], - absent_from: [Phase0], + carried_by: [Altair, Bellatrix, Capella, Deneb, Electra, Fulu, Gloas, Eip8483], + absent_from: [Phase0, Pq], ) } } @@ -1927,6 +2115,13 @@ pub enum SignedBeaconBlock { /// [`electra::SignedBeaconBlock`] instead of a `fulu` type. Fulu(electra::SignedBeaconBlock), Gloas(gloas::SignedBeaconBlock), + Eip8483(eip8483::SignedBeaconBlock), + /// The PQ fork's block, signed with leanXMSS: [`SignedBeaconBlock::signature`] + /// does not answer for it, [`SignedBeaconBlock::xmss_signature`] does. + /// + /// Boxed: its 2404-byte signature is inline, and unboxed it would make + /// every fork's block several kilobytes. + Pq(Box), /// The Lean consensus protocol's block. /// @@ -1982,7 +2177,12 @@ impl SignedBeaconBlock { /// (`is_payload_verified(store, root)`) rather than this accessor. pub fn execution_block_hash(&self) -> Option { match self { - Self::Phase0(_) | Self::Altair(_) | Self::Gloas(_) | Self::Lean(_) => None, + Self::Phase0(_) + | Self::Altair(_) + | Self::Gloas(_) + | Self::Eip8483(_) + | Self::Pq(_) + | Self::Lean(_) => None, Self::Bellatrix(block) => Some(block.message.body.execution_payload.block_hash), Self::Capella(block) => Some(block.message.body.execution_payload.block_hash), Self::Deneb(block) => Some(block.message.body.execution_payload.block_hash), @@ -2037,6 +2237,20 @@ impl SignedBeaconBlock { .message .blob_kzg_commitments .len(), + Self::Eip8483(block) => block + .message + .body + .signed_execution_payload_bid + .message + .blob_kzg_commitments + .len(), + Self::Pq(block) => block + .message + .body + .signed_execution_payload_bid + .message + .blob_kzg_commitments + .len(), } } @@ -2050,7 +2264,12 @@ impl SignedBeaconBlock { /// would describe. pub fn execution_payload_timestamp(&self) -> Option { match self { - Self::Phase0(_) | Self::Altair(_) | Self::Gloas(_) | Self::Lean(_) => None, + Self::Phase0(_) + | Self::Altair(_) + | Self::Gloas(_) + | Self::Eip8483(_) + | Self::Pq(_) + | Self::Lean(_) => None, Self::Bellatrix(block) => Some(block.message.body.execution_payload.timestamp), Self::Capella(block) => Some(block.message.body.execution_payload.timestamp), Self::Deneb(block) => Some(block.message.body.execution_payload.timestamp), @@ -2076,6 +2295,8 @@ impl SignedBeaconBlock { SignedBeaconBlock::Electra(_) => ForkName::Electra, SignedBeaconBlock::Fulu(_) => ForkName::Fulu, SignedBeaconBlock::Gloas(_) => ForkName::Gloas, + SignedBeaconBlock::Eip8483(_) => ForkName::Eip8483, + SignedBeaconBlock::Pq(_) => ForkName::Pq, SignedBeaconBlock::Lean(_) => ForkName::Lean, } } @@ -2112,6 +2333,12 @@ impl SignedBeaconBlock { ForkName::Gloas => Ok(SignedBeaconBlock::Gloas( gloas::SignedBeaconBlock::from_ssz_bytes(bytes)?, )), + ForkName::Eip8483 => Ok(SignedBeaconBlock::Eip8483( + eip8483::SignedBeaconBlock::from_ssz_bytes(bytes)?, + )), + ForkName::Pq => Ok(SignedBeaconBlock::Pq(Box::new( + pq::SignedBeaconBlock::from_ssz_bytes(bytes)?, + ))), ForkName::Lean => Ok(SignedBeaconBlock::Lean( crate::block::SignedBlock::from_ssz_bytes(bytes)?, )), @@ -2207,11 +2434,76 @@ signed_beacon_block_accessors!( (parent_root, Root), (state_root, Root), ], - outer: [ - (signature, BlsSignature), - ], + outer: [], ); +impl SignedBeaconBlock { + /// The proposer's BLS signature, at every fork that has one. + /// + /// # Panics + /// + /// On [`SignedBeaconBlock::Pq`], whose block is signed with leanXMSS + /// ([`SignedBeaconBlock::xmss_signature`]), and on + /// [`SignedBeaconBlock::Lean`]. Every BLS verification path dispatches on + /// the fork before it gets here, so reaching either arm is a caller bug, + /// named the way `dispatch_block!`'s lean arm names one. + pub fn signature(&self) -> BlsSignature { + match self { + SignedBeaconBlock::Pq(_) => { + crate::beacon::xmss_fork_has_no_bls("SignedBeaconBlock::signature") + } + other => dispatch_block!(other, "signature", |block| bls_signature_of(block)), + } + } + + /// The proposer's leanXMSS signature: `Some` only for + /// [`SignedBeaconBlock::Pq`]. + pub fn xmss_signature(&self) -> Option<&XmssSignature> { + match self { + SignedBeaconBlock::Pq(block) => Some(&block.signature), + _ => None, + } + } +} + +/// The `signature` field of a block whose fork signs with BLS. A trait rather +/// than a field read in `dispatch_block!`'s body, since the PQ block's field +/// has another type and the macro inserts one body into every arm. +trait BlsSigned { + fn bls_signature(&self) -> BlsSignature; +} + +macro_rules! bls_signed { + ($($ty:ty),+ $(,)?) => { + $(impl BlsSigned for $ty { + fn bls_signature(&self) -> BlsSignature { + self.signature + } + })+ + }; +} + +bls_signed!( + phase0::SignedBeaconBlock, + altair::SignedBeaconBlock, + bellatrix::SignedBeaconBlock, + capella::SignedBeaconBlock, + deneb::SignedBeaconBlock, + electra::SignedBeaconBlock, + gloas::SignedBeaconBlock, + eip8483::SignedBeaconBlock, +); + +impl BlsSigned for Box { + fn bls_signature(&self) -> BlsSignature { + crate::beacon::xmss_fork_has_no_bls("SignedBeaconBlock::signature") + } +} + +fn bls_signature_of(block: &impl BlsSigned) -> BlsSignature { + block.bls_signature() +} + /// A data column sidecar in either shape: fulu's carries a signed header and /// an inclusion proof, gloas's names its block by root and reads its /// commitments from that block's bid. @@ -2271,9 +2563,10 @@ impl DataColumnSidecar { pub fn from_ssz(fork: ForkName, bytes: &[u8]) -> Result { match fork { ForkName::Fulu => Ok(Self::Fulu(fulu::DataColumnSidecar::from_ssz_bytes(bytes)?)), - ForkName::Gloas => Ok(Self::Gloas(gloas::DataColumnSidecar::from_ssz_bytes( - bytes, - )?)), + // Neither feature fork changes the sidecar: theirs is gloas's. + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => Ok(Self::Gloas( + gloas::DataColumnSidecar::from_ssz_bytes(bytes)?, + )), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -2296,6 +2589,186 @@ impl DataColumnSidecar { } } +/// A signed execution payload envelope (EIP-7732) of any fork that reveals +/// payloads in one: gloas's, EIP-8483's (gloas's with wider execution +/// requests) and PQ's (EIP-8483's requests replaced, signed with leanXMSS). +/// +/// The three share every message field but the requests, and the payload is +/// gloas's in all of them, so the accessors below answer for any; the bytes, +/// the signature and the requests stay the fork's own. Like a data column +/// sidecar's, the fork comes from context: a topic's digest, a chunk's context +/// bytes, or the slot of the block the envelope names. +#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize)] +#[serde(untagged)] +pub enum SignedExecutionPayloadEnvelope { + Gloas(gloas::SignedExecutionPayloadEnvelope), + Eip8483(eip8483::SignedExecutionPayloadEnvelope), + /// Boxed for its inline 2404-byte signature. + Pq(Box), +} + +macro_rules! dispatch_envelope { + ($envelope:expr, |$inner:ident| $body:expr) => { + match $envelope { + SignedExecutionPayloadEnvelope::Gloas($inner) => $body, + SignedExecutionPayloadEnvelope::Eip8483($inner) => $body, + SignedExecutionPayloadEnvelope::Pq($inner) => $body, + } + }; +} + +impl SignedExecutionPayloadEnvelope { + /// The fork whose container this is. + pub fn fork(&self) -> ForkName { + match self { + Self::Gloas(_) => ForkName::Gloas, + Self::Eip8483(_) => ForkName::Eip8483, + Self::Pq(_) => ForkName::Pq, + } + } + + /// Decodes an envelope of a known fork. A fork without envelopes answers + /// with an error, since the fork comes off the wire. + pub fn from_ssz(fork: ForkName, bytes: &[u8]) -> Result { + match fork { + ForkName::Gloas => Ok(Self::Gloas( + gloas::SignedExecutionPayloadEnvelope::from_ssz_bytes(bytes)?, + )), + ForkName::Eip8483 => Ok(Self::Eip8483( + eip8483::SignedExecutionPayloadEnvelope::from_ssz_bytes(bytes)?, + )), + ForkName::Pq => Ok(Self::Pq(Box::new( + pq::SignedExecutionPayloadEnvelope::from_ssz_bytes(bytes)?, + ))), + ForkName::Phase0 + | ForkName::Altair + | ForkName::Bellatrix + | ForkName::Capella + | ForkName::Deneb + | ForkName::Electra + | ForkName::Fulu + | ForkName::Lean => Err(Error::UnsupportedForFork { + function: "SignedExecutionPayloadEnvelope", + fork, + }), + } + } + + pub fn to_ssz(&self) -> Vec { + match self { + Self::Gloas(inner) => libssz::SszEncode::to_ssz(inner), + Self::Eip8483(inner) => libssz::SszEncode::to_ssz(inner), + Self::Pq(inner) => libssz::SszEncode::to_ssz(inner.as_ref()), + } + } + + /// The root of the block whose payload this reveals. + pub fn beacon_block_root(&self) -> Root { + dispatch_envelope!(self, |inner| inner.message.beacon_block_root) + } + + pub fn parent_beacon_block_root(&self) -> Root { + dispatch_envelope!(self, |inner| inner.message.parent_beacon_block_root) + } + + pub fn builder_index(&self) -> gloas::BuilderIndex { + dispatch_envelope!(self, |inner| inner.message.builder_index) + } + + /// The execution payload, gloas's at every fork that has envelopes. + pub fn payload(&self) -> &gloas::ExecutionPayload { + dispatch_envelope!(self, |inner| &inner.message.payload) + } + + /// The hash tree root of the unsigned message, which the signature is + /// over. + pub fn message_root(&self) -> Root { + dispatch_envelope!(self, |inner| inner.message.hash_tree_root()) + } + + /// The hash tree root of the whole signed envelope, which tells two + /// envelopes for one block apart. + pub fn signed_root(&self) -> Root { + dispatch_envelope!(self, |inner| inner.hash_tree_root()) + } + + /// The hash tree root of the fork's own execution requests, which the + /// bid commits to. + pub fn execution_requests_root(&self) -> Root { + dispatch_envelope!(self, |inner| inner + .message + .execution_requests + .hash_tree_root()) + } + + /// The deposit requests, which every fork's requests carry first. + pub fn deposit_count(&self) -> usize { + dispatch_envelope!(self, |inner| inner + .message + .execution_requests + .deposits + .len()) + } + + /// The gloas envelope this is, if it is one. + pub fn as_gloas(&self) -> Option<&gloas::SignedExecutionPayloadEnvelope> { + match self { + Self::Gloas(inner) => Some(inner), + Self::Eip8483(_) | Self::Pq(_) => None, + } + } +} + +/// What storing an envelope reads: the block it reveals the payload of and +/// its bytes. Implemented by the fork-tagged enum and by a bare gloas +/// envelope, so a caller holding either stores it without converting. +pub trait StorableEnvelope { + fn beacon_block_root(&self) -> Root; + fn to_ssz(&self) -> Vec; +} + +impl StorableEnvelope for SignedExecutionPayloadEnvelope { + fn beacon_block_root(&self) -> Root { + SignedExecutionPayloadEnvelope::beacon_block_root(self) + } + fn to_ssz(&self) -> Vec { + SignedExecutionPayloadEnvelope::to_ssz(self) + } +} + +impl StorableEnvelope for gloas::SignedExecutionPayloadEnvelope { + fn beacon_block_root(&self) -> Root { + self.message.beacon_block_root + } + fn to_ssz(&self) -> Vec { + libssz::SszEncode::to_ssz(self) + } +} + +impl PartialEq for SignedExecutionPayloadEnvelope { + fn eq(&self, other: &gloas::SignedExecutionPayloadEnvelope) -> bool { + self.as_gloas() == Some(other) + } +} + +impl From for SignedExecutionPayloadEnvelope { + fn from(envelope: gloas::SignedExecutionPayloadEnvelope) -> Self { + Self::Gloas(envelope) + } +} + +impl From for SignedExecutionPayloadEnvelope { + fn from(envelope: eip8483::SignedExecutionPayloadEnvelope) -> Self { + Self::Eip8483(envelope) + } +} + +impl From for SignedExecutionPayloadEnvelope { + fn from(envelope: pq::SignedExecutionPayloadEnvelope) -> Self { + Self::Pq(Box::new(envelope)) + } +} + #[cfg(test)] pub(crate) mod json_tests; diff --git a/crates/common/types/src/beacon/containers/pq.rs b/crates/common/types/src/beacon/containers/pq.rs new file mode 100644 index 000000000..9eb528a42 --- /dev/null +++ b/crates/common/types/src/beacon/containers/pq.rs @@ -0,0 +1,1184 @@ +//! Containers whose shape is specific to the PQ fork (consensus-specs +//! `specs/_features/pq`, on the `pq` branch at +//! `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`). +//! +//! The fork replaces BLS with leanXMSS. Every signed container trades its +//! `BLSSignature` for an [`XmssSignature`]; [`Attestation`] loses its signature +//! altogether, a block instead carrying one [`AttestationProof`] of every +//! attestation's signatures; the RANDAO reveal becomes a [`VrfProof`]; sync +//! committees, voluntary exits, BLS-to-execution changes and payload +//! attestations leave the block body; and the state trades the sync committees +//! and EIP-8483's registrations for preregistrations and the aggregation +//! committee window. +//! +//! # Gaps +//! +//! The big containers are EIP-7495 progressive containers, and a field the fork +//! removes leaves a gap in the merkle tree rather than shifting every later +//! field's position. The `active_fields` attribute on each container below +//! spells the specification's `active_fields(width=.., gaps=(..))` out bit by +//! bit; serialization only ever sees the active fields. +//! +//! # Two records with the shared shape in memory +//! +//! [`Validator`] and the builders in [`Builders`] hold the specification's +//! 32-byte [`XmssPubkey`] where every earlier fork holds a 48-byte BLS key. In +//! memory they keep the earlier forks' records ([`shared::Validator`], +//! [`gloas::Builder`]) with the key in the 48-byte form an execution-layer +//! request names it by, 16 zero bytes then the XMSS key +//! ([`XmssPubkey::to_request_key`]), which is the form the specification itself +//! compares request keys in. Their SSZ encoding and merkleization are written by +//! hand, through the specification's own field list (`ValidatorRecord`, +//! `BuilderRecord`), so the bytes and roots are the PQ containers' exactly. +//! +//! That is what lets every function that reads a validator's balance, epochs or +//! credentials, the bulk of the state transition, run unchanged on a PQ state: +//! the registry hands out [`shared::Validator`] references at every fork. A +//! [`Validator`]'s `vrf_root` sits beside that record and is read only by the +//! PQ code that needs it. + +use libssz::{DecodeError, SszDecode, SszEncode}; +use libssz_derive::{HashTreeRoot, SszDecode, SszEncode}; +use libssz_merkle::{HashTreeRoot, Node, Sha256Hasher}; +use libssz_types::{ProgressiveBitlist, ProgressiveList, SszBitvector, SszList, SszVector}; + +use super::fulu::ProposerLookahead; +use super::gloas::{ + self, BuilderDepositRequests, BuilderExitRequests, BuilderIndex, BuilderPendingPayments, + BuilderPendingWithdrawals, ConsolidationRequests, DepositRequests, Deposits, + EpochParticipation, ExecutionPayload, ExecutionPayloadAvailability, ExecutionPayloadBid, + InactivityScores, PayloadAttestationData, PayloadTimelinessCommitteeWindow, + PendingConsolidations, PendingDeposits, PendingPartialWithdrawals, ProposerPreferences, + WithdrawalRequests, Withdrawals, +}; +use super::shared::{ + self, AttestationData, BeaconBlockHeader, BlockRoots, Checkpoint, Eth1Data, Eth1DataVotes, + Fork, HistoricalRoots, HistoricalSummaries, JustificationBits, ProgressiveBalances, + RandaoMixes, Slashings, StateRoots, +}; +use crate::beacon::constants::VRF_TREE_DEPTH; +use crate::beacon::preset; +use crate::beacon::primitives::{ + Bytes32, CommitteeIndex, Epoch, ExecutionAddress, ExecutionBlockHash, Gwei, Root, Slot, + ValidatorIndex, WithdrawalIndex, XmssPubkey, XmssSignature, +}; + +// --------------------------------------------------------------------------- +// Types +// --------------------------------------------------------------------------- + +/// Bits marking which of a slot's (at most `MAX_COMMITTEES_PER_SLOT_PQ`) +/// committees take part in an attestation. +pub type CommitteeBits = SszBitvector<{ preset::MAX_COMMITTEES_PER_SLOT_PQ }>; + +/// A leaf of the tree under a validator's VRF root, then the leaf's Merkle +/// branch: the PQ fork's RANDAO reveal. +pub type VrfProof = SszVector; + +/// A proof of the XMSS signatures of attestations. In the specification's +/// reference encoding, which [`ethlambda_crypto`'s `stark`] reproduces, the +/// signatures concatenated. +pub type AttestationProof = SszList; + +pub type AggregationBits = ProgressiveBitlist; +pub type AttestingIndices = ProgressiveList; +pub type Attestations = ProgressiveList; +pub type AttesterSlashings = ProgressiveList; +pub type ProposerSlashings = ProgressiveList; + +pub type ValidatorPreregistrationRequests = ProgressiveList; +pub type BuilderPreregistrationRequests = ProgressiveList; +pub type ValidatorPreregistrations = ProgressiveList; +pub type BuilderPreregistrations = ProgressiveList; +pub type VoluntaryExitRequests = ProgressiveList; + +/// The aggregators of one beacon committee. +pub type AggregationCommittee = ProgressiveList; +/// The aggregation committee of each beacon committee of a slot, in committee +/// order. +pub type AggregationCommittees = ProgressiveList; +/// The aggregation committees of the previous, current, and lookahead epochs. +pub type AggregationCommitteeWindow = + SszVector; + +/// The PQ validator registry: tree-backed and progressively merkleized like +/// gloas's [`shared::ProgressiveValidators`], over this fork's [`Validator`]. +pub type Validators = + ethlambda_ssz_tree::ProgressiveList>; + +// --------------------------------------------------------------------------- +// The registry element +// --------------------------------------------------------------------------- + +/// A PQ registry entry: the shared record, holding the XMSS key in its 48-byte +/// request form, and the validator's VRF root. See the module doc. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct Validator { + /// Every field but `vrf_root`, with `pubkey` as + /// [`XmssPubkey::to_request_key`] of the validator's XMSS key. + pub record: shared::Validator, + /// The root of the tree of per-slot secrets the validator's RANDAO reveals + /// open (`is_valid_vrf_proof`). + pub vrf_root: Root, +} + +impl Validator { + /// The validator's XMSS public key. + pub fn pubkey(&self) -> XmssPubkey { + XmssPubkey::from_request_key(&self.record.pubkey) + } + + fn to_wire(&self) -> ValidatorRecord { + let record = &self.record; + ValidatorRecord { + pubkey: self.pubkey(), + vrf_root: self.vrf_root, + withdrawal_credentials: record.withdrawal_credentials, + effective_balance: record.effective_balance, + slashed: record.slashed, + activation_eligibility_epoch: record.activation_eligibility_epoch, + activation_epoch: record.activation_epoch, + exit_epoch: record.exit_epoch, + withdrawable_epoch: record.withdrawable_epoch, + } + } + + fn from_wire(wire: ValidatorRecord) -> Self { + Self { + record: shared::Validator { + pubkey: wire.pubkey.to_request_key(), + withdrawal_credentials: wire.withdrawal_credentials, + effective_balance: wire.effective_balance, + slashed: wire.slashed, + activation_eligibility_epoch: wire.activation_eligibility_epoch, + activation_epoch: wire.activation_epoch, + exit_epoch: wire.exit_epoch, + withdrawable_epoch: wire.withdrawable_epoch, + }, + vrf_root: wire.vrf_root, + } + } +} + +/// The specification's PQ `Validator` container, field for field: what a +/// [`Validator`] encodes and merkleizes as. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct ValidatorRecord { + pub pubkey: XmssPubkey, + pub vrf_root: Root, + pub withdrawal_credentials: Bytes32, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub effective_balance: Gwei, + pub slashed: bool, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub activation_eligibility_epoch: Epoch, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub activation_epoch: Epoch, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub exit_epoch: Epoch, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub withdrawable_epoch: Epoch, +} + +impl From for Validator { + fn from(wire: ValidatorRecord) -> Self { + Self::from_wire(wire) + } +} + +impl From<&Validator> for ValidatorRecord { + fn from(validator: &Validator) -> Self { + validator.to_wire() + } +} + +impl SszEncode for Validator { + fn is_fixed_size() -> bool { + ::is_fixed_size() + } + + fn fixed_size() -> usize { + ::fixed_size() + } + + fn encoded_len(&self) -> usize { + ::fixed_size() + } + + fn ssz_append(&self, buf: &mut Vec) { + self.to_wire().ssz_append(buf); + } +} + +impl SszDecode for Validator { + fn is_fixed_size() -> bool { + ::is_fixed_size() + } + + fn fixed_size() -> usize { + ::fixed_size() + } + + fn from_ssz_bytes(bytes: &[u8]) -> Result { + ValidatorRecord::from_ssz_bytes(bytes).map(Self::from_wire) + } +} + +impl HashTreeRoot for Validator { + fn hash_tree_root(&self, hasher: &impl Sha256Hasher) -> Node { + self.to_wire().hash_tree_root(hasher) + } +} + +impl serde::Serialize for Validator { + fn serialize(&self, serializer: S) -> Result { + self.to_wire().serialize(serializer) + } +} + +// --------------------------------------------------------------------------- +// Builders +// --------------------------------------------------------------------------- + +/// The specification's PQ `Builder` container, field for field: what each of +/// [`Builders`]' records encodes and merkleizes as. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct BuilderRecord { + pub pubkey: XmssPubkey, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub version: u8, + pub execution_address: ExecutionAddress, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub balance: Gwei, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub deposit_epoch: Epoch, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub withdrawable_epoch: Epoch, +} + +impl From<&gloas::Builder> for BuilderRecord { + fn from(builder: &gloas::Builder) -> Self { + Self { + pubkey: XmssPubkey::from_request_key(&builder.pubkey), + version: builder.version, + execution_address: builder.execution_address, + balance: builder.balance, + deposit_epoch: builder.deposit_epoch, + withdrawable_epoch: builder.withdrawable_epoch, + } + } +} + +impl From for gloas::Builder { + fn from(record: BuilderRecord) -> Self { + Self { + pubkey: record.pubkey.to_request_key(), + version: record.version, + execution_address: record.execution_address, + balance: record.balance, + deposit_epoch: record.deposit_epoch, + withdrawable_epoch: record.withdrawable_epoch, + } + } +} + +/// The PQ builder registry: gloas's records in memory, each builder's XMSS key +/// in its 48-byte request form, encoded and merkleized as a +/// `ProgressiveList[Builder]` of [`BuilderRecord`]s. Derefs to the gloas list, +/// so the builder code every gloas-family fork shares reads it unchanged. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct Builders(pub gloas::Builders); + +impl Builders { + fn records(&self) -> ProgressiveList { + self.0 + .iter() + .map(BuilderRecord::from) + .collect::>() + .into() + } +} + +impl core::ops::Deref for Builders { + type Target = gloas::Builders; + + fn deref(&self) -> &gloas::Builders { + &self.0 + } +} + +impl core::ops::DerefMut for Builders { + fn deref_mut(&mut self) -> &mut gloas::Builders { + &mut self.0 + } +} + +impl SszEncode for Builders { + fn is_fixed_size() -> bool { + false + } + + fn fixed_size() -> usize { + 0 + } + + fn encoded_len(&self) -> usize { + self.0.len() * ::fixed_size() + } + + fn ssz_append(&self, buf: &mut Vec) { + for builder in self.0.iter() { + BuilderRecord::from(builder).ssz_append(buf); + } + } +} + +impl SszDecode for Builders { + fn is_fixed_size() -> bool { + false + } + + fn fixed_size() -> usize { + 0 + } + + fn from_ssz_bytes(bytes: &[u8]) -> Result { + let records = ProgressiveList::::from_ssz_bytes(bytes)?; + let builders: Vec = records.iter().cloned().map(Into::into).collect(); + Ok(Self(builders.into())) + } +} + +impl HashTreeRoot for Builders { + fn hash_tree_root(&self, hasher: &impl Sha256Hasher) -> Node { + self.records().hash_tree_root(hasher) + } +} + +impl serde::Serialize for Builders { + fn serialize(&self, serializer: S) -> Result { + serializer.collect_seq(self.0.iter().map(BuilderRecord::from)) + } +} + +// --------------------------------------------------------------------------- +// Execution-layer requests +// --------------------------------------------------------------------------- + +/// A binding of an XMSS key to its withdrawal credentials and VRF root, which a +/// deposit for the key must match while it is active (EIP-8205). The system +/// contract verifies the key's signature before emitting it, so it carries +/// none. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct ValidatorPreregistrationRequest { + pub pubkey: XmssPubkey, + pub withdrawal_credentials: Bytes32, + pub vrf_root: Root, +} + +/// A stored validator preregistration, enforced until `expiry_slot`. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct ValidatorPreregistration { + pub pubkey: XmssPubkey, + pub withdrawal_credentials: Bytes32, + pub vrf_root: Root, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub expiry_slot: Slot, +} + +/// The builder counterpart of [`ValidatorPreregistrationRequest`]: a builder +/// has no VRF root. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct BuilderPreregistrationRequest { + pub pubkey: XmssPubkey, + pub withdrawal_credentials: Bytes32, +} + +/// A stored builder preregistration, enforced until `expiry_slot`. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct BuilderPreregistration { + pub pubkey: XmssPubkey, + pub withdrawal_credentials: Bytes32, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub expiry_slot: Slot, +} + +/// A voluntary exit, now an execution-layer request: the system contract +/// verifies the validator's signature at a reserved leaf before emitting it. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct VoluntaryExitRequest { + pub pubkey: XmssPubkey, +} + +/// Gloas's five request lists, then the preregistrations and voluntary exits. +/// EIP-8483's two registration lists are gaps (`width=10, gaps=(5, 6)`). +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +#[ssz(progressive_container, active_fields = [1, 1, 1, 1, 1, 0, 0, 1, 1, 1])] +pub struct ExecutionRequests { + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub deposits: DepositRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub withdrawals: WithdrawalRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub consolidations: ConsolidationRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_deposits: BuilderDepositRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_exits: BuilderExitRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub validator_preregistrations: ValidatorPreregistrationRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_preregistrations: BuilderPreregistrationRequests, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub voluntary_exits: VoluntaryExitRequests, +} + +// --------------------------------------------------------------------------- +// Signed containers +// --------------------------------------------------------------------------- + +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SignedBeaconBlockHeader { + pub message: BeaconBlockHeader, + pub signature: XmssSignature, +} + +/// Two headers a proposer signed at one slot. Both are signed at the same leaf, +/// which exposes that leaf of the proposer's key; the proposer is slashed. +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct ProposerSlashing { + pub signed_header_1: SignedBeaconBlockHeader, + pub signed_header_2: SignedBeaconBlockHeader, +} + +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SignedExecutionPayloadBid { + pub message: ExecutionPayloadBid, + pub signature: XmssSignature, +} + +/// Gloas's envelope, carrying the PQ fork's [`ExecutionRequests`]. +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +#[ssz(progressive_container)] +pub struct ExecutionPayloadEnvelope { + pub payload: ExecutionPayload, + pub execution_requests: ExecutionRequests, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub builder_index: BuilderIndex, + pub beacon_block_root: Root, + pub parent_beacon_block_root: Root, +} + +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SignedExecutionPayloadEnvelope { + pub message: ExecutionPayloadEnvelope, + pub signature: XmssSignature, +} + +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct PayloadAttestationMessage { + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub validator_index: ValidatorIndex, + pub data: PayloadAttestationData, + pub signature: XmssSignature, +} + +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SingleAttestation { + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub committee_index: CommitteeIndex, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub attester_index: ValidatorIndex, + pub data: AttestationData, + pub signature: XmssSignature, +} + +#[derive( + Debug, + Clone, + Default, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SignedProposerPreferences { + pub message: ProposerPreferences, + pub signature: XmssSignature, +} + +// --------------------------------------------------------------------------- +// Attestations +// --------------------------------------------------------------------------- + +/// Gloas's aggregate attestation without its signature (`width=4, gaps=(2,)`): +/// the signatures travel in a separate [`AttestationProof`]. +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +#[ssz(progressive_container, active_fields = [1, 1, 0, 1])] +pub struct Attestation { + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub aggregation_bits: AggregationBits, + pub data: AttestationData, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub committee_bits: CommitteeBits, +} + +/// An attestation with its attesters named, and the proof of their signatures +/// in place of the aggregate signature (`width=3`: `proof` takes the removed +/// `signature`'s position). +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +#[ssz(progressive_container)] +pub struct IndexedAttestation { + #[serde( + serialize_with = "crate::beacon::serde_helpers::quoted_u64_seq::serialize", + deserialize_with = "crate::beacon::serde_helpers::quoted_u64_seq::deserialize_progressive" + )] + pub attesting_indices: AttestingIndices, + pub data: AttestationData, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub proof: AttestationProof, +} + +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct AttesterSlashing { + pub attestation_1: IndexedAttestation, + pub attestation_2: IndexedAttestation, +} + +/// An aggregate as gossiped: the aggregate and the proof of its attesters' +/// signatures. The selection proof is gone, aggregators being the state's +/// aggregation committee rather than self-selected. +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct AggregateAndProof { + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub aggregator_index: ValidatorIndex, + pub aggregate: Attestation, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub proof: AttestationProof, +} + +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SignedAggregateAndProof { + pub message: AggregateAndProof, + pub signature: XmssSignature, +} + +/// What the Beacon API's `aggregate_attestation` answers at a PQ slot: a +/// committee's network aggregate and the proof of its attesters' +/// signatures, which the aggregator wraps in its `AggregateAndProof`. Not a +/// consensus container; the shape is Lighthouse's PQ branch's `PqAggregate`. +#[derive( + Debug, Clone, PartialEq, Eq, serde::Serialize, serde::Deserialize, SszEncode, SszDecode, +)] +pub struct ProvenAggregate { + pub aggregate: Attestation, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub proof: AttestationProof, +} + +// --------------------------------------------------------------------------- +// Blocks +// --------------------------------------------------------------------------- + +/// The PQ block body (`width=14, gaps=(7, 8, 9, 11)`): gloas's without +/// `voluntary_exits`, `sync_aggregate`, `bls_to_execution_changes` and +/// `payload_attestations`, with a VRF proof for its RANDAO reveal and the +/// `proof` of every attestation's signatures appended. +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +#[ssz( + progressive_container, + active_fields = [1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 1, 0, 1, 1] +)] +pub struct BeaconBlockBody { + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub randao_reveal: VrfProof, + pub eth1_data: Eth1Data, + pub graffiti: Bytes32, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub proposer_slashings: ProposerSlashings, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub attester_slashings: AttesterSlashings, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub attestations: Attestations, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub deposits: Deposits, + pub signed_execution_payload_bid: SignedExecutionPayloadBid, + pub parent_execution_requests: ExecutionRequests, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub proof: AttestationProof, +} + +/// An all-zero [`VrfProof`], which [`SszVector`] has no `Default` for. +pub fn empty_vrf_proof() -> VrfProof { + VrfProof::try_from(vec![Bytes32::default(); VRF_TREE_DEPTH + 1]) + .expect("VRF_TREE_DEPTH + 1 roots fit VrfProof's exact length") +} + +impl BeaconBlockBody { + /// An empty body: no operations, a zero VRF proof, an empty proof. + pub fn empty() -> Self { + Self { + randao_reveal: empty_vrf_proof(), + eth1_data: Default::default(), + graffiti: Default::default(), + proposer_slashings: Default::default(), + attester_slashings: Default::default(), + attestations: Default::default(), + deposits: Default::default(), + signed_execution_payload_bid: Default::default(), + parent_execution_requests: Default::default(), + proof: Default::default(), + } + } +} + +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct BeaconBlock { + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub slot: Slot, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub proposer_index: ValidatorIndex, + pub parent_root: Root, + pub state_root: Root, + pub body: BeaconBlockBody, +} + +#[derive( + Debug, + Clone, + PartialEq, + Eq, + serde::Serialize, + serde::Deserialize, + SszEncode, + SszDecode, + HashTreeRoot, +)] +pub struct SignedBeaconBlock { + pub message: BeaconBlock, + pub signature: XmssSignature, +} + +// --------------------------------------------------------------------------- +// State +// --------------------------------------------------------------------------- + +/// The PQ beacon state (`width=51, gaps=(22, 23, 46, 47)`): gloas's fields +/// without the two sync committees, EIP-8483's registrations removed in turn, +/// and the preregistrations and aggregation committee window appended. 47 +/// fields, in the specification's order. +#[derive(Debug, Clone, PartialEq, Eq, serde::Serialize, SszEncode, SszDecode, HashTreeRoot)] +#[ssz( + progressive_container, + active_fields = [ + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, + 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1 + ] +)] +pub struct BeaconState { + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub genesis_time: u64, + pub genesis_validators_root: Root, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub slot: Slot, + pub fork: Fork, + pub latest_block_header: BeaconBlockHeader, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub block_roots: BlockRoots, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub state_roots: StateRoots, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub historical_roots: HistoricalRoots, + pub eth1_data: Eth1Data, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub eth1_data_votes: Eth1DataVotes, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub eth1_deposit_index: u64, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub validators: Validators, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub balances: ProgressiveBalances, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub randao_mixes: RandaoMixes, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub slashings: Slashings, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub previous_epoch_participation: EpochParticipation, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub current_epoch_participation: EpochParticipation, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub justification_bits: JustificationBits, + pub previous_justified_checkpoint: Checkpoint, + pub current_justified_checkpoint: Checkpoint, + pub finalized_checkpoint: Checkpoint, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub inactivity_scores: InactivityScores, + // Gaps: `current_sync_committee` (22), `next_sync_committee` (23). + pub latest_block_hash: ExecutionBlockHash, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub next_withdrawal_index: WithdrawalIndex, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub next_withdrawal_validator_index: ValidatorIndex, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub historical_summaries: HistoricalSummaries, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub deposit_requests_start_index: u64, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub deposit_balance_to_consume: Gwei, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub exit_balance_to_consume: Gwei, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub earliest_exit_epoch: Epoch, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub consolidation_balance_to_consume: Gwei, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub earliest_consolidation_epoch: Epoch, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub pending_deposits: PendingDeposits, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub pending_partial_withdrawals: PendingPartialWithdrawals, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub pending_consolidations: PendingConsolidations, + #[serde(with = "crate::beacon::serde_helpers::quoted_u64_seq")] + pub proposer_lookahead: ProposerLookahead, + pub builders: Builders, + #[serde(with = "crate::beacon::serde_helpers::quoted_or_bare")] + pub next_withdrawal_builder_index: BuilderIndex, + #[serde(with = "crate::beacon::serde_helpers::ssz_hex")] + pub execution_payload_availability: ExecutionPayloadAvailability, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_pending_payments: BuilderPendingPayments, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_pending_withdrawals: BuilderPendingWithdrawals, + pub latest_execution_payload_bid: ExecutionPayloadBid, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub payload_expected_withdrawals: Withdrawals, + #[serde(serialize_with = "crate::beacon::serde_helpers::nested_quoted_u64_seq::serialize")] + pub ptc_window: PayloadTimelinessCommitteeWindow, + // Gaps: `validator_pq_registrations` (46), `builder_pq_registrations` (47). + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub validator_preregistrations: ValidatorPreregistrations, + #[serde(with = "crate::beacon::serde_helpers::seq")] + pub builder_preregistrations: BuilderPreregistrations, + #[serde(serialize_with = "serialize_window")] + pub aggregation_committee_window: AggregationCommitteeWindow, +} + +/// The Beacon API's JSON for the window: per slot, per committee, the +/// aggregators' indices as quoted integers. +fn serialize_window( + window: &AggregationCommitteeWindow, + serializer: S, +) -> Result { + struct Committees<'a>(&'a AggregationCommittees); + impl serde::Serialize for Committees<'_> { + fn serialize(&self, serializer: S) -> Result { + serializer.collect_seq(self.0.iter().map(Committee)) + } + } + struct Committee<'a>(&'a AggregationCommittee); + impl serde::Serialize for Committee<'_> { + fn serialize(&self, serializer: S) -> Result { + serializer.collect_seq(self.0.iter().map(u64::to_string)) + } + } + serializer.collect_seq(window.iter().map(Committees)) +} + +/// An all-empty [`AggregationCommitteeWindow`], which [`SszVector`] has no +/// `Default` for: what the fork upgrade starts from before +/// `initialize_aggregation_committee_window` fills it. +pub fn empty_aggregation_committee_window() -> AggregationCommitteeWindow { + let empty = vec![AggregationCommittees::default(); preset::AGGREGATION_COMMITTEE_WINDOW_LENGTH]; + AggregationCommitteeWindow::try_from(empty) + .expect("AGGREGATION_COMMITTEE_WINDOW_LENGTH entries fit the window's exact length") +} + +#[cfg(test)] +mod tests { + use libssz::{SszDecode as _, SszEncode as _}; + use libssz_merkle::{HashTreeRoot as _, Sha2Hasher}; + + use super::*; + + fn validator() -> Validator { + Validator { + record: shared::Validator { + pubkey: XmssPubkey([9; 32]).to_request_key(), + withdrawal_credentials: Bytes32::repeat_byte(1), + effective_balance: 32_000_000_000, + slashed: false, + activation_eligibility_epoch: 1, + activation_epoch: 2, + exit_epoch: 3, + withdrawable_epoch: 4, + }, + vrf_root: Root::repeat_byte(5), + } + } + + /// A [`Validator`] encodes and merkleizes exactly as the specification's + /// nine-field container does, built here independently field by field. + #[test] + fn validator_is_the_specifications_container() { + let validator = validator(); + let mut expected = Vec::new(); + expected.extend_from_slice(&[9; 32]); + expected.extend_from_slice(&[5; 32]); + expected.extend_from_slice(&[1; 32]); + expected.extend_from_slice(&32_000_000_000u64.to_le_bytes()); + expected.push(0); + for epoch in 1u64..=4 { + expected.extend_from_slice(&epoch.to_le_bytes()); + } + assert_eq!(validator.to_ssz(), expected); + assert_eq!(::fixed_size(), 137); + assert_eq!(Validator::from_ssz_bytes(&expected).unwrap(), validator); + assert_eq!( + validator.hash_tree_root(&Sha2Hasher), + ValidatorRecord::from(&validator).hash_tree_root(&Sha2Hasher) + ); + } + + #[test] + fn builders_round_trip_through_their_records() { + let builder = gloas::Builder { + pubkey: XmssPubkey([3; 32]).to_request_key(), + version: 0, + execution_address: Default::default(), + balance: 7, + deposit_epoch: 1, + withdrawable_epoch: u64::MAX, + }; + let builders = Builders(vec![builder.clone(), builder].into()); + let bytes = builders.to_ssz(); + assert_eq!(bytes.len(), 2 * 77); + assert_eq!(Builders::from_ssz_bytes(&bytes).unwrap(), builders); + assert_eq!( + builders.hash_tree_root(&Sha2Hasher), + builders.records().hash_tree_root(&Sha2Hasher) + ); + } + + #[test] + fn the_window_round_trips() { + let mut window = empty_aggregation_committee_window(); + window[3] = vec![vec![1u64, 2].into(), vec![].into()].into(); + let bytes = window.to_ssz(); + assert_eq!( + AggregationCommitteeWindow::from_ssz_bytes(&bytes).unwrap(), + window + ); + } + + /// The largest attestation proof a block or aggregate can carry. + fn largest_proof() -> AttestationProof { + AttestationProof::try_from(vec![0u8; preset::MAX_STARK_PROOF_SIZE]) + .expect("MAX_STARK_PROOF_SIZE bytes fit the proof's bound") + } + + /// The most attesters one attestation can name: every member of every + /// committee of a slot. + const MAX_ATTESTERS: usize = + preset::MAX_VALIDATORS_PER_COMMITTEE * preset::MAX_COMMITTEES_PER_SLOT_PQ; + + fn largest_indexed_attestation() -> IndexedAttestation { + IndexedAttestation { + attesting_indices: vec![0; MAX_ATTESTERS].into(), + data: Default::default(), + proof: largest_proof(), + } + } + + /// The specification's own maximum encodings (`pq.yaml`'s type-specific + /// SSZ bounds), against values built at every bound: the offsets, field + /// widths and the fields the fork removed all show up in these lengths. + /// Serialization only; `active_fields` changes roots, not bytes. + #[test] + fn the_largest_values_encode_to_the_specifications_bounds() { + let slashing = AttesterSlashing { + attestation_1: largest_indexed_attestation(), + attestation_2: largest_indexed_attestation(), + }; + assert_eq!( + slashing.to_ssz().len(), + preset::MAX_ATTESTER_SLASHING_SIZE_PQ + ); + + let aggregate = SignedAggregateAndProof { + message: AggregateAndProof { + aggregator_index: 0, + aggregate: Attestation { + aggregation_bits: ProgressiveBitlist::with_length(MAX_ATTESTERS), + data: Default::default(), + committee_bits: Default::default(), + }, + proof: largest_proof(), + }, + signature: Default::default(), + }; + assert_eq!( + aggregate.to_ssz().len(), + preset::MAX_SIGNED_AGGREGATE_AND_PROOF_SIZE_PQ + ); + + let commitments = vec![Default::default(); preset::MAX_BLOB_COMMITMENTS_PER_BLOCK]; + let bid = SignedExecutionPayloadBid { + message: ExecutionPayloadBid { + blob_kzg_commitments: commitments.into(), + ..Default::default() + }, + signature: Default::default(), + }; + assert_eq!( + bid.to_ssz().len(), + preset::MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE_PQ + ); + } +} + +/// What block production hands a PQ proposer: the unsigned block and the +/// unsigned envelope revealing its payload, which the proposer signs at their +/// own leaves. ethlambda's own validator-client exchange: Lighthouse's PQ +/// branch has an equivalent (`PqBlockAndEnvelope`), the Beacon API none yet. +#[derive(Debug, Clone, PartialEq, Eq, SszEncode, SszDecode)] +pub struct BlockContents { + pub block: BeaconBlock, + pub execution_payload_envelope: ExecutionPayloadEnvelope, +} diff --git a/crates/common/types/src/beacon/fork.rs b/crates/common/types/src/beacon/fork.rs index 1d090ab9f..a604a2296 100644 --- a/crates/common/types/src/beacon/fork.rs +++ b/crates/common/types/src/beacon/fork.rs @@ -41,7 +41,18 @@ const BEACON_SNAPSHOT_INTERVAL: u64 = preset::SLOTS_PER_EPOCH; /// a target of the beacon STF's `upgrade`-style traversal. See /// [`ForkName::ALL`]. /// -/// Forks after gloas exist upstream but are out of scope for this crate. +/// [`ForkName::Eip8483`] and [`ForkName::Pq`] are consensus-specs feature +/// forks (`specs/_features/{eip8483,pq}` on the `pq` branch, pinned at +/// `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`), not mainnet upgrades: the +/// PQ fork replaces BLS with leanXMSS, and needs the EIP-8483 registrations +/// made before it, so the two are scheduled in that order after gloas. Other +/// forks after gloas exist upstream but are out of scope for this crate. +/// +/// Each builds on the fork before it, as every fork here does: EIP-8483 is +/// gloas plus its two registration requests, and PQ is EIP-8483 with BLS +/// replaced. So a rule gated `fork >= ForkName::Gloas` holds at both, one +/// gated `>= ForkName::Eip8483` holds at PQ, and a rule a later fork replaces +/// is that fork's own match arm. #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] pub enum ForkName { Phase0, @@ -52,6 +63,12 @@ pub enum ForkName { Electra, Fulu, Gloas, + /// EIP-8483: validators and builders register post-quantum key material + /// through execution-layer requests, which the PQ fork then adopts. + Eip8483, + /// The post-quantum fork: leanXMSS signatures replace BLS, sync + /// committees are removed, and RANDAO reveals become VRF proofs. + Pq, /// The Lean consensus protocol, which this repository implements alongside /// the Beacon Chain. Not a Beacon Chain fork, and not in [`ForkName::ALL`]. Lean, @@ -65,7 +82,7 @@ impl ForkName { /// array, its absence is what makes `parse("lean")` return `None`, keeps /// `Gloas.next()` at `None`, and stops any fixture directory from resolving /// to a lean case. - pub const ALL: [ForkName; 8] = [ + pub const ALL: [ForkName; 10] = [ ForkName::Phase0, ForkName::Altair, ForkName::Bellatrix, @@ -74,6 +91,8 @@ impl ForkName { ForkName::Electra, ForkName::Fulu, ForkName::Gloas, + ForkName::Eip8483, + ForkName::Pq, ]; /// The lowercase name the specification and its fixture paths use. @@ -87,6 +106,8 @@ impl ForkName { ForkName::Electra => "electra", ForkName::Fulu => "fulu", ForkName::Gloas => "gloas", + ForkName::Eip8483 => "eip8483", + ForkName::Pq => "pq", ForkName::Lean => "lean", } } @@ -134,7 +155,17 @@ impl ForkName { | ForkName::Deneb | ForkName::Electra | ForkName::Fulu - | ForkName::Gloas => true, + | ForkName::Gloas + // PQ: its blocks and envelopes are gossiped, requested and + // imported like gloas's, its execution requests go to the engine + // as gloas's do, and its attestation proofs need the prover + // backend the network runs (`ethlambda_crypto::stark`). Run + // against Lighthouse on a PQ-at-genesis devnet. + | ForkName::Pq => true, + // Not yet: its block's cheap gossip rules, its envelope topic's + // decode and rules, and its fetched envelopes' checks still take + // gloas's shapes. Nothing schedules it: the PQ devnet starts on PQ. + ForkName::Eip8483 => false, ForkName::Lean => super::lean_fork_unreachable("ForkName::is_followed"), } } @@ -148,7 +179,7 @@ impl ForkName { /// gloas must be listed here, so the compiler forces the decision. pub const fn has_payload_envelopes(self) -> bool { match self { - ForkName::Gloas => true, + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => true, ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -179,6 +210,8 @@ impl ForkName { ForkName::Electra => 5, ForkName::Fulu => 6, ForkName::Gloas => 7, + ForkName::Eip8483 => 8, + ForkName::Pq => 9, ForkName::Lean => 255, } } @@ -204,7 +237,9 @@ impl ForkName { | ForkName::Deneb | ForkName::Electra | ForkName::Fulu - | ForkName::Gloas => BEACON_SNAPSHOT_INTERVAL, + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => BEACON_SNAPSHOT_INTERVAL, } } @@ -222,6 +257,8 @@ impl ForkName { 5 => Some(ForkName::Electra), 6 => Some(ForkName::Fulu), 7 => Some(ForkName::Gloas), + 8 => Some(ForkName::Eip8483), + 9 => Some(ForkName::Pq), 255 => Some(ForkName::Lean), _ => None, } @@ -263,18 +300,20 @@ mod tests { } #[test] - fn gloas_is_the_last_beacon_fork() { + fn pq_is_the_last_beacon_fork() { assert_eq!(ForkName::Fulu.next(), Some(ForkName::Gloas)); + assert_eq!(ForkName::Gloas.next(), Some(ForkName::Eip8483)); + assert_eq!(ForkName::Eip8483.next(), Some(ForkName::Pq)); // Also guards the reason Lean is kept out of ALL: adding it there // would make this None into Some(Lean) and let `upgrade` walk off // the end. - assert_eq!(ForkName::Gloas.next(), None); + assert_eq!(ForkName::Pq.next(), None); } #[test] fn neighbours_terminate_at_the_ends() { assert_eq!(ForkName::Phase0.previous(), None); - assert_eq!(ForkName::Gloas.next(), None); + assert_eq!(ForkName::Pq.next(), None); assert_eq!(ForkName::Altair.previous(), Some(ForkName::Phase0)); assert_eq!(ForkName::Altair.next(), Some(ForkName::Bellatrix)); } @@ -332,9 +371,9 @@ mod tests { } #[test] - fn every_beacon_fork_through_gloas_is_followed() { + fn every_beacon_fork_but_eip8483_is_followed() { for fork in ForkName::ALL { - assert!(fork.is_followed(), "{fork:?}"); + assert_eq!(fork.is_followed(), fork != ForkName::Eip8483, "{fork:?}"); } } @@ -352,9 +391,11 @@ mod tests { assert_eq!(ForkName::Phase0.selector(), 0); assert_eq!(ForkName::Fulu.selector(), 6); assert_eq!(ForkName::Gloas.selector(), 7); + assert_eq!(ForkName::Eip8483.selector(), 8); + assert_eq!(ForkName::Pq.selector(), 9); // Lean sits at the top of the byte range so heze can keep taking the // next free value after gloas. assert_eq!(ForkName::Lean.selector(), 255); - assert_eq!(ForkName::from_selector(8), None); + assert_eq!(ForkName::from_selector(10), None); } } diff --git a/crates/common/types/src/beacon/mod.rs b/crates/common/types/src/beacon/mod.rs index 2dbab74ea..e0a3b9f5e 100644 --- a/crates/common/types/src/beacon/mod.rs +++ b/crates/common/types/src/beacon/mod.rs @@ -74,6 +74,23 @@ pub(crate) fn lean_block_unreachable(function: &str) -> ! { ) } +/// Panics, naming the BLS-only accessor a PQ value reached. +/// +/// The PQ fork signs with leanXMSS, so a block, attestation or key of that +/// fork has no BLS counterpart to hand back. Every BLS verification path +/// dispatches on the fork first; reaching this means one did not. Public, +/// unlike the lean helpers, because the state transition's own PQ arms of BLS +/// paths panic the same way. +#[cold] +#[track_caller] +pub fn xmss_fork_has_no_bls(function: &str) -> ! { + unreachable!( + "a PQ value reached a BLS-only function ({function}); \ + the PQ fork signs with leanXMSS, so the caller must dispatch on the \ + fork before verifying a BLS signature" + ) +} + /// Panics, naming the beacon fork a lean-only caller reached. /// /// The mirror image of [`lean_state_unreachable`] and [`lean_block_unreachable`]: diff --git a/crates/common/types/src/beacon/preset.rs b/crates/common/types/src/beacon/preset.rs index afa81596b..76301ab8b 100644 --- a/crates/common/types/src/beacon/preset.rs +++ b/crates/common/types/src/beacon/preset.rs @@ -561,6 +561,99 @@ pub mod mainnet { /// (`Vector`): /// builder payments owed for the previous and current epoch. pub const BUILDER_PENDING_PAYMENTS_LENGTH: usize = 2 * SLOTS_PER_EPOCH as usize; + + // ================================================================ + // EIP-8483 (consensus-specs `_features/eip8483`) + // ================================================================ + + // --- Execution --- + + /// Bounds `ExecutionRequests.validator_pq_registrations` per payload. + pub const MAX_VALIDATOR_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD: u64 = 1_024; + + /// Bounds `ExecutionRequests.builder_pq_registrations` per payload. + pub const MAX_BUILDER_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD: u64 = 16; + + // ================================================================ + // PQ (consensus-specs `_features/pq`) + // ================================================================ + + // --- Misc --- + + /// The PQ fork's cap on a slot's committees, which replaces + /// [`MAX_COMMITTEES_PER_SLOT`] in `get_committee_count_per_slot` and + /// bounds `Attestation.committee_bits` (`Bitvector`). + /// A quarter of `MAX_COMMITTEES_PER_SLOT`. + pub const MAX_COMMITTEES_PER_SLOT_PQ: usize = 16; + + // --- Time parameters --- + + /// Replaces [`MIN_ATTESTATION_INCLUSION_DELAY`] from PQ: a block includes + /// attestations from at least two slots before it, which gives the + /// proposer a slot to receive aggregates and merge their proofs. The + /// timely-head flag needs exactly this delay. + pub const MIN_ATTESTATION_INCLUSION_DELAY_PQ: u64 = 2; + + // --- XMSS --- + + /// Bounds `AttestationProof` (`ByteList`). A + /// placeholder for a STARK's size: the reference concatenation fits + /// `MAX_STARK_PROOF_SIZE / XMSS_SIGNATURE_SIZE` signatures. + pub const MAX_STARK_PROOF_SIZE: usize = 1_048_576; + + // --- Max operations per block --- + + /// Replaces [`MAX_ATTESTATIONS_ELECTRA`] as the PQ block's attestation cap. + pub const MAX_ATTESTATIONS_PQ: u64 = 16; + + // --- Execution --- + + /// Bounds `ExecutionRequests.validator_preregistrations` per payload. + pub const MAX_VALIDATOR_PREREGISTRATION_REQUESTS_PER_PAYLOAD: u64 = 64; + + /// Bounds `ExecutionRequests.builder_preregistrations` per payload. + pub const MAX_BUILDER_PREREGISTRATION_REQUESTS_PER_PAYLOAD: u64 = 4; + + /// Bounds `ExecutionRequests.voluntary_exits` per payload. + pub const MAX_VOLUNTARY_EXIT_REQUESTS_PER_PAYLOAD: u64 = 16; + + // --- Type-specific SSZ bounds --- + + /// The largest SSZ encoding of a PQ `SignedAggregateAndProof`: a + /// `MAX_VALIDATORS_PER_COMMITTEE * MAX_COMMITTEES_PER_SLOT_PQ`-bit + /// aggregate and a `MAX_STARK_PROOF_SIZE` proof. follows + /// `MAX_COMMITTEES_PER_SLOT_PQ`. + pub const MAX_SIGNED_AGGREGATE_AND_PROOF_SIZE_PQ: usize = 1_055_231; + + /// The largest SSZ encoding of a PQ `AttesterSlashing`: two indexed + /// attestations, each naming `MAX_VALIDATORS_PER_COMMITTEE * + /// MAX_COMMITTEES_PER_SLOT_PQ` attesters with a `MAX_STARK_PROOF_SIZE` + /// proof. follows `MAX_COMMITTEES_PER_SLOT_PQ`. + pub const MAX_ATTESTER_SLASHING_SIZE_PQ: usize = 2_621_720; + + /// The largest SSZ encoding of a PQ `SignedExecutionPayloadBid`: + /// `MAX_BLOB_COMMITMENTS_PER_BLOCK` commitments and an XMSS signature. + pub const MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE_PQ: usize = 199_240; + + /// How many active preregistrations of one kind the state admits. + /// From `presets/mainnet/eip8205.yaml`: the PQ preset file does not list + /// it, and the specification's build loads every preset file of the + /// directory. + pub const PREREGISTRATIONS_LIMIT: u64 = 524_288; + + /// How long a preregistration binds its key, in slots from the slot it was + /// stored at. From `presets/mainnet/eip8205.yaml`: the PQ preset file does not list + /// it, and the specification's build loads every preset file of the + /// directory. + pub const PREREGISTRATION_EXPIRY_SLOTS: u64 = 262_144; + + // --- State list lengths --- + + /// `(MIN_SEED_LOOKAHEAD + 2) * SLOTS_PER_EPOCH`, the length of + /// `BeaconState.aggregation_committee_window`: the aggregation committees + /// of the previous, current, and lookahead epochs. + pub const AGGREGATION_COMMITTEE_WINDOW_LENGTH: usize = + (MIN_SEED_LOOKAHEAD as usize + 2) * SLOTS_PER_EPOCH as usize; } /// The minimal preset: the same shape as [`mainnet`], scaled down so spec test @@ -1069,6 +1162,99 @@ pub mod minimal { /// (`Vector`): /// builder payments owed for the previous and current epoch. pub const BUILDER_PENDING_PAYMENTS_LENGTH: usize = 2 * SLOTS_PER_EPOCH as usize; + + // ================================================================ + // EIP-8483 (consensus-specs `_features/eip8483`) + // ================================================================ + + // --- Execution --- + + /// Bounds `ExecutionRequests.validator_pq_registrations` per payload. + pub const MAX_VALIDATOR_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD: u64 = 1_024; + + /// Bounds `ExecutionRequests.builder_pq_registrations` per payload. + pub const MAX_BUILDER_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD: u64 = 16; + + // ================================================================ + // PQ (consensus-specs `_features/pq`) + // ================================================================ + + // --- Misc --- + + /// The PQ fork's cap on a slot's committees, which replaces + /// [`MAX_COMMITTEES_PER_SLOT`] in `get_committee_count_per_slot` and + /// bounds `Attestation.committee_bits` (`Bitvector`). + /// `[customized]`: minimal's `MAX_COMMITTEES_PER_SLOT` is 4 too. + pub const MAX_COMMITTEES_PER_SLOT_PQ: usize = 4; + + // --- Time parameters --- + + /// Replaces [`MIN_ATTESTATION_INCLUSION_DELAY`] from PQ: a block includes + /// attestations from at least two slots before it, which gives the + /// proposer a slot to receive aggregates and merge their proofs. The + /// timely-head flag needs exactly this delay. + pub const MIN_ATTESTATION_INCLUSION_DELAY_PQ: u64 = 2; + + // --- XMSS --- + + /// Bounds `AttestationProof` (`ByteList`). A + /// placeholder for a STARK's size: the reference concatenation fits + /// `MAX_STARK_PROOF_SIZE / XMSS_SIGNATURE_SIZE` signatures. + pub const MAX_STARK_PROOF_SIZE: usize = 1_048_576; + + // --- Max operations per block --- + + /// Replaces [`MAX_ATTESTATIONS_ELECTRA`] as the PQ block's attestation cap. + pub const MAX_ATTESTATIONS_PQ: u64 = 16; + + // --- Execution --- + + /// Bounds `ExecutionRequests.validator_preregistrations` per payload. + pub const MAX_VALIDATOR_PREREGISTRATION_REQUESTS_PER_PAYLOAD: u64 = 64; + + /// Bounds `ExecutionRequests.builder_preregistrations` per payload. + pub const MAX_BUILDER_PREREGISTRATION_REQUESTS_PER_PAYLOAD: u64 = 4; + + /// Bounds `ExecutionRequests.voluntary_exits` per payload. + pub const MAX_VOLUNTARY_EXIT_REQUESTS_PER_PAYLOAD: u64 = 16; + + // --- Type-specific SSZ bounds --- + + /// The largest SSZ encoding of a PQ `SignedAggregateAndProof`: a + /// `MAX_VALIDATORS_PER_COMMITTEE * MAX_COMMITTEES_PER_SLOT_PQ`-bit + /// aggregate and a `MAX_STARK_PROOF_SIZE` proof. `[customized]`: follows + /// `MAX_COMMITTEES_PER_SLOT_PQ`. + pub const MAX_SIGNED_AGGREGATE_AND_PROOF_SIZE_PQ: usize = 1_052_158; + + /// The largest SSZ encoding of a PQ `AttesterSlashing`: two indexed + /// attestations, each naming `MAX_VALIDATORS_PER_COMMITTEE * + /// MAX_COMMITTEES_PER_SLOT_PQ` attesters with a `MAX_STARK_PROOF_SIZE` + /// proof. `[customized]`: follows `MAX_COMMITTEES_PER_SLOT_PQ`. + pub const MAX_ATTESTER_SLASHING_SIZE_PQ: usize = 2_228_504; + + /// The largest SSZ encoding of a PQ `SignedExecutionPayloadBid`: + /// `MAX_BLOB_COMMITMENTS_PER_BLOCK` commitments and an XMSS signature. + pub const MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE_PQ: usize = 199_240; + + /// How many active preregistrations of one kind the state admits. + /// `[customized]`, from `presets/minimal/eip8205.yaml`: the PQ preset file + /// does not list it, and the specification's build loads every preset + /// file of the directory. + pub const PREREGISTRATIONS_LIMIT: u64 = 64; + + /// How long a preregistration binds its key, in slots from the slot it was + /// stored at. `[customized]`, from `presets/minimal/eip8205.yaml`: the PQ preset file + /// does not list it, and the specification's build loads every preset + /// file of the directory. + pub const PREREGISTRATION_EXPIRY_SLOTS: u64 = 64; + + // --- State list lengths --- + + /// `(MIN_SEED_LOOKAHEAD + 2) * SLOTS_PER_EPOCH`, the length of + /// `BeaconState.aggregation_committee_window`: the aggregation committees + /// of the previous, current, and lookahead epochs. + pub const AGGREGATION_COMMITTEE_WINDOW_LENGTH: usize = + (MIN_SEED_LOOKAHEAD as usize + 2) * SLOTS_PER_EPOCH as usize; } #[cfg(not(feature = "preset-minimal"))] @@ -1195,7 +1381,9 @@ pub mod retuned { | ForkName::Deneb | ForkName::Electra | ForkName::Fulu - | ForkName::Gloas => super::PROPORTIONAL_SLASHING_MULTIPLIER_BELLATRIX, + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => super::PROPORTIONAL_SLASHING_MULTIPLIER_BELLATRIX, ForkName::Lean => lean_fork_unreachable("proportional_slashing_multiplier"), } } @@ -1217,9 +1405,11 @@ pub mod retuned { ForkName::Bellatrix | ForkName::Capella | ForkName::Deneb => { super::MIN_SLASHING_PENALTY_QUOTIENT_BELLATRIX } - ForkName::Electra | ForkName::Fulu | ForkName::Gloas => { - super::MIN_SLASHING_PENALTY_QUOTIENT_ELECTRA - } + ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => super::MIN_SLASHING_PENALTY_QUOTIENT_ELECTRA, ForkName::Lean => lean_fork_unreachable("min_slashing_penalty_quotient"), } } @@ -1233,9 +1423,11 @@ pub mod retuned { /// small one. pub fn whistleblower_reward_quotient(fork: ForkName) -> u64 { match fork { - ForkName::Electra | ForkName::Fulu | ForkName::Gloas => { - super::WHISTLEBLOWER_REWARD_QUOTIENT_ELECTRA - } + ForkName::Electra + | ForkName::Fulu + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => super::WHISTLEBLOWER_REWARD_QUOTIENT_ELECTRA, ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -1261,7 +1453,9 @@ pub mod retuned { | ForkName::Deneb | ForkName::Electra | ForkName::Fulu - | ForkName::Gloas => super::INACTIVITY_PENALTY_QUOTIENT_BELLATRIX, + | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq => super::INACTIVITY_PENALTY_QUOTIENT_BELLATRIX, ForkName::Lean => lean_fork_unreachable("inactivity_penalty_quotient"), } } @@ -1641,5 +1835,49 @@ mod tests { MAX_BUILDER_EXIT_REQUESTS_PER_PAYLOAD, MAX_BUILDERS_PER_WITHDRAWALS_SWEEP, PAYLOAD_TIMELY_THRESHOLD, DATA_AVAILABILITY_TIMELY_THRESHOLD, ); + + in_both!(u64: + MAX_VALIDATOR_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD, + MAX_BUILDER_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD, + ); + in_both!(usize: + MAX_COMMITTEES_PER_SLOT_PQ, MAX_STARK_PROOF_SIZE, AGGREGATION_COMMITTEE_WINDOW_LENGTH, + MAX_SIGNED_AGGREGATE_AND_PROOF_SIZE_PQ, MAX_ATTESTER_SLASHING_SIZE_PQ, + MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE_PQ, + ); + in_both!(u64: + MAX_ATTESTATIONS_PQ, MIN_ATTESTATION_INCLUSION_DELAY_PQ, + MAX_VALIDATOR_PREREGISTRATION_REQUESTS_PER_PAYLOAD, + MAX_BUILDER_PREREGISTRATION_REQUESTS_PER_PAYLOAD, MAX_VOLUNTARY_EXIT_REQUESTS_PER_PAYLOAD, + PREREGISTRATIONS_LIMIT, PREREGISTRATION_EXPIRY_SLOTS, + ); + } + + /// The PQ preset's values, pinned from `presets/{mainnet,minimal}/pq.yaml`, + /// `eip8483.yaml` and `eip8205.yaml` at consensus-specs' `pq` pin + /// `a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`. + #[test] + fn pq_pinned_values() { + assert_eq!(mainnet::MAX_COMMITTEES_PER_SLOT_PQ, 16); + assert_eq!(minimal::MAX_COMMITTEES_PER_SLOT_PQ, 4); + assert_eq!(mainnet::MIN_ATTESTATION_INCLUSION_DELAY_PQ, 2); + assert_eq!(minimal::MIN_ATTESTATION_INCLUSION_DELAY_PQ, 2); + assert_eq!(mainnet::PREREGISTRATIONS_LIMIT, 1 << 19); + assert_eq!(minimal::PREREGISTRATIONS_LIMIT, 64); + assert_eq!(mainnet::PREREGISTRATION_EXPIRY_SLOTS, 1 << 18); + assert_eq!(minimal::PREREGISTRATION_EXPIRY_SLOTS, 64); + assert_eq!(mainnet::MAX_STARK_PROOF_SIZE, 1 << 20); + assert_eq!( + mainnet::MAX_VALIDATOR_PQ_REGISTRATION_REQUESTS_PER_PAYLOAD, + 1 << 10 + ); + assert_eq!(mainnet::AGGREGATION_COMMITTEE_WINDOW_LENGTH, 96); + assert_eq!(minimal::AGGREGATION_COMMITTEE_WINDOW_LENGTH, 24); + assert_eq!(mainnet::MAX_SIGNED_AGGREGATE_AND_PROOF_SIZE_PQ, 1_055_231); + assert_eq!(minimal::MAX_SIGNED_AGGREGATE_AND_PROOF_SIZE_PQ, 1_052_158); + assert_eq!(mainnet::MAX_ATTESTER_SLASHING_SIZE_PQ, 2_621_720); + assert_eq!(minimal::MAX_ATTESTER_SLASHING_SIZE_PQ, 2_228_504); + assert_eq!(mainnet::MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE_PQ, 199_240); + assert_eq!(minimal::MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE_PQ, 199_240); } } diff --git a/crates/common/types/src/beacon/primitives.rs b/crates/common/types/src/beacon/primitives.rs index b19309a8f..d1ad0c5ed 100644 --- a/crates/common/types/src/beacon/primitives.rs +++ b/crates/common/types/src/beacon/primitives.rs @@ -21,6 +21,7 @@ use core::{cmp::Ordering, fmt}; use libssz_derive::{HashTreeRoot, SszDecode, SszEncode}; +pub use super::constants::{XMSS_PUBKEY_SIZE, XMSS_SIGNATURE_SIZE}; use super::serde_helpers::HexPrefixed; pub use crate::primitives::{H256, HashTreeRoot}; @@ -416,6 +417,76 @@ impl fmt::Debug for BlsSignature { } } +/// A leanXMSS public key (the PQ fork's `XMSSPubkey`): the top tree's Merkle +/// root followed by the public parameter, 16 bytes each. +/// +/// The SSZ shape of a `Bytes32`, and a separate type from [`Root`] so a key +/// and a root cannot be passed for one another. +#[derive( + Clone, Copy, Default, PartialEq, Eq, Hash, PartialOrd, Ord, SszEncode, SszDecode, HashTreeRoot, +)] +#[ssz(transparent)] +pub struct XmssPubkey(pub [u8; XMSS_PUBKEY_SIZE]); + +impl XmssPubkey { + /// The 48-byte form an execution-layer request or a pending deposit names + /// this key by: `XMSS_DEPOSIT_PREFIX` (16 zero bytes), then the key. + pub fn to_request_key(&self) -> BlsPubkey { + let mut out = [0u8; BLS_PUBKEY_SIZE]; + out[BLS_PUBKEY_SIZE - XMSS_PUBKEY_SIZE..].copy_from_slice(&self.0); + BlsPubkey(out) + } + + /// The key a 48-byte request field names: its last 32 bytes, the + /// specification's `XMSSPubkey(pubkey[16:])`. Reads the bytes whatever the + /// first 16 are; whether they mark a legacy BLS key is the caller's + /// question (`is_legacy_pubkey`). + pub fn from_request_key(key: &BlsPubkey) -> Self { + let mut out = [0u8; XMSS_PUBKEY_SIZE]; + out.copy_from_slice(&key.0[BLS_PUBKEY_SIZE - XMSS_PUBKEY_SIZE..]); + Self(out) + } +} + +impl AsRef<[u8]> for XmssPubkey { + fn as_ref(&self) -> &[u8] { + &self.0 + } +} + +impl fmt::Debug for XmssPubkey { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + debug_byte_vector(f, "XmssPubkey", &self.0) + } +} + +/// A leanXMSS signature (the PQ fork's `XMSSSignature`): the leaf index, four +/// bytes little endian, then one signature per layer. +/// +/// `Clone` but not `Copy`, unlike [`BlsSignature`]: at 2404 bytes an implicit +/// copy is worth seeing at the call site. +#[derive(Clone, PartialEq, Eq, Hash, SszEncode, SszDecode, HashTreeRoot)] +#[ssz(transparent)] +pub struct XmssSignature(pub [u8; XMSS_SIGNATURE_SIZE]); + +impl Default for XmssSignature { + fn default() -> Self { + Self([0; XMSS_SIGNATURE_SIZE]) + } +} + +impl AsRef<[u8]> for XmssSignature { + fn as_ref(&self) -> &[u8] { + &self.0 + } +} + +impl fmt::Debug for XmssSignature { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + debug_byte_vector(f, "XmssSignature", &self.0) + } +} + /// A KZG commitment to a blob. /// /// The same width as [`KzgProof`] and as [`BlsPubkey`], and a separate type from @@ -498,7 +569,15 @@ macro_rules! impl_hex_serialize { }; } -impl_hex_serialize!(H160, BlsPubkey, BlsSignature, KzgCommitment, KzgProof); +impl_hex_serialize!( + H160, + BlsPubkey, + BlsSignature, + KzgCommitment, + KzgProof, + XmssPubkey, + XmssSignature, +); /// The `Deserialize` counterpart of [`impl_hex_serialize`], for the types the /// Beacon API accepts in a request body: a validator's public key (as a @@ -521,7 +600,15 @@ macro_rules! impl_hex_deserialize { }; } -impl_hex_deserialize!(BlsPubkey, BlsSignature, H160, KzgCommitment, KzgProof); +impl_hex_deserialize!( + BlsPubkey, + BlsSignature, + H160, + KzgCommitment, + KzgProof, + XmssPubkey, + XmssSignature, +); #[cfg(test)] mod tests { diff --git a/crates/net/api/src/lib.rs b/crates/net/api/src/lib.rs index 0d62efd5d..04db10e35 100644 --- a/crates/net/api/src/lib.rs +++ b/crates/net/api/src/lib.rs @@ -4,12 +4,15 @@ use std::time::Instant; use ethlambda_types::{ attestation::{SignedAggregatedAttestation, SignedAttestation}, beacon::containers::{ - DataColumnSidecar, SignedAggregateAndProof, SignedBeaconBlock, + AttestationData, DataColumnSidecar, SignedAggregateAndProof, SignedBeaconBlock, + SignedExecutionPayloadEnvelope, altair::{SignedContributionAndProof, SyncCommitteeMessage}, electra::SingleAttestation, - gloas::{ - PayloadAttestationMessage, SignedExecutionPayloadBid, SignedExecutionPayloadEnvelope, - SignedProposerPreferences, + gloas::{PayloadAttestationMessage, SignedExecutionPayloadBid, SignedProposerPreferences}, + pq::{ + PayloadAttestationMessage as PqPayloadAttestationMessage, + SignedAggregateAndProof as PqSignedAggregateAndProof, + SingleAttestation as PqSingleAttestation, }, }, beacon::operation::BeaconOperation, @@ -351,6 +354,24 @@ pub trait P2PToBlockChain: Send + Sync { message: PayloadAttestationMessage, arrival: BlockArrival, ) -> Result<(), ActorError>; + /// A PQ aggregate that passed gossip validation (its aggregator's XMSS + /// signature and its STARK proof included), as the vote it carries and + /// the attesting indices validation resolved: what fork choice applies. + /// Not the aggregate itself, whose proof and leanXMSS signature nothing + /// past gossip reads. + fn new_pq_aggregate( + &self, + data: AttestationData, + attesting_indices: Vec, + arrival: AggregateArrival, + ) -> Result<(), ActorError>; + /// A PQ payload attestation message that passed gossip validation, for + /// fork choice. Not queued, like gloas's. + fn new_pq_payload_attestation_message( + &self, + message: Box, + arrival: BlockArrival, + ) -> Result<(), ActorError>; } /// When an aggregate reached this node, for the one metric that cannot be @@ -400,6 +421,24 @@ impl AggregateArrival { /// these requests come from a validator client through the HTTP server, not /// from the chain actor, and a beacon node's chain actor has nothing to publish /// on its own behalf. +/// A validator client's PQ message for [`RpcToP2P::publish_pq_gossip`]. +#[derive(Debug, Clone)] +pub enum PqGossipObject { + /// For `beacon_attestation_{subnet_id}`, one of PQ's 16 subnets. + Attestation { + subnet_id: u64, + attestation: Box, + }, + /// For `beacon_aggregate_and_proof`, with the attesting indices + /// validation resolved, which fork choice applies. + Aggregate { + aggregate: Box, + attesting_indices: Vec, + }, + /// For `payload_attestation_message`. + PayloadAttestation(Box), +} + #[protocol] pub trait RpcToP2P: Send + Sync { /// Gossip one unaggregated attestation on `beacon_attestation_{subnet_id}`. @@ -427,10 +466,19 @@ pub trait RpcToP2P: Send + Sync { ) -> Result<(), ActorError>; /// Gossip one operation on its topic, already validated by the caller. fn publish_beacon_operation(&self, operation: BeaconOperation) -> Result<(), ActorError>; + /// Gossip one of a validator client's PQ messages, already validated by + /// the caller with PQ's gossip rules, on its PQ-digest topic, and hand + /// what fork choice reads to the chain actor, since gossip never delivers + /// a node its own messages. + fn publish_pq_gossip(&self, object: PqGossipObject) -> Result<(), ActorError>; /// Join attestation subnets a validator client's aggregators need, each /// until the end of the paired slot, so their committees' attestations /// reach this node's pool. `(subnet_id, slot)` pairs. fn subscribe_attestation_subnets(&self, subnets: Vec<(u64, u64)>) -> Result<(), ActorError>; + /// [`Self::subscribe_attestation_subnets`] for a PQ slot's committees: + /// `(subnet_id, slot)` pairs among PQ's 16 subnets, joined under PQ + /// digests only. + fn subscribe_pq_attestation_subnets(&self, subnets: Vec<(u64, u64)>) -> Result<(), ActorError>; /// Gossip one sync committee message on `sync_committee_{id}` for every id /// in `subnet_ids` (every subnet its validator has a seat in, computed by /// the caller from the head state), and mark it seen. Checked by the caller. diff --git a/crates/net/p2p/src/beacon/column_checks.rs b/crates/net/p2p/src/beacon/column_checks.rs index 1e476beab..5e8d404e9 100644 --- a/crates/net/p2p/src/beacon/column_checks.rs +++ b/crates/net/p2p/src/beacon/column_checks.rs @@ -399,9 +399,22 @@ mod tests { } fn new_execution_payload_envelope( &self, - _envelope: Box< - ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope, - >, + _envelope: Box, + _arrival: BlockArrival, + ) -> Result<(), ActorError> { + Ok(()) + } + fn new_pq_aggregate( + &self, + _data: ethlambda_types::beacon::containers::AttestationData, + _attesting_indices: Vec, + _arrival: AggregateArrival, + ) -> Result<(), ActorError> { + Ok(()) + } + fn new_pq_payload_attestation_message( + &self, + _message: Box, _arrival: BlockArrival, ) -> Result<(), ActorError> { Ok(()) diff --git a/crates/net/p2p/src/beacon/decode.rs b/crates/net/p2p/src/beacon/decode.rs index 376bbce22..08c77e3f0 100644 --- a/crates/net/p2p/src/beacon/decode.rs +++ b/crates/net/p2p/src/beacon/decode.rs @@ -25,10 +25,14 @@ //! | `data_column_sidecar_{subnet_id}` | Yes, at gloas, by topic digest | //! | `execution_payload`, `payload_attestation_message` | No, gloas onward | //! | `execution_payload_bid`, `proposer_preferences` | No, gloas onward | +//! +//! A PQ digest's topics are the exception to all of the above: every one +//! whose container PQ reshapes (all but the block and column topics) decodes +//! as PQ's by the topic's digest alone, through [`decode_pq_gossip`]. use ethlambda_types::beacon::config::Config; use ethlambda_types::beacon::containers::{ - DataColumnSidecar, SignedBeaconBlock, altair, capella, electra, gloas, phase0, shared, + DataColumnSidecar, SignedBeaconBlock, altair, capella, electra, gloas, phase0, pq, shared, }; use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::preset; @@ -283,8 +287,15 @@ pub fn decode_aggregate_and_proof( } // Electra's bytes, but a progressive `Attestation` (EIP-7688), so the // root the aggregator signs differs; `data.index` is the payload flag. - ForkName::Gloas => gloas::SignedAggregateAndProof::from_ssz_bytes(bytes) - .map(SignedAggregateAndProof::Gloas), + ForkName::Gloas | ForkName::Eip8483 => { + gloas::SignedAggregateAndProof::from_ssz_bytes(bytes) + .map(SignedAggregateAndProof::Gloas) + } + // The PQ fork's aggregate carries a proof and an XMSS signature. A PQ + // digest's topic decodes it by the digest instead + // (`decode_pq_gossip`); by the slot, here, it is not a container + // this decoder returns. + ForkName::Pq => return Err(DecodeError::UnsupportedFork), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -336,9 +347,12 @@ pub fn decode_attestation(fork: ForkName, bytes: &[u8]) -> Result { + ForkName::Electra | ForkName::Fulu | ForkName::Gloas | ForkName::Eip8483 => { electra::SingleAttestation::from_ssz_bytes(bytes).map(Attestation::Electra) } + // XMSS-signed, decoded by a PQ digest's topic: see + // `decode_aggregate_and_proof`. + ForkName::Pq => return Err(DecodeError::UnsupportedFork), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -373,9 +387,12 @@ pub fn decode_gossip( ForkName::Electra | ForkName::Fulu => { electra::AttesterSlashing::from_ssz_bytes(bytes).map(AttesterSlashing::Electra) } - ForkName::Gloas => { + ForkName::Gloas | ForkName::Eip8483 => { gloas::AttesterSlashing::from_ssz_bytes(bytes).map(AttesterSlashing::Gloas) } + // Proof-carrying, decoded by a PQ digest's topic: see + // `decode_aggregate_and_proof`. + ForkName::Pq => return Err(DecodeError::UnsupportedFork), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -411,6 +428,96 @@ pub fn decode_gossip( } } +/// A message on one of the topics whose container the PQ fork reshapes, +/// decoded as PQ's. Every one carries an XMSS signature where gloas's carries +/// a BLS one; an aggregate also carries a STARK proof in place of a +/// selection proof. Kept apart from [`BeaconGossip`] and the shared +/// containers, which the chain actor, the pools and the Beacon API read: the +/// node does not follow PQ, and nothing past gossip consumes these. +#[derive(Debug, Clone, PartialEq)] +pub enum PqGossip { + Aggregate(Box), + Attestation(Box), + ProposerSlashing(Box), + AttesterSlashing(Box), + PayloadAttestation(Box), + ExecutionPayloadBid(Box), + ProposerPreferences(Box), + Envelope(Box), +} + +/// Whether `kind` is a topic whose container the PQ fork reshapes, the ones +/// [`decode_pq_gossip`] decodes. A block, whose fork the payload names, and a +/// data column sidecar, unchanged since gloas, are not. +pub fn is_pq_reshaped(kind: &str) -> bool { + matches!( + kind, + topics::BEACON_AGGREGATE_AND_PROOF + | topics::PROPOSER_SLASHING + | topics::ATTESTER_SLASHING + | topics::PAYLOAD_ATTESTATION_MESSAGE + | topics::EXECUTION_PAYLOAD_BID + | topics::PROPOSER_PREFERENCES + | topics::EXECUTION_PAYLOAD + ) || topics::attestation_subnet(kind).is_some() +} + +/// Decode a message from a PQ digest's topic `kind` as PQ's container. +/// +/// The fork comes from the topic's digest, not the payload: every container +/// here changed shape at PQ, so the digest is what says which shape the bytes +/// are. The three with a type-specific bound in `pq/p2p-interface.md` +/// (`MAX_*_SIZE_PQ`, sized for XMSS signatures) are refused past it before +/// any decode. +pub fn decode_pq_gossip(kind: &str, bytes: &[u8]) -> Result { + fn bounded(bytes: &[u8], max: usize) -> Result<&[u8], DecodeError> { + if bytes.len() > max { + Err(DecodeError::Ssz) + } else { + Ok(bytes) + } + } + let ssz = |_| DecodeError::Ssz; + match kind { + topics::BEACON_AGGREGATE_AND_PROOF => pq::SignedAggregateAndProof::from_ssz_bytes(bounded( + bytes, + preset::MAX_SIGNED_AGGREGATE_AND_PROOF_SIZE_PQ, + )?) + .map(|value| PqGossip::Aggregate(Box::new(value))) + .map_err(ssz), + topics::PROPOSER_SLASHING => pq::ProposerSlashing::from_ssz_bytes(bytes) + .map(|value| PqGossip::ProposerSlashing(Box::new(value))) + .map_err(ssz), + topics::ATTESTER_SLASHING => pq::AttesterSlashing::from_ssz_bytes(bounded( + bytes, + preset::MAX_ATTESTER_SLASHING_SIZE_PQ, + )?) + .map(|value| PqGossip::AttesterSlashing(Box::new(value))) + .map_err(ssz), + topics::PAYLOAD_ATTESTATION_MESSAGE => pq::PayloadAttestationMessage::from_ssz_bytes(bytes) + .map(|value| PqGossip::PayloadAttestation(Box::new(value))) + .map_err(ssz), + topics::EXECUTION_PAYLOAD_BID => pq::SignedExecutionPayloadBid::from_ssz_bytes(bounded( + bytes, + preset::MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE_PQ, + )?) + .map(|value| PqGossip::ExecutionPayloadBid(Box::new(value))) + .map_err(ssz), + topics::PROPOSER_PREFERENCES => pq::SignedProposerPreferences::from_ssz_bytes(bytes) + .map(|value| PqGossip::ProposerPreferences(Box::new(value))) + .map_err(ssz), + topics::EXECUTION_PAYLOAD => pq::SignedExecutionPayloadEnvelope::from_ssz_bytes(bytes) + .map(|value| PqGossip::Envelope(Box::new(value))) + .map_err(ssz), + kind if topics::attestation_subnet(kind).is_some() => { + pq::SingleAttestation::from_ssz_bytes(bytes) + .map(|value| PqGossip::Attestation(Box::new(value))) + .map_err(ssz) + } + _ => Err(DecodeError::UnknownTopic), + } +} + #[cfg(test)] mod tests { use super::*; @@ -804,4 +911,84 @@ mod tests { } } } + + /// Each reshaped topic decodes as PQ's own container, by the topic alone. + #[test] + fn pq_topics_decode_as_pq_containers() { + let slashing = pq::ProposerSlashing::default(); + assert_eq!( + decode_pq_gossip(topics::PROPOSER_SLASHING, &slashing.to_ssz()), + Ok(PqGossip::ProposerSlashing(Box::new(slashing))) + ); + let attestation = pq::SingleAttestation::default(); + let subnet = topics::attestation_topic_name([0; 4], 3); + let kind = topics::topic_kind(&subnet).unwrap(); + assert_eq!( + decode_pq_gossip(kind, &attestation.to_ssz()), + Ok(PqGossip::Attestation(Box::new(attestation))) + ); + let message = pq::PayloadAttestationMessage::default(); + assert_eq!( + decode_pq_gossip(topics::PAYLOAD_ATTESTATION_MESSAGE, &message.to_ssz()), + Ok(PqGossip::PayloadAttestation(Box::new(message))) + ); + let bid = pq::SignedExecutionPayloadBid::default(); + assert_eq!( + decode_pq_gossip(topics::EXECUTION_PAYLOAD_BID, &bid.to_ssz()), + Ok(PqGossip::ExecutionPayloadBid(Box::new(bid))) + ); + let preferences = pq::SignedProposerPreferences::default(); + assert_eq!( + decode_pq_gossip(topics::PROPOSER_PREFERENCES, &preferences.to_ssz()), + Ok(PqGossip::ProposerPreferences(Box::new(preferences))) + ); + // A gloas container is not a PQ one: its BLS signature is the wrong + // width. + let gloas_slashing = shared::ProposerSlashing::default(); + assert_eq!( + decode_pq_gossip(topics::PROPOSER_SLASHING, &gloas_slashing.to_ssz()), + Err(DecodeError::Ssz) + ); + } + + /// The type-specific bounds `pq/p2p-interface.md` sizes for XMSS + /// signatures refuse a larger payload before it is decoded. + #[test] + fn pq_payloads_past_their_size_bound_are_refused() { + for (kind, bound) in [ + ( + topics::BEACON_AGGREGATE_AND_PROOF, + preset::MAX_SIGNED_AGGREGATE_AND_PROOF_SIZE_PQ, + ), + ( + topics::ATTESTER_SLASHING, + preset::MAX_ATTESTER_SLASHING_SIZE_PQ, + ), + ( + topics::EXECUTION_PAYLOAD_BID, + preset::MAX_SIGNED_EXECUTION_PAYLOAD_BID_SIZE_PQ, + ), + ] { + assert_eq!( + decode_pq_gossip(kind, &vec![0u8; bound + 1]), + Err(DecodeError::Ssz), + "{kind}" + ); + } + assert_eq!( + decode_pq_gossip(topics::VOLUNTARY_EXIT, &[]), + Err(DecodeError::UnknownTopic) + ); + } + + /// Blocks and columns are not reshaped: they take the shared path. + #[test] + fn only_the_reshaped_topics_are_pq_decoded() { + assert!(is_pq_reshaped(topics::BEACON_AGGREGATE_AND_PROOF)); + assert!(is_pq_reshaped("beacon_attestation_15")); + assert!(is_pq_reshaped(topics::EXECUTION_PAYLOAD)); + assert!(!is_pq_reshaped(topics::BEACON_BLOCK)); + assert!(!is_pq_reshaped("data_column_sidecar_3")); + assert!(!is_pq_reshaped(topics::VOLUNTARY_EXIT)); + } } diff --git a/crates/net/p2p/src/beacon/encoding.rs b/crates/net/p2p/src/beacon/encoding.rs index ea7df9bee..46dfb9a23 100644 --- a/crates/net/p2p/src/beacon/encoding.rs +++ b/crates/net/p2p/src/beacon/encoding.rs @@ -36,7 +36,7 @@ use std::io; use ethlambda_types::beacon::config::Config; use ethlambda_types::beacon::containers::DataColumnSidecar; use ethlambda_types::beacon::containers::SignedBeaconBlock; -use ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope; +use ethlambda_types::beacon::containers::SignedExecutionPayloadEnvelope; use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::fork_digest::compute_fork_digest; use ethlambda_types::beacon::preset; @@ -359,7 +359,7 @@ where { for envelope in envelopes { let encoded = envelope.to_ssz(); - let slot = envelope.message.payload.slot_number; + let slot = envelope.payload().slot_number; if encoded.len() > MAX_PAYLOAD_SIZE - 1024 { warn!( slot, @@ -411,16 +411,17 @@ where read_chunked_response(io, protocol_label, limits, |context, payload| { let fork = <[u8; 4]>::try_from(context) .ok() - .and_then(|digest| schedule.fork_for_digest(digest)); - if !fork.is_some_and(carries_envelopes) { - return Err(invalid(format!( - "execution payload envelope chunk context {} is not a gloas-or-later fork digest", - hex::encode(context), - ))); - } - let envelope = SignedExecutionPayloadEnvelope::from_ssz_bytes(payload) + .and_then(|digest| schedule.fork_for_digest(digest)) + .filter(|fork| carries_envelopes(*fork)) + .ok_or_else(|| { + invalid(format!( + "execution payload envelope chunk context {} is not a gloas-or-later fork digest", + hex::encode(context), + )) + })?; + let envelope = SignedExecutionPayloadEnvelope::from_ssz(fork, payload) .map_err(|err| invalid(format!("execution payload envelope chunk: {err:?}")))?; - let slot = envelope.message.payload.slot_number; + let slot = envelope.payload().slot_number; let expected = schedule.digest_at(slot / preset::SLOTS_PER_EPOCH); if context != expected { warn!( diff --git a/crates/net/p2p/src/beacon/envelope_checks.rs b/crates/net/p2p/src/beacon/envelope_checks.rs index 119b2c8e7..d9c29a6dd 100644 --- a/crates/net/p2p/src/beacon/envelope_checks.rs +++ b/crates/net/p2p/src/beacon/envelope_checks.rs @@ -24,12 +24,12 @@ use std::num::NonZeroUsize; use ethlambda_network_api::BlockArrival; -use ethlambda_state_transition::beacon::gossip::Outcome; use ethlambda_state_transition::beacon::gossip::envelope::{ SeenEnvelopes, cheap_checks, stateful_checks, }; +use ethlambda_state_transition::beacon::gossip::{self, Outcome}; use ethlambda_storage::Store; -use ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope; +use ethlambda_types::beacon::containers::SignedExecutionPayloadEnvelope; use tracing::{debug, error, warn}; use crate::beacon::payload_hash; @@ -49,14 +49,27 @@ fn judge(store: &Store, envelope: &SignedExecutionPayloadEnvelope) -> Outcome { // A fresh, empty cache: a fetched envelope is not a gossip duplicate of // anything, so the cache's one rule never fires. let seen = SeenEnvelopes::new(NonZeroUsize::MIN); - if let Err(outcome) = cheap_checks(&seen, store, envelope) { - return outcome; - } // The same stricter-than-the-specification rule the gossip topic runs. if let Err(outcome) = payload_hash::reject_envelope_on_mismatch(envelope) { return outcome; } - stateful_checks(store, envelope) + match envelope { + SignedExecutionPayloadEnvelope::Gloas(inner) => { + if let Err(outcome) = cheap_checks(&seen, store, inner) { + return outcome; + } + stateful_checks(store, inner) + } + SignedExecutionPayloadEnvelope::Pq(inner) => { + if let Err(outcome) = gossip::pq::envelope_cheap_checks(&seen, store, inner) { + return outcome; + } + gossip::pq::envelope_stateful_checks(store, inner) + } + // No gossip rules of its own: the chain actor verifies every envelope + // against its block's post-state. + SignedExecutionPayloadEnvelope::Eip8483(_) => Outcome::Accept, + } } /// Run the gossip rules on `envelopes` and send the chain actor the ones that @@ -137,7 +150,7 @@ pub(crate) fn check_and_forward_then( } }; if !forwards(&outcome) { - let block_root = envelope.message.beacon_block_root; + let block_root = envelope.beacon_block_root(); let (_, reason) = outcome.labels(); debug!( block_root = %ethlambda_types::ShortRoot(&block_root.0), @@ -227,15 +240,30 @@ pub(crate) mod tests { } fn new_execution_payload_envelope( &self, - envelope: Box, + envelope: Box, _: BlockArrival, ) -> Result<(), ActorError> { let _ = self.0.send(Seen::Envelope( - envelope.message.beacon_block_root, - envelope.message.payload.slot_number, + envelope.beacon_block_root(), + envelope.payload().slot_number, )); Ok(()) } + fn new_pq_aggregate( + &self, + _data: ethlambda_types::beacon::containers::AttestationData, + _attesting_indices: Vec, + _arrival: AggregateArrival, + ) -> Result<(), ActorError> { + Ok(()) + } + fn new_pq_payload_attestation_message( + &self, + _message: Box, + _arrival: BlockArrival, + ) -> Result<(), ActorError> { + Ok(()) + } fn new_payload_attestation_message( &self, _: ethlambda_types::beacon::containers::gloas::PayloadAttestationMessage, @@ -248,10 +276,12 @@ pub(crate) mod tests { pub(crate) fn envelope(slot: u64, byte: u8) -> SignedExecutionPayloadEnvelope { // Sealed: the fetched-envelope rules refuse a block hash that is not // the payload's header's. - payload_hash::seal_envelope(crate::beacon::encoding::test_support::envelope( - H256::repeat_byte(byte), - slot, - H256::ZERO, + SignedExecutionPayloadEnvelope::Gloas(payload_hash::seal_envelope( + crate::beacon::encoding::test_support::envelope( + H256::repeat_byte(byte), + slot, + H256::ZERO, + ), )) } diff --git a/crates/net/p2p/src/beacon/handler.rs b/crates/net/p2p/src/beacon/handler.rs index 2e566f2dc..18af6a1d3 100644 --- a/crates/net/p2p/src/beacon/handler.rs +++ b/crates/net/p2p/src/beacon/handler.rs @@ -170,7 +170,14 @@ pub(crate) fn advertised_metadata(wire: &BeaconWire) -> AdvertisedMetaData { /// answer than setting none. fn attnets(wire: &BeaconWire) -> AttnetsBits { let mut attnets = AttnetsBits::default(); - for &subnet_id in &wire.attestation_subnets { + // Under a PQ digest, its own backbone: only those bits may be set + // (`pq/p2p-interface.md`). + let backbone = super::topics::backbone_subnets( + wire.fork, + &wire.attestation_subnets, + &wire.attestation_subnets_pq, + ); + for &subnet_id in backbone { let _ = attnets.set(subnet_id as usize, true); } attnets @@ -269,6 +276,7 @@ mod tests { metadata: Default::default(), custody_columns: Vec::new(), attestation_subnets: Vec::new(), + attestation_subnets_pq: Vec::new(), sync_committee_subnets: Default::default(), } } diff --git a/crates/net/p2p/src/beacon/mod.rs b/crates/net/p2p/src/beacon/mod.rs index 17710ee8f..a5f59593c 100644 --- a/crates/net/p2p/src/beacon/mod.rs +++ b/crates/net/p2p/src/beacon/mod.rs @@ -23,6 +23,7 @@ pub mod fork_schedule; pub mod handler; pub mod messages; pub mod payload_hash; +pub mod pq; pub mod protocols; pub mod scoring; pub mod subnets; @@ -85,6 +86,10 @@ pub struct BeaconWire { /// Read by `build_metadata` for the `attnets` bitfield it advertises, so /// what this node claims to serve is what it actually subscribed to. pub attestation_subnets: Vec, + /// The attestation subnets this node backbones under a PQ digest, among + /// `ATTESTATION_SUBNET_COUNT_PQ`. `attnets` advertises them while the + /// current digest is PQ's. + pub attestation_subnets_pq: Vec, /// The sync committee subnets joined on a validator client's request, each /// with the epoch (exclusive) it is needed until. /// @@ -193,6 +198,11 @@ pub mod constants { /// though this node subscribes to none of them. pub const ATTESTATION_SUBNET_COUNT: u64 = 64; + /// `ATTESTATION_SUBNET_COUNT_PQ`: the attestation subnets a PQ digest + /// carries. `attnets` stays [`ATTESTATION_SUBNET_COUNT`] wide; only the + /// first this many bits may be set under PQ. + pub const ATTESTATION_SUBNET_COUNT_PQ: u64 = 16; + /// The width of `MetaData`'s `syncnets`. Re-exported so the bitfield and /// the sync subcommittee size divide by one value. pub use ethlambda_types::beacon::constants::SYNC_COMMITTEE_SUBNET_COUNT; diff --git a/crates/net/p2p/src/beacon/payload_hash.rs b/crates/net/p2p/src/beacon/payload_hash.rs index b0e33b905..1de0150f4 100644 --- a/crates/net/p2p/src/beacon/payload_hash.rs +++ b/crates/net/p2p/src/beacon/payload_hash.rs @@ -27,9 +27,13 @@ //! which holds `base_fee_per_gas` as a `u64` where the payload has 256 bits. use ethlambda_state_transition::beacon::gossip::{Outcome, RejectReason}; -use ethlambda_state_transition::beacon::stf::{electra, gloas as gloas_stf}; +use ethlambda_state_transition::beacon::stf::{ + eip8483 as eip8483_stf, electra, gloas as gloas_stf, pq as pq_stf, +}; use ethlambda_types::ShortRoot; -use ethlambda_types::beacon::containers::{SignedBeaconBlock, capella, gloas}; +use ethlambda_types::beacon::containers::{ + SignedBeaconBlock, SignedExecutionPayloadEnvelope, capella, gloas, +}; use ethlambda_types::beacon::primitives::Root; use ethrex_common::types::requests::{EncodedRequests, compute_requests_hash}; use ethrex_common::types::{Withdrawal, compute_withdrawals_root}; @@ -276,16 +280,50 @@ pub fn verify_block(block: &SignedBeaconBlock) -> Result<(), HashMismatch> { SignedBeaconBlock::Phase0(_) | SignedBeaconBlock::Altair(_) | SignedBeaconBlock::Gloas(_) + | SignedBeaconBlock::Eip8483(_) + | SignedBeaconBlock::Pq(_) | SignedBeaconBlock::Lean(_) => Ok(()), } } -/// A gloas envelope's payload hash against its header. -pub fn verify_envelope( +/// An envelope's payload hash against its header, at any fork with +/// envelopes: the header's `requests_hash` covers the fork's own typed +/// requests. +pub fn verify_envelope(envelope: &SignedExecutionPayloadEnvelope) -> Result<(), HashMismatch> { + let requests = match envelope { + SignedExecutionPayloadEnvelope::Gloas(inner) => { + gloas_stf::get_execution_requests_list(&inner.message.execution_requests) + } + SignedExecutionPayloadEnvelope::Eip8483(inner) => { + eip8483_stf::get_execution_requests_list(&inner.message.execution_requests) + } + SignedExecutionPayloadEnvelope::Pq(inner) => { + pq_stf::get_execution_requests_list(&inner.message.execution_requests) + } + }; + verify_envelope_parts( + envelope.payload(), + envelope.parent_beacon_block_root(), + requests, + ) +} + +/// [`verify_envelope`] for a gloas envelope, without wrapping it. +pub fn verify_gloas_envelope( envelope: &gloas::SignedExecutionPayloadEnvelope, ) -> Result<(), HashMismatch> { - let message = &envelope.message; - let payload = &message.payload; + verify_envelope_parts( + &envelope.message.payload, + envelope.message.parent_beacon_block_root, + gloas_stf::get_execution_requests_list(&envelope.message.execution_requests), + ) +} + +fn verify_envelope_parts( + payload: &gloas::ExecutionPayload, + parent_beacon_block_root: Root, + requests: Vec>, +) -> Result<(), HashMismatch> { Fields { parent_hash: payload.parent_hash.0, fee_recipient: payload.fee_recipient.0, @@ -303,10 +341,8 @@ pub fn verify_envelope( transactions: payload.transactions.iter().map(|tx| &tx[..]).collect(), withdrawals: Some(payload.withdrawals.as_ref()), blob_gas: Some((payload.blob_gas_used, payload.excess_blob_gas)), - parent_beacon_block_root: Some(message.parent_beacon_block_root.0), - requests: Some(gloas_stf::get_execution_requests_list( - &message.execution_requests, - )), + parent_beacon_block_root: Some(parent_beacon_block_root.0), + requests: Some(requests), amsterdam: Some((payload.block_access_list.as_ref(), payload.slot_number)), } .verify() @@ -318,7 +354,7 @@ pub fn verify_envelope( pub(crate) fn seal_envelope( mut envelope: gloas::SignedExecutionPayloadEnvelope, ) -> gloas::SignedExecutionPayloadEnvelope { - if let Err(mismatch) = verify_envelope(&envelope) { + if let Err(mismatch) = verify_gloas_envelope(&envelope) { envelope.message.payload.block_hash = mismatch.computed.into(); } envelope @@ -336,9 +372,23 @@ pub(crate) fn reject_block_on_mismatch( /// [`verify_envelope`] as a verdict, shared by the gossip topic and the /// fetched-envelope checks. pub(crate) fn reject_envelope_on_mismatch( - envelope: &gloas::SignedExecutionPayloadEnvelope, + envelope: &SignedExecutionPayloadEnvelope, ) -> Result<(), Outcome> { verify_envelope(envelope).map_err(|mismatch| { + refuse( + "envelope", + envelope.payload().slot_number, + envelope.beacon_block_root(), + mismatch, + ) + }) +} + +/// [`reject_envelope_on_mismatch`] for a gloas envelope, without wrapping it. +pub(crate) fn reject_gloas_envelope_on_mismatch( + envelope: &gloas::SignedExecutionPayloadEnvelope, +) -> Result<(), Outcome> { + verify_gloas_envelope(envelope).map_err(|mismatch| { refuse( "envelope", envelope.message.payload.slot_number, diff --git a/crates/net/p2p/src/beacon/payload_hash/tests.rs b/crates/net/p2p/src/beacon/payload_hash/tests.rs index 500be82c8..8eb0bfdd4 100644 --- a/crates/net/p2p/src/beacon/payload_hash/tests.rs +++ b/crates/net/p2p/src/beacon/payload_hash/tests.rs @@ -68,7 +68,7 @@ fn every_real_payload_matches_its_hash() { let block = block(fork, bytes); assert_eq!(verify_block(&block), Ok(()), "{fork:?}"); } - assert_eq!(verify_envelope(&envelope()), Ok(()), "gloas"); + assert_eq!(verify_gloas_envelope(&envelope()), Ok(()), "gloas"); } #[test] @@ -209,7 +209,7 @@ fn a_changed_gloas_field_is_refused() { } _ => message.payload.base_fee_per_gas.0[31] ^= 1, } - assert_refused(verify_envelope(&changed), what); + assert_refused(verify_gloas_envelope(&changed), what); } } @@ -227,7 +227,7 @@ fn a_pre_merge_bellatrix_payload_has_nothing_to_check() { fn a_mismatch_is_a_reject_with_its_own_reason() { let mut changed = envelope(); changed.message.payload.gas_used += 1; - let outcome = reject_envelope_on_mismatch(&changed).expect_err("a changed envelope"); + let outcome = reject_gloas_envelope_on_mismatch(&changed).expect_err("a changed envelope"); assert_eq!(outcome.labels(), ("reject", "payload_block_hash_mismatch")); - assert!(reject_envelope_on_mismatch(&envelope()).is_ok()); + assert!(reject_gloas_envelope_on_mismatch(&envelope()).is_ok()); } diff --git a/crates/net/p2p/src/beacon/pq.rs b/crates/net/p2p/src/beacon/pq.rs new file mode 100644 index 000000000..42c951d16 --- /dev/null +++ b/crates/net/p2p/src/beacon/pq.rs @@ -0,0 +1,545 @@ +//! Gossip on a PQ fork digest's topics whose container the fork reshapes: +//! decode by the topic's digest, the cheap rules inline, and the stateful ones +//! on a blocking thread. The rules are +//! `ethlambda_state_transition::beacon::gossip::pq`'s, and the bid and +//! preference rules gloas's own through their fork traits. +//! +//! An accepted message is relayed and recorded in the seen caches the other +//! forks' messages share (and its bid or preferences in the PQ builder +//! market). The chain actor then takes what fork choice reads: an aggregate's +//! vote and attesting indices, a payload attestation message, and an envelope +//! (queued ones too, as for gloas). Blocks and data column sidecars take the +//! shared path. + +use std::time::Instant; + +use std::time::Instant as StdInstant; + +use ethlambda_network_api::{AggregateArrival, BlockArrival}; +use ethlambda_state_transition::beacon::builder_market::SharedPqBuilderMarket; +use ethlambda_state_transition::beacon::gossip::{self, IgnoreReason, Outcome}; +use ethlambda_state_transition::beacon::helpers::pq::get_committee_indices; +use ethlambda_storage::{Store, pools::PooledPqAggregate}; +use ethlambda_types::beacon::containers::{SignedExecutionPayloadEnvelope, pq}; +use ethlambda_types::beacon::primitives::ValidatorIndex; +use ethlambda_types::time::unix_now_ms; +use tracing::{debug, trace, warn}; + +use crate::beacon::builder_market::wall_slot; +use crate::beacon::decode::{self as beacon_decode, PqGossip}; +use crate::beacon::topics as beacon_topics; +use crate::beacon::verdict::{Dispatch, Validated}; +use crate::{P2PServer, metrics}; + +/// A PQ message whose stateful checks run on a blocking thread. The one +/// [`Validated`] variant for every PQ topic, so the shared variants, which +/// the chain actor, the pools and the event stream read, carry no PQ shape. +pub(crate) enum PqValidated { + Aggregate { + signed: Box, + /// Filled in by [`Self::stateful_checks`] on `Accept`. + attesting_indices: Vec, + }, + Attestation { + attestation: Box, + subnet_id: u64, + }, + ProposerSlashing(Box), + AttesterSlashing(Box), + PayloadAttestation(Box), + /// Carries the market because the stateful checks receive only the store. + ExecutionPayloadBid { + bid: Box, + market: SharedPqBuilderMarket, + }, + ProposerPreferences(Box), + Envelope(Box), +} + +impl PqValidated { + pub(crate) fn stateful_checks(&mut self, store: &Store) -> Outcome { + match self { + Self::Aggregate { + signed, + attesting_indices, + } => match gossip::pq::aggregate_stateful_checks(store, signed) { + Ok(indices) => { + *attesting_indices = indices; + Outcome::Accept + } + Err(outcome) => outcome, + }, + Self::Attestation { + attestation, + subnet_id, + } => gossip::pq::attestation_stateful_checks(store, attestation, *subnet_id), + Self::ProposerSlashing(slashing) => { + gossip::pq::proposer_slashing_stateful_checks(store, slashing) + } + Self::AttesterSlashing(slashing) => { + gossip::pq::attester_slashing_stateful_checks(store, slashing) + } + Self::PayloadAttestation(message) => { + gossip::pq::payload_attestation_stateful_checks(store, message) + } + Self::ExecutionPayloadBid { bid, market } => { + gossip::execution_payload_bid::stateful_checks(store, &**market, &**bid) + } + Self::ProposerPreferences(preferences) => { + gossip::proposer_preferences::stateful_checks(store, &**preferences) + } + Self::Envelope(envelope) => gossip::pq::envelope_stateful_checks(store, envelope), + } + } + + /// Record this message as the first valid one for its key, in the seen + /// caches the other forks' messages of its topic share. `false` when + /// another verdict recorded one first. + pub(crate) fn record_seen(&self, server: &mut P2PServer) -> bool { + match self { + Self::Aggregate { signed, .. } => { + let Some(key) = gossip::pq::aggregate_key(signed) else { + return false; + }; + let bits = gossip::pq::aggregation_bits(signed); + server.seen_aggregates.record_parts(&key, &bits) + } + Self::Attestation { attestation, .. } => server + .seen_attestations + .record_key(attestation.data.target.epoch, attestation.attester_index), + Self::ProposerSlashing(slashing) => server + .seen_operations + .record_proposer_slashing(slashing.signed_header_1.message.proposer_index), + Self::AttesterSlashing(slashing) => server + .seen_operations + .record_attester_slashing(&gossip::pq::slashed_indices(slashing)), + Self::PayloadAttestation(message) => server + .seen_payload_attestations + .record(message.data.slot, message.validator_index), + Self::ExecutionPayloadBid { bid, .. } => { + server.pq_builder_market.record_bid((**bid).clone()) + } + Self::ProposerPreferences(preferences) => server + .pq_builder_market + .record_preferences((**preferences).clone(), wall_slot(server)), + Self::Envelope(envelope) => { + let recorded = server.seen_envelopes.record( + envelope.message.beacon_block_root, + envelope.message.builder_index, + ); + if recorded { + // A PQ bid's parent payload is known once its envelope + // passed gossip, as a gloas bid's is. + server + .pq_builder_market + .record_pq_execution_payload(&envelope.message); + } + recorded + } + } + } + + /// Hand the parts fork choice reads to the chain actor, given the gossip + /// `outcome`: an aggregate's vote and a payload attestation only when + /// accepted, an envelope also when queued (the actor holds one whose block + /// it has not imported). Subnet attestations, slashings, bids and + /// preferences stop at relay, as gloas's do. + pub(crate) fn forward(self, server: &P2PServer, received_at: StdInstant, outcome: Outcome) { + if let Self::Aggregate { + signed, + attesting_indices, + } = &self + && outcome == Outcome::Accept + { + pool_aggregate(&server.store, signed, attesting_indices); + } + if let Self::Attestation { attestation, .. } = &self + && outcome == Outcome::Accept + && let Some(wire) = server.wire.beacon() + { + let now = crate::gossipsub::beacon_wall_slot(wire); + server.store.pq_signature_pool().insert(now, attestation); + } + let Some(blockchain) = &server.blockchain else { + return; + }; + match self { + Self::Aggregate { + signed, + attesting_indices, + } if outcome == Outcome::Accept => { + let arrival = AggregateArrival { + decode_start: received_at, + handed_off: StdInstant::now(), + }; + let data = signed.message.aggregate.data; + let _ = blockchain + .new_pq_aggregate(data, attesting_indices, arrival) + .inspect_err(|err| warn!(%err, "Failed to forward a PQ aggregate")); + } + Self::PayloadAttestation(message) if outcome == Outcome::Accept => { + let arrival = BlockArrival { + decode_start: Some(received_at), + handed_off: StdInstant::now(), + deferred_from: None, + }; + let _ = blockchain + .new_pq_payload_attestation_message(message, arrival) + .inspect_err(|err| warn!(%err, "Failed to forward a PQ payload attestation")); + } + Self::Envelope(envelope) if matches!(outcome, Outcome::Accept | Outcome::Queue(_)) => { + let arrival = BlockArrival { + decode_start: Some(received_at), + handed_off: StdInstant::now(), + deferred_from: None, + }; + let envelope = Box::new(SignedExecutionPayloadEnvelope::Pq(envelope)); + let _ = blockchain + .new_execution_payload_envelope(envelope, arrival) + .inspect_err(|err| warn!(%err, "Failed to forward a PQ envelope")); + } + _ => {} + } + } + + /// The permit pool its stateful checks draw from, as the matching gloas + /// object's does. + pub(crate) fn pool(&self) -> &'static str { + match self { + Self::Envelope(_) => "gossip", + Self::Aggregate { .. } + | Self::Attestation { .. } + | Self::PayloadAttestation(_) + | Self::ProposerSlashing(_) + | Self::AttesterSlashing(_) => "attestation", + Self::ExecutionPayloadBid { .. } | Self::ProposerPreferences(_) => "builder", + } + } +} + +/// Hold an accepted aggregate for block production, which merges the proofs +/// of the ones it packs (PQ `validator.md`, "Proof"). +fn pool_aggregate( + store: &Store, + signed: &pq::SignedAggregateAndProof, + attesting_indices: &[ValidatorIndex], +) { + let aggregate = &signed.message.aggregate; + let committees = get_committee_indices(&aggregate.committee_bits); + // Gossip validation accepts single-committee aggregates only. + let [committee_index] = committees[..] else { + return; + }; + store.pq_aggregate_pool().insert(PooledPqAggregate { + attestation: aggregate.clone(), + committee_index, + attesting_indices: attesting_indices.to_vec(), + proof: signed.message.proof.clone(), + }); +} + +/// Decode a message from a PQ digest's topic `kind` and run its cheap rules, +/// or `None` when `kind` is not one whose container PQ reshapes (a block or a +/// data column sidecar), which the shared path takes. +/// +/// A message that does not decode is ignored rather than rejected, as on +/// every topic of a fork this node does not follow: its containers are +/// modelled from a moving specification branch, and a peer must not be +/// scored down for this build's gap. +pub(crate) fn triage( + server: &P2PServer, + kind: &str, + payload: &[u8], + received_at: Instant, +) -> Option { + if !beacon_decode::is_pq_reshaped(kind) { + return None; + } + let metric_kind = beacon_topics::metric_kind(kind).unwrap_or(kind); + let decoded = match beacon_decode::decode_pq_gossip(kind, payload) { + Ok(decoded) => decoded, + Err(err) => { + metrics::inc_beacon_gossip(metric_kind, "unsupported_fork"); + debug!(kind, %err, bytes = payload.len(), "PQ gossip decode failed"); + return Some(Dispatch::Report(Outcome::Ignore( + IgnoreReason::UnsupportedFork, + ))); + } + }; + metrics::inc_beacon_gossip(metric_kind, "decoded"); + trace!( + kind, + bytes = payload.len(), + elapsed_ms = received_at.elapsed().as_millis() as u64, + "PQ gossip decoded" + ); + let now_ms = unix_now_ms(); + let store = &server.store; + let cheap = match &decoded { + PqGossip::Aggregate(signed) => { + gossip::pq::aggregate_cheap_checks(&server.seen_aggregates, store, signed, now_ms) + } + PqGossip::Attestation(attestation) => gossip::pq::attestation_cheap_checks( + &server.seen_attestations, + store, + attestation, + now_ms, + ), + PqGossip::ProposerSlashing(slashing) => { + gossip::pq::proposer_slashing_cheap_checks(&server.seen_operations, slashing) + } + PqGossip::AttesterSlashing(slashing) => { + gossip::pq::attester_slashing_cheap_checks(&server.seen_operations, slashing) + } + PqGossip::PayloadAttestation(message) => gossip::pq::payload_attestation_cheap_checks( + &server.seen_payload_attestations, + store, + message, + now_ms, + ), + PqGossip::ExecutionPayloadBid(bid) => gossip::execution_payload_bid::cheap_checks( + &*server.pq_builder_market, + store, + &**bid, + now_ms, + ), + PqGossip::ProposerPreferences(preferences) => gossip::proposer_preferences::cheap_checks( + &*server.pq_builder_market, + store, + &**preferences, + now_ms, + ), + PqGossip::Envelope(envelope) => { + gossip::pq::envelope_cheap_checks(&server.seen_envelopes, store, envelope) + } + }; + if let Err(outcome) = cheap { + return Some(Dispatch::Report(outcome)); + } + let object = match decoded { + PqGossip::Aggregate(signed) => PqValidated::Aggregate { + signed, + attesting_indices: Vec::new(), + }, + PqGossip::Attestation(attestation) => { + let subnet_id = beacon_topics::attestation_subnet(kind) + .expect("decode_pq_gossip decodes an attestation only off a subnet topic"); + PqValidated::Attestation { + attestation, + subnet_id, + } + } + PqGossip::ProposerSlashing(slashing) => PqValidated::ProposerSlashing(slashing), + PqGossip::AttesterSlashing(slashing) => PqValidated::AttesterSlashing(slashing), + PqGossip::PayloadAttestation(message) => PqValidated::PayloadAttestation(message), + PqGossip::ExecutionPayloadBid(bid) => PqValidated::ExecutionPayloadBid { + bid, + market: server.pq_builder_market.clone(), + }, + PqGossip::ProposerPreferences(preferences) => PqValidated::ProposerPreferences(preferences), + PqGossip::Envelope(envelope) => PqValidated::Envelope(envelope), + }; + Some(Dispatch::Validate(Validated::Pq(Box::new(object)))) +} + +/// Gossip one of a validator client's PQ messages, which the Beacon API +/// validated with PQ's gossip rules, on its topic under the digest of its +/// slot, record it in the seen caches gossip validation reads, and hand the +/// chain actor what fork choice takes: gossip never delivers a node its own +/// messages. +pub(crate) fn publish_pq_gossip( + server: &mut P2PServer, + object: ethlambda_network_api::PqGossipObject, +) { + use ethlambda_network_api::PqGossipObject; + use libp2p::gossipsub::IdentTopic; + use libssz::SszEncode as _; + + let slot = match &object { + PqGossipObject::Attestation { attestation, .. } => attestation.data.slot, + PqGossipObject::Aggregate { aggregate, .. } => aggregate.message.aggregate.data.slot, + PqGossipObject::PayloadAttestation(message) => message.data.slot, + }; + let Some(beacon) = server.wire.beacon() else { + warn!(slot, "A PQ message reached a lean node; dropping it"); + return; + }; + let Some(digest) = beacon.publish_digest(slot) else { + warn!( + slot, + "No held fork digest covers this PQ message's slot; not publishing" + ); + return; + }; + let (topic, bytes) = match &object { + PqGossipObject::Attestation { + subnet_id, + attestation, + } => { + server + .seen_attestations + .record_key(attestation.data.target.epoch, attestation.attester_index); + ( + beacon_topics::attestation_topic_name(digest, *subnet_id), + attestation.to_ssz(), + ) + } + PqGossipObject::Aggregate { + aggregate, + attesting_indices, + } => { + // Gossip never hands a node its own aggregate back. + pool_aggregate(&server.store, aggregate, attesting_indices); + if let Some(key) = gossip::pq::aggregate_key(aggregate) { + let bits = gossip::pq::aggregation_bits(aggregate); + server.seen_aggregates.record_parts(&key, &bits); + } + ( + beacon_topics::topic_name(digest, beacon_topics::BEACON_AGGREGATE_AND_PROOF), + aggregate.to_ssz(), + ) + } + PqGossipObject::PayloadAttestation(message) => { + server + .seen_payload_attestations + .record(message.data.slot, message.validator_index); + ( + beacon_topics::topic_name(digest, beacon_topics::PAYLOAD_ATTESTATION_MESSAGE), + message.to_ssz(), + ) + } + }; + server.swarm_handle.publish( + IdentTopic::new(topic), + crate::gossipsub::compress_message(&bytes), + ); + debug!(slot, "Published a validator client's PQ message"); + + let Some(blockchain) = server.blockchain.clone() else { + return; + }; + match object { + PqGossipObject::Attestation { .. } => {} + PqGossipObject::Aggregate { + aggregate, + attesting_indices, + } => { + let arrival = AggregateArrival { + decode_start: StdInstant::now(), + handed_off: StdInstant::now(), + }; + let _ = blockchain + .new_pq_aggregate(aggregate.message.aggregate.data, attesting_indices, arrival) + .inspect_err(|err| warn!(%err, "Failed to hand over a published PQ aggregate")); + } + PqGossipObject::PayloadAttestation(message) => { + let _ = blockchain + .new_pq_payload_attestation_message(message, BlockArrival::now()) + .inspect_err( + |err| warn!(%err, "Failed to hand over a published PQ payload attestation"), + ); + } + } +} + +#[cfg(test)] +mod tests { + use ethlambda_state_transition::beacon::gossip::RejectReason; + use ethlambda_types::beacon::config::Config; + use ethlambda_types::beacon::containers::shared::BeaconBlockHeader; + use ethlambda_types::beacon::fork::ForkName; + use libssz::SszEncode as _; + + use super::*; + use crate::test_support::unconnected_beacon_server; + + fn config() -> Config { + Config::mainnet() + .with_fork_epoch(ForkName::Gloas, 0) + .with_fork_epoch(ForkName::Eip8483, 0) + .with_fork_epoch(ForkName::Pq, 0) + } + + fn triage_now(server: &P2PServer, kind: &str, payload: &[u8]) -> Option { + triage(server, kind, payload, Instant::now()) + } + + /// A message this build cannot decode under a PQ digest is this build's + /// gap, as on every topic of a fork the node does not follow: ignored, + /// so no peer is scored down for it. + #[tokio::test] + async fn an_undecodable_pq_message_is_ignored_not_rejected() { + let server = unconnected_beacon_server(config(), 0).await; + let dispatch = triage_now( + &server, + beacon_topics::BEACON_AGGREGATE_AND_PROOF, + &[0xff; 9], + ); + assert!(matches!( + dispatch, + Some(Dispatch::Report(Outcome::Ignore( + IgnoreReason::UnsupportedFork + ))) + )); + } + + /// A block or a column takes the shared path under a PQ digest too. + #[tokio::test] + async fn a_pq_digests_block_topic_is_not_triaged_here() { + let server = unconnected_beacon_server(config(), 0).await; + assert!(triage_now(&server, beacon_topics::BEACON_BLOCK, &[]).is_none()); + } + + /// A slashing that passes the cheap rules goes on to the stateful ones as + /// a PQ object; one whose headers do not match is rejected inline. + #[tokio::test] + async fn a_pq_proposer_slashing_is_judged_as_pq() { + let server = unconnected_beacon_server(config(), 0).await; + let header = |state_root| pq::SignedBeaconBlockHeader { + message: BeaconBlockHeader { + slot: 5, + proposer_index: 2, + state_root, + ..Default::default() + }, + signature: Default::default(), + }; + let slashing = pq::ProposerSlashing { + signed_header_1: header(ethlambda_types::beacon::primitives::Root::repeat_byte(1)), + signed_header_2: header(ethlambda_types::beacon::primitives::Root::repeat_byte(2)), + }; + let dispatch = triage_now( + &server, + beacon_topics::PROPOSER_SLASHING, + &slashing.to_ssz(), + ); + assert!(matches!( + dispatch, + Some(Dispatch::Validate(Validated::Pq(ref object))) + if matches!(**object, PqValidated::ProposerSlashing(_)) + )); + + let same = pq::ProposerSlashing { + signed_header_1: slashing.signed_header_1.clone(), + signed_header_2: slashing.signed_header_1, + }; + assert!(matches!( + triage_now(&server, beacon_topics::PROPOSER_SLASHING, &same.to_ssz()), + Some(Dispatch::Report(Outcome::Reject( + RejectReason::InvalidOperation + ))) + )); + } + + /// The PQ node does not follow PQ, so its head state is not a PQ one: a + /// slashing that reaches the stateful rules is ignored. + #[tokio::test] + async fn a_pq_slashing_against_a_non_pq_head_is_ignored() { + let server = unconnected_beacon_server(config(), 0).await; + let mut object = PqValidated::ProposerSlashing(Box::default()); + assert_eq!( + object.stateful_checks(&server.store), + Outcome::Ignore(IgnoreReason::StateUnavailable) + ); + } +} diff --git a/crates/net/p2p/src/beacon/scoring.rs b/crates/net/p2p/src/beacon/scoring.rs index 4f17065cb..7f85428b1 100644 --- a/crates/net/p2p/src/beacon/scoring.rs +++ b/crates/net/p2p/src/beacon/scoring.rs @@ -77,15 +77,20 @@ use std::cmp::Ordering; use std::collections::HashSet; use std::time::Duration; -use ethlambda_state_transition::beacon::helpers::accessors::committee_count_per_slot; +use ethlambda_state_transition::beacon::helpers::accessors::{ + committee_count_per_slot, committee_count_per_slot_for, +}; use ethlambda_types::beacon::config::Config; -use ethlambda_types::beacon::constants::TARGET_AGGREGATORS_PER_COMMITTEE; +use ethlambda_types::beacon::constants::{ + AGGREGATORS_PER_COMMITTEE, TARGET_AGGREGATORS_PER_COMMITTEE, +}; +use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::preset::SLOTS_PER_EPOCH; use ethlambda_types::beacon::primitives::ForkDigest; use libp2p::gossipsub::{IdentTopic, PeerScoreParams, PeerScoreThresholds, TopicScoreParams}; use tracing::info; -use crate::beacon::constants::ATTESTATION_SUBNET_COUNT; +use crate::beacon::constants::{ATTESTATION_SUBNET_COUNT, ATTESTATION_SUBNET_COUNT_PQ}; use crate::beacon::topics::{ ATTESTER_SLASHING, BEACON_AGGREGATE_AND_PROOF, BEACON_BLOCK, PROPOSER_SLASHING, VOLUNTARY_EXIT, attestation_topic_name, topic_name, @@ -324,6 +329,47 @@ impl ScoreSettings { topics } + /// [`Self::digest_topic_params`] for a digest of `fork`. A PQ digest + /// scores its own topic set: no `voluntary_exit`, and + /// `ATTESTATION_SUBNET_COUNT_PQ` attestation subnets, each carrying that + /// share of the attesters, with aggregates from aggregation committees. + pub fn digest_topic_params_at( + &self, + fork: ForkName, + fork_digest: ForkDigest, + role: DigestRole, + inputs: ScoreInputs, + ) -> Vec<(IdentTopic, TopicScoreParams)> { + if fork != ForkName::Pq { + return self.digest_topic_params(fork_digest, role, inputs); + } + let inputs = ScoreInputs { + mesh_deliveries_scored: inputs.mesh_deliveries_scored && role == DigestRole::Current, + ..inputs + }; + let voluntary_exit = IdentTopic::new(topic_name(fork_digest, VOLUNTARY_EXIT)).hash(); + let mut topics: Vec<_> = self + .fixed_topic_params(fork_digest) + .into_iter() + .filter(|(topic, _)| topic.hash() != voluntary_exit) + .collect(); + let active = inputs.active_validators.max(PLACEHOLDER_ACTIVE_VALIDATORS); + let (aggregators_per_slot, committees_per_slot) = expected_pq_aggregators_per_slot(active); + topics.extend(self.dynamic_topic_params_with( + fork_digest, + inputs, + aggregators_per_slot, + committees_per_slot, + ATTESTATION_SUBNET_COUNT_PQ, + )); + if role == DigestRole::Previous { + for (_, params) in &mut topics { + params.topic_weight = 0.0; + } + } + topics + } + /// Exits and slashings: rare enough that their rates do not depend on /// the validator count, and never mesh-delivery scored. fn fixed_topic_params(&self, fork_digest: ForkDigest) -> Vec<(IdentTopic, TopicScoreParams)> { @@ -362,10 +408,30 @@ impl ScoreSettings { // it, which gossipsub rejects. let active = inputs.active_validators.max(PLACEHOLDER_ACTIVE_VALIDATORS); let (aggregators_per_slot, committees_per_slot) = expected_aggregators_per_slot(active); + self.dynamic_topic_params_with( + fork_digest, + inputs, + aggregators_per_slot, + committees_per_slot, + ATTESTATION_SUBNET_COUNT, + ) + } + + /// [`Self::dynamic_topic_params`] for `subnet_count` attestation subnets + /// and an expected aggregate rate, the parts a PQ digest changes. + fn dynamic_topic_params_with( + &self, + fork_digest: ForkDigest, + inputs: ScoreInputs, + aggregators_per_slot: f64, + committees_per_slot: u64, + subnet_count: u64, + ) -> Vec<(IdentTopic, TopicScoreParams)> { + let active = inputs.active_validators.max(PLACEHOLDER_ACTIVE_VALIDATORS); // Whether each subnet sees more than one committee's burst an epoch, // which shortens its decay windows. let multiple_bursts_per_subnet_per_epoch = - committees_per_slot >= 2 * ATTESTATION_SUBNET_COUNT / SLOTS_PER_EPOCH; + committees_per_slot >= 2 * subnet_count / SLOTS_PER_EPOCH; // Lighthouse's young-chain rule and this node's sync gate: P3 needs // both a chain older than the topic's decay window and a node that // has kept up with it. @@ -416,9 +482,16 @@ impl ScoreSettings { } else { self.epoch * 4 }; + // The subnets share one unit of weight between them, as lighthouse's + // do, so fewer subnets weigh each more and the sum is unchanged. + let attestation_subnet_weight = if subnet_count == ATTESTATION_SUBNET_COUNT { + self.attestation_subnet_weight + } else { + 1.0 / subnet_count.max(1) as f64 + }; let attestation = self.topic_params( - self.attestation_subnet_weight, - active as f64 / ATTESTATION_SUBNET_COUNT as f64 / SLOTS_PER_EPOCH as f64, + attestation_subnet_weight, + active as f64 / subnet_count.max(1) as f64 / SLOTS_PER_EPOCH as f64, attestation_fmd_decay_time, Some(&attestation_mesh), scored(attestation_mesh.decay_slots), @@ -434,7 +507,7 @@ impl ScoreSettings { aggregate, ), ]; - topics.extend((0..ATTESTATION_SUBNET_COUNT).map(|subnet_id| { + topics.extend((0..subnet_count).map(|subnet_id| { let topic = IdentTopic::new(attestation_topic_name(fork_digest, subnet_id)); (topic, attestation.clone()) })); @@ -539,6 +612,19 @@ fn expected_aggregators_per_slot(active: u64) -> (f64, u64) { (per_epoch / SLOTS_PER_EPOCH as f64, committees_per_slot) } +/// [`expected_aggregators_per_slot`] at PQ: every committee, under PQ's cap, +/// has an aggregation committee of up to `AGGREGATORS_PER_COMMITTEE` members, +/// fewer only when the committee itself is smaller. +fn expected_pq_aggregators_per_slot(active: u64) -> (f64, u64) { + let committees_per_slot = committee_count_per_slot_for(ForkName::Pq, active); + let committee_size = active / (committees_per_slot * SLOTS_PER_EPOCH).max(1); + let per_committee = AGGREGATORS_PER_COMMITTEE.min(committee_size.max(1)); + ( + (committees_per_slot * per_committee) as f64, + committees_per_slot, + ) +} + /// Whether mesh-delivery scoring (P3) is safe to switch on: the head has /// kept up with the wall clock long enough that the delivery counters /// reflect what mesh peers sent, not what this node was able to accept. @@ -590,8 +676,9 @@ pub(crate) struct PeerScoring { active_validators: u64, /// The gate's last verdict, so a change is logged once. mesh_deliveries_scored: bool, - /// The fork digests whose topics the node holds, in activation order. - held: Vec, + /// The fork digests whose topics the node holds, in activation order, + /// each with its fork, whose topic set it scores. + held: Vec<(ForkDigest, ForkName)>, /// The one among [`Self::held`] this node publishes under. current: ForkDigest, } @@ -599,13 +686,13 @@ pub(crate) struct PeerScoring { impl PeerScoring { /// `fork_digest` is the digest the swarm was built on, the only one held /// until the fork schedule is first applied. - pub(crate) fn new(config: &Config, fork_digest: ForkDigest) -> Self { + pub(crate) fn new(config: &Config, fork_digest: ForkDigest, fork: ForkName) -> Self { Self { settings: ScoreSettings::new(config, crate::MESH_N), gate: MeshDeliveryGate::default(), active_validators: PLACEHOLDER_ACTIVE_VALIDATORS, mesh_deliveries_scored: false, - held: vec![fork_digest], + held: vec![(fork_digest, fork)], current: fork_digest, } } @@ -618,7 +705,7 @@ impl PeerScoring { /// swarm's one command channel, so sending these first is enough. pub(crate) fn hold( &mut self, - held: Vec, + held: Vec<(ForkDigest, ForkName)>, current: ForkDigest, wall_slot: u64, swarm: &SwarmHandle, @@ -651,18 +738,22 @@ impl PeerScoring { }; // `apply` checks that the current digest is held before it calls // `hold`, so `None` (every digest previous) is never reached. - let current_index = self.held.iter().position(|&digest| digest == self.current); + let current_index = self + .held + .iter() + .position(|&(digest, _)| digest == self.current); let topics = self .held .iter() .enumerate() - .flat_map(|(index, &digest)| { + .flat_map(|(index, &(digest, fork))| { let role = match current_index.map(|current| index.cmp(¤t)) { Some(Ordering::Equal) => DigestRole::Current, Some(Ordering::Greater) => DigestRole::Upcoming, Some(Ordering::Less) | None => DigestRole::Previous, }; - self.settings.digest_topic_params(digest, role, inputs) + self.settings + .digest_topic_params_at(fork, digest, role, inputs) }) .collect(); swarm.set_topic_score_params(topics); @@ -962,6 +1053,43 @@ mod tests { /// Mainnet's committees saturate at `MAX_COMMITTEES_PER_SLOT`, so about /// sixteen aggregators each: lighthouse's figure of 1041.67 per slot. + /// A PQ digest is scored for its own topics: no `voluntary_exit`, and + /// `ATTESTATION_SUBNET_COUNT_PQ` subnets sharing one unit of weight. + #[test] + fn a_pq_digest_scores_its_own_topics() { + let settings = settings(crate::MESH_N); + let topics = settings.digest_topic_params_at( + ForkName::Pq, + DIGEST, + DigestRole::Current, + inputs(1_000_000, true), + ); + assert_eq!( + topics.len(), + 2 + 2 + crate::beacon::constants::ATTESTATION_SUBNET_COUNT_PQ as usize + ); + let exit = IdentTopic::new(topic_name(DIGEST, VOLUNTARY_EXIT)).hash(); + assert!(topics.iter().all(|(topic, _)| topic.hash() != exit)); + let subnet_weight: f64 = topics + .iter() + .filter(|(topic, _)| topic.to_string().contains("beacon_attestation_")) + .map(|(_, params)| params.topic_weight) + .sum(); + assert_close(subnet_weight, 1.0, "the PQ subnets' summed weight"); + for (_, params) in &topics { + params.validate().expect("a valid topic parameter set"); + } + + // Every fork before PQ keeps the shared set. + let gloas = settings.digest_topic_params_at( + ForkName::Gloas, + DIGEST, + DigestRole::Current, + inputs(1_000_000, true), + ); + assert_eq!(gloas.len(), 2 + 3 + ATTESTATION_SUBNET_COUNT as usize); + } + #[test] fn a_million_validators_expect_about_a_thousand_aggregators_a_slot() { let (per_slot, committees_per_slot) = expected_aggregators_per_slot(1_000_000); @@ -1138,7 +1266,7 @@ mod tests { let (swarm_handle, commands) = SwarmHandle::capturing(); server.swarm_handle = swarm_handle; // The digest `unconnected_beacon_server` builds its swarm on. - server.peer_scoring = Some(PeerScoring::new(config, [0; 4])); + server.peer_scoring = Some(PeerScoring::new(config, [0; 4], ForkName::Fulu)); (server, commands) } @@ -1326,6 +1454,7 @@ mod tests { genesis_validators_root: Root::ZERO, custody_columns: Vec::new(), attestation_subnets: Vec::new(), + attestation_subnets_pq: Vec::new(), })), }) .expect("beacon swarm builds") @@ -1429,6 +1558,7 @@ mod tests { genesis_validators_root: Root::ZERO, custody_columns: vec![3], attestation_subnets: vec![5], + attestation_subnets_pq: Vec::new(), }, )))) .expect("beacon swarm builds"); diff --git a/crates/net/p2p/src/beacon/subnets.rs b/crates/net/p2p/src/beacon/subnets.rs index 169ec87ec..c45292ff9 100644 --- a/crates/net/p2p/src/beacon/subnets.rs +++ b/crates/net/p2p/src/beacon/subnets.rs @@ -91,8 +91,8 @@ pub fn compute_subnet_for_attestation( /// to mainnet's 6 and would shuffle over a space its own subnet count does not /// match. `the_shipped_configs_agree_with_the_derivation` pins the two /// together for the configurations that do carry it. -fn attestation_subnet_prefix_bits(config: &Config) -> u32 { - let count = config.attestation_subnet_count.max(1); +fn attestation_subnet_prefix_bits(config: &Config, subnet_count: u64) -> u32 { + let count = subnet_count.max(1); // ceillog2: how many bits it takes to index `count` values. Exact for the // powers of two every shipped configuration uses, and rounds up for // anything else, which is what `ceillog2` means. @@ -122,7 +122,27 @@ pub fn compute_subscribed_subnet( index: u64, config: &Config, ) -> Result { - let prefix_bits = attestation_subnet_prefix_bits(config); + compute_subscribed_subnet_among( + node_id, + epoch, + index, + config.attestation_subnet_count, + config, + ) +} + +/// [`compute_subscribed_subnet`] over `subnet_count` subnets: the +/// specification's own for `ATTESTATION_SUBNET_COUNT`, and its PQ +/// modification for `ATTESTATION_SUBNET_COUNT_PQ`, which changes nothing but +/// the count (`compute_attestation_subnet_prefix_bits` and the final modulo). +fn compute_subscribed_subnet_among( + node_id: [u8; 32], + epoch: u64, + index: u64, + subnet_count: u64, + config: &Config, +) -> Result { + let prefix_bits = attestation_subnet_prefix_bits(config, subnet_count); // Strictly less than `u64::BITS`: `1u64 << prefix_bits` below requires a // shift strictly less than the type's width, and `prefix_bits == // u64::BITS` would overflow it (a panic in debug, `1` itself in release, @@ -145,7 +165,7 @@ pub fn compute_subscribed_subnet( // `compute_shuffled_index`'s own precondition. let permutated_prefix = compute_shuffled_index(node_id_prefix, 1u64 << prefix_bits, permutation_seed)?; - Ok(permutated_prefix.saturating_add(index) % config.attestation_subnet_count) + Ok(permutated_prefix.saturating_add(index) % subnet_count.max(1)) } /// Every subnet `node_id` subscribes to at `epoch`, sorted ascending and @@ -163,10 +183,30 @@ pub fn compute_subscribed_subnets( node_id: [u8; 32], epoch: u64, config: &Config, +) -> Result> { + subscribed_subnets_among(node_id, epoch, config.attestation_subnet_count, config) +} + +/// The subnets `node_id` backbones under a PQ digest, among +/// `ATTESTATION_SUBNET_COUNT_PQ` (`compute_subscribed_subnet` modified in PQ). +/// Sorted and deduplicated, as [`compute_subscribed_subnets`]. +pub fn compute_subscribed_subnets_pq( + node_id: [u8; 32], + epoch: u64, + config: &Config, +) -> Result> { + subscribed_subnets_among(node_id, epoch, config.attestation_subnet_count_pq, config) +} + +fn subscribed_subnets_among( + node_id: [u8; 32], + epoch: u64, + subnet_count: u64, + config: &Config, ) -> Result> { let mut subnets = Vec::with_capacity(config.subnets_per_node as usize); for index in 0..config.subnets_per_node { - let subnet = compute_subscribed_subnet(node_id, epoch, index, config)?; + let subnet = compute_subscribed_subnet_among(node_id, epoch, index, subnet_count, config)?; if !subnets.contains(&subnet) { subnets.push(subnet); } @@ -220,7 +260,10 @@ mod tests { fn the_shipped_configs_agree_with_the_derivation() { for config in [Config::mainnet(), Config::minimal()] { assert_eq!( - u64::from(attestation_subnet_prefix_bits(&config)), + u64::from(attestation_subnet_prefix_bits( + &config, + config.attestation_subnet_count + )), config.attestation_subnet_prefix_bits, "the derived prefix must match the configured one" ); @@ -240,14 +283,20 @@ mod tests { attestation_subnet_extra_bits: 58, ..Config::mainnet() }; - assert_eq!(attestation_subnet_prefix_bits(&config), u64::BITS); + assert_eq!( + attestation_subnet_prefix_bits(&config, config.attestation_subnet_count), + u64::BITS + ); assert!(compute_subscribed_subnet([0u8; 32], 0, 0, &config).is_err()); } #[test] fn mainnet_takes_the_top_six_bits_of_the_node_id() { let config = Config::mainnet(); - assert_eq!(attestation_subnet_prefix_bits(&config), 6); + assert_eq!( + attestation_subnet_prefix_bits(&config, config.attestation_subnet_count), + 6 + ); // 0b1010_1100: the top six bits are 0b101011, which is 43. let mut node_id = [0u8; 32]; node_id[0] = 0b1010_1100; @@ -374,4 +423,21 @@ mod tests { } } } + + /// The PQ backbone is the same function over `ATTESTATION_SUBNET_COUNT_PQ`: + /// every subnet it names is one of those. + #[test] + fn the_pq_backbone_stays_within_the_pq_subnets() { + let config = Config::mainnet(); + for seed in 0..32u8 { + let subnets = compute_subscribed_subnets_pq([seed; 32], 0, &config).unwrap(); + assert!(!subnets.is_empty()); + assert!( + subnets + .iter() + .all(|&subnet| subnet < config.attestation_subnet_count_pq), + "{subnets:?}" + ); + } + } } diff --git a/crates/net/p2p/src/beacon/swarm.rs b/crates/net/p2p/src/beacon/swarm.rs index bb90cdbdf..5ef61bd06 100644 --- a/crates/net/p2p/src/beacon/swarm.rs +++ b/crates/net/p2p/src/beacon/swarm.rs @@ -172,6 +172,9 @@ pub struct BeaconWireConfig { /// set, and a peer computes the same set from this node's id, so deriving /// it twice is how the two would come to disagree. pub attestation_subnets: Vec, + /// The attestation subnets this node backbones under a PQ digest, from + /// the same node id among `ATTESTATION_SUBNET_COUNT_PQ`. + pub attestation_subnets_pq: Vec, } #[cfg(test)] @@ -284,6 +287,7 @@ mod tests { genesis_validators_root: Root::ZERO, custody_columns: custody_columns.clone(), attestation_subnets: Vec::new(), + attestation_subnets_pq: Vec::new(), })), }) .expect("swarm builds"); diff --git a/crates/net/p2p/src/beacon/sync_committee.rs b/crates/net/p2p/src/beacon/sync_committee.rs index bfba3355a..830aed528 100644 --- a/crates/net/p2p/src/beacon/sync_committee.rs +++ b/crates/net/p2p/src/beacon/sync_committee.rs @@ -143,9 +143,10 @@ fn join(server: &mut P2PServer, subnets: Vec<(u64, Epoch)>) -> Vec { } // Under every digest the node holds, like the aggregator subnets: while a // boundary's window is open, messages are published on either side of it. + // Not a PQ digest's, which carries no sync committee. let mut subscribed = Vec::new(); for &subnet_id in &joined { - for held in wire.held_topics() { + for held in wire.held_topics().filter(|held| !held.pq) { let name = topics::sync_committee_topic_name(held.fork_digest, subnet_id); server.swarm_handle.subscribe(IdentTopic::new(name.clone())); subscribed.push(name); @@ -188,7 +189,7 @@ fn leave_expired(server: &mut P2PServer, epoch: Epoch) -> Vec { let mut unsubscribed = Vec::new(); for subnet_id in &expired { wire.sync_committee_subnets.remove(subnet_id); - for held in wire.held_topics() { + for held in wire.held_topics().filter(|held| !held.pq) { let name = topics::sync_committee_topic_name(held.fork_digest, *subnet_id); server .swarm_handle diff --git a/crates/net/p2p/src/beacon/topics.rs b/crates/net/p2p/src/beacon/topics.rs index 67a99a423..d1690a450 100644 --- a/crates/net/p2p/src/beacon/topics.rs +++ b/crates/net/p2p/src/beacon/topics.rs @@ -2,9 +2,11 @@ //! //! Seven global topics (four more from gloas: the execution payload envelope, //! the payload attestation message, the builder bid and the proposer -//! preferences), plus two families this node's own node id selects a +//! preferences; three fewer at PQ, which drops the exit, BLS-change and sync +//! contribution topics), plus two families this node's own node id selects a //! narrow slice of: the data column subnets it custodies, and the -//! `SUBNETS_PER_NODE` attestation subnets it backbones. +//! `SUBNETS_PER_NODE` attestation subnets it backbones (among +//! `ATTESTATION_SUBNET_COUNT_PQ` under a PQ digest). //! //! "Only what it consumes" is no longer the whole rule, and //! `beacon_attestation_{subnet_id}` is where it stops applying. @@ -70,8 +72,24 @@ pub const GLOAS_TOPIC_KINDS: [&str; 4] = [ PROPOSER_PREFERENCES, ]; -/// Every topic kind this node subscribes to at every fork, in the order they -/// are subscribed. [`GLOAS_TOPIC_KINDS`] follow from gloas. +/// The global topic kinds a PQ digest carries (`pq/p2p-interface.md`): +/// gloas's, without `voluntary_exit` (exits are execution-layer requests), +/// `bls_to_execution_change` (disabled) and +/// `sync_committee_contribution_and_proof` (sync committees are removed). +pub const PQ_TOPIC_KINDS: [&str; 8] = [ + BEACON_BLOCK, + BEACON_AGGREGATE_AND_PROOF, + PROPOSER_SLASHING, + ATTESTER_SLASHING, + EXECUTION_PAYLOAD, + PAYLOAD_ATTESTATION_MESSAGE, + EXECUTION_PAYLOAD_BID, + PROPOSER_PREFERENCES, +]; + +/// Every topic kind this node subscribes to at every fork before PQ, in the +/// order they are subscribed. [`GLOAS_TOPIC_KINDS`] follow from gloas; +/// [`PQ_TOPIC_KINDS`] replace both at PQ. pub const SUBSCRIBED_TOPIC_KINDS: [&str; 7] = [ BEACON_BLOCK, BEACON_AGGREGATE_AND_PROOF, @@ -240,8 +258,10 @@ pub fn data_column_subnet(kind: &str) -> Option { #[derive(Debug, Clone)] pub struct BeaconTopics { pub fork_digest: ForkDigest, - /// Parallel to [`SUBSCRIBED_TOPIC_KINDS`], then `column_topics`'s values in - /// ascending subnet order. Derived from `column_topics` rather than built + /// The fork's global topic kinds ([`SUBSCRIBED_TOPIC_KINDS`] and, from + /// gloas, [`GLOAS_TOPIC_KINDS`]; [`PQ_TOPIC_KINDS`] at PQ), then + /// `column_topics`'s values in ascending subnet order, then + /// `attestation_topics`'s. Derived from `column_topics` rather than built /// alongside it, so the two cannot disagree about how many column topics /// there are: pushing one entry per `column_subnets` element here, while a /// map dedupes the same input, is exactly how the two would drift the @@ -266,6 +286,11 @@ pub struct BeaconTopics { /// /// A `BTreeMap` for the reason `column_topics` is one. pub attestation_topics: BTreeMap, + /// Whether these are a PQ digest's topics. Sync committees are removed at + /// PQ, and its attestation subnets are `ATTESTATION_SUBNET_COUNT_PQ`, so + /// the sync committee and aggregator subnets a validator client asks for, + /// both from BLS duties, are never joined under one. + pub pq: bool, } impl BeaconTopics { @@ -290,12 +315,26 @@ impl BeaconTopics { | ForkName::Electra | ForkName::Fulu => Self::new(fork_digest, column_subnets, attestation_subnets), // Gloas adds the payload envelope and the payload attestation - // topics, which the follower consumes. - ForkName::Gloas => Self::with_extra_kinds( + // topics, which the follower consumes. EIP-8483 builds on it and + // keeps its topics: its gossip containers are gloas's, bar the + // block and envelope. + ForkName::Gloas | ForkName::Eip8483 => Self::with_kinds( + fork_digest, + SUBSCRIBED_TOPIC_KINDS.iter().chain(&GLOAS_TOPIC_KINDS), + column_subnets, + attestation_subnets, + false, + ), + // PQ's own set, and `attestation_subnets` is its own backbone + // (see [`backbone_subnets`]). The node does not follow PQ, so + // its blocks, columns and envelopes are ignored there, while the + // rest are judged by `gossip::pq` and relayed. + ForkName::Pq => Self::with_kinds( fork_digest, - &GLOAS_TOPIC_KINDS, + PQ_TOPIC_KINDS.iter(), column_subnets, attestation_subnets, + true, ), ForkName::Lean => { unreachable!("a beacon topic's fork is never Lean: it is absent from ForkName::ALL") @@ -303,21 +342,29 @@ impl BeaconTopics { } } + /// A fork before gloas's topics: [`SUBSCRIBED_TOPIC_KINDS`] and the two + /// subnet families. pub fn new( fork_digest: ForkDigest, column_subnets: &[u64], attestation_subnets: &[u64], ) -> Self { - Self::with_extra_kinds(fork_digest, &[], column_subnets, attestation_subnets) + Self::with_kinds( + fork_digest, + SUBSCRIBED_TOPIC_KINDS.iter(), + column_subnets, + attestation_subnets, + false, + ) } - /// [`Self::new`] plus the global topic `extra_kinds` a fork adds, placed - /// after [`SUBSCRIBED_TOPIC_KINDS`] and ahead of the subnet families. - fn with_extra_kinds( + /// The global topic `kinds`, in order and ahead of the subnet families. + fn with_kinds<'a>( fork_digest: ForkDigest, - extra_kinds: &[&str], + kinds: impl Iterator, column_subnets: &[u64], attestation_subnets: &[u64], + pq: bool, ) -> Self { // Built first so the map's own key semantics do the deduplication; // `topics` below only ever sees what survived that. @@ -335,9 +382,7 @@ impl BeaconTopics { .or_insert_with(|| IdentTopic::new(attestation_topic_name(fork_digest, subnet_id))); } - let mut topics: Vec = SUBSCRIBED_TOPIC_KINDS - .iter() - .chain(extra_kinds) + let mut topics: Vec = kinds .map(|kind| IdentTopic::new(topic_name(fork_digest, kind))) .collect(); topics.extend(column_topics.values().cloned()); @@ -348,10 +393,26 @@ impl BeaconTopics { topics, column_topics, attestation_topics, + pq, } } } +/// The attestation subnets this node backbones under a digest of `fork`: +/// PQ's own set (`compute_subscribed_subnet` modified in PQ, over +/// `ATTESTATION_SUBNET_COUNT_PQ`) at PQ, the shared one before it. +pub fn backbone_subnets<'a>( + fork: ForkName, + subnets: &'a [u64], + subnets_pq: &'a [u64], +) -> &'a [u64] { + if fork == ForkName::Pq { + subnets_pq + } else { + subnets + } +} + #[cfg(test)] mod tests { use super::*; @@ -384,6 +445,48 @@ mod tests { assert!(name.starts_with("/eth2/0abcdef0/"), "got {name}"); } + /// `pq/p2p-interface.md`: no `voluntary_exit`, `bls_to_execution_change` + /// or `sync_committee_contribution_and_proof` under a PQ digest, and its + /// own backbone. + #[test] + fn a_pq_digest_holds_its_own_topic_set_and_backbone() { + let pq = BeaconTopics::for_fork( + ForkName::Pq, + MAINNET, + &[], + backbone_subnets(ForkName::Pq, &[40], &[7]), + ); + assert!(pq.pq); + let names: Vec = pq.topics.iter().map(ToString::to_string).collect(); + for kind in PQ_TOPIC_KINDS { + assert!(names.contains(&topic_name(MAINNET, kind)), "{kind}"); + } + for kind in [ + VOLUNTARY_EXIT, + BLS_TO_EXECUTION_CHANGE, + SYNC_COMMITTEE_CONTRIBUTION_AND_PROOF, + ] { + assert!(!names.contains(&topic_name(MAINNET, kind)), "{kind}"); + } + assert_eq!( + pq.attestation_topics.keys().copied().collect::>(), + [7] + ); + + let gloas = BeaconTopics::for_fork( + ForkName::Gloas, + MAINNET, + &[], + backbone_subnets(ForkName::Gloas, &[40], &[7]), + ); + assert!(!gloas.pq); + assert_eq!(gloas.topics.len(), 7 + 4 + 1); + assert_eq!( + gloas.attestation_topics.keys().copied().collect::>(), + [40] + ); + } + #[test] fn subscriptions_are_exactly_the_seven_global_topics() { let topics = BeaconTopics::new(MAINNET, &[], &[]); diff --git a/crates/net/p2p/src/beacon/transition.rs b/crates/net/p2p/src/beacon/transition.rs index 38c11e599..18959375d 100644 --- a/crates/net/p2p/src/beacon/transition.rs +++ b/crates/net/p2p/src/beacon/transition.rs @@ -139,7 +139,10 @@ pub(crate) fn apply(server: &mut P2PServer, epoch: Epoch) -> Changes { // topic had parameters when the peer was grafted, and joining grafts at // once. if let Some(scoring) = &mut server.peer_scoring { - let digests = held.iter().map(|entry| entry.digest).collect(); + let digests = held + .iter() + .map(|entry| (entry.digest, entry.fork)) + .collect(); scoring.hold( digests, current.digest, @@ -164,21 +167,36 @@ pub(crate) fn apply(server: &mut P2PServer, epoch: Epoch) -> Changes { entry.fork, entry.digest, &column_subnets, - &wire.attestation_subnets, + topics::backbone_subnets( + entry.fork, + &wire.attestation_subnets, + &wire.attestation_subnets_pq, + ), ); for topic in &joined.topics { server.swarm_handle.subscribe(topic.clone()); changes.subscribed.push(topic.to_string()); } - for &subnet_id in server.aggregator_subnets.keys() { - let topic = attestation_topic(entry.digest, subnet_id); - changes.subscribed.push(topic.to_string()); - server.swarm_handle.subscribe(topic); - } - for &subnet_id in wire.sync_committee_subnets.keys() { - let topic = sync_committee_topic(entry.digest, subnet_id); - changes.subscribed.push(topic.to_string()); - server.swarm_handle.subscribe(topic); + // A validator client's aggregator and sync committee subnets come from + // BLS duties, which a PQ digest has neither of; its PQ aggregators' + // subnets are among PQ's 16 and joined under PQ digests only. + if joined.pq { + for &subnet_id in server.aggregator_subnets_pq.keys() { + let topic = attestation_topic(entry.digest, subnet_id); + changes.subscribed.push(topic.to_string()); + server.swarm_handle.subscribe(topic); + } + } else { + for &subnet_id in server.aggregator_subnets.keys() { + let topic = attestation_topic(entry.digest, subnet_id); + changes.subscribed.push(topic.to_string()); + server.swarm_handle.subscribe(topic); + } + for &subnet_id in wire.sync_committee_subnets.keys() { + let topic = sync_committee_topic(entry.digest, subnet_id); + changes.subscribed.push(topic.to_string()); + server.swarm_handle.subscribe(topic); + } } info!( fork_digest = %hex::encode(entry.digest), @@ -196,15 +214,23 @@ pub(crate) fn apply(server: &mut P2PServer, epoch: Epoch) -> Changes { server.swarm_handle.unsubscribe(topic.clone()); changes.unsubscribed.push(topic.to_string()); } - for &subnet_id in server.aggregator_subnets.keys() { - let topic = attestation_topic(left.fork_digest, subnet_id); - changes.unsubscribed.push(topic.to_string()); - server.swarm_handle.unsubscribe(topic); - } - for &subnet_id in wire.sync_committee_subnets.keys() { - let topic = sync_committee_topic(left.fork_digest, subnet_id); - changes.unsubscribed.push(topic.to_string()); - server.swarm_handle.unsubscribe(topic); + if left.pq { + for &subnet_id in server.aggregator_subnets_pq.keys() { + let topic = attestation_topic(left.fork_digest, subnet_id); + changes.unsubscribed.push(topic.to_string()); + server.swarm_handle.unsubscribe(topic); + } + } else { + for &subnet_id in server.aggregator_subnets.keys() { + let topic = attestation_topic(left.fork_digest, subnet_id); + changes.unsubscribed.push(topic.to_string()); + server.swarm_handle.unsubscribe(topic); + } + for &subnet_id in wire.sync_committee_subnets.keys() { + let topic = sync_committee_topic(left.fork_digest, subnet_id); + changes.unsubscribed.push(topic.to_string()); + server.swarm_handle.unsubscribe(topic); + } } info!( fork_digest = %hex::encode(left.fork_digest), @@ -425,6 +451,47 @@ mod tests { assert!(has_no_gloas_kinds(&wire.window_topics[0])); } + /// A PQ digest is joined with PQ's own topic set and backbone, and without + /// the aggregator and sync committee subnets a validator client asked for, + /// which come from BLS duties. + #[tokio::test] + async fn a_pq_digest_is_joined_with_its_own_topics_and_no_duty_subnets() { + const EIP8483: Epoch = 6_000; + const PQ: Epoch = 7_000; + let config = config() + .with_fork_epoch(ForkName::Eip8483, EIP8483) + .with_fork_epoch(ForkName::Pq, PQ); + let mut server = unconnected_beacon_server(config, 0).await; + { + let Wire::Beacon(wire) = &mut server.wire else { + unreachable!("a beacon wire") + }; + wire.attestation_subnets = vec![40]; + wire.attestation_subnets_pq = vec![7]; + wire.sync_committee_subnets.insert(2, u64::MAX); + } + server.aggregator_subnets.insert(9, u64::MAX); + apply(&mut server, PQ - 2); + let joined = apply(&mut server, PQ - 1).subscribed; + let wire = server.wire.beacon().unwrap(); + let pq = wire.schedule.digest_at(PQ); + assert!(wire.window_topics[0].pq); + + for kind in topics::PQ_TOPIC_KINDS { + assert!(joined.contains(&topics::topic_name(pq, kind)), "{kind}"); + } + for absent in [ + topics::topic_name(pq, topics::VOLUNTARY_EXIT), + topics::topic_name(pq, topics::SYNC_COMMITTEE_CONTRIBUTION_AND_PROOF), + topics::attestation_topic_name(pq, 40), + topics::attestation_topic_name(pq, 9), + topics::sync_committee_topic_name(pq, 2), + ] { + assert!(!joined.contains(&absent), "{absent}"); + } + assert!(joined.contains(&topics::attestation_topic_name(pq, 7))); + } + #[tokio::test] async fn a_message_takes_its_fork_from_its_topic() { let config = config(); diff --git a/crates/net/p2p/src/beacon/verdict.rs b/crates/net/p2p/src/beacon/verdict.rs index 57b67d53e..d18f84748 100644 --- a/crates/net/p2p/src/beacon/verdict.rs +++ b/crates/net/p2p/src/beacon/verdict.rs @@ -42,6 +42,7 @@ use spawned_concurrency::message::Message; use spawned_concurrency::tasks::{Context, Handler}; use tracing::{debug, error, warn}; +use crate::beacon::pq::PqValidated; use crate::beacon::{builder_market, column_checks}; use crate::gossipsub::operation_kind; use crate::{P2PServer, metrics}; @@ -112,6 +113,10 @@ pub(crate) enum Validated { }, /// A `sync_committee_contribution_and_proof`. SyncContribution(Box), + /// A message off a PQ digest's topic, in the PQ fork's container. Never + /// forwarded, pooled or announced: the node does not follow PQ. See + /// [`crate::beacon::pq`]. + Pq(Box), } impl Validated { @@ -151,7 +156,7 @@ impl Validated { gossip::execution_payload_bid::stateful_checks(store, market, bid) } Self::ProposerPreferences(preferences) => { - gossip::proposer_preferences::stateful_checks(store, preferences) + gossip::proposer_preferences::stateful_checks(store, &**preferences) } Self::PayloadAttestation(message) => { gossip::payload_attestation::stateful_checks(store, message) @@ -173,6 +178,7 @@ impl Validated { Self::SyncContribution(signed) => { gossip::sync_committee::contribution_stateful_checks(store, signed) } + Self::Pq(object) => object.stateful_checks(store), } } @@ -230,6 +236,7 @@ impl Validated { .record(message.slot, message.validator_index, *subnet_id) } Self::SyncContribution(signed) => server.seen_sync_contributions.record(signed), + Self::Pq(object) => object.record_seen(server), } } @@ -269,6 +276,12 @@ impl Validated { /// validators it names as live for its target epoch; see /// [`record_liveness`]. fn forward(self, server: &P2PServer, received_at: Instant, outcome: Outcome) { + // Relayed on `Accept` by gossipsub itself; the chain actor takes what + // fork choice reads (see `PqValidated::forward`). + if let Self::Pq(object) = self { + object.forward(server, received_at, outcome); + return; + } if outcome == Outcome::Accept { record_liveness(server, &self); publish_gossip_event(server, &self); @@ -392,6 +405,11 @@ impl Validated { handed_off: Instant::now(), deferred_from: None, }; + let envelope = Box::new( + ethlambda_types::beacon::containers::SignedExecutionPayloadEnvelope::Gloas( + *envelope, + ), + ); let _ = blockchain .new_execution_payload_envelope(envelope, arrival) .inspect_err(|err| warn!(%err, "Failed to forward a gossip envelope")); @@ -415,7 +433,8 @@ impl Validated { | Self::PayloadAttestation(_) | Self::Operation(_) | Self::SyncCommitteeMessage { .. } - | Self::SyncContribution(_) => {} + | Self::SyncContribution(_) + | Self::Pq(_) => {} } } } @@ -490,7 +509,8 @@ fn record_liveness(server: &P2PServer, object: &Validated) { | Validated::ProposerPreferences(_) | Validated::Operation(_) | Validated::SyncCommitteeMessage { .. } - | Validated::SyncContribution(_) => {} + | Validated::SyncContribution(_) + | Validated::Pq(_) => {} } } @@ -677,6 +697,7 @@ fn pool_of(object: &Validated) -> &'static str { | Validated::Operation(_) => "attestation", Validated::ExecutionPayloadBid { .. } | Validated::ProposerPreferences(_) => "builder", Validated::SyncCommitteeMessage { .. } | Validated::SyncContribution(_) => "sync", + Validated::Pq(object) => object.pool(), } } @@ -1284,12 +1305,27 @@ mod tests { } fn new_execution_payload_envelope( &self, - _envelope: Box, + _envelope: Box, _arrival: BlockArrival, ) -> Result<(), ActorError> { let _ = self.0.send(Sent::Envelope); Ok(()) } + fn new_pq_aggregate( + &self, + _data: ethlambda_types::beacon::containers::AttestationData, + _attesting_indices: Vec, + _arrival: AggregateArrival, + ) -> Result<(), ActorError> { + Ok(()) + } + fn new_pq_payload_attestation_message( + &self, + _message: Box, + _arrival: BlockArrival, + ) -> Result<(), ActorError> { + Ok(()) + } fn new_payload_attestation_message( &self, _message: PayloadAttestationMessage, @@ -1658,7 +1694,12 @@ mod tests { let hash = signed.message.payload.block_hash; assert!(server.builder_market.known_payload(hash).is_none()); - crate::gossipsub::publish_execution_payload_envelope(&mut server, signed, Vec::new()).await; + crate::gossipsub::publish_execution_payload_envelope( + &mut server, + signed.into(), + Vec::new(), + ) + .await; assert!(server.builder_market.known_payload(hash).is_some()); } diff --git a/crates/net/p2p/src/gossipsub/handler.rs b/crates/net/p2p/src/gossipsub/handler.rs index 4ca0ad31c..208541d91 100644 --- a/crates/net/p2p/src/gossipsub/handler.rs +++ b/crates/net/p2p/src/gossipsub/handler.rs @@ -248,7 +248,14 @@ fn handle_beacon_gossip( let topic_fork = beacon_topics::topic_digest(topic) .and_then(|digest| wire.schedule.fork_for_digest(digest)) .unwrap_or(wire.fork); - let dispatch = if kind == beacon_topics::BEACON_BLOCK { + // A PQ digest's reshaped topics decode as PQ's containers; its block and + // column topics take the shared path below. + let pq_dispatch = (topic_fork == ForkName::Pq) + .then(|| crate::beacon::pq::triage(server, kind, payload, id.received_at)) + .flatten(); + let dispatch = if let Some(dispatch) = pq_dispatch { + dispatch + } else if kind == beacon_topics::BEACON_BLOCK { triage_block(server, wire, payload) } else if let Some(subnet_id) = beacon_topics::data_column_subnet(kind) { triage_data_column(server, topic_fork, payload, subnet_id) @@ -536,7 +543,7 @@ fn triage_envelope(server: &P2PServer, payload: &[u8]) -> Dispatch { { return Dispatch::Report(outcome); } - if let Err(outcome) = payload_hash::reject_envelope_on_mismatch(&envelope) { + if let Err(outcome) = payload_hash::reject_gloas_envelope_on_mismatch(&envelope) { return Dispatch::Report(outcome); } Dispatch::Validate(Validated::Envelope(Box::new(envelope))) @@ -1036,16 +1043,23 @@ pub async fn publish_beacon_block( /// peers. pub async fn publish_execution_payload_envelope( server: &mut P2PServer, - envelope: ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope, + envelope: ethlambda_types::beacon::containers::SignedExecutionPayloadEnvelope, sidecars: Vec, ) { - let slot = envelope.message.payload.slot_number; - let block_root = envelope.message.beacon_block_root; + use ethlambda_types::beacon::containers::SignedExecutionPayloadEnvelope as Envelope; + let slot = envelope.payload().slot_number; + let block_root = envelope.beacon_block_root(); // Gossip never echoes a node's own message, so its own envelope is a // known payload for bid validation only because it is recorded here. - server - .builder_market - .record_execution_payload(&envelope.message); + match &envelope { + Envelope::Gloas(inner) => server + .builder_market + .record_execution_payload(&inner.message), + Envelope::Pq(inner) => server + .pq_builder_market + .record_pq_execution_payload(&inner.message), + Envelope::Eip8483(_) => {} + } let Some(beacon) = server.wire.beacon() else { error!( slot, @@ -1069,7 +1083,7 @@ pub async fn publish_execution_payload_envelope( .publish(topic, compress_message(&envelope.to_ssz())); info!( slot, - builder_index = envelope.message.builder_index, + builder_index = envelope.builder_index(), block_root = %ShortRoot(&block_root.0), "Published execution payload envelope to gossipsub" ); @@ -1197,9 +1211,10 @@ pub fn join_aggregator_subnets(server: &mut P2PServer, subnets: Vec<(u64, u64)>) *until = (*until).max(slot); } // Under every digest the node holds: while a boundary's window is open, - // attestations are published on either side of it. + // attestations are published on either side of it. Not a PQ digest's: the + // subnet comes from a BLS duty over `ATTESTATION_SUBNET_COUNT` subnets. for &subnet_id in &joined { - for held in wire.held_topics() { + for held in wire.held_topics().filter(|held| !held.pq) { let topic = beacon_topics::attestation_topic_name(held.fork_digest, subnet_id); server.swarm_handle.subscribe(IdentTopic::new(topic)); } @@ -1209,6 +1224,40 @@ pub fn join_aggregator_subnets(server: &mut P2PServer, subnets: Vec<(u64, u64)>) } } +/// [`join_aggregator_subnets`] for a PQ slot's committees: `(subnet_id, +/// slot)` pairs among PQ's 16 subnets, joined under every held PQ digest. A +/// subnet of PQ's backbone is joined there already. +pub fn join_pq_aggregator_subnets(server: &mut P2PServer, subnets: Vec<(u64, u64)>) { + let Some(wire) = server.wire.beacon() else { + return; + }; + let mut joined = Vec::new(); + for (subnet_id, slot) in subnets { + if subnet_id >= wire.config.attestation_subnet_count_pq + || wire.attestation_subnets_pq.contains(&subnet_id) + { + continue; + } + let until = server + .aggregator_subnets_pq + .entry(subnet_id) + .or_insert_with(|| { + joined.push(subnet_id); + slot + }); + *until = (*until).max(slot); + } + for &subnet_id in &joined { + for held in wire.held_topics().filter(|held| held.pq) { + let topic = beacon_topics::attestation_topic_name(held.fork_digest, subnet_id); + server.swarm_handle.subscribe(IdentTopic::new(topic)); + } + } + if !joined.is_empty() { + info!(?joined, "Joined PQ attestation subnets for aggregation"); + } +} + /// Gossip one operation on its topic, handed over by the Beacon API after /// validation. This node is subscribed to all four operation topics, so it /// reaches the mesh rather than relying on fanout. @@ -1284,7 +1333,20 @@ pub fn leave_expired_aggregator_subnets(server: &mut P2PServer) { .collect(); for subnet_id in &expired { server.aggregator_subnets.remove(subnet_id); - for held in wire.held_topics() { + for held in wire.held_topics().filter(|held| !held.pq) { + let topic = beacon_topics::attestation_topic_name(held.fork_digest, *subnet_id); + server.swarm_handle.unsubscribe(IdentTopic::new(topic)); + } + } + let expired_pq: Vec = server + .aggregator_subnets_pq + .iter() + .filter(|&(_, &until)| until < now) + .map(|(&subnet_id, _)| subnet_id) + .collect(); + for subnet_id in &expired_pq { + server.aggregator_subnets_pq.remove(subnet_id); + for held in wire.held_topics().filter(|held| held.pq) { let topic = beacon_topics::attestation_topic_name(held.fork_digest, *subnet_id); server.swarm_handle.unsubscribe(IdentTopic::new(topic)); } diff --git a/crates/net/p2p/src/gossipsub/mod.rs b/crates/net/p2p/src/gossipsub/mod.rs index 7ba46286c..43c72a658 100644 --- a/crates/net/p2p/src/gossipsub/mod.rs +++ b/crates/net/p2p/src/gossipsub/mod.rs @@ -6,10 +6,10 @@ pub use encoding::{compress_message, decompress_message}; pub(crate) use handler::beacon_wall_slot; pub(crate) use handler::operation_kind; pub use handler::{ - handle_gossip_message, join_aggregator_subnets, leave_expired_aggregator_subnets, - prune_attestation_pool, prune_operation_pool, publish_aggregated_attestation, - publish_attestation, publish_beacon_aggregate, publish_beacon_attestation, - publish_beacon_block, publish_beacon_operation, publish_block, + handle_gossip_message, join_aggregator_subnets, join_pq_aggregator_subnets, + leave_expired_aggregator_subnets, prune_attestation_pool, prune_operation_pool, + publish_aggregated_attestation, publish_attestation, publish_beacon_aggregate, + publish_beacon_attestation, publish_beacon_block, publish_beacon_operation, publish_block, publish_execution_payload_envelope, publish_payload_attestation_message, }; pub use messages::{aggregation_topic, attestation_subnet_topic, block_topic, topic_kind}; diff --git a/crates/net/p2p/src/lib.rs b/crates/net/p2p/src/lib.rs index 4f670185b..53e857e13 100644 --- a/crates/net/p2p/src/lib.rs +++ b/crates/net/p2p/src/lib.rs @@ -47,9 +47,9 @@ use ethlambda_network_api::{ rpc_to_p2p::{ PublishBeaconAggregate, PublishBeaconAttestation, PublishBeaconBlock, PublishBeaconOperation, PublishExecutionPayloadBid, PublishExecutionPayloadEnvelope, - PublishPayloadAttestationMessage, PublishProposerPreferences, + PublishPayloadAttestationMessage, PublishPqGossip, PublishProposerPreferences, PublishSyncCommitteeContribution, PublishSyncCommitteeMessage, SubscribeAttestationSubnets, - SubscribeSyncCommitteeSubnets, + SubscribePqAttestationSubnets, SubscribeSyncCommitteeSubnets, }, }; use ethlambda_state_transition::beacon::aggregate::MAX_AGGREGATES_PER_SLOT; @@ -63,7 +63,8 @@ use ethlambda_state_transition::beacon::gossip::{ sync_committee::{SeenSyncCommitteeMessages, SeenSyncContributions}, }; use ethlambda_state_transition::beacon::{ - builder_market::SharedBuilderMarket, payload_attestation_pool::SharedPayloadAttestationPool, + builder_market::{SharedBuilderMarket, SharedPqBuilderMarket}, + payload_attestation_pool::SharedPayloadAttestationPool, sync_committee_pool::SharedSyncCommitteePool, }; use ethlambda_storage::{Chain, Store}; @@ -1091,7 +1092,11 @@ pub fn build_swarm(config: SwarmConfig) -> Result { beacon.fork, beacon.fork_digest, &column_subnets, - &beacon.attestation_subnets, + beacon::topics::backbone_subnets( + beacon.fork, + &beacon.attestation_subnets, + &beacon.attestation_subnets_pq, + ), ); for topic in &topics.topics { swarm.behaviour_mut().gossipsub.subscribe(topic)?; @@ -1128,6 +1133,7 @@ pub fn build_swarm(config: SwarmConfig) -> Result { metadata_seq_number: 0, custody_columns: beacon.custody_columns, attestation_subnets: beacon.attestation_subnets, + attestation_subnets_pq: beacon.attestation_subnets_pq, sync_committee_subnets: std::collections::BTreeMap::new(), metadata: MetaDataController::default(), }; @@ -1212,10 +1218,9 @@ impl P2P { .wire .beacon() .map_or(0, |beacon| beacon.attestation_subnets.len()); - let peer_scoring = built - .wire - .beacon() - .map(|beacon| beacon::scoring::PeerScoring::new(&beacon.config, beacon.fork_digest)); + let peer_scoring = built.wire.beacon().map(|beacon| { + beacon::scoring::PeerScoring::new(&beacon.config, beacon.fork_digest, beacon.fork) + }); let local_metadata = built.wire.beacon().map(|beacon| beacon.metadata.clone()); let peer_table = PeerTableController::default(); @@ -1266,12 +1271,14 @@ impl P2P { )), payload_attestation_pool, builder_market, + pq_builder_market: Default::default(), builder_validation_permits: Arc::new(tokio::sync::Semaphore::new( BUILDER_VALIDATION_PERMITS, )), sync_committee_pool, gossip_events, aggregator_subnets: HashMap::new(), + aggregator_subnets_pq: HashMap::new(), peer_scoring, }; let scores_peers = server.peer_scoring.is_some(); @@ -1466,6 +1473,10 @@ pub struct P2PServer { /// Gossip's stateful checks read it from blocking threads, so it cannot /// live in the chain actor; lean never touches it. pub(crate) builder_market: SharedBuilderMarket, + /// The PQ fork's bids and preferences, kept apart from + /// [`Self::builder_market`]: they carry XMSS signatures, and block + /// production must never read one. Only gossip touches it. + pub(crate) pq_builder_market: SharedPqBuilderMarket, /// Permits for `execution_payload_bid` and `proposer_preferences` stateful /// checks, see [`BUILDER_VALIDATION_PERMITS`]. pub(crate) builder_validation_permits: Arc, @@ -1485,6 +1496,11 @@ pub struct P2PServer { /// backbone subnets are separate and never left. pub(crate) aggregator_subnets: HashMap, + /// [`Self::aggregator_subnets`] for PQ slots: among PQ's 16 subnets, and + /// joined under PQ digests only, where the other map is joined under + /// every other digest. + pub(crate) aggregator_subnets_pq: HashMap, + /// Gossipsub scoring's inputs and gate. `None` on lean, which runs /// without peer scoring. See [`beacon::scoring`]. pub(crate) peer_scoring: Option, @@ -1841,6 +1857,12 @@ impl Handler for P2PServer { } } +impl Handler for P2PServer { + async fn handle(&mut self, msg: PublishPqGossip, _ctx: &Context) { + beacon::pq::publish_pq_gossip(self, msg.object); + } +} + impl Handler for P2PServer { async fn handle(&mut self, msg: PublishBeaconBlock, _ctx: &Context) { publish_beacon_block(self, msg.block, msg.sidecars).await; @@ -1877,6 +1899,12 @@ impl Handler for P2PServer { } } +impl Handler for P2PServer { + async fn handle(&mut self, msg: SubscribePqAttestationSubnets, _ctx: &Context) { + gossipsub::join_pq_aggregator_subnets(self, msg.subnets); + } +} + impl Handler for P2PServer { async fn handle(&mut self, msg: PublishSyncCommitteeMessage, _ctx: &Context) { beacon::sync_committee::publish_sync_committee_message(self, msg.subnet_ids, msg.message); @@ -2962,9 +2990,22 @@ pub(crate) mod test_support { } fn new_execution_payload_envelope( &self, - _envelope: Box< - ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope, - >, + _envelope: Box, + _arrival: ethlambda_network_api::BlockArrival, + ) -> Result<(), spawned_concurrency::error::ActorError> { + Ok(()) + } + fn new_pq_aggregate( + &self, + _data: ethlambda_types::beacon::containers::AttestationData, + _attesting_indices: Vec, + _arrival: ethlambda_network_api::AggregateArrival, + ) -> Result<(), spawned_concurrency::error::ActorError> { + Ok(()) + } + fn new_pq_payload_attestation_message( + &self, + _message: Box, _arrival: ethlambda_network_api::BlockArrival, ) -> Result<(), spawned_concurrency::error::ActorError> { Ok(()) @@ -3049,6 +3090,7 @@ pub(crate) mod test_support { genesis_validators_root: Root::ZERO, custody_columns: Vec::new(), attestation_subnets: Vec::new(), + attestation_subnets_pq: Vec::new(), })), }) .expect("swarm builds"); @@ -3169,12 +3211,14 @@ pub(crate) mod test_support { )), payload_attestation_pool: Default::default(), builder_market: Default::default(), + pq_builder_market: Default::default(), builder_validation_permits: std::sync::Arc::new(tokio::sync::Semaphore::new( crate::BUILDER_VALIDATION_PERMITS, )), sync_committee_pool: Default::default(), gossip_events: None, aggregator_subnets: HashMap::new(), + aggregator_subnets_pq: HashMap::new(), peer_scoring: None, } } @@ -3540,6 +3584,7 @@ mod tests { genesis_validators_root: Root::ZERO, custody_columns: Vec::new(), attestation_subnets: Vec::new(), + attestation_subnets_pq: Vec::new(), })), }) .expect("swarm builds") @@ -4285,6 +4330,7 @@ mod tests { genesis_validators_root: Root::ZERO, custody_columns: Vec::new(), attestation_subnets: Vec::new(), + attestation_subnets_pq: Vec::new(), })), }) .expect("swarm builds"); diff --git a/crates/net/p2p/src/req_resp/codec.rs b/crates/net/p2p/src/req_resp/codec.rs index 9bd77f103..44f14a15a 100644 --- a/crates/net/p2p/src/req_resp/codec.rs +++ b/crates/net/p2p/src/req_resp/codec.rs @@ -1020,7 +1020,7 @@ mod tests { &stream_protocol, &mut buffer, Response::success(ResponsePayload::ExecutionPayloadEnvelopes( - envelopes.clone(), + envelopes.iter().cloned().map(Into::into).collect(), )), ) .await diff --git a/crates/net/p2p/src/req_resp/envelope_client.rs b/crates/net/p2p/src/req_resp/envelope_client.rs index e10f16529..a71668b79 100644 --- a/crates/net/p2p/src/req_resp/envelope_client.rs +++ b/crates/net/p2p/src/req_resp/envelope_client.rs @@ -8,7 +8,7 @@ use std::collections::HashSet; use std::time::Duration; use ethlambda_types::beacon::config::Config; -use ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope; +use ethlambda_types::beacon::containers::SignedExecutionPayloadEnvelope; use ethlambda_types::primitives::H256; use libp2p::PeerId; use spawned_concurrency::tasks::{Context, send_after}; @@ -159,7 +159,7 @@ fn retain_requested_envelopes( let received = envelopes.len(); let kept: Vec<_> = envelopes .into_iter() - .filter(|envelope| envelope.message.beacon_block_root == block_root) + .filter(|envelope| envelope.beacon_block_root() == block_root) .collect(); let dropped = received - kept.len(); (kept, dropped) @@ -323,9 +323,9 @@ fn retain_envelopes_in_range( let received = envelopes.len(); let mut kept: Vec<_> = envelopes .into_iter() - .filter(|envelope| (start_slot..=end_slot).contains(&envelope.message.payload.slot_number)) + .filter(|envelope| (start_slot..=end_slot).contains(&envelope.payload().slot_number)) .collect(); - kept.sort_by_key(|envelope| envelope.message.payload.slot_number); + kept.sort_by_key(|envelope| envelope.payload().slot_number); let dropped = received - kept.len(); (kept, dropped) } @@ -386,10 +386,8 @@ mod tests { fn envelope_for(slot: u64, block_root: H256) -> SignedExecutionPayloadEnvelope { // Sealed: the fetched-envelope rules refuse a block hash that is not // the payload's header's. - crate::beacon::payload_hash::seal_envelope(crate::beacon::encoding::test_support::envelope( - block_root, - slot, - H256::ZERO, + SignedExecutionPayloadEnvelope::Gloas(crate::beacon::payload_hash::seal_envelope( + crate::beacon::encoding::test_support::envelope(block_root, slot, H256::ZERO), )) } @@ -433,7 +431,7 @@ mod tests { let slots: Vec = kept .iter() - .map(|envelope| envelope.message.payload.slot_number) + .map(|envelope| envelope.payload().slot_number) .collect(); assert_eq!(slots, vec![10, 11]); assert_eq!(dropped, 2); diff --git a/crates/net/p2p/src/req_resp/envelopes.rs b/crates/net/p2p/src/req_resp/envelopes.rs index c7f2249c9..42fff92f4 100644 --- a/crates/net/p2p/src/req_resp/envelopes.rs +++ b/crates/net/p2p/src/req_resp/envelopes.rs @@ -3,7 +3,7 @@ //! answering peers out of `Table::ExecutionPayloadEnvelopes`. use ethlambda_storage::Store; -use ethlambda_types::beacon::containers::gloas; +use ethlambda_types::beacon::containers::SignedExecutionPayloadEnvelope; use ethlambda_types::primitives::H256; use libp2p::PeerId; use libp2p::request_response::ResponseChannel; @@ -84,7 +84,7 @@ fn envelopes_by_range( store: &Store, start_slot: u64, count: u64, -) -> Result, &'static str> { +) -> Result, &'static str> { if start_slot < store.anchor_slot() { return Err("requested range starts before this node's earliest available slot"); } @@ -146,7 +146,7 @@ pub(super) async fn handle_execution_payload_envelopes_by_root_request( /// The held envelopes among `roots`, in the order asked, looking at no more /// than `MAX_REQUEST_PAYLOADS` of them. -fn envelopes_by_root(store: &Store, roots: &[H256]) -> Vec { +fn envelopes_by_root(store: &Store, roots: &[H256]) -> Vec { roots .iter() .take(MAX_REQUEST_PAYLOADS as usize) @@ -173,6 +173,7 @@ mod tests { use ethlambda_storage::{ForkCheckpoints, backend::InMemoryBackend}; use ethlambda_types::beacon::config::Config; use ethlambda_types::beacon::containers::SignedBeaconBlock; + use ethlambda_types::beacon::containers::gloas; use ethlambda_types::beacon::fork_choice::PayloadStatus; use ethlambda_types::checkpoint::Checkpoint; use std::sync::Arc; @@ -345,12 +346,7 @@ mod tests { let served = envelopes_by_range(&store, 1, total).expect("in window"); assert_eq!(served.len() as u64, MAX_REQUEST_PAYLOADS); - let last_slot = served - .last() - .expect("non-empty") - .message - .payload - .slot_number; + let last_slot = served.last().expect("non-empty").payload().slot_number; assert_eq!(last_slot, MAX_REQUEST_PAYLOADS); // A count past the cap is cut to it, not refused. let wide = envelopes_by_range(&store, 1, u64::MAX).expect("in window"); diff --git a/crates/net/p2p/src/req_resp/handlers.rs b/crates/net/p2p/src/req_resp/handlers.rs index 3327ee4af..80e14f186 100644 --- a/crates/net/p2p/src/req_resp/handlers.rs +++ b/crates/net/p2p/src/req_resp/handlers.rs @@ -1465,7 +1465,7 @@ fn range_needs_columns( | ForkName::Capella | ForkName::Deneb | ForkName::Electra => false, - ForkName::Fulu | ForkName::Gloas => true, + ForkName::Fulu | ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => true, ForkName::Lean => { unreachable!("fork_at_slot never returns Lean: it is absent from ForkName::ALL") } @@ -2846,12 +2846,14 @@ pub(crate) mod tests { )), payload_attestation_pool: Default::default(), builder_market: Default::default(), + pq_builder_market: Default::default(), builder_validation_permits: std::sync::Arc::new(tokio::sync::Semaphore::new( crate::BUILDER_VALIDATION_PERMITS, )), sync_committee_pool: Default::default(), gossip_events: None, aggregator_subnets: HashMap::new(), + aggregator_subnets_pq: HashMap::new(), peer_scoring: None, } } diff --git a/crates/net/p2p/src/req_resp/messages.rs b/crates/net/p2p/src/req_resp/messages.rs index a94f6dac6..4670be556 100644 --- a/crates/net/p2p/src/req_resp/messages.rs +++ b/crates/net/p2p/src/req_resp/messages.rs @@ -1,8 +1,8 @@ use ethlambda_types::ShortRoot; use ethlambda_types::beacon::containers::DataColumnSidecar; use ethlambda_types::beacon::containers::SignedBeaconBlock; +use ethlambda_types::beacon::containers::SignedExecutionPayloadEnvelope; use ethlambda_types::beacon::containers::fulu::DataColumnsByRootIdentifier; -use ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope; use ethlambda_types::beacon::primitives::Root; use libssz_types::SszList; diff --git a/crates/net/rpc/src/beacon/builders.rs b/crates/net/rpc/src/beacon/builders.rs index 36a5d8546..55e07c7a7 100644 --- a/crates/net/rpc/src/beacon/builders.rs +++ b/crates/net/rpc/src/beacon/builders.rs @@ -15,7 +15,7 @@ use ethlambda_state_transition::beacon::helpers::gloas::is_active_builder; use ethlambda_storage::Store; use ethlambda_types::beacon::{ constants::FAR_FUTURE_EPOCH, - containers::{BeaconState, gloas::Builder}, + containers::gloas::Builder, primitives::{BlsPubkey, ValidatorIndex}, }; use serde::{Deserialize, Serialize}; @@ -139,7 +139,10 @@ async fn post_builders( Ok(found) => found, Err(err) => return err.into_response(), }; - let BeaconState::Gloas(inner) = state.as_ref() else { + // Gloas or a fork built on it. A PQ builder's key is its XMSS key in the + // 48-byte form an execution-layer request names it by, which is how the + // state holds it (see `containers::pq`). + let Some(inner) = state.gloas_family() else { return ApiError::BadRequest("the requested state is prior to Gloas").into_response(); }; @@ -151,7 +154,7 @@ async fn post_builders( } let status = if builder.withdrawable_epoch != FAR_FUTURE_EPOCH { BuilderStatus::Exited - } else if is_active_builder(inner, index).unwrap_or(false) { + } else if is_active_builder(&inner, index).unwrap_or(false) { BuilderStatus::Active } else { BuilderStatus::Pending @@ -182,7 +185,7 @@ mod tests { http::{Request, StatusCode}, }; use ethlambda_state_transition::beacon::helpers::test_state::with_signing_validators_at; - use ethlambda_types::beacon::{config::Config, fork::ForkName}; + use ethlambda_types::beacon::{config::Config, containers::BeaconState, fork::ForkName}; use http_body_util::BodyExt as _; use tower::ServiceExt as _; diff --git a/crates/net/rpc/src/beacon/envelopes.rs b/crates/net/rpc/src/beacon/envelopes.rs index 5d077c881..37bfa397e 100644 --- a/crates/net/rpc/src/beacon/envelopes.rs +++ b/crates/net/rpc/src/beacon/envelopes.rs @@ -14,7 +14,6 @@ use axum::{ }; use ethlambda_storage::Store; use ethlambda_types::beacon::fork::ForkName; -use libssz::SszEncode; use crate::{ beacon::{ApiError, Envelope as ResponseEnvelope, blocks::is_finalized}, @@ -55,20 +54,21 @@ async fn get_envelope( Err(_) => return ApiError::Internal("store read failed").into_response(), }; // Verified against its block's bid, so the payload's slot is the block's. - let slot = envelope.message.payload.slot_number; + let slot = envelope.payload().slot_number; + let fork: ForkName = envelope.fork(); let accept = headers.get(header::ACCEPT).and_then(|v| v.to_str().ok()); let response = match Encoding::from_accept(accept) { Encoding::Ssz => ssz_response(envelope.to_ssz()), Encoding::Json => crate::json_response(ResponseEnvelope { - version: ForkName::Gloas.as_str(), + version: fork.as_str(), execution_optimistic: envelope_is_optimistic(&store, root), finalized: is_finalized(&store, slot), data: envelope, }), }; - with_consensus_version(response, ForkName::Gloas) + with_consensus_version(response, fork) } #[cfg(test)] @@ -150,7 +150,10 @@ mod tests { "gloas" ); let body = response.into_body().collect().await.unwrap().to_bytes(); - assert_eq!(body.as_ref(), envelope.to_ssz().as_slice()); + assert_eq!( + body.as_ref(), + libssz::SszEncode::to_ssz(&envelope).as_slice() + ); } #[tokio::test] diff --git a/crates/net/rpc/src/beacon/gloas_proposal.rs b/crates/net/rpc/src/beacon/gloas_proposal.rs index 0845d0930..be8d89f22 100644 --- a/crates/net/rpc/src/beacon/gloas_proposal.rs +++ b/crates/net/rpc/src/beacon/gloas_proposal.rs @@ -683,8 +683,11 @@ fn prepare( .ok_or(ApiError::ServiceUnavailable( "the parent's payload envelope is not held to build on", ))? + .as_gloas() + .ok_or(ApiError::Internal("a gloas head with a non-gloas envelope"))? .message .execution_requests + .clone() } else { ExecutionRequests::default() }; @@ -759,7 +762,8 @@ async fn build_local( prepared: &Prepared, slot: Slot, ) -> Result { - let built = build_payload(store, engine, fee_recipients, custody, prepared).await?; + let fee_recipient = fee_recipient_for(fee_recipients, prepared); + let built = build_payload(store, engine, fee_recipient, custody, &prepared.inputs).await?; let bundle = &built.blobs_bundle; if bundle.commitments.len() != bundle.blobs.len() || bundle.proofs.len() @@ -818,15 +822,13 @@ fn fee_recipient_for(fee_recipients: &FeeRecipients, prepared: &Prepared) -> Exe /// execution client starts with a valid payload and improves it, so an early /// `getPayload` always answers, and the validator client asks at the start of /// the slot, when the block is due. -async fn build_payload( +pub(crate) async fn build_payload( store: &Store, engine: &EngineClient, - fee_recipients: &FeeRecipients, + fee_recipient: ExecutionAddress, custody: &NodeCustodyColumns, - prepared: &Prepared, + inputs: &GloasPayloadInputs, ) -> Result { - let inputs = &prepared.inputs; - let fee_recipient = fee_recipient_for(fee_recipients, prepared); let forkchoice = ForkchoiceStateV1 { head_block_hash: inputs.head_block_hash, safe_block_hash: checkpoint_hash(store, store.beacon_justified_checkpoint().root), @@ -1078,8 +1080,11 @@ async fn post_envelope_emitting( body: &[u8], block_wait: Duration, ) -> Response { + if consensus_version(headers) == Some(ForkName::Pq) { + return post_pq_envelope(store, p2p, events, headers, body, block_wait).await; + } if consensus_version(headers) != Some(ForkName::Gloas) { - return ApiError::BadRequest("Eth-Consensus-Version must be gloas").into_response(); + return ApiError::BadRequest("Eth-Consensus-Version must be gloas or pq").into_response(); } let encoding = match BodyEncoding::from_headers(headers) { Ok(encoding) => encoding, @@ -1209,7 +1214,79 @@ async fn post_envelope_emitting( block_root, }; if p2p - .publish_execution_payload_envelope(Box::new(signed), sidecars) + .publish_execution_payload_envelope(Box::new(signed.into()), sidecars) + .is_err() + { + return ApiError::Internal("the network actor is not running").into_response(); + } + emit_accepted(events, || accepted); + StatusCode::OK.into_response() +} + +/// The PQ half of `publishExecutionPayloadEnvelope`: PQ's container, signed +/// with leanXMSS at the slot's envelope leaf. PQ block production builds no +/// blobs yet, so none are taken and no columns are gossiped. +async fn post_pq_envelope( + store: &Store, + p2p: &RpcToP2PRef, + events: Option<&EventBus>, + headers: &HeaderMap, + body: &[u8], + block_wait: Duration, +) -> Response { + let encoding = match BodyEncoding::from_headers(headers) { + Ok(encoding) => encoding, + Err(err) => return err.into_response(), + }; + let Some(signed) = encoding.decode::(body) + else { + return ApiError::BadRequest("the body is not a pq SignedExecutionPayloadEnvelope") + .into_response(); + }; + let block_root = signed.message.beacon_block_root; + let Some((block, post_state)) = wait_for_block(store, block_root, block_wait).await else { + return ApiError::BadRequest("unknown block: the envelope's block is not held") + .into_response(); + }; + let containers::SignedBeaconBlock::Pq(block) = block else { + return ApiError::BadRequest("the envelope's block is not a pq block").into_response(); + }; + let block_slot = block.message.slot; + let bid = &block.message.body.signed_execution_payload_bid.message; + if signed.message.builder_index != bid.builder_index + || signed.message.payload.block_hash != bid.block_hash + { + return ApiError::BadRequest("the envelope does not fulfill the block's bid") + .into_response(); + } + if !bid.blob_kzg_commitments.is_empty() { + return ApiError::BadRequest("pq envelopes with blobs are not published yet") + .into_response(); + } + let envelope_for_check = signed.clone(); + let checked = tokio::task::spawn_blocking(move || { + ethlambda_state_transition::beacon::stf::pq::verify_execution_payload_envelope_signature( + &post_state, + &envelope_for_check, + ) + }) + .await; + match checked { + Ok(Ok(true)) => {} + Ok(Ok(false)) => return ApiError::BadRequest("invalid envelope signature").into_response(), + Ok(Err(_)) => { + return ApiError::BadRequest("the envelope's signer is unknown").into_response(); + } + Err(_) => return ApiError::Internal("verifying the envelope failed").into_response(), + } + let accepted = GossipEvent::ExecutionPayloadGossip { + slot: block_slot, + builder_index: signed.message.builder_index, + block_hash: signed.message.payload.block_hash, + block_root, + }; + if p2p + .publish_execution_payload_envelope(Box::new(signed.into()), Vec::new()) .is_err() { return ApiError::Internal("the network actor is not running").into_response(); diff --git a/crates/net/rpc/src/beacon/headers.rs b/crates/net/rpc/src/beacon/headers.rs index 8f68dc3dd..d68a44ae0 100644 --- a/crates/net/rpc/src/beacon/headers.rs +++ b/crates/net/rpc/src/beacon/headers.rs @@ -62,9 +62,14 @@ async fn get_header(Path(block_id): Path, State(store): State) -> .flatten() .is_some_and(|canonical| canonical == root); - // `signature()` panics on a lean block by design (`dispatch_block!`), - // which is unreachable here: this router is only ever mounted on a beacon - // store. + // A PQ block's proposer signs with leanXMSS, and `signature()` (BLS only) + // panics on it. It also panics on a lean block by design + // (`dispatch_block!`), which is unreachable here: this router is only + // ever mounted on a beacon store. + let signature = match block.xmss_signature() { + Some(xmss) => serde_json::json!(xmss), + None => serde_json::json!(block.signature()), + }; crate::json_response(serde_json::json!({ "execution_optimistic": crate::shared::optimistic::block_is_optimistic(&store, root), "finalized": is_finalized(&store, block.slot()), @@ -73,7 +78,7 @@ async fn get_header(Path(block_id): Path, State(store): State) -> "canonical": canonical, "header": { "message": message, - "signature": block.signature(), + "signature": signature, } } })) diff --git a/crates/net/rpc/src/beacon/mod.rs b/crates/net/rpc/src/beacon/mod.rs index 2c37e9ffc..ef715b5ec 100644 --- a/crates/net/rpc/src/beacon/mod.rs +++ b/crates/net/rpc/src/beacon/mod.rs @@ -33,6 +33,8 @@ pub(crate) mod headers; pub(crate) mod node; pub(crate) mod operations; pub(crate) mod pool; +pub(crate) mod pq_aggregation; +pub(crate) mod pq_proposal; pub(crate) mod proposal; pub(crate) mod proposer_preferences; pub(crate) mod ptc; @@ -216,6 +218,7 @@ pub(crate) fn routes(version: &'static str, peer_id: String) -> Router { .merge(operations::routes()) .merge(proposal::routes()) .merge(gloas_proposal::routes()) + .merge(pq_proposal::routes()) .merge(ptc::routes()) .merge(bids::routes()) .merge(proposer_preferences::routes()) diff --git a/crates/net/rpc/src/beacon/pool.rs b/crates/net/rpc/src/beacon/pool.rs index 46317d8ff..9c6a644a1 100644 --- a/crates/net/rpc/src/beacon/pool.rs +++ b/crates/net/rpc/src/beacon/pool.rs @@ -96,6 +96,13 @@ async fn post_pool_attestations( headers: HeaderMap, body: Bytes, ) -> Response { + let is_pq = headers + .get("eth-consensus-version") + .and_then(|value| value.to_str().ok()) + == Some(ForkName::Pq.as_str()); + if is_pq { + return post_pq_attestations(&store, &p2p, &headers, &body); + } let header_fork = match submission_fork(&headers) { Ok(fork) => fork, Err(err) => return err.into_response(), @@ -164,6 +171,83 @@ async fn post_pool_attestations( ) } +/// The PQ half of `submitPoolAttestationsV2`: PQ's `SingleAttestation`, +/// signed with leanXMSS, judged by PQ's gossip rules (its signature at the +/// target epoch's leaf, its committee, its subnet among PQ's 16) and +/// gossiped on that subnet. +fn post_pq_attestations( + store: &Store, + p2p: &RpcToP2PRef, + headers: &HeaderMap, + body: &[u8], +) -> Response { + use ethlambda_state_transition::beacon::gossip::{SeenAttestations, pq as pq_gossip}; + let attestations = match decode_list::( + headers, body, + ) { + Ok(attestations) => attestations, + Err(err) => return err.into_response(), + }; + let (_head_root, state) = match head(store) { + Ok(found) => found, + Err(err) => return err.into_response(), + }; + let now_ms = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|elapsed| elapsed.as_millis() as u64) + .unwrap_or(0); + let config = store.config(); + // A fresh cache: the node gossips its own validator client's votes once. + let seen = SeenAttestations::new(std::num::NonZeroUsize::MIN); + let mut failures = Vec::new(); + for (index, attestation) in attestations.into_iter().enumerate() { + let slot = attestation.data.slot; + let validator = attestation.attester_index; + let committees_per_slot = store + .committee_cache() + .committees(&state, attestation.data.target.epoch) + .committees_per_slot(); + let subnet_id = pq_gossip::compute_subnet_for_attestation( + committees_per_slot, + slot, + attestation.committee_index, + &config, + ); + let verdict = pq_gossip::attestation_cheap_checks(&seen, store, &attestation, now_ms) + .err() + .unwrap_or_else(|| { + pq_gossip::attestation_stateful_checks(store, &attestation, subnet_id) + }); + let published = match verdict { + Outcome::Accept => { + // This node's own aggregator may be collecting the committee. + let now = wall_slot(&config, now_ms); + store.pq_signature_pool().insert(now, &attestation); + p2p.publish_pq_gossip(ethlambda_network_api::PqGossipObject::Attestation { + subnet_id, + attestation: Box::new(attestation), + }) + .map_err(|_| "the network actor is not running".to_string()) + } + other => Err(format!("{other:?}")), + }; + match published { + Ok(()) => debug!(%slot, validator, subnet_id, "Accepted a PQ attestation for gossip"), + Err(message) => { + warn!(%slot, validator, reason = %message, "Refused a submitted PQ attestation"); + failures.push(Failure { + index, + message: message.into(), + }); + } + } + } + batch_response( + failures, + "some attestations failed validation and were not published", + ) +} + /// Where `attestation` belongs, if it passes every gossip condition /// this node can check, or the first one it fails. /// @@ -199,9 +283,11 @@ fn validate( return Err("attestation is older than the previous epoch"); } - // [REJECT] data.index == 0 (electra), or at a gloas slot 0 or 1, the - // payload-present flag; the target epoch is the slot's. - let at_gloas = config.fork_at_epoch(attestation_epoch) == ForkName::Gloas; + // [REJECT] data.index == 0 (electra), or from gloas on 0 or 1, the + // payload-present flag; the target epoch is the slot's. The forks after + // gloas build on it, so they keep the flag; `fork_at_epoch` never names + // lean, which sorts after them. + let at_gloas = config.fork_at_epoch(attestation_epoch) >= ForkName::Gloas; if at_gloas { if data.index > 1 { return Err("data.index must be 0 or 1 from gloas on"); @@ -314,6 +400,9 @@ fn submission_fork(headers: &HeaderMap) -> Result { | ForkName::Bellatrix | ForkName::Capella | ForkName::Deneb + // Not followed: their attestations are not gloas's to submit. + | ForkName::Eip8483 + | ForkName::Pq | ForkName::Lean, ) | None => Err(ApiError::BadRequest( @@ -322,13 +411,21 @@ fn submission_fork(headers: &HeaderMap) -> Result { } } -/// Whether `header` is a version the fork at `slot` accepts: gloas's slots need -/// the gloas header, and electra's and fulu's, whose containers are the same, -/// accept either of theirs. +/// Whether `header` is a version the fork at `slot` accepts: a slot from gloas +/// on needs its own fork's header, and electra's and fulu's, whose containers +/// are the same, accept either of theirs. +/// +/// A slot of a fork after gloas is a gloas slot too, never an electra one, so +/// an older header is refused there rather than judged by electra's rules. +/// [`submission_fork`] refuses the EIP-8483 and PQ headers, which leaves no +/// version that submits at their slots: the node does not follow them. fn header_matches_slot(store: &Store, header: ForkName, slot: Slot) -> bool { - let slot_is_gloas = - store.config().fork_at_epoch(compute_epoch_at_slot(slot)) == ForkName::Gloas; - slot_is_gloas == (header == ForkName::Gloas) + let slot_fork = store.config().fork_at_epoch(compute_epoch_at_slot(slot)); + if slot_fork >= ForkName::Gloas { + header == slot_fork + } else { + header < ForkName::Gloas + } } /// `200` when nothing failed, else the Beacon API's `IndexedErrorMessage` @@ -359,6 +456,13 @@ async fn post_aggregate_and_proofs( headers: HeaderMap, body: Bytes, ) -> Response { + let is_pq = headers + .get("eth-consensus-version") + .and_then(|value| value.to_str().ok()) + == Some(ForkName::Pq.as_str()); + if is_pq { + return post_pq_aggregate_and_proofs(&store, &p2p, &headers, &body); + } let header_fork = match submission_fork(&headers) { Ok(fork) => fork, Err(err) => return err.into_response(), @@ -444,6 +548,66 @@ async fn post_aggregate_and_proofs( ) } +/// [`post_aggregate_and_proofs`] at a PQ slot: aggregates proved with +/// STARKs and signed with leanXMSS, judged by PQ's `beacon_aggregate_and_proof` +/// rules (the aggregator's signature at the slot's aggregate leaf, its seat +/// in the aggregation committee, the proof) and gossiped. The network actor +/// hands each to the chain actor and to block production's pool. +fn post_pq_aggregate_and_proofs( + store: &Store, + p2p: &RpcToP2PRef, + headers: &HeaderMap, + body: &[u8], +) -> Response { + use ethlambda_state_transition::beacon::gossip::pq as pq_gossip; + use ethlambda_types::beacon::containers::pq; + let aggregates = match decode_list::(headers, body) { + Ok(aggregates) => aggregates, + Err(err) => return err.into_response(), + }; + let now_ms = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|elapsed| elapsed.as_millis() as u64) + .unwrap_or(0); + let capacity = std::num::NonZeroUsize::MIN; + let mut failures = Vec::new(); + for (index, signed) in aggregates.into_iter().enumerate() { + let slot = signed.message.aggregate.data.slot; + let aggregator = signed.message.aggregator_index; + // A fresh seen-cache, as for the other forks' aggregates. + let seen = aggregate::SeenAggregates::new(capacity, capacity); + let checked = pq_gossip::aggregate_cheap_checks(&seen, store, &signed, now_ms) + .and_then(|()| pq_gossip::aggregate_stateful_checks(store, &signed)) + .map_err(|outcome| format!("{outcome:?}")); + let published = checked.and_then(|attesting_indices| { + p2p.publish_pq_gossip(ethlambda_network_api::PqGossipObject::Aggregate { + aggregate: Box::new(signed), + attesting_indices, + }) + .map_err(|_| "the network actor is not running".to_string()) + }); + match published { + Ok(()) => debug!(%slot, aggregator, "Accepted a PQ aggregate for gossip"), + Err(message) => { + warn!(%slot, aggregator, reason = %message, "Refused a submitted PQ aggregate"); + failures.push(Failure { + index, + message: message.into(), + }); + } + } + } + batch_response( + failures, + "some aggregates failed validation and were not published", + ) +} + +/// The wall clock's slot at `now_ms` (`0` before genesis). +fn wall_slot(config: ðlambda_types::beacon::config::Config, now_ms: u64) -> Slot { + now_ms.saturating_sub(config.genesis_time.saturating_mul(1000)) / config.slot_duration_ms.max(1) +} + #[derive(Debug, Deserialize)] struct AggregateQuery { attestation_data_root: Root, @@ -465,6 +629,7 @@ async fn get_aggregate_attestation( State(store): State, Extension(engine): Extension>, Query(query): Query, + headers: HeaderMap, ) -> Response { if let Err(err) = require_execution_client(&engine) { return err.into_response(); @@ -472,6 +637,9 @@ async fn get_aggregate_attestation( let fork = store .config() .fork_at_epoch(compute_epoch_at_slot(query.slot)); + if fork == ForkName::Pq { + return get_pq_aggregate(&store, &query, &headers).await; + } let aggregate = store.attestation_pool().aggregate( query.attestation_data_root, query.slot, @@ -483,9 +651,11 @@ async fn get_aggregate_attestation( if let Err(err) = require_validated(&store, aggregate.data.beacon_block_root) { return err.into_response(); } - // The pool holds electra-shaped votes; a gloas slot is served the same - // vote in gloas's container, whose JSON is the same shape. - let data = if fork == ForkName::Gloas { + // The pool holds electra-shaped votes; a slot from gloas on is served the + // same vote in gloas's container, whose JSON is the same shape. EIP-8483 + // keeps that container. PQ replaces it, but no PQ vote reaches the pool: + // `header_matches_slot` leaves no version that submits at a PQ slot. + let data = if fork >= ForkName::Gloas { serde_json::json!(gloas::Attestation::from(&aggregate)) } else { serde_json::json!(aggregate) @@ -497,6 +667,56 @@ async fn get_aggregate_attestation( crate::shared::content::with_consensus_version(response, fork) } +/// [`get_aggregate_attestation`] at a PQ slot: the committee's aggregate +/// this node proved from the votes it collected for its aggregators (see +/// [`crate::beacon::pq_aggregation`]), with its proof. A request racing the +/// proof waits for it until the slot's signatures are sealed and proved, for +/// at most a few seconds; a 404 when there is none. +async fn get_pq_aggregate(store: &Store, query: &AggregateQuery, headers: &HeaderMap) -> Response { + use ethlambda_types::beacon::containers::pq; + use libssz::SszEncode as _; + const WAIT: std::time::Duration = std::time::Duration::from_secs(3); + const POLL: std::time::Duration = std::time::Duration::from_millis(100); + let started = std::time::Instant::now(); + let aggregate = loop { + let proven = store.pq_signature_pool().proven( + query.slot, + query.attestation_data_root, + query.committee_index, + ); + if let Some(aggregate) = proven { + break aggregate; + } + let collecting = store + .pq_signature_pool() + .is_subscribed(query.slot, query.committee_index); + if !collecting || started.elapsed() >= WAIT { + return ApiError::NotFound("no PQ aggregate proved for this committee").into_response(); + } + tokio::time::sleep(POLL).await; + }; + if let Err(err) = require_validated(store, aggregate.attestation.data.beacon_block_root) { + return err.into_response(); + } + let answer = pq::ProvenAggregate { + aggregate: aggregate.attestation.clone(), + proof: aggregate.proof.clone(), + }; + let accept = headers + .get(axum::http::header::ACCEPT) + .and_then(|value| value.to_str().ok()); + let response = match crate::shared::content::Encoding::from_accept(accept) { + crate::shared::content::Encoding::Ssz => { + crate::shared::content::ssz_response(answer.to_ssz()) + } + crate::shared::content::Encoding::Json => crate::json_response(serde_json::json!({ + "version": ForkName::Pq.as_str(), + "data": answer, + })), + }; + crate::shared::content::with_consensus_version(response, ForkName::Pq) +} + /// The root of the latest block at or before `epoch`'s first slot on the chain /// ending in `root`: the spec's `get_checkpoint_block`, walked through the /// store's block index. `None` if the walk leaves the stored chain. @@ -581,10 +801,18 @@ mod tests { /// slot, as [`fixture`]'s does, so it is its own checkpoint. Its votes are /// gloas's, so it submits under that version. fn fork_boundary_fixture() -> Fixture { + fork_boundary_fixture_at(&[ForkName::Gloas]) + } + + /// [`fork_boundary_fixture`], with each of `forks` scheduled at the wall + /// clock's epoch. + fn fork_boundary_fixture_at(forks: &[ForkName]) -> Fixture { let mut state = with_signing_validators_at(ForkName::Fulu, 64); let (probe, _) = crate::test_utils::beacon_store_at(state.clone()); let wall_epoch = compute_epoch_at_slot(crate::beacon::node::wall_slot(&probe)); - let config = Config::mainnet().with_fork_epoch(ForkName::Gloas, wall_epoch); + let config = forks.iter().fold(Config::mainnet(), |config, &fork| { + config.with_fork_epoch(fork, wall_epoch) + }); *state.slot_mut() = compute_start_slot_at_epoch(wall_epoch - 1); *state.fork_mut() = Fork { previous_version: config.fork_version(ForkName::Electra), @@ -1135,6 +1363,39 @@ mod tests { assert!(fulu.network.published.lock().unwrap().is_empty()); } + /// The forks after gloas build on it, so their slots are gloas slots, not + /// electra ones: a vote there under fulu's header is refused rather than + /// judged by electra's rules. Gloas's header names a different fork, and + /// the node follows neither EIP-8483 nor PQ, so no header submits there. + #[tokio::test] + async fn no_header_submits_at_a_slot_after_gloas() { + let schedules: [&[ForkName]; 2] = [ + &[ForkName::Gloas, ForkName::Eip8483], + &[ForkName::Gloas, ForkName::Eip8483, ForkName::Pq], + ]; + for forks in schedules { + let fixture = fork_boundary_fixture_at(forks); + let slot = first_slot_of_the_fork(&fixture); + let roots = (fixture.head_root, fixture.head_root); + let vote = attestation_for(&fixture, slot, roots, 0, 0, 0); + for version in ["fulu", "gloas"] { + let (status, json) = + submit_as(&fixture, std::slice::from_ref(&vote), version).await; + assert_eq!( + status, + StatusCode::BAD_REQUEST, + "{forks:?} {version}: {json}" + ); + assert_eq!( + json["failures"][0]["message"], + "Eth-Consensus-Version does not match the attestation slot's fork", + "{forks:?} {version}" + ); + } + assert!(fixture.network.published.lock().unwrap().is_empty()); + } + } + #[tokio::test] async fn a_gloas_vote_in_the_voted_blocks_own_slot_is_published_with_index_zero() { let fixture = fixture_at(ForkName::Gloas, 0); diff --git a/crates/net/rpc/src/beacon/pq_aggregation.rs b/crates/net/rpc/src/beacon/pq_aggregation.rs new file mode 100644 index 000000000..000047fe0 --- /dev/null +++ b/crates/net/rpc/src/beacon/pq_aggregation.rs @@ -0,0 +1,226 @@ +//! PQ aggregation for this node's validator client. +//! +//! A PQ aggregator proves its committee's signatures instead of aggregating +//! them (PQ `validator.md`, "Aggregate stark"), and proving takes the +//! sidecar seconds, too long to run inside the `aggregate_attestation` +//! request. So, as Lighthouse's PQ branch does, the node proves ahead: the +//! validator client names the committees it aggregates for through +//! `beacon_committee_subscriptions`, the node collects their validated votes' +//! signatures ([`ethlambda_storage::pools::PqSignaturePool`]), and once a slot +//! the task here seals that slot's signatures halfway between the +//! attestation and aggregate deadlines and proves each committee's before the +//! aggregate deadline, largest first. The endpoint then answers with what was +//! proved, and the aggregate also goes into the pool block production packs +//! from, since gossip never hands a node its own messages. + +use std::sync::Arc; +use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; + +use ethlambda_state_transition::beacon::{ + fork_choice::{get_attestation_due_ms, get_slot_component_duration_ms}, + helpers::{accessors::CommitteeCacheExt as _, pq::get_attestation_statements}, +}; +use ethlambda_storage::{ + Store, + pools::{CommitteeSignatures, PooledPqAggregate}, +}; +use ethlambda_types::beacon::{ + config::Config, + containers::{BeaconState, pq}, + fork::ForkName, + primitives::Slot, + signing::compute_epoch_at_slot, +}; +use tokio_util::sync::CancellationToken; +use tracing::{debug, info, warn}; + +use crate::beacon::validator::head; + +/// Start proving PQ aggregates once a slot, if the network schedules PQ. +/// Stops with `shutdown`. +pub(crate) fn spawn(store: Store, shutdown: CancellationToken) { + let config = store.config(); + if config.fork_epoch(ForkName::Pq) == u64::MAX { + return; + } + tokio::spawn(async move { + loop { + let slot = wall_slot(&config) + 1; + let cutoff = collection_cutoff_ms(&config, slot); + let wait = Duration::from_millis(cutoff.saturating_sub(now_ms())); + tokio::select! { + () = shutdown.cancelled() => return, + () = tokio::time::sleep(wait) => {} + } + if config.fork_at_epoch(compute_epoch_at_slot(slot)) != ForkName::Pq { + continue; + } + prove_slot(&store, &config, slot).await; + } + }); +} + +/// Seal `slot`'s collected signatures and prove what the time to its +/// aggregate deadline allows. +async fn prove_slot(store: &Store, config: &Config, slot: Slot) { + let groups = store.pq_signature_pool().seal(slot); + if groups.is_empty() { + return; + } + let deadline_ms = slot_start_ms(config, slot) + aggregate_due_ms(config, slot); + let Some(budget) = deadline_ms.checked_sub(now_ms()).filter(|&left| left > 0) else { + warn!(%slot, "No time left to prove this slot's PQ aggregates"); + return; + }; + let deadline = Instant::now() + Duration::from_millis(budget); + let store = store.clone(); + let job = tokio::task::spawn_blocking(move || { + let state = match head(&store) { + Ok((_, state)) => state, + Err(_) => return, + }; + let mut groups = groups; + // The larger groups first, in case not every one fits before the + // deadline. + groups.sort_by_key(|group| std::cmp::Reverse(group.signatures.len())); + for group in groups { + if Instant::now() >= deadline { + warn!(%slot, "The PQ aggregate deadline passed with committees left to prove"); + break; + } + let committee = group.committee_index; + let started = Instant::now(); + match prove_committee(&store, &state, group) { + Ok(aggregate) if Instant::now() < deadline => { + info!( + %slot, + committee, + attesters = aggregate.attesting_indices.len(), + proof_bytes = aggregate.proof.len(), + took_ms = started.elapsed().as_millis() as u64, + "Proved a PQ aggregate" + ); + store.pq_aggregate_pool().insert((*aggregate).clone()); + store.pq_signature_pool().insert_proven(aggregate); + } + Ok(_) => { + warn!(%slot, committee, "A PQ aggregate was proved after its deadline"); + break; + } + Err(err) => debug!(%slot, committee, %err, "Could not prove a PQ aggregate"), + } + } + }); + let _ = job.await; +} + +/// The network aggregate of `group`'s committee and the proof of its +/// signatures, over the committee `state` assigns: bits at the attesters' +/// positions, signatures in committee order, which is the order +/// `get_attestation_statements` lists their keys in. +fn prove_committee( + store: &Store, + state: &BeaconState, + group: CommitteeSignatures, +) -> Result, String> { + let committees = store.committee_cache(); + let epoch_committees = committees.committees(state, group.data.target.epoch); + let committee = epoch_committees + .committee(group.data.slot, group.committee_index) + .map_err(|err| err.to_string())?; + let mut aggregation_bits = pq::AggregationBits::with_length(committee.len()); + let mut attesting_indices = Vec::new(); + let mut signatures = Vec::new(); + for (position, &validator) in committee.iter().enumerate() { + if let Some(signature) = group.signatures.get(&validator) { + aggregation_bits + .set(position, true) + .map_err(|_| "a position past the committee".to_string())?; + attesting_indices.push(validator); + signatures.push(signature.0.to_vec()); + } + } + if signatures.is_empty() { + return Err("no collected signature belongs to the committee".into()); + } + let mut committee_bits = pq::CommitteeBits::default(); + committee_bits + .set(group.committee_index as usize, true) + .map_err(|_| "a committee index past MAX_COMMITTEES_PER_SLOT_PQ".to_string())?; + let attestation = pq::Attestation { + aggregation_bits, + data: group.data, + committee_bits, + }; + let statements = get_attestation_statements(state, &attestation, &committees) + .map_err(|err| err.to_string())?; + let proof = ethlambda_state_transition::beacon::helpers::pq::prove_aggregates( + &statements, + &[signatures], + ) + .map_err(|err| err.to_string())?; + attesting_indices.sort_unstable(); + Ok(Arc::new(PooledPqAggregate { + attestation, + committee_index: group.committee_index, + attesting_indices, + proof: proof + .try_into() + .map_err(|_| "the proof exceeds MAX_STARK_PROOF_SIZE".to_string())?, + })) +} + +fn now_ms() -> u64 { + SystemTime::now() + .duration_since(UNIX_EPOCH) + .map_or(0, |elapsed| elapsed.as_millis() as u64) +} + +fn slot_start_ms(config: &Config, slot: Slot) -> u64 { + config + .genesis_time + .saturating_mul(1000) + .saturating_add(slot.saturating_mul(config.slot_duration_ms)) +} + +/// The wall clock's slot (`0` before genesis). +fn wall_slot(config: &Config) -> Slot { + now_ms().saturating_sub(config.genesis_time.saturating_mul(1000)) + / config.slot_duration_ms.max(1) +} + +/// How far into `slot` its aggregates are due. +fn aggregate_due_ms(config: &Config, slot: Slot) -> u64 { + let gloas_on = config.fork_at_epoch(compute_epoch_at_slot(slot)) >= ForkName::Gloas; + let basis_points = if gloas_on { + config.aggregate_due_bps_gloas + } else { + config.aggregate_due_bps + }; + get_slot_component_duration_ms(basis_points, config) +} + +/// When `slot`'s signatures stop being collected: halfway from the +/// attestation deadline to the aggregate deadline. +fn collection_cutoff_ms(config: &Config, slot: Slot) -> u64 { + let attestation_due = get_attestation_due_ms(compute_epoch_at_slot(slot), config); + let aggregate_due = aggregate_due_ms(config, slot); + let window = aggregate_due.saturating_sub(attestation_due); + slot_start_ms(config, slot) + attestation_due + window / 2 +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn signatures_are_collected_until_halfway_to_the_aggregate_deadline() { + let mut config = Config::mainnet(); + config.genesis_time = 100; + config.gloas_fork_epoch = 0; + // Gloas timing: attestations at 25% of 12 s, aggregates at 50%. + assert_eq!(slot_start_ms(&config, 2), 100_000 + 24_000); + assert_eq!(aggregate_due_ms(&config, 2), 6_000); + assert_eq!(collection_cutoff_ms(&config, 2), 124_000 + 3_000 + 1_500); + } +} diff --git a/crates/net/rpc/src/beacon/pq_proposal.rs b/crates/net/rpc/src/beacon/pq_proposal.rs new file mode 100644 index 000000000..ed65ccd6b --- /dev/null +++ b/crates/net/rpc/src/beacon/pq_proposal.rs @@ -0,0 +1,382 @@ +//! PQ block production for ethlambda's own validator client. +//! +//! `POST /eth/v1/validator/pq_blocks/{slot}` takes the proposer's VRF proof +//! for the slot (its RANDAO reveal) and answers the unsigned block and the +//! unsigned envelope revealing its self-built payload, as SSZ +//! `pq::BlockContents`. The Beacon API has no PQ production endpoint yet; +//! Lighthouse's PQ branch has an equivalent of its own. The proposer signs +//! both at their own leaves and publishes them through the usual block and +//! envelope endpoints. +//! +//! The build is gloas's self-build (`gloas_proposal`) with PQ's shapes: the +//! parent's requests are PQ's, the bid carries the empty `XMSSSignature()`, +//! and the block carries no sync aggregate or payload attestations. No +//! builder bids are considered. +//! +//! The attestations come from the pool of validated network aggregates, and +//! their proofs are merged into the block's one `proof` while the execution +//! client builds the payload. A merge runs on the proving sidecar and costs +//! seconds when there is more than one child, so it is bounded by a deadline +//! three quarters of the way to the slot's attestation deadline (Lighthouse's +//! PQ branch uses the same one). A merge that misses it or fails is dropped +//! for the most valuable single-committee attestation alone, whose proof +//! needs no merging; a block `process_stark` refuses is built again without +//! attestations, so the block goes out rather than not at all. + +use std::time::{Duration, Instant, SystemTime, UNIX_EPOCH}; + +use axum::{ + Extension, Json, Router, + extract::{Path, State}, + response::{IntoResponse, Response}, + routing::post, +}; +use ethlambda_blockchain::{SyncStatusController, metrics::SyncStatus}; +use ethlambda_engine::EngineClient; +use ethlambda_state_transition::beacon::{ + block_production::advance_to_slot, + fork_choice::{ForkChoiceNode, get_attestation_due_ms, get_head_node, should_build_on_full}, + helpers::{accessors::get_beacon_proposer_index, pq::is_valid_vrf_proof}, + pq_block_production::{ + PqBlockInputs, PqPacked, assemble_pq_block, parse_pq_execution_requests, pq_payload_inputs, + select_pq_attestations, + }, +}; +use ethlambda_storage::Store; +use ethlambda_types::beacon::{ + config::Config, + containers::{BeaconState, SignedBeaconBlock, SignedExecutionPayloadEnvelope, pq}, + fork::ForkName, + preset::MIN_ATTESTATION_INCLUSION_DELAY_PQ, + primitives::{Bytes32, ExecutionAddress, Root, Slot}, + signing::compute_epoch_at_slot, +}; +use libssz::SszEncode as _; +use serde::Deserialize; +use tokio::sync::Semaphore; +use tracing::{info, warn}; + +use crate::CustodyColumns as NodeCustodyColumns; +use crate::beacon::{ + ApiError, + gloas_proposal::build_payload, + proposal::refuse_future_slot, + validator::{FeeRecipients, head}, +}; +use crate::shared::content::{ssz_response, with_consensus_version}; + +pub(crate) fn routes() -> Router { + Router::new().route( + "/eth/v1/validator/pq_blocks/{slot}", + post(post_produce_pq_block), + ) +} + +#[derive(Debug, Deserialize)] +struct ProducePqBlock { + /// The proposer's VRF proof for the slot: the leaf, then its branch. + randao_reveal: Vec, + #[serde(default)] + graffiti: Option, +} + +async fn post_produce_pq_block( + Path(slot): Path, + State(store): State, + Extension(engine): Extension>, + Extension(fee_recipients): Extension, + Extension(custody): Extension, + Extension(sync_status): Extension, + Json(request): Json, +) -> Response { + let Ok(slot) = slot.parse::() else { + return ApiError::BadRequest("invalid slot").into_response(); + }; + let config = store.config(); + if config.fork_at_epoch(compute_epoch_at_slot(slot)) != ForkName::Pq { + return ApiError::BadRequest("this endpoint serves PQ slots only").into_response(); + } + if let Err(err) = refuse_future_slot(&config, slot) { + return err.into_response(); + } + let Ok(randao_reveal) = pq::VrfProof::try_from(request.randao_reveal) else { + return ApiError::BadRequest("randao_reveal is VRF_TREE_DEPTH + 1 roots").into_response(); + }; + if sync_status.get() == SyncStatus::Syncing { + return ApiError::ServiceUnavailable("the node is syncing").into_response(); + } + let Some(engine) = engine else { + return ApiError::ServiceUnavailable( + "no execution client configured to build a payload with", + ) + .into_response(); + }; + match produce( + &store, + &engine, + &fee_recipients, + &custody, + slot, + randao_reveal, + request.graffiti.unwrap_or(Bytes32::ZERO), + ) + .await + { + Ok(contents) => with_consensus_version(ssz_response(contents.to_ssz()), ForkName::Pq), + Err(err) => err.into_response(), + } +} + +async fn produce( + store: &Store, + engine: &EngineClient, + fee_recipients: &FeeRecipients, + custody: &NodeCustodyColumns, + slot: Slot, + randao_reveal: pq::VrfProof, + graffiti: Bytes32, +) -> Result { + let config = store.config(); + let (head_root, head_state) = head(store)?; + if slot <= head_state.slot() { + return Err(ApiError::BadRequest("slot is not after the head block")); + } + let payload_status = match store.head_payload_status() { + Some(status) => status, + None => { + get_head_node(store, &config) + .map_err(|_| ApiError::Internal("fork choice could not name the head"))? + .payload_status + } + }; + let node = ForkChoiceNode { + root: head_root, + payload_status, + }; + let build_on_full = should_build_on_full(store, node, slot) + .map_err(|_| ApiError::Internal("could not decide which parent payload to build on"))?; + let head_is_pq = matches!( + store.get_signed_block(&head_root), + Ok(Some(SignedBeaconBlock::Pq(_))) + ); + let parent_requests = if build_on_full && head_is_pq { + match store + .get_execution_payload_envelope(&head_root) + .map_err(|_| ApiError::Internal("store read failed"))? + { + Some(SignedExecutionPayloadEnvelope::Pq(envelope)) => { + envelope.message.execution_requests.clone() + } + Some(_) => return Err(ApiError::Internal("a PQ head with a non-PQ envelope")), + None => { + return Err(ApiError::ServiceUnavailable( + "the parent's payload envelope is not held to build on", + )); + } + } + } else { + pq::ExecutionRequests::default() + }; + + let state = advance_to_slot(&head_state, slot, &config).map_err(|err| { + warn!(%slot, %err, "Advancing the head state to the PQ proposal slot failed"); + ApiError::Internal("advancing the head state failed") + })?; + let proposer = get_beacon_proposer_index(&state) + .map_err(|_| ApiError::Internal("no proposer for the slot"))?; + if !is_valid_vrf_proof(&state, proposer, slot, &randao_reveal).unwrap_or(false) { + return Err(ApiError::BadRequest("invalid randao_reveal")); + } + let inputs = pq_payload_inputs(&state, build_on_full, &parent_requests, &config) + .map_err(|_| ApiError::Internal("computing the payload attributes failed"))?; + let fee_recipient = fee_recipients + .lock() + .expect("fee recipient lock poisoned") + .get(&proposer) + .copied() + .unwrap_or(ExecutionAddress::ZERO); + let (built, (attestations, proof)) = tokio::join!( + build_payload(store, engine, fee_recipient, custody, &inputs), + pack_attestations(store, &state, &config), + ); + let built = built?; + if !built.blobs_bundle.blobs.is_empty() { + // The envelope's column sidecars are not built for PQ yet: a payload + // with blobs would reveal data no node could sample. + warn!(%slot, blobs = built.blobs_bundle.blobs.len(), "Refusing a PQ build with blobs"); + return Err(ApiError::ServiceUnavailable( + "PQ block production does not carry blobs yet", + )); + } + let execution_requests = parse_pq_execution_requests(&built.execution_requests) + .map_err(|_| ApiError::Internal("the execution client's request list is malformed"))?; + let inputs = PqBlockInputs { + randao_reveal, + graffiti, + attestations, + proof, + parent_execution_requests: parent_requests, + execution_payload: built.execution_payload, + blob_kzg_commitments: built.blobs_bundle.commitments, + execution_requests, + }; + // `process_block` runs `process_stark` on the merged proof: one that does + // not prove the packed attestations fails here, and the block is built + // again without them. + let fallback = (!inputs.attestations.is_empty()).then(|| PqBlockInputs { + attestations: Vec::new(), + proof: Vec::new(), + ..inputs.clone() + }); + let produced = match assemble_pq_block(&state, inputs, &config, Default::default()) { + Ok(produced) => produced, + Err(err) => { + let Some(fallback) = fallback else { + warn!(%slot, %err, "The PQ block failed to build"); + return Err(ApiError::Internal("the block failed to build")); + }; + warn!(%slot, %err, "The PQ block failed to build with its attestations; building it without"); + assemble_pq_block(&state, fallback, &config, Default::default()).map_err(|err| { + warn!(%slot, %err, "The PQ block failed to build"); + ApiError::Internal("the block failed to build") + })? + } + }; + let attestations = produced.block.body.attestations.len(); + info!(%slot, proposer, build_on_full, attestations, "Produced a PQ block"); + Ok(pq::BlockContents { + block: produced.block, + execution_payload_envelope: produced.envelope, + }) +} + +/// One merge at a time: the sidecar proves one request after another, so a +/// second merge started while one runs would only push both past their +/// deadlines. A merge that missed its deadline keeps the permit until it +/// ends. +static MERGE_PERMIT: Semaphore = Semaphore::const_new(1); + +/// The attestations a PQ block at `state`'s slot packs and the merged proof +/// of their aggregates, or none when there are none, the merge misses its +/// deadline, or it fails. +async fn pack_attestations( + store: &Store, + state: &BeaconState, + config: &Config, +) -> (Vec, Vec) { + let slot = state.slot(); + let latest = slot.saturating_sub(MIN_ATTESTATION_INCLUSION_DELAY_PQ); + let candidates = store.pq_aggregate_pool().candidates(latest); + if candidates.is_empty() { + return Default::default(); + } + let selected = { + let (store, state) = (store.clone(), state.clone()); + tokio::task::spawn_blocking(move || { + select_pq_attestations(&state, candidates, &store.committee_cache()) + }) + .await + }; + let packed = match selected { + Ok(Ok(packed)) => packed, + Ok(Err(err)) => { + warn!(%slot, %err, "Selecting the PQ attestations failed"); + return Default::default(); + } + Err(err) => { + warn!(%slot, %err, "The PQ attestation selection task failed"); + return Default::default(); + } + }; + if packed.attestations.is_empty() { + return Default::default(); + } + // One child's proof is the block's as it is; more must be merged. + let fallback = packed.first_unmerged(); + if packed.children.len() == 1 { + return unmerged(slot, packed); + } + let children = packed.children.len(); + let fall_back = |reason: &str| { + warn!(%slot, children, reason, "Packing one PQ attestation instead of merging every proof"); + fallback + .clone() + .map_or_else(Default::default, |single| unmerged(slot, single)) + }; + let Some(remaining) = merge_budget(config, slot) else { + return fall_back("no time left to merge"); + }; + let Ok(permit) = MERGE_PERMIT.try_acquire() else { + return fall_back("a merge is still running"); + }; + let job = tokio::task::spawn_blocking(move || { + let _permit = permit; + let started = Instant::now(); + let proof = packed.merge().map_err(|err| err.to_string()); + // Logged when it ends, deadline or not: what a merge costs here. + info!( + %slot, + children, + took_ms = started.elapsed().as_millis() as u64, + ok = proof.is_ok(), + "Merged PQ attestation proofs" + ); + proof.map(|proof| (packed, proof)) + }); + match tokio::time::timeout(remaining, job).await { + Ok(Ok(Ok((packed, proof)))) => { + info!( + %slot, + attestations = packed.attestations.len(), + children, + proof_bytes = proof.len(), + "Packed PQ attestations" + ); + (packed.attestations, proof) + } + Ok(Ok(Err(err))) => fall_back(&format!("the merge failed: {err}")), + Ok(Err(err)) => fall_back(&format!("the merge task failed: {err}")), + Err(_) => fall_back(&format!( + "the merge missed its {} ms budget", + remaining.as_millis() + )), + } +} + +/// `packed` with its one child's proof as the block's, nothing merged. +fn unmerged(slot: Slot, packed: PqPacked) -> (Vec, Vec) { + match packed.merge() { + Ok(proof) => { + info!( + %slot, + attestations = packed.attestations.len(), + proof_bytes = proof.len(), + "Packed PQ attestations" + ); + (packed.attestations, proof) + } + Err(err) => { + warn!(%slot, %err, "A single PQ proof would not pass through the merge"); + Default::default() + } + } +} + +/// How long a merge for a block at `slot` may take: until three quarters of +/// the slot's attestation deadline, so the block still reaches the attesters +/// in time. `None` once that point has passed. +fn merge_budget(config: &Config, slot: Slot) -> Option { + let due_ms = get_attestation_due_ms(compute_epoch_at_slot(slot), config); + let deadline_ms = config + .genesis_time + .saturating_mul(1000) + .saturating_add(slot.saturating_mul(config.slot_duration_ms)) + .saturating_add(due_ms.saturating_mul(3) / 4); + let now_ms = SystemTime::now() + .duration_since(UNIX_EPOCH) + .map_or(0, |elapsed| elapsed.as_millis() as u64); + deadline_ms + .checked_sub(now_ms) + .filter(|&left| left > 0) + .map(Duration::from_millis) +} diff --git a/crates/net/rpc/src/beacon/proposal.rs b/crates/net/rpc/src/beacon/proposal.rs index 7356f9d89..11767b4c5 100644 --- a/crates/net/rpc/src/beacon/proposal.rs +++ b/crates/net/rpc/src/beacon/proposal.rs @@ -162,15 +162,16 @@ async fn post_block( .get("eth-consensus-version") .and_then(|value| value.to_str().ok()) .and_then(ForkName::parse); - if !matches!(fork, Some(ForkName::Fulu | ForkName::Gloas)) { - return ApiError::BadRequest("Eth-Consensus-Version must be fulu or gloas").into_response(); + if !matches!(fork, Some(ForkName::Fulu | ForkName::Gloas | ForkName::Pq)) { + return ApiError::BadRequest("Eth-Consensus-Version must be fulu, gloas or pq") + .into_response(); } let encoding = match BodyEncoding::from_headers(&headers) { Ok(encoding) => encoding, Err(err) => return err.into_response(), }; - if fork == Some(ForkName::Gloas) { - return post_gloas_block(&store, &p2p, &published, encoding, &body).await; + if let Some(fork @ (ForkName::Gloas | ForkName::Pq)) = fork { + return post_gloas_block(&store, &p2p, &published, fork, encoding, &body).await; } let Some(contents) = encoding.decode::(&body) else { return ApiError::BadRequest("the body is not fulu SignedBlockContents").into_response(); @@ -289,22 +290,39 @@ async fn post_block( /// slot's, and the proposer's signature verifies against the parent's state /// advanced to the slot. The advance and the signature are CPU-bound and run /// off the runtime. +/// +/// A PQ block (`fork` is PQ) takes the same path: it carries no payload or +/// blobs either, and its proposer's leanXMSS signature is checked at its +/// slot's proposer leaf. async fn post_gloas_block( store: &Store, p2p: &RpcToP2PRef, published: &PublishedProposals, + fork: ForkName, encoding: BodyEncoding, body: &[u8], ) -> Response { - let Some(signed) = encoding.decode::(body) else { - return ApiError::BadRequest("the body is not a gloas SignedBeaconBlock").into_response(); + let block = if fork == ForkName::Pq { + let Some(signed) = encoding.decode::(body) else { + return ApiError::BadRequest("the body is not a pq SignedBeaconBlock").into_response(); + }; + containers::SignedBeaconBlock::Pq(Box::new(signed)) + } else { + let Some(signed) = encoding.decode::(body) else { + return ApiError::BadRequest("the body is not a gloas SignedBeaconBlock") + .into_response(); + }; + containers::SignedBeaconBlock::Gloas(signed) }; - let block = containers::SignedBeaconBlock::Gloas(signed); let config = store.config(); - if let Err(err) = require_gloas_slot( + let slot_fork = config.fork_at_epoch(compute_epoch_at_slot(block.slot())); + if slot_fork != fork { + return ApiError::BadRequest("the block's fork is not its slot's").into_response(); + } + if let Err(err) = require_gloas_family_slot( &config, block.slot(), - "blocks are accepted for gloas slots only", + "blocks are accepted for gloas and pq slots only", ) .and_then(|()| refuse_future_slot(&config, block.slot())) .and_then(|()| refuse_finalized_slot(store, block.slot())) @@ -368,6 +386,18 @@ async fn post_gloas_block( StatusCode::OK.into_response() } +/// Refuses a slot the schedule places at neither gloas nor PQ. +fn require_gloas_family_slot( + config: &Config, + slot: Slot, + message: &'static str, +) -> Result<(), ApiError> { + match config.fork_at_epoch(compute_epoch_at_slot(slot)) { + ForkName::Pq => Ok(()), + _ => require_gloas_slot(config, slot, message), + } +} + /// Refuses a slot the schedule does not place at gloas. pub(crate) fn require_gloas_slot( config: &Config, @@ -383,6 +413,9 @@ pub(crate) fn require_gloas_slot( | ForkName::Deneb | ForkName::Electra | ForkName::Fulu + // Not followed, and no block production for them. + | ForkName::Eip8483 + | ForkName::Pq | ForkName::Lean => Err(ApiError::BadRequest(message)), } } @@ -448,6 +481,8 @@ pub(super) fn require_fulu_slot( | ForkName::Deneb | ForkName::Electra | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq | ForkName::Lean => Err(ApiError::BadRequest(message)), } } diff --git a/crates/net/rpc/src/beacon/ptc.rs b/crates/net/rpc/src/beacon/ptc.rs index 962bdd355..3a633cf9a 100644 --- a/crates/net/rpc/src/beacon/ptc.rs +++ b/crates/net/rpc/src/beacon/ptc.rs @@ -192,7 +192,8 @@ fn ptc_duties( )); } - let readable_from_head = matches!(*head_state, BeaconState::Gloas(_)) + // A head of a fork after gloas holds the PTC window too. + let readable_from_head = head_state.fork_name().has_payload_envelopes() && epoch <= state_epoch + preset::MIN_SEED_LOOKAHEAD; let state: Arc = if readable_from_head { head_state @@ -295,6 +296,16 @@ fn bid_commitment_count(block: &SignedBeaconBlock) -> Option { .blob_kzg_commitments .len(), ), + // PQ keeps gloas's bid. + SignedBeaconBlock::Pq(inner) => Some( + inner + .message + .body + .signed_execution_payload_bid + .message + .blob_kzg_commitments + .len(), + ), _ => None, } } @@ -394,6 +405,13 @@ async fn post_pool_payload_attestations( headers: HeaderMap, body: Bytes, ) -> Response { + if headers + .get("eth-consensus-version") + .and_then(|value| value.to_str().ok()) + == Some(ForkName::Pq.as_str()) + { + return post_pq_payload_attestations(&store, &p2p, &headers, &body); + } if let Err(err) = require_gloas_or_absent(&headers) { return err.into_response(); } @@ -463,6 +481,68 @@ async fn post_pool_payload_attestations( response } +/// The PQ half: PQ's message, signed with leanXMSS at the slot's PTC leaf, +/// judged by PQ's gossip rules, gossiped and handed to the chain actor by the +/// publish path. Not pooled: a PQ block carries no payload attestations. +fn post_pq_payload_attestations( + store: &Store, + p2p: &RpcToP2PRef, + headers: &HeaderMap, + body: &[u8], +) -> Response { + use ethlambda_state_transition::beacon::gossip::pq as pq_gossip; + let messages = match decode_list::< + ethlambda_types::beacon::containers::pq::PayloadAttestationMessage, + >(headers, body) + { + Ok(messages) => messages, + Err(err) => return err.into_response(), + }; + let now_ms = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|elapsed| elapsed.as_millis() as u64) + .unwrap_or(0); + let seen = SeenPayloadAttestations::new(std::num::NonZeroUsize::MIN); + let mut failures = Vec::new(); + for (index, message) in messages.into_iter().enumerate() { + let slot = message.data.slot; + let validator = message.validator_index; + let verdict = + match pq_gossip::payload_attestation_cheap_checks(&seen, store, &message, now_ms) { + Ok(()) => pq_gossip::payload_attestation_stateful_checks(store, &message), + Err(outcome) => outcome, + }; + if verdict != Outcome::Accept { + let (outcome, reason) = verdict.labels(); + warn!(%slot, validator, outcome, reason, "Refused a submitted PQ payload attestation"); + failures.push(Failure { + index, + message: format!("{outcome}: {reason}"), + }); + continue; + } + let object = ethlambda_network_api::PqGossipObject::PayloadAttestation(Box::new(message)); + match p2p.publish_pq_gossip(object) { + Ok(()) => debug!(%slot, validator, "Accepted a PQ payload attestation for gossip"), + Err(_) => failures.push(Failure { + index, + message: "the network actor is not running".to_string(), + }), + } + } + if failures.is_empty() { + return StatusCode::OK.into_response(); + } + let body = serde_json::json!({ + "code": 400, + "message": "some payload attestations failed validation and were not published", + "failures": failures, + }); + let mut response = crate::json_response(body); + *response.status_mut() = StatusCode::BAD_REQUEST; + response +} + /// `Eth-Consensus-Version` is optional here, and must name gloas when given: /// the container exists from that fork alone. fn require_gloas_or_absent(headers: &HeaderMap) -> Result<(), ApiError> { diff --git a/crates/net/rpc/src/beacon/validator.rs b/crates/net/rpc/src/beacon/validator.rs index ac3294986..2ce533c2e 100644 --- a/crates/net/rpc/src/beacon/validator.rs +++ b/crates/net/rpc/src/beacon/validator.rs @@ -39,8 +39,10 @@ use ethlambda_state_transition::beacon::{ checkpoint_state, compute_shuffling_dependent_slot, get_current_store_epoch, get_head_node, }, gossip::attestation::compute_subnet_for_attestation, + gossip::pq::compute_subnet_for_attestation as pq_compute_subnet_for_attestation, helpers::accessors::{CommitteeCacheExt as _, get_block_root_at_slot}, helpers::altair::compute_sync_committee_period, + helpers::pq::get_aggregation_committee, }; use crate::beacon::ApiError; @@ -311,9 +313,10 @@ async fn post_committee_subscriptions( Json(subscriptions): Json>, ) -> Response { let config = store.config(); + let is_pq_slot = |slot: Slot| config.fork_at_epoch(compute_epoch_at_slot(slot)) >= ForkName::Pq; let subnets: Vec<(u64, Slot)> = subscriptions .iter() - .filter(|entry| entry.is_aggregator) + .filter(|entry| entry.is_aggregator && !is_pq_slot(entry.slot)) .map(|entry| { let subnet_id = compute_subnet_for_attestation( entry.committees_at_slot, @@ -324,9 +327,37 @@ async fn post_committee_subscriptions( (subnet_id, entry.slot) }) .collect(); + // A PQ slot's committees travel on PQ's 16 subnets, and its aggregators + // prove what this node collects for them (see `pq_aggregation`). + let now_ms = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map_or(0, |elapsed| elapsed.as_millis() as u64); + let now = now_ms.saturating_sub(config.genesis_time.saturating_mul(1000)) + / config.slot_duration_ms.max(1); + let pq_subnets: Vec<(u64, Slot)> = subscriptions + .iter() + .filter(|entry| entry.is_aggregator && is_pq_slot(entry.slot)) + .filter(|entry| { + store + .pq_signature_pool() + .subscribe(now, entry.slot, entry.committee_index) + }) + .map(|entry| { + let subnet_id = pq_compute_subnet_for_attestation( + entry.committees_at_slot, + entry.slot, + entry.committee_index, + &config, + ); + (subnet_id, entry.slot) + }) + .collect(); if !subnets.is_empty() && p2p.subscribe_attestation_subnets(subnets).is_err() { return ApiError::Internal("the network actor is not running").into_response(); } + if !pq_subnets.is_empty() && p2p.subscribe_pq_attestation_subnets(pq_subnets).is_err() { + return ApiError::Internal("the network actor is not running").into_response(); + } axum::http::StatusCode::OK.into_response() } @@ -581,12 +612,15 @@ fn proposer_duties( // Gloas keeps fulu's lookahead as it is, and `upgrade_to_gloas` carries it // over, so a fulu head already holds the first gloas epoch's proposers. + // The forks built on gloas keep the lookahead too. let proposer_lookahead = match state.as_ref() { BeaconState::Fulu(fulu) => &fulu.proposer_lookahead, BeaconState::Gloas(gloas) => &gloas.proposer_lookahead, + BeaconState::Eip8483(state) => &state.proposer_lookahead, + BeaconState::Pq(state) => &state.proposer_lookahead, _ => { return Err(ApiError::BadRequest( - "proposer duties are served from the proposer lookahead of fulu and gloas only", + "proposer duties are served from the proposer lookahead of fulu on", )); } }; @@ -636,6 +670,12 @@ struct AttesterDuty { validator_committee_index: u64, #[serde(with = "ethlambda_types::beacon::serde_helpers::quoted_or_bare")] slot: Slot, + /// At a PQ slot, whether the validator sits on its committee's + /// aggregation committee, which is what makes a PQ aggregator (PQ + /// `validator.md`, "Aggregation selection"); absent before PQ, where + /// aggregators select themselves. Lighthouse's PQ branch's field. + #[serde(skip_serializing_if = "Option::is_none")] + is_aggregator: Option, } /// `POST /eth/v1/validator/duties/attester/{epoch}`, with the validator @@ -702,6 +742,7 @@ fn attester_duties( let committees = store.committee_cache().committees(&state, epoch); let committees_at_slot = committees.committees_per_slot(); let first_slot = compute_start_slot_at_epoch(epoch); + let is_pq = store.config().fork_at_epoch(epoch) == ForkName::Pq; let mut duties = Vec::new(); for slot in first_slot..first_slot + preset::SLOTS_PER_EPOCH { for committee_index in 0..committees_at_slot { @@ -715,6 +756,12 @@ fn attester_duties( let validator = state .validator(validator_index) .map_err(|_| ApiError::Internal("committee names an unknown validator"))?; + // A state not yet at PQ (the fork's first epoch, asked + // early) names no aggregation committee: not an aggregator. + let is_aggregator = is_pq.then(|| { + get_aggregation_committee(&state, slot, committee_index) + .is_ok_and(|members| members.contains(&validator_index)) + }); duties.push(AttesterDuty { pubkey: validator.pubkey, validator_index, @@ -723,6 +770,7 @@ fn attester_duties( committees_at_slot, validator_committee_index: position as u64, slot, + is_aggregator, }); } } @@ -815,7 +863,8 @@ fn attestation_data(store: &Store, slot: Slot) -> Result= ForkName::Gloas { payload_present_index(store, &state, slot)? } else { 0 @@ -851,7 +900,8 @@ fn payload_present_index( if head_state.slot() == slot { return Ok(0); } - if !matches!(head_state, BeaconState::Gloas(_)) { + // A head of a fork after gloas has a payload status too. + if !head_state.fork_name().has_payload_envelopes() { return Ok(1); } let status = match store.head_payload_status() { diff --git a/crates/net/rpc/src/lib.rs b/crates/net/rpc/src/lib.rs index 12e1521e0..bb27e8a0a 100644 --- a/crates/net/rpc/src/lib.rs +++ b/crates/net/rpc/src/lib.rs @@ -309,6 +309,7 @@ pub async fn start_beacon_rpc_server( sync_status.clone(), )) }); + beacon::pq_aggregation::spawn(store.clone(), shutdown.clone()); let api_router = build_beacon_api_router(store, config.version, peer_id) .layer(Extension(sync_status)) .layer(Extension(handles.p2p)) @@ -535,6 +536,7 @@ pub(crate) mod test_utils { pub(crate) operations: std::sync::Mutex>, pub(crate) subscriptions: std::sync::Mutex>, + pub(crate) pq_subscriptions: std::sync::Mutex>, pub(crate) blocks: std::sync::Mutex>, pub(crate) sidecars: @@ -571,6 +573,13 @@ pub(crate) mod test_utils { } impl ethlambda_network_api::RpcToP2P for RecordingNetwork { + fn publish_pq_gossip( + &self, + _object: ethlambda_network_api::PqGossipObject, + ) -> Result<(), spawned_concurrency::error::ActorError> { + Ok(()) + } + fn publish_beacon_attestation( &self, subnet_id: u64, @@ -617,6 +626,14 @@ pub(crate) mod test_utils { Ok(()) } + fn subscribe_pq_attestation_subnets( + &self, + subnets: Vec<(u64, u64)>, + ) -> Result<(), spawned_concurrency::error::ActorError> { + self.pq_subscriptions.lock().unwrap().extend(subnets); + Ok(()) + } + fn publish_beacon_block( &self, block: ethlambda_types::beacon::containers::SignedBeaconBlock, @@ -629,12 +646,14 @@ pub(crate) mod test_utils { fn publish_execution_payload_envelope( &self, - envelope: Box< - ethlambda_types::beacon::containers::gloas::SignedExecutionPayloadEnvelope, - >, + envelope: Box, sidecars: Vec, ) -> Result<(), spawned_concurrency::error::ActorError> { - self.envelopes.lock().unwrap().push((*envelope, sidecars)); + let envelope = envelope + .as_gloas() + .expect("the rpc tests publish gloas envelopes") + .clone(); + self.envelopes.lock().unwrap().push((envelope, sidecars)); Ok(()) } diff --git a/crates/net/rpc/src/shared/optimistic.rs b/crates/net/rpc/src/shared/optimistic.rs index e7b127ad1..85f8da264 100644 --- a/crates/net/rpc/src/shared/optimistic.rs +++ b/crates/net/rpc/src/shared/optimistic.rs @@ -72,6 +72,9 @@ fn payload_link(store: &Store, root: H256) -> Option { }; Some(BlockPayloadLink::Gloas { parent_status }) } + // Not followed (`ForkName::is_followed`): no store holds one, and + // no payload link is recorded for one. + SignedBeaconBlock::Eip8483(_) | SignedBeaconBlock::Pq(_) => None, // A lean block is never in a beacon store. SignedBeaconBlock::Lean(_) => None, } diff --git a/crates/storage/src/committee_cache.rs b/crates/storage/src/committee_cache.rs index a2361b9c8..ec34724e6 100644 --- a/crates/storage/src/committee_cache.rs +++ b/crates/storage/src/committee_cache.rs @@ -53,6 +53,12 @@ use ethlambda_types::beacon::primitives::{Epoch, Root}; /// since. Epoch and decision root together are what makes a cross-state /// cache sound where `epoch` or `seed` alone would not be. /// +/// `max_committees_per_slot` is the third input, constant within a fork: the +/// PQ fork caps a slot's committees at `MAX_COMMITTEES_PER_SLOT_PQ` rather +/// than `MAX_COMMITTEES_PER_SLOT`, and recomputes even the previous epoch's +/// committees under its own cap, so a state either side of the fork boundary +/// names the same epoch and deciding block but a different shuffling. +/// /// Opaque to this module: `epoch` and `decision_root` are read only for /// equality and (for eviction) ordering by `epoch`. `ethlambda-state-transition`'s /// `beacon::helpers::accessors::shuffling_key` is what derives one from a @@ -62,6 +68,7 @@ use ethlambda_types::beacon::primitives::{Epoch, Root}; pub struct ShufflingKey { pub epoch: Epoch, pub decision_root: Root, + pub max_committees_per_slot: u64, } /// How many distinct shufflings stay resident in a [`CommitteeCache`]. @@ -375,6 +382,7 @@ mod tests { ShufflingKey { epoch, decision_root: Root::repeat_byte(decision_root), + max_committees_per_slot: 64, } } diff --git a/crates/storage/src/pools/mod.rs b/crates/storage/src/pools/mod.rs index 7255f21eb..9a97e0ae7 100644 --- a/crates/storage/src/pools/mod.rs +++ b/crates/storage/src/pools/mod.rs @@ -3,6 +3,10 @@ mod attestation; mod operation; +mod pq_aggregate; +mod pq_signature; pub use attestation::{AttestationPool, single_committee}; pub use operation::OperationPool; +pub use pq_aggregate::{PooledPqAggregate, PqAggregatePool}; +pub use pq_signature::{CommitteeSignatures, PqSignaturePool}; diff --git a/crates/storage/src/pools/pq_aggregate.rs b/crates/storage/src/pools/pq_aggregate.rs new file mode 100644 index 000000000..3e08bfa65 --- /dev/null +++ b/crates/storage/src/pools/pq_aggregate.rs @@ -0,0 +1,167 @@ +//! PQ network aggregates this node has validated, held until a block packs +//! them. +//! +//! A PQ attestation carries no signature: each network aggregate covers one +//! committee and travels with an `AttestationProof` of its attesters' +//! signatures, and a proposer merges the proofs of the aggregates it packs +//! into the block's single `proof` (PQ `validator.md`, "Attestations" and +//! "Proof"). A proof proves an exact set of signers, so two aggregates for +//! one committee cannot be combined the way BLS aggregates are. The pool +//! keeps the one covering the most attesters per `(AttestationData, +//! committee)` instead. +//! +//! Only aggregates that passed gossip validation go in, and each keeps the +//! attesting indices it was validated with, so a proposer on another branch +//! can tell whether the committee it read still names the same signers. In +//! memory only. + +use std::{ + collections::{BTreeMap, HashMap}, + sync::Arc, +}; + +use ethlambda_types::{ + beacon::{ + containers::pq::{Attestation, AttestationProof}, + preset, + primitives::{Root, Slot, ValidatorIndex}, + }, + primitives::HashTreeRoot as _, +}; + +/// How many slots of aggregates the pool holds below the newest one it has +/// seen: an attestation is includable through the end of the epoch after its +/// own, so two epochs cover every one a block could still pack. +const RETAINED_SLOTS: Slot = 2 * preset::SLOTS_PER_EPOCH; + +/// One validated network aggregate. +#[derive(Debug, Clone)] +pub struct PooledPqAggregate { + /// Covers exactly one committee. + pub attestation: Attestation, + pub committee_index: u64, + /// The attesters the aggregate was validated with, sorted. + pub attesting_indices: Vec, + /// The proof of their signatures over the attestation's message. + pub proof: AttestationProof, +} + +/// Validated PQ aggregates by slot, then by `(data root, committee)`. +#[derive(Debug, Default)] +pub struct PqAggregatePool { + by_slot: BTreeMap>>, +} + +impl PqAggregatePool { + /// Hold `aggregate` unless the pool already has one for its data and + /// committee covering at least as many attesters. Returns whether it was + /// kept. Drops everything more than [`RETAINED_SLOTS`] below the newest + /// slot held. + pub fn insert(&mut self, mut aggregate: PooledPqAggregate) -> bool { + aggregate.attesting_indices.sort_unstable(); + let slot = aggregate.attestation.data.slot; + let newest = self + .by_slot + .last_key_value() + .map_or(slot, |(&newest, _)| newest.max(slot)); + let floor = newest.saturating_sub(RETAINED_SLOTS); + if slot < floor { + return false; + } + self.by_slot = self.by_slot.split_off(&floor); + let key = ( + aggregate.attestation.data.hash_tree_root(), + aggregate.committee_index, + ); + let held = self.by_slot.entry(slot).or_default(); + let better = held.get(&key).is_none_or(|current| { + aggregate.attesting_indices.len() > current.attesting_indices.len() + }); + if better { + held.insert(key, Arc::new(aggregate)); + } + better + } + + /// Every aggregate held for a slot at or below `latest_slot`. + pub fn candidates(&self, latest_slot: Slot) -> Vec> { + self.by_slot + .range(..=latest_slot) + .flat_map(|(_, held)| held.values().cloned()) + .collect() + } + + /// How many aggregates the pool holds. + pub fn len(&self) -> usize { + self.by_slot.values().map(HashMap::len).sum() + } + + /// Whether the pool holds nothing. + pub fn is_empty(&self) -> bool { + self.by_slot.is_empty() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use ethlambda_types::beacon::containers::{pq::AggregationBits, shared::AttestationData}; + + fn aggregate(slot: Slot, committee: u64, attesters: &[ValidatorIndex]) -> PooledPqAggregate { + PooledPqAggregate { + attestation: Attestation { + aggregation_bits: AggregationBits::default(), + data: AttestationData { + slot, + ..Default::default() + }, + committee_bits: Default::default(), + }, + committee_index: committee, + attesting_indices: attesters.to_vec(), + proof: AttestationProof::default(), + } + } + + #[test] + fn the_aggregate_covering_more_attesters_wins_its_committee() { + let mut pool = PqAggregatePool::default(); + assert!(pool.insert(aggregate(5, 0, &[3]))); + assert!(pool.insert(aggregate(5, 0, &[3, 1]))); + assert!(!pool.insert(aggregate(5, 0, &[2]))); + assert!(!pool.insert(aggregate(5, 0, &[4, 2]))); + // Another committee of the same data is kept beside it. + assert!(pool.insert(aggregate(5, 1, &[9]))); + + let held = pool.candidates(5); + assert_eq!(held.len(), 2); + let committee_zero = held.iter().find(|held| held.committee_index == 0).unwrap(); + assert_eq!(committee_zero.attesting_indices, vec![1, 3]); + } + + #[test] + fn candidates_stop_at_the_latest_slot() { + let mut pool = PqAggregatePool::default(); + pool.insert(aggregate(3, 0, &[1])); + pool.insert(aggregate(4, 0, &[2])); + pool.insert(aggregate(6, 0, &[3])); + let slots: Vec = pool + .candidates(4) + .iter() + .map(|held| held.attestation.data.slot) + .collect(); + assert_eq!(slots.len(), 2); + assert!(slots.iter().all(|&slot| slot <= 4)); + } + + #[test] + fn slots_two_epochs_below_the_newest_are_dropped() { + let mut pool = PqAggregatePool::default(); + pool.insert(aggregate(1, 0, &[1])); + pool.insert(aggregate(1 + RETAINED_SLOTS, 0, &[2])); + assert_eq!(pool.len(), 2); + pool.insert(aggregate(2 + RETAINED_SLOTS, 0, &[3])); + assert_eq!(pool.len(), 2); + assert!(!pool.insert(aggregate(1, 0, &[4, 5]))); + } +} diff --git a/crates/storage/src/pools/pq_signature.rs b/crates/storage/src/pools/pq_signature.rs new file mode 100644 index 000000000..7185d98e1 --- /dev/null +++ b/crates/storage/src/pools/pq_signature.rs @@ -0,0 +1,230 @@ +//! The XMSS signatures this node's PQ aggregators prove aggregates from. +//! +//! A PQ aggregator is a member of its committee's aggregation committee +//! (PQ `validator.md`, "Aggregation selection") and proves its committee's +//! signatures with an `AttestationProof` instead of aggregating them. The +//! validator client names the committees it aggregates for through +//! `beacon_committee_subscriptions`; this pool keeps the signatures of +//! validated single attestations for those committees only, seals each slot's +//! once its collection window closes, and holds what the node proved from +//! them for the aggregate endpoint to answer with. +//! +//! In memory only, and a few slots deep: an aggregate is published within its +//! own slot. + +use std::collections::{BTreeMap, BTreeSet}; +use std::sync::Arc; + +use ethlambda_types::{ + beacon::{ + containers::{pq::SingleAttestation, shared::AttestationData}, + preset, + primitives::{Root, Slot, ValidatorIndex, XmssSignature}, + }, + primitives::HashTreeRoot as _, +}; + +use super::PooledPqAggregate; + +/// How many slots below the current one the pool keeps anything for. +const RETAINED_SLOTS: Slot = 3; + +/// How many distinct `(AttestationData, committee)` groups one slot may hold: +/// four votes per committee leaves room for honest disagreement and bounds +/// what a peer can make the node store. +const MAX_GROUPS_PER_SLOT: usize = preset::MAX_COMMITTEES_PER_SLOT_PQ * 4; + +/// `(slot, data root, committee index)`. +type Key = (Slot, Root, u64); + +/// One committee's collected votes on one `AttestationData`. +#[derive(Debug, Clone)] +pub struct CommitteeSignatures { + pub data: AttestationData, + pub committee_index: u64, + /// By attester. Positions in the committee are resolved when proving, + /// against the state the proof is made for. + pub signatures: BTreeMap, +} + +/// Signatures awaiting proof and the aggregates proved from them. +#[derive(Debug, Default)] +pub struct PqSignaturePool { + /// `(slot, committee)` this node's validators aggregate for. + subscriptions: BTreeSet<(Slot, u64)>, + raw: BTreeMap, + proven: BTreeMap>, + /// The latest slot whose signatures were handed to the prover: later + /// signatures for it are not collected. + sealed_slot: Option, +} + +impl PqSignaturePool { + fn prune(&mut self, now: Slot) { + let earliest = now.saturating_sub(RETAINED_SLOTS - 1); + self.raw.retain(|(slot, _, _), _| *slot >= earliest); + self.proven.retain(|(slot, _, _), _| *slot >= earliest); + self.subscriptions.retain(|(slot, _)| *slot >= now); + } + + /// Collect `committee`'s signatures at `slot`, which one of this node's + /// validators aggregates for. Refused for a past slot, one more than two + /// epochs ahead, or a committee index PQ cannot have. + pub fn subscribe(&mut self, now: Slot, slot: Slot, committee: u64) -> bool { + self.prune(now); + let accepted = slot >= now + && slot <= now.saturating_add(2 * preset::SLOTS_PER_EPOCH) + && committee < preset::MAX_COMMITTEES_PER_SLOT_PQ as u64; + if accepted { + self.subscriptions.insert((slot, committee)); + } + accepted + } + + /// Whether this node collects `committee`'s signatures at `slot`. + pub fn is_subscribed(&self, slot: Slot, committee: u64) -> bool { + self.subscriptions.contains(&(slot, committee)) + } + + /// Keep a validated attestation's signature, if its committee is one this + /// node aggregates for and its slot is neither past nor sealed. The first + /// signature per attester wins. Returns whether it was kept. + pub fn insert(&mut self, now: Slot, attestation: &SingleAttestation) -> bool { + self.prune(now); + let slot = attestation.data.slot; + let committee = attestation.committee_index; + if slot < now + || self.sealed_slot.is_some_and(|sealed| slot <= sealed) + || !self.subscriptions.contains(&(slot, committee)) + { + return false; + } + let key = (slot, attestation.data.hash_tree_root(), committee); + if !self.raw.contains_key(&key) { + let groups = self + .raw + .range((slot, Root::ZERO, 0)..) + .take_while(|((held, _, _), _)| *held == slot) + .count(); + if groups >= MAX_GROUPS_PER_SLOT { + return false; + } + } + let group = self.raw.entry(key).or_insert_with(|| CommitteeSignatures { + data: attestation.data, + committee_index: committee, + signatures: BTreeMap::new(), + }); + let mut kept = false; + group + .signatures + .entry(attestation.attester_index) + .or_insert_with(|| { + kept = true; + attestation.signature.clone() + }); + kept + } + + /// Hand `slot`'s collected signatures to the prover, at most once per + /// slot. Signatures arriving for it afterwards are not collected. + pub fn seal(&mut self, slot: Slot) -> Vec { + self.prune(slot); + if self.sealed_slot.is_some_and(|sealed| slot <= sealed) { + return Vec::new(); + } + self.sealed_slot = Some(slot); + let later = self.raw.split_off(&(slot + 1, Root::ZERO, 0)); + std::mem::replace(&mut self.raw, later) + .into_iter() + .filter(|((held, _, _), _)| *held == slot) + .map(|(_, group)| group) + .collect() + } + + /// Hold an aggregate this node proved, for the aggregate endpoint. + pub fn insert_proven(&mut self, aggregate: Arc) { + let data = &aggregate.attestation.data; + let key = (data.slot, data.hash_tree_root(), aggregate.committee_index); + self.proven.insert(key, aggregate); + } + + /// The aggregate this node proved for `committee`'s vote on the data + /// whose root is `data_root` at `slot`. + pub fn proven( + &self, + slot: Slot, + data_root: Root, + committee: u64, + ) -> Option> { + self.proven.get(&(slot, data_root, committee)).cloned() + } + + /// Whether `slot`'s signatures were already handed to the prover. + pub fn is_sealed(&self, slot: Slot) -> bool { + self.sealed_slot.is_some_and(|sealed| slot <= sealed) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn vote(slot: Slot, committee: u64, attester: ValidatorIndex, root: u8) -> SingleAttestation { + SingleAttestation { + committee_index: committee, + attester_index: attester, + data: AttestationData { + slot, + beacon_block_root: Root::repeat_byte(root), + ..Default::default() + }, + signature: XmssSignature::default(), + } + } + + #[test] + fn signatures_need_a_subscription_and_a_slot_seals_once() { + let mut pool = PqSignaturePool::default(); + assert!(!pool.insert(3, &vote(3, 0, 7, 1))); + assert!(pool.subscribe(3, 3, 0)); + assert!(pool.insert(3, &vote(3, 0, 7, 1))); + // The first signature per attester wins. + assert!(!pool.insert(3, &vote(3, 0, 7, 1))); + assert!(pool.insert(3, &vote(3, 0, 8, 1))); + // Another committee of the slot is not subscribed. + assert!(!pool.insert(3, &vote(3, 1, 9, 1))); + + let sealed = pool.seal(3); + let [group] = sealed.as_slice() else { + panic!("one group to prove"); + }; + assert_eq!( + group.signatures.keys().copied().collect::>(), + vec![7, 8] + ); + // Nothing more is collected for a sealed slot, nor sealed twice. + assert!(!pool.insert(3, &vote(3, 0, 10, 1))); + assert!(pool.seal(3).is_empty()); + } + + #[test] + fn a_slot_holds_a_bounded_number_of_groups() { + let mut pool = PqSignaturePool::default(); + pool.subscribe(3, 3, 0); + for root in 0..=MAX_GROUPS_PER_SLOT as u8 { + pool.insert(3, &vote(3, 0, root as u64, root)); + } + assert_eq!(pool.seal(3).len(), MAX_GROUPS_PER_SLOT); + } + + #[test] + fn subscriptions_are_bounded_in_time_and_committee() { + let mut pool = PqSignaturePool::default(); + assert!(!pool.subscribe(10, 9, 0)); + assert!(!pool.subscribe(10, 10 + 2 * preset::SLOTS_PER_EPOCH + 1, 0)); + assert!(!pool.subscribe(10, 10, preset::MAX_COMMITTEES_PER_SLOT_PQ as u64)); + assert!(pool.subscribe(10, 12, 1)); + assert!(pool.is_subscribed(12, 1)); + } +} diff --git a/crates/storage/src/store.rs b/crates/storage/src/store.rs index f3d79299c..e003c9bc3 100644 --- a/crates/storage/src/store.rs +++ b/crates/storage/src/store.rs @@ -11,7 +11,7 @@ use crate::api::{StorageBackend, StorageReadView, StorageReadViewExt, StorageWri use crate::committee_cache::CommitteeCache; use crate::error::Error; use crate::liveness::ObservedLiveness; -use crate::pools::{AttestationPool, OperationPool}; +use crate::pools::{AttestationPool, OperationPool, PqAggregatePool, PqSignaturePool}; use crate::proto_array::{ProtoArray, ProtoArrayError, RebuildReason, WeightRules}; use crate::pubkey_index_cache::PubkeyIndexCache; use crate::target_balance_cache::TargetBalanceCache; @@ -28,7 +28,7 @@ use ethlambda_types::{ constants::NUM_BLOCK_TIMELINESS_DEADLINES, containers::{ BeaconState, Checkpoint as BeaconCheckpoint, DataColumnSidecar, SignedBeaconBlock, - gloas, + SignedExecutionPayloadEnvelope, StorableEnvelope, }, fork::ForkName, fork_choice::{ @@ -222,7 +222,13 @@ const KEY_STATES_PRUNED_BELOW: &[u8] = b"states_pruned_below"; /// [`Store::prune_finalized_states`]). A directory written by the previous /// version holds rows no lookup can find, and there is no migration, by the /// same policy every previous change followed. -pub const DB_VERSION: u64 = 7; +/// +/// 8 widened `Config` with the EIP-8483 and PQ fork versions and epochs, +/// `UNREGISTERED_VALIDATOR_WITHDRAWABILITY_DELAY` and +/// `ATTESTATION_SUBNET_COUNT_PQ`, so a directory written by +/// version 7 decodes into the wrong fields, the same consequence as 3, 4 and +/// 6. +pub const DB_VERSION: u64 = 8; /// The consensus protocol a data directory holds. /// @@ -1085,6 +1091,24 @@ fn builds_on_payload(block: &SignedBeaconBlock, payload_block_hash: &ExecutionBl .parent_block_hash == *payload_block_hash } + SignedBeaconBlock::Eip8483(block) => { + block + .message + .body + .signed_execution_payload_bid + .message + .parent_block_hash + == *payload_block_hash + } + SignedBeaconBlock::Pq(block) => { + block + .message + .body + .signed_execution_payload_bid + .message + .parent_block_hash + == *payload_block_hash + } // A pre-gloas block cannot follow a gloas one. SignedBeaconBlock::Phase0(_) | SignedBeaconBlock::Altair(_) @@ -1231,6 +1255,10 @@ pub struct Store { attestation_pool: Arc>, /// Beacon operation pool. See [`Store::operation_pool`]. operation_pool: Arc>, + /// PQ network aggregates. See [`Store::pq_aggregate_pool`]. + pq_aggregate_pool: Arc>, + /// PQ aggregators' signatures. See [`Store::pq_signature_pool`]. + pq_signature_pool: Arc>, /// LRU memoization of states by block root, shared across `Store` clones. /// /// Holds the same fork-ladder enum the `States` table stores, so a lean @@ -1940,6 +1968,8 @@ impl Store { ))), attestation_pool: Default::default(), operation_pool: Default::default(), + pq_aggregate_pool: Default::default(), + pq_signature_pool: Default::default(), state_cache, pending_states, state_rebuilds, @@ -1974,6 +2004,23 @@ impl Store { .expect("operation pool lock poisoned") } + /// The PQ network aggregates block production packs. Same sharing and + /// locking rules as [`Store::attestation_pool`]. + pub fn pq_aggregate_pool(&self) -> MutexGuard<'_, PqAggregatePool> { + self.pq_aggregate_pool + .lock() + .expect("PQ aggregate pool lock poisoned") + } + + /// The signatures this node's PQ aggregators prove aggregates from, and + /// what it proved. Same sharing and locking rules as + /// [`Store::attestation_pool`]. + pub fn pq_signature_pool(&self) -> MutexGuard<'_, PqSignaturePool> { + self.pq_signature_pool + .lock() + .expect("PQ signature pool lock poisoned") + } + // ============ Metadata Helpers ============ /// Reads an SSZ metadata value that the store's bootstrap path guarantees @@ -4051,12 +4098,8 @@ impl Store { /// `slot` is the slot of the block the envelope fulfills, which the /// caller has already read while verifying the envelope against it; the /// envelope carries none and the row is keyed by it. - pub fn insert_verified_payload( - &mut self, - slot: u64, - envelope: &gloas::SignedExecutionPayloadEnvelope, - ) { - let root = envelope.message.beacon_block_root; + pub fn insert_verified_payload(&mut self, slot: u64, envelope: &impl StorableEnvelope) { + let root = envelope.beacon_block_root(); debug_assert_eq!( beacon_block_slot(self.backend.as_ref(), &root), Some(slot), @@ -4107,11 +4150,24 @@ impl Store { .collect() } + /// The fork of an envelope stored at `slot`: the fork of the block it + /// reveals the payload of. Gloas's when the configuration schedules no + /// fork with envelopes there, which only a store fed gloas envelopes under + /// another schedule (a test) holds. + fn envelope_fork(&self, slot: u64) -> ForkName { + let fork = self.config.fork_at_epoch(slot / SLOTS_PER_EPOCH); + if fork.has_payload_envelopes() { + fork + } else { + ForkName::Gloas + } + } + /// Gloas: the verified envelope stored for `root`, if any. pub fn get_execution_payload_envelope( &self, root: &H256, - ) -> Result, Error> { + ) -> Result, Error> { let Some(slot) = beacon_block_slot(self.backend.as_ref(), root) else { return Ok(None); }; @@ -4123,7 +4179,7 @@ impl Store { ) .expect("get"); Ok(bytes.map(|bytes| { - gloas::SignedExecutionPayloadEnvelope::from_ssz_bytes(&bytes) + SignedExecutionPayloadEnvelope::from_ssz(self.envelope_fork(slot), &bytes) .expect("a stored envelope decodes") })) } @@ -4149,7 +4205,7 @@ impl Store { &self, start_slot: u64, end_slot: u64, - ) -> Result, Error> { + ) -> Result, Error> { if self.chain != Chain::Beacon { Self::lean_only("canonical_execution_payload_envelopes"); } @@ -4203,8 +4259,9 @@ impl Store { let Some(bytes) = envelope_row(slot, &root) else { continue; }; - let envelope = gloas::SignedExecutionPayloadEnvelope::from_ssz_bytes(&bytes) - .expect("a stored envelope decodes"); + let envelope = + SignedExecutionPayloadEnvelope::from_ssz(self.envelope_fork(slot), &bytes) + .expect("a stored envelope decodes"); let builds_on_it = match window.get(index + 1) { // No successor: this is the head, whose envelope is part of // the chain only on the FULL node. @@ -4215,9 +4272,7 @@ impl Store { .expect("get") .map(|bytes| decode_beacon_block_value(&bytes)); match next { - Some(next) => { - builds_on_payload(&next, &envelope.message.payload.block_hash) - } + Some(next) => builds_on_payload(&next, &envelope.payload().block_hash), None => false, } } @@ -4297,7 +4352,7 @@ impl Store { .and_then(|bytes| bytes.first().copied()) .and_then(ForkName::from_selector); let is_gloas = match fork { - Some(ForkName::Gloas) => true, + Some(ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq) => true, Some( ForkName::Phase0 | ForkName::Altair @@ -5578,6 +5633,7 @@ mod tests { use crate::committee_cache::{Lookup, ShufflingKey}; use ethlambda_types::beacon::committees::EpochCommittees; use ethlambda_types::beacon::containers::Checkpoint as BeaconCheckpoint; + use ethlambda_types::beacon::containers::gloas; // Only the tests name a status variant: the store itself stores and hands // back whole `PayloadStatusV1` values without ever reading the tag. use ethlambda_types::beacon::fork_choice::PayloadStatusEnum; @@ -7581,6 +7637,7 @@ mod tests { let key = ShufflingKey { epoch: 1, decision_root: H256::from([1u8; 32]), + max_committees_per_slot: 64, }; let (first, first_lookup) = clone @@ -9298,7 +9355,7 @@ mod tests { reopened .get_execution_payload_envelope(&root) .expect("read"), - Some(envelope) + Some(envelope.into()) ); } @@ -10584,7 +10641,7 @@ mod tests { /// slot a full decode yields, for every fork shape the table can hold. #[test] fn peeking_a_beacon_block_slot_equals_decoding_it() { - use ethlambda_types::beacon::containers::{altair, electra, gloas}; + use ethlambda_types::beacon::containers::{altair, eip8483, electra, gloas, pq}; let altair_block = SignedBeaconBlock::Altair(altair::SignedBeaconBlock { message: altair::BeaconBlock { @@ -10622,10 +10679,32 @@ mod tests { }, signature: Default::default(), }), + SignedBeaconBlock::Eip8483(eip8483::SignedBeaconBlock { + message: eip8483::BeaconBlock { + slot: 0x0e0f, + proposer_index: 6, + parent_root: root(8), + state_root: root(9), + body: eip8483::BeaconBlockBody::empty(), + }, + signature: Default::default(), + }), + // An XMSS signature widens the fixed part ahead of `message`'s + // offset, which the peek reads rather than assumes. + SignedBeaconBlock::Pq(Box::new(pq::SignedBeaconBlock { + message: pq::BeaconBlock { + slot: 0x1011_1213, + proposer_index: 7, + parent_root: root(10), + state_root: root(11), + body: pq::BeaconBlockBody::empty(), + }, + signature: Default::default(), + })), ]; // The fork tag is part of the value; the newest fork in `ALL` is among // the blocks above. - assert_eq!(blocks[5].fork_name(), *ForkName::ALL.last().unwrap()); + assert_eq!(blocks[7].fork_name(), *ForkName::ALL.last().unwrap()); for block in blocks { let value = encode_beacon_block_value(&block); assert_eq!( diff --git a/crates/validator/Cargo.toml b/crates/validator/Cargo.toml index 403e7f1f4..b9c5def53 100644 --- a/crates/validator/Cargo.toml +++ b/crates/validator/Cargo.toml @@ -12,6 +12,8 @@ version.workspace = true [dependencies] ethlambda-types.workspace = true ethlambda-metrics.workspace = true +# leanXMSS signing and VRF proofs for the PQ fork (`pq`). +ethlambda-crypto.workspace = true # A produced block is decoded and re-published as SSZ, not JSON: signing it # needs its `hash_tree_root`, which only the typed container can give. See diff --git a/crates/validator/src/aggregation.rs b/crates/validator/src/aggregation.rs index 3a0a32dcd..3d11a5e66 100644 --- a/crates/validator/src/aggregation.rs +++ b/crates/validator/src/aggregation.rs @@ -63,6 +63,14 @@ pub struct AggregationService { } impl AggregationService { + /// The epoch the PQ fork activates at, from which this service does not + /// aggregate (`FAR_FUTURE_EPOCH` when unscheduled). + pub fn pq_fork_epoch(&self) -> u64 { + self.context + .config + .fork_epoch(ethlambda_types::beacon::fork::ForkName::Pq) + } + pub fn new(beacon_node: Arc, context: Arc) -> Self { Self { beacon_node, @@ -79,6 +87,89 @@ impl AggregationService { /// Returns how many aggregates were published, which is zero whenever none /// of this client's validators was selected. That is the ordinary case: a /// validator aggregates a few times a day. + /// PQ's aggregation: each of this client's validators the node names an + /// aggregator (its seat in the committee's aggregation committee) wraps + /// the aggregate the node proved for its committee and signs the wrapper + /// with leanXMSS at the slot's aggregate leaf (PQ `validator.md`, + /// "Broadcast aggregate"). There is no selection proof, and the proof of + /// the attesters' signatures is the node's: it collects them and proves + /// ahead, since proving takes seconds. + pub async fn aggregate_pq( + &self, + slot: Slot, + data: &AttestationData, + duties: &[AttesterDutyDto], + ) -> Result { + use ethlambda_types::beacon::constants::LEAF_OFFSET_AGGREGATE_AND_PROOF; + use ethlambda_types::beacon::containers::pq; + + let selected: Vec<&AttesterDutyDto> = duties + .iter() + .filter(|duty| duty.is_aggregator == Some(true)) + .collect(); + if selected.is_empty() { + return Ok(0); + } + // One fetch per committee, as for the other forks. + let data_root = data.hash_tree_root(); + let mut aggregates = BTreeMap::new(); + for duty in &selected { + if aggregates.contains_key(&duty.committee_index) { + continue; + } + match self + .beacon_node + .pq_aggregate(slot, data_root, duty.committee_index) + .await + { + Ok(aggregate) => { + aggregates.insert(duty.committee_index, aggregate); + } + Err(err) => warn!( + %slot, + committee = duty.committee_index, + %err, + "No PQ aggregate available for this committee" + ), + } + } + + let mut signed = Vec::new(); + for duty in &selected { + let Some(proven) = aggregates.get(&duty.committee_index) else { + continue; + }; + let pubkey = match parse_pubkey(&duty.pubkey) { + Ok(pubkey) => pubkey, + Err(err) => { + warn!(%slot, validator = duty.validator_index, %err, "Duty carried an unreadable pubkey"); + continue; + } + }; + let message = pq::AggregateAndProof { + aggregator_index: duty.validator_index, + aggregate: proven.aggregate.clone(), + proof: proven.proof.clone(), + }; + let root = self + .context + .aggregate_and_proof_signing_root(message.hash_tree_root(), slot); + let leaf = crate::pq::slot_leaf(slot, LEAF_OFFSET_AGGREGATE_AND_PROOF); + match crate::pq::sign(&pubkey, leaf, root) { + Ok(signature) => signed.push(pq::SignedAggregateAndProof { message, signature }), + Err(err) => { + warn!(%slot, validator = duty.validator_index, %err, "Failed to sign a PQ aggregate"); + } + } + } + if signed.is_empty() { + return Ok(0); + } + self.beacon_node.publish_pq_aggregates(&signed).await?; + info!(%slot, count = signed.len(), "Published PQ aggregates"); + Ok(signed.len()) + } + pub async fn aggregate( &self, slot: Slot, @@ -327,6 +418,7 @@ mod tests { committees_at_slot: 64, validator_committee_index: 7, slot, + is_aggregator: None, } } diff --git a/crates/validator/src/attestation.rs b/crates/validator/src/attestation.rs index a857db8b2..b29a88f04 100644 --- a/crates/validator/src/attestation.rs +++ b/crates/validator/src/attestation.rs @@ -147,6 +147,8 @@ impl AttestationService { .as_str(); let data_dto = AttestationDataOutDto::from(&data); + let is_pq = self.context.config.fork_at_epoch(data.target.epoch) + == ethlambda_types::beacon::fork::ForkName::Pq; let candidates: Vec<(&AttesterDutyDto, BlsPubkey)> = duties .iter() @@ -245,7 +247,22 @@ impl AttestationService { } // One validator failing must not cost the others their attestation. - let signature = match self.context.sign_attestation(&store, &pubkey, &data) { + // From PQ a leanXMSS signature at the target epoch's leaf, over + // the same signing root: the slashing protection record above + // already guards that leaf (one per target epoch). + let signature = if is_pq { + let root = self.context.attestation_signing_root(&data); + let leaf = crate::pq::attestation_leaf(data.target.epoch); + crate::pq::sign(&pubkey, leaf, root) + .map(|signature| encode_hex(&signature.0)) + .map_err(|err| err.to_string()) + } else { + self.context + .sign_attestation(&store, &pubkey, &data) + .map(|signature| encode_hex(&signature.0)) + .map_err(|err| err.to_string()) + }; + let signature = match signature { Ok(signature) => signature, Err(err) => { error!(%slot, validator = duty.validator_index, %err, "Failed to sign attestation"); @@ -258,7 +275,7 @@ impl AttestationService { committee_index: duty.committee_index, attester_index: duty.validator_index, data: data_dto.clone(), - signature: encode_hex(&signature.0), + signature, }); } } @@ -351,6 +368,7 @@ mod tests { committees_at_slot: 64, validator_committee_index: 7, slot, + is_aggregator: None, } } diff --git a/crates/validator/src/beacon_node/block_contents.rs b/crates/validator/src/beacon_node/block_contents.rs index f39d349a5..09cfdffc0 100644 --- a/crates/validator/src/beacon_node/block_contents.rs +++ b/crates/validator/src/beacon_node/block_contents.rs @@ -359,6 +359,14 @@ impl ProducedBlock { "a gloas block cannot be decoded without Eth-Execution-Payload-Included" .to_string(), )), + // The consensus-specs feature forks after gloas: the PQ fork signs + // with leanXMSS, which this client has no keys for, and EIP-8483 + // precedes it only to register them. + ForkName::Eip8483 | ForkName::Pq => Err(Error::InconsistentResponse(format!( + "beacon node produced a block for fork {}, which this client does not support \ + (it signs with BLS only)", + fork.as_str() + ))), ForkName::Phase0 | ForkName::Altair | ForkName::Bellatrix @@ -398,6 +406,8 @@ impl ProducedBlock { | ForkName::Capella | ForkName::Deneb | ForkName::Gloas + | ForkName::Eip8483 + | ForkName::Pq | ForkName::Lean => Err(Error::InconsistentResponse(format!( "beacon node produced a blinded block for fork {}; blinded blocks exist for \ electra and fulu only", diff --git a/crates/validator/src/beacon_node/dto.rs b/crates/validator/src/beacon_node/dto.rs index 6a55aa378..29578ea4e 100644 --- a/crates/validator/src/beacon_node/dto.rs +++ b/crates/validator/src/beacon_node/dto.rs @@ -212,6 +212,11 @@ pub struct AttesterDutyDto { pub validator_committee_index: u64, #[serde(with = "quoted_u64")] pub slot: Slot, + /// At a PQ slot, whether the node counts the validator on its committee's + /// aggregation committee, which is what makes a PQ aggregator; absent + /// before PQ, where aggregators select themselves with a BLS proof. + #[serde(default)] + pub is_aggregator: Option, } /// One proposer duty, from `GET /eth/v1/validator/duties/proposer/{epoch}`. @@ -459,6 +464,10 @@ pub fn parse_signature(text: &str) -> Result { Ok(BlsSignature(array)) } +/// A 48-byte pubkey from the beacon node. On the PQ fork the registry names a +/// validator by its XMSS key in request form (16 zero bytes, then the key); +/// one of this client's own is answered as the BLS key that owns it +/// (`crate::pq`), so every duty still matches a local key by BLS key. pub fn parse_pubkey(text: &str) -> Result { let bytes = hex::decode(text.strip_prefix("0x").unwrap_or(text)) .map_err(|err| Error::Decode(format!("bad pubkey hex: {err}")))?; @@ -468,7 +477,7 @@ pub fn parse_pubkey(text: &str) -> Result { got.len() )) })?; - Ok(BlsPubkey(array)) + Ok(crate::pq::bls_for_request_key(&array).unwrap_or(BlsPubkey(array))) } /// `0x`-prefixed hex, through the same adapter the beacon node writes its own diff --git a/crates/validator/src/beacon_node/fallback.rs b/crates/validator/src/beacon_node/fallback.rs index 24a199095..e6c103f42 100644 --- a/crates/validator/src/beacon_node/fallback.rs +++ b/crates/validator/src/beacon_node/fallback.rs @@ -597,6 +597,57 @@ impl BeaconNodeApi for FallbackBeaconNode { .await } + async fn pq_aggregate( + &self, + slot: Slot, + attestation_data_root: Root, + committee_index: u64, + ) -> Result { + self.try_each("pq_aggregate", |node| { + node.pq_aggregate(slot, attestation_data_root, committee_index) + }) + .await + } + + async fn publish_pq_aggregates( + &self, + aggregates: &[ethlambda_types::beacon::containers::pq::SignedAggregateAndProof], + ) -> Result<()> { + self.try_each("publish_pq_aggregates", |node| { + node.publish_pq_aggregates(aggregates) + }) + .await + } + + async fn submit_pq_payload_attestations( + &self, + messages: &[ethlambda_types::beacon::containers::pq::PayloadAttestationMessage], + ) -> Result { + self.try_each("submit_pq_payload_attestations", |node| { + node.submit_pq_payload_attestations(messages) + }) + .await + } + + async fn produce_pq_block( + &self, + slot: Slot, + randao_reveal: &[Root], + graffiti: Option, + ) -> Result { + self.try_each("produce_pq_block", |node| { + node.produce_pq_block(slot, randao_reveal, graffiti) + }) + .await + } + + async fn publish_pq_execution_payload_envelope(&self, body: &[u8]) -> Result<()> { + self.try_each("publish_pq_execution_payload_envelope", |node| { + node.publish_pq_execution_payload_envelope(body) + }) + .await + } + async fn submit_attestations( &self, attestations: &[SingleAttestationDto], diff --git a/crates/validator/src/beacon_node/http.rs b/crates/validator/src/beacon_node/http.rs index 9c405cac7..8be760e75 100644 --- a/crates/validator/src/beacon_node/http.rs +++ b/crates/validator/src/beacon_node/http.rs @@ -678,8 +678,17 @@ impl BeaconNodeApi for HttpBeaconNode { struct Body { ids: Vec, } + // Each key in both forms the registry may name it by: its BLS key, and + // on the PQ fork its XMSS key's request form (`crate::pq`). The answer + // names whichever the head state holds, which `parse_pubkey` maps back. let body = Body { - ids: pubkeys.iter().map(|key| encode_hex(&key.0)).collect(), + ids: pubkeys + .iter() + .flat_map(|key| { + std::iter::once(encode_hex(&key.0)) + .chain(crate::pq::request_key(key).map(|request| encode_hex(&request))) + }) + .collect(), }; let response: DataResponse> = self .post("/eth/v1/beacon/states/head/validators", &body, None) @@ -1068,6 +1077,116 @@ impl BeaconNodeApi for HttpBeaconNode { Self::handle_pool_submission(status, body, messages.len()) } + async fn submit_pq_payload_attestations( + &self, + messages: &[ethlambda_types::beacon::containers::pq::PayloadAttestationMessage], + ) -> Result { + let url = format!("{}/eth/v1/beacon/pool/payload_attestations", self.base_url); + let response = self + .client + .post(&url) + .header("Eth-Consensus-Version", ForkName::Pq.as_str()) + .json(messages) + .send() + .await + .map_err(|err| Error::BeaconNode { + url: url.clone(), + failure: BeaconNodeFailure::classify(&err), + detail: err.to_string(), + })?; + let status = response.status(); + if status.is_success() { + return Ok(messages.len()); + } + let body = read_error_body(response).await; + Self::handle_pool_submission(status, body, messages.len()) + } + + async fn produce_pq_block( + &self, + slot: Slot, + randao_reveal: &[Root], + graffiti: Option, + ) -> Result { + #[derive(Serialize)] + struct Body { + randao_reveal: Vec, + #[serde(skip_serializing_if = "Option::is_none")] + graffiti: Option, + } + let body = Body { + randao_reveal: randao_reveal + .iter() + .map(|node| encode_hex(&node.0)) + .collect(), + graffiti: graffiti.map(|graffiti| encode_hex(&graffiti.0)), + }; + let url = format!("{}/eth/v1/validator/pq_blocks/{slot}", self.base_url); + let response = self + .client + .post(&url) + .header(reqwest::header::ACCEPT, "application/octet-stream") + .json(&body) + .send() + .await + .map_err(|err| Error::BeaconNode { + url: url.clone(), + failure: BeaconNodeFailure::classify(&err), + detail: err.to_string(), + })?; + let status = response.status(); + if !status.is_success() { + let body = error_body(response).await; + return Err(Error::BeaconNodeStatus { + status: status.as_u16(), + body, + }); + } + let bytes = response.bytes().await.map_err(|err| Error::BeaconNode { + url: url.clone(), + failure: BeaconNodeFailure::classify(&err), + detail: err.to_string(), + })?; + let contents = + ::from_ssz_bytes(&bytes) + .map_err(|err| Error::Decode(format!("pq block contents: {err:?}")))?; + let block_root = + ethlambda_types::beacon::primitives::HashTreeRoot::hash_tree_root(&contents.block); + if contents.block.slot != slot + || contents.execution_payload_envelope.beacon_block_root != block_root + { + return Err(Error::InconsistentResponse(format!( + "asked for a PQ block at slot {slot}, node answered one at slot {} or an envelope for another block", + contents.block.slot + ))); + } + Ok(contents) + } + + async fn publish_pq_execution_payload_envelope(&self, body: &[u8]) -> Result<()> { + let (url, sent) = self + .post_ssz_body( + "/eth/v1/beacon/execution_payload_envelopes", + ForkName::Pq, + body, + ) + .await; + let response = sent.map_err(|err| Error::BeaconNode { + url: url.clone(), + failure: BeaconNodeFailure::classify(&err), + detail: err.to_string(), + })?; + let status = response.status(); + if status.is_success() { + return Ok(()); + } + let body = error_body(response).await; + Err(Error::BeaconNodeStatus { + status: status.as_u16(), + body, + }) + } + /// Distinct from `post_no_content`: this endpoint's 400s are not always /// total failures. See [`Self::handle_pool_submission`]. async fn submit_attestations( @@ -1153,6 +1272,13 @@ impl BeaconNodeApi for HttpBeaconNode { .map_err(|err| Error::Decode(format!("gloas aggregate: {err}")))?; AggregateKind::Gloas(attestation) } + // Not supported by this client: see `ProducedBlock::from_ssz`. + ForkName::Eip8483 | ForkName::Pq => { + return Err(Error::InconsistentResponse(format!( + "node produced a {} aggregate, which this client does not publish", + fork.as_str() + ))); + } }; // The same contract the other two fetches carry, enforced here so a // node answering about the wrong slot is failed over from rather than @@ -1171,6 +1297,46 @@ impl BeaconNodeApi for HttpBeaconNode { /// JSON, because Lighthouse answers an SSZ body here with 415. See the /// contract on [`BeaconNodeApi::publish_aggregates`]. + async fn pq_aggregate( + &self, + slot: Slot, + attestation_data_root: Root, + committee_index: u64, + ) -> Result { + let path = Self::aggregate_path(slot, attestation_data_root, committee_index); + let response: VersionedResponse = + self.get(&path).await?; + if !response.version.eq_ignore_ascii_case(ForkName::Pq.as_str()) { + return Err(Error::InconsistentResponse(format!( + "asked for a PQ aggregate, node answered a {} one", + response.version + ))); + } + let aggregate = response.data; + if aggregate.aggregate.data.slot != slot + || ethlambda_types::primitives::HashTreeRoot::hash_tree_root(&aggregate.aggregate.data) + != attestation_data_root + { + return Err(Error::InconsistentResponse( + "the PQ aggregate is for another vote".into(), + )); + } + Ok(aggregate) + } + + async fn publish_pq_aggregates( + &self, + aggregates: &[ethlambda_types::beacon::containers::pq::SignedAggregateAndProof], + ) -> Result<()> { + // The PQ container serialises as the endpoint's JSON itself. + self.post_no_content( + "/eth/v2/validator/aggregate_and_proofs", + &aggregates, + Some(ForkName::Pq.as_str()), + ) + .await + } + async fn publish_aggregates( &self, fork: ForkName, diff --git a/crates/validator/src/beacon_node/mock.rs b/crates/validator/src/beacon_node/mock.rs index 9bde4d283..048c0c3bc 100644 --- a/crates/validator/src/beacon_node/mock.rs +++ b/crates/validator/src/beacon_node/mock.rs @@ -1116,6 +1116,49 @@ impl BeaconNodeApi for MockBeaconNode { Ok(Some(data)) } + async fn pq_aggregate( + &self, + _slot: Slot, + _attestation_data_root: Root, + _committee_index: u64, + ) -> Result { + self.guard("pq_aggregate")?; + Err(Error::InconsistentResponse( + "the mock proves no PQ aggregates".into(), + )) + } + + async fn publish_pq_aggregates( + &self, + _aggregates: &[ethlambda_types::beacon::containers::pq::SignedAggregateAndProof], + ) -> Result<()> { + self.guard("publish_pq_aggregates") + } + + async fn submit_pq_payload_attestations( + &self, + messages: &[ethlambda_types::beacon::containers::pq::PayloadAttestationMessage], + ) -> Result { + self.guard("submit_pq_payload_attestations")?; + Ok(messages.len()) + } + + async fn produce_pq_block( + &self, + _slot: Slot, + _randao_reveal: &[Root], + _graffiti: Option, + ) -> Result { + self.guard("produce_pq_block")?; + Err(Error::InconsistentResponse( + "the mock produces no PQ blocks".into(), + )) + } + + async fn publish_pq_execution_payload_envelope(&self, _body: &[u8]) -> Result<()> { + self.guard("publish_pq_execution_payload_envelope") + } + async fn submit_payload_attestations( &self, messages: &[gloas::PayloadAttestationMessage], diff --git a/crates/validator/src/beacon_node/mod.rs b/crates/validator/src/beacon_node/mod.rs index 49e3b5890..78bf758fd 100644 --- a/crates/validator/src/beacon_node/mod.rs +++ b/crates/validator/src/beacon_node/mod.rs @@ -130,7 +130,8 @@ fn highest_attestation_index(fork: ForkName) -> u64 { // a block whose payload was seen. Which of the two is right depends // on the node's view of the block, which this client does not have, // so only the range is checked. - ForkName::Gloas => 1, + // EIP-8483 and PQ keep gloas's payload signal. + ForkName::Gloas | ForkName::Eip8483 | ForkName::Pq => 1, // Not a Beacon Chain fork: `Config::fork_at_epoch` never names it, so // a caller that does has a bug of its own. A panic rather than an // `Err`, because this runs after the node has answered: an `Err` @@ -672,6 +673,29 @@ pub trait BeaconNodeApi: Send + Sync { messages: &[gloas::PayloadAttestationMessage], ) -> Result; + /// The PQ fork's [`Self::submit_payload_attestations`]: leanXMSS-signed + /// messages, under the `pq` header. + async fn submit_pq_payload_attestations( + &self, + messages: &[ethlambda_types::beacon::containers::pq::PayloadAttestationMessage], + ) -> Result; + + /// The unsigned PQ block for `slot` and the unsigned envelope revealing + /// its self-built payload (ethlambda's `POST + /// /eth/v1/validator/pq_blocks/{slot}`), given the proposer's VRF proof + /// for the slot. + /// + /// # Contract: the block is for `slot`, and the envelope names it + async fn produce_pq_block( + &self, + slot: Slot, + randao_reveal: &[Root], + graffiti: Option, + ) -> Result; + + /// Publish a signed PQ envelope, given its SSZ body. + async fn publish_pq_execution_payload_envelope(&self, body: &[u8]) -> Result<()>; + /// Submits signed attestations to the node's pool, so they reach gossip. /// `fork_name` names the fork the attestations were produced under, since /// the endpoint requires it as a header rather than inferring it. @@ -721,6 +745,22 @@ pub trait BeaconNodeApi: Send + Sync { async fn publish_aggregates(&self, fork: ForkName, aggregates: &SignedAggregates) -> Result<()>; + /// The aggregate the node proved for `committee_index`'s vote on the data + /// whose root is `attestation_data_root` at a PQ slot, with its proof: what + /// a PQ aggregator wraps and signs. A 404 when it proved none. + async fn pq_aggregate( + &self, + slot: Slot, + attestation_data_root: Root, + committee_index: u64, + ) -> Result; + + /// Publish signed PQ aggregates. + async fn publish_pq_aggregates( + &self, + aggregates: &[ethlambda_types::beacon::containers::pq::SignedAggregateAndProof], + ) -> Result<()>; + /// Tells the node where to pay each validator's execution-layer block /// rewards, so it has somewhere to send them when it builds a payload. /// diff --git a/crates/validator/src/duties.rs b/crates/validator/src/duties.rs index b1073811a..fb8057199 100644 --- a/crates/validator/src/duties.rs +++ b/crates/validator/src/duties.rs @@ -433,6 +433,7 @@ mod tests { committees_at_slot: 64, validator_committee_index: 7, slot, + is_aggregator: None, } } diff --git a/crates/validator/src/lib.rs b/crates/validator/src/lib.rs index 0c10065f7..28277e99c 100644 --- a/crates/validator/src/lib.rs +++ b/crates/validator/src/lib.rs @@ -36,6 +36,7 @@ pub(crate) mod in_flight; pub mod keys; pub mod metrics; pub mod payload_attestation; +pub mod pq; pub mod preferences; pub mod proposal; pub mod proposer_settings; @@ -77,6 +78,7 @@ use crate::registration::RegistrationService; use crate::signing::SigningContext; use crate::slot_clock::SlotClock; use crate::sync_committee::SyncCommitteeService; +use ethlambda_types::beacon::fork::ForkName; /// Everything the client is configured with at startup. #[derive(Debug, Clone)] @@ -263,6 +265,48 @@ pub async fn run(config: ValidatorConfig) -> Result<()> { genesis_validators_root: genesis.genesis_validators_root, }); + // A network scheduling the PQ fork: derive every validator's leanXMSS key + // and VRF tree before any duty runs, since index lookup already needs the + // keys. About a second and a half a key, the keys in parallel. + let pq_epoch = context + .config + .fork_epoch(ethlambda_types::beacon::fork::ForkName::Pq); + if pq_epoch != ethlambda_types::beacon::constants::FAR_FUTURE_EPOCH { + let secrets: Vec<(BlsPubkey, [u8; 32])> = { + let store = store.read().await; + store + .pubkeys() + .into_iter() + .filter_map(|pubkey| match store.get(&pubkey)? { + keys::store::SigningMethod::LocalKeystore { secret_key } => { + Some((pubkey, secret_key.to_bytes())) + } + }) + .collect() + }; + let count = secrets.len(); + let current_slot = clock.now().unwrap_or(0); + let started = std::time::Instant::now(); + info!(count, pq_epoch, "Deriving the validators' PQ keys"); + tokio::task::spawn_blocking(move || { + std::thread::scope(|scope| { + for (pubkey, secret) in &secrets { + scope.spawn(move || pq::derive_and_register(*pubkey, secret, current_slot)); + } + }); + }) + .await + .map_err(|err| Error::Io { + path: "PQ key derivation".into(), + source: std::io::Error::other(err.to_string()), + })?; + info!( + count, + elapsed_ms = started.elapsed().as_millis() as u64, + "Derived the validators' PQ keys" + ); + } + let mut duties = DutiesService::new(beacon_node.clone(), Vec::new()); let attestation = AttestationService::new(beacon_node.clone(), context.clone(), protection.clone()); @@ -446,7 +490,11 @@ pub async fn run(config: ValidatorConfig) -> Result<()> { // should not compete with it producing this slot's block. Off the slot // loop, since it is single-flight and a slow node must not delay this // slot's duties. - tokio::spawn(preferences.clone().poll(slot, store.clone())); + // PQ preferences are signed with leanXMSS, which this client's + // preferences service does not do yet: a PQ proposal self-builds. + if clock_fork(&context, slot) != ForkName::Pq { + tokio::spawn(preferences.clone().poll(slot, store.clone())); + } // The rest of the way to the first of the slot's remaining duties: // the attester offset, or the sync committee offset when the network @@ -800,8 +848,14 @@ async fn aggregate( store: &RwLock, ) { let budget = clock.remaining_in(slot, SystemTime::now()); - let attempt = tokio::time::timeout(budget, aggregation.aggregate(slot, data, duties, store)); - match attempt.await { + // PQ aggregators are the aggregation committee and publish what the node + // proved; see `AggregationService::aggregate_pq`. + let outcome = if data.target.epoch >= aggregation.pq_fork_epoch() { + tokio::time::timeout(budget, aggregation.aggregate_pq(slot, data, duties)).await + } else { + tokio::time::timeout(budget, aggregation.aggregate(slot, data, duties, store)).await + }; + match outcome { Ok(Ok(_)) => {} Ok(Err(err)) => { error!(%slot, %err, "Failed to publish this slot's aggregates"); @@ -975,6 +1029,8 @@ async fn refresh_epoch( refresh_ptc(duties, epoch, context).await; refresh_sync(beacon_node, duties, epoch, context).await; metrics::set_duties_held(duties.all().len() as u64); + // At a PQ slot a duty's aggregator role is the node's `is_aggregator` + // rather than a BLS selection proof; `subscribe` reads either. if changed { subscriptions::subscribe(beacon_node, &duties.all(), store, context).await?; } @@ -1050,7 +1106,9 @@ async fn refresh_sync( use ethlambda_types::beacon::fork::ForkName; use ethlambda_types::beacon::preset::EPOCHS_PER_SYNC_COMMITTEE_PERIOD as PERIOD_EPOCHS; - if context.config.fork_at_epoch(epoch) < ForkName::Altair { + // Sync committees start at altair, and PQ removes them. + let fork = context.config.fork_at_epoch(epoch); + if fork < ForkName::Altair || fork == ForkName::Pq { return; } let period = crate::duties::sync_committee_period(epoch); @@ -1082,6 +1140,15 @@ async fn refresh_sync( } } +/// The fork of `slot` by the client's own schedule. +fn clock_fork(context: &SigningContext, slot: u64) -> ForkName { + context + .config + .fork_at_epoch(ethlambda_types::beacon::signing::compute_epoch_at_slot( + slot, + )) +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/validator/src/payload_attestation.rs b/crates/validator/src/payload_attestation.rs index 0870c038a..7dc9ac3bf 100644 --- a/crates/validator/src/payload_attestation.rs +++ b/crates/validator/src/payload_attestation.rs @@ -117,6 +117,15 @@ impl PayloadAttestationService { ))); } + // From PQ a leanXMSS signature at the slot's PTC leaf, which + // `crate::pq` guards against a second message. + let is_pq = self.context.config.fork_at_epoch( + ethlambda_types::beacon::signing::compute_epoch_at_slot(slot), + ) == ethlambda_types::beacon::fork::ForkName::Pq; + if is_pq { + return self.attest_pq(slot, &pending, data).await; + } + let mut messages = Vec::with_capacity(pending.len()); { let _timing = crate::metrics::time_signing(); @@ -181,6 +190,58 @@ impl PayloadAttestationService { Ok(published) } + /// [`Self::attest`] at a PQ slot. + async fn attest_pq( + &self, + slot: Slot, + pending: &[&PtcDutyDto], + data: ethlambda_types::beacon::containers::gloas::PayloadAttestationData, + ) -> Result { + let root = self.context.payload_attestation_signing_root(&data); + let leaf = crate::pq::slot_leaf(slot, crate::pq::LEAF_OFFSET_PTC_ATTESTER); + let mut messages = Vec::with_capacity(pending.len()); + for duty in pending { + let pubkey = match parse_pubkey(&duty.pubkey) { + Ok(pubkey) => pubkey, + Err(err) => { + error!(%slot, validator = duty.validator_index, %err, "Duty carried an unreadable pubkey"); + continue; + } + }; + if !self.record(duty.validator_index, slot) { + continue; + } + match crate::pq::sign(&pubkey, leaf, root) { + Ok(signature) => messages.push( + ethlambda_types::beacon::containers::pq::PayloadAttestationMessage { + validator_index: duty.validator_index, + data, + signature, + }, + ), + Err(err) => { + error!(%slot, validator = duty.validator_index, %err, "Failed to sign a PQ payload attestation"); + crate::metrics::inc_signing_failures(); + } + } + } + if messages.is_empty() { + return Ok(0); + } + let published = self + .beacon_node + .submit_pq_payload_attestations(&messages) + .await?; + info!( + %slot, + count = published, + payload_present = data.payload_present, + "Published PQ payload attestations" + ); + crate::metrics::inc_payload_attestations_published(published as u64); + Ok(published) + } + /// Record that `validator` is being served for `slot`. `false` when it /// already was, or when the record is unusable, which is treated as a /// refusal for the reason the other guards' poisoned locks are. diff --git a/crates/validator/src/pq.rs b/crates/validator/src/pq.rs new file mode 100644 index 000000000..d56c9fceb --- /dev/null +++ b/crates/validator/src/pq.rs @@ -0,0 +1,225 @@ +//! The validator client on the PQ fork (`specs/_features/pq/validator.md`). +//! +//! At PQ a validator signs with leanXMSS and reveals RANDAO with a VRF proof, +//! both derived from its BLS signing key the way the PQ devnet's genesis +//! tooling registered them (`ethlambda_crypto::pq_keys`), so a keystore is +//! still all a validator needs. This module holds that material per key, +//! translates between a validator's BLS key and the form the PQ registry names +//! it by, and signs at the leaf each message is assigned. +//! +//! **The registry's form.** A PQ state names each validator by its 32-byte +//! XMSS key in the 48-byte form an execution-layer request names one: 16 zero +//! bytes, then the key. Duty responses carry that form, and no BLS key starts +//! with 16 zero bytes (its first byte carries the compression flag), so +//! [`bls_for_request_key`] maps one back to the local key that owns it, and +//! the client keeps working with BLS keys everywhere else. +//! +//! **Leaves.** XMSS is stateful: signing two messages at one leaf exposes it. +//! Every message has a leaf of its own (an attestation one per target epoch, +//! everything else one per slot and kind), so the slashing protection record +//! already covers blocks (one per slot) and attestations (one per target). +//! The other per-slot messages (payload attestations, aggregates, +//! envelopes) are guarded here, in memory: one signing root per leaf. The +//! specification asks for that record to persist across restarts; this one +//! does not, so a client restarted within a slot could sign a second +//! different message at one of those leaves. + +use std::collections::HashMap; +use std::sync::{Arc, Mutex, OnceLock, RwLock}; + +use ethlambda_crypto::{lean_xmss, pq_keys}; +use ethlambda_types::beacon::constants::{ + XMSS_ATTESTATION_LEAVES, XMSS_BUILDER_BID_LEAVES, XMSS_LEAVES_PER_SLOT, XMSS_RESERVED_LEAVES, +}; +use ethlambda_types::beacon::primitives::{BlsPubkey, Epoch, Root, Slot, XmssSignature}; + +pub use ethlambda_types::beacon::constants::{ + LEAF_OFFSET_AGGREGATE_AND_PROOF, LEAF_OFFSET_BEACON_PROPOSER, + LEAF_OFFSET_EXECUTION_PAYLOAD_ENVELOPE, LEAF_OFFSET_PTC_ATTESTER, +}; + +/// A validator's PQ key material. +pub struct PqKey { + xmss: lean_xmss::SecretKey, + vrf: pq_keys::VrfTree, + /// The 48-byte form the PQ registry names the validator by. + pub request_key: [u8; 48], +} + +#[derive(Default)] +struct Registry { + by_bls: HashMap>, + by_request: HashMap<[u8; 48], BlsPubkey>, +} + +fn registry() -> &'static RwLock { + static REGISTRY: OnceLock> = OnceLock::new(); + REGISTRY.get_or_init(Default::default) +} + +fn signed_leaves() -> &'static Mutex> { + static SIGNED: OnceLock>> = OnceLock::new(); + SIGNED.get_or_init(Default::default) +} + +/// Derives and registers the PQ key of the validator whose BLS key is +/// `pubkey` and secret key `secret` (32 bytes, big-endian, as a keystore +/// holds it). Slow: a leanXMSS key generation (about a second and a half +/// across cores) and a VRF tree (two million hashes), plus building the +/// bottom trees its next attestation and per-slot signatures fall in. Run it +/// off the async runtime. +pub fn derive_and_register(pubkey: BlsPubkey, secret: &[u8; 32], current_slot: Slot) { + let xmss = pq_keys::xmss_key(secret); + let vrf = pq_keys::VrfTree::from_bls(secret); + let mut request_key = [0u8; 48]; + request_key[16..].copy_from_slice(&xmss.public_key().to_bytes()); + // Warm the two bottom trees duties sign in, so the first signature of each + // kind does not pay for one at its deadline. + let epoch = current_slot / ethlambda_types::beacon::preset::SLOTS_PER_EPOCH; + let _ = xmss.prepare(attestation_leaf(epoch) as u32); + let _ = xmss.prepare(slot_leaf(current_slot, 0) as u32); + let key = Arc::new(PqKey { + xmss, + vrf, + request_key, + }); + let mut registry = registry().write().expect("PQ key registry poisoned"); + registry.by_request.insert(request_key, pubkey); + registry.by_bls.insert(pubkey, key); +} + +/// Whether any key is registered: the network schedules PQ and this client +/// derived its keys. +pub fn enabled() -> bool { + !registry() + .read() + .expect("PQ key registry poisoned") + .by_bls + .is_empty() +} + +/// The local BLS key whose PQ key `request_key` names, if any. +pub fn bls_for_request_key(request_key: &[u8; 48]) -> Option { + if request_key[..16] != [0; 16] { + return None; + } + registry() + .read() + .expect("PQ key registry poisoned") + .by_request + .get(request_key) + .copied() +} + +/// The request-key form of `pubkey`'s PQ key, if it has one. +pub fn request_key(pubkey: &BlsPubkey) -> Option<[u8; 48]> { + key(pubkey).map(|key| key.request_key) +} + +fn key(pubkey: &BlsPubkey) -> Option> { + registry() + .read() + .expect("PQ key registry poisoned") + .by_bls + .get(pubkey) + .cloned() +} + +/// `compute_xmss_attestation_leaf_index`. +pub fn attestation_leaf(target_epoch: Epoch) -> u64 { + XMSS_RESERVED_LEAVES + target_epoch +} + +/// `compute_xmss_leaf_index`. +pub fn slot_leaf(slot: Slot, offset: u64) -> u64 { + XMSS_RESERVED_LEAVES + + XMSS_ATTESTATION_LEAVES + + XMSS_BUILDER_BID_LEAVES + + slot * XMSS_LEAVES_PER_SLOT + + offset +} + +#[derive(Debug, thiserror::Error)] +pub enum PqSignError { + #[error("no PQ key for this validator")] + UnknownValidator, + #[error("leaf {0} already signed a different message")] + LeafUsed(u64), + #[error("leanXMSS: {0:?}")] + Sign(lean_xmss::SignError), +} + +/// Signs `root` at `leaf` with `pubkey`'s leanXMSS key. Refuses a leaf this +/// process already signed a different root at; the same root again is +/// re-signed, which yields the same bytes (signing is deterministic). +/// Blocking: building a bottom tree takes about a second and a half. +pub fn sign(pubkey: &BlsPubkey, leaf: u64, root: Root) -> Result { + let key = key(pubkey).ok_or(PqSignError::UnknownValidator)?; + { + let mut signed = signed_leaves().lock().expect("PQ leaf record poisoned"); + match signed.get(&(*pubkey, leaf)) { + Some(previous) if *previous != root => return Err(PqSignError::LeafUsed(leaf)), + Some(_) => {} + None => { + signed.insert((*pubkey, leaf), root); + } + } + } + let leaf_index = u32::try_from(leaf).map_err(|_| PqSignError::LeafUsed(leaf))?; + let signature = key + .xmss + .sign(&root.0, leaf_index) + .map_err(PqSignError::Sign)?; + let bytes: [u8; ethlambda_types::beacon::constants::XMSS_SIGNATURE_SIZE] = signature + .to_bytes() + .try_into() + .expect("a leanXMSS signature is XMSS_SIGNATURE_SIZE bytes"); + Ok(XmssSignature(bytes)) +} + +/// `pubkey`'s VRF proof for `slot`, its RANDAO reveal at PQ. +pub fn vrf_proof(pubkey: &BlsPubkey, slot: Slot) -> Result, PqSignError> { + let key = key(pubkey).ok_or(PqSignError::UnknownValidator)?; + Ok(key.vrf.proof(slot).into_iter().map(Root::from).collect()) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn leaves_follow_the_specification() { + assert_eq!(attestation_leaf(3), (1 << 15) + 3); + assert_eq!( + slot_leaf(10, LEAF_OFFSET_PTC_ATTESTER), + (1 << 15) + (1 << 29) + (1 << 30) + 80 + 2 + ); + // The slot leaves start halfway into a bottom tree, so the first 4096 + // slots share one (and every later 8192 another), attestations another. + assert_eq!(slot_leaf(0, 0) >> 16, slot_leaf(4095, 7) >> 16); + assert_ne!(slot_leaf(0, 0) >> 16, slot_leaf(4096, 0) >> 16); + assert_ne!(attestation_leaf(0) >> 16, slot_leaf(0, 0) >> 16); + } + + #[test] + fn a_request_key_maps_back_and_a_leaf_signs_once() { + let secret = [7u8; 32]; + let pubkey = BlsPubkey([9; 48]); + derive_and_register(pubkey, &secret, 0); + let request = request_key(&pubkey).unwrap(); + assert_eq!(request[..16], [0; 16]); + assert_eq!(bls_for_request_key(&request), Some(pubkey)); + assert_eq!(bls_for_request_key(&pubkey.0), None); + + let leaf = slot_leaf(5, LEAF_OFFSET_PTC_ATTESTER); + let first = sign(&pubkey, leaf, Root::from([1; 32])).unwrap(); + assert_eq!(sign(&pubkey, leaf, Root::from([1; 32])).unwrap(), first); + assert!(matches!( + sign(&pubkey, leaf, Root::from([2; 32])), + Err(PqSignError::LeafUsed(_)) + )); + let public = request[16..].try_into().unwrap(); + assert!(lean_xmss::verify(&public, leaf, &[1; 32], &first.0)); + assert_eq!(vrf_proof(&pubkey, 5).unwrap().len(), 32); + } +} diff --git a/crates/validator/src/proposal.rs b/crates/validator/src/proposal.rs index 87ed960e2..010628e53 100644 --- a/crates/validator/src/proposal.rs +++ b/crates/validator/src/proposal.rs @@ -395,6 +395,14 @@ impl ProposalService { store: &RwLock, deadline: Option, ) -> Result { + if self + .context + .config + .fork_at_epoch(compute_epoch_at_slot(slot)) + == ethlambda_types::beacon::fork::ForkName::Pq + { + return self.propose_pq(slot, duty, deadline).await; + } // Nothing awaits between signing and the end of `produce_and_sign`, // and a completed future is never discarded at the deadline, so a // signed block always reaches the publish step below. A deadline @@ -420,6 +428,106 @@ impl ProposalService { .map(Proposed::Full) } + /// A proposal at a PQ slot: the RANDAO reveal is the proposer's VRF proof, + /// the beacon node builds a self-built block and its envelope (ethlambda's + /// `pq_blocks` endpoint), and both are signed with leanXMSS at the slot's + /// proposer and envelope leaves. The slashing protection record guards + /// the block's leaf (one block per slot), `crate::pq` the envelope's. An + /// envelope failure is logged, not the proposal's: the block is out. + async fn propose_pq( + &self, + slot: Slot, + duty: &ProposerDutyDto, + deadline: Option, + ) -> Result { + use ethlambda_types::beacon::containers::pq; + use ethlambda_types::beacon::primitives::HashTreeRoot as _; + use libssz::SszEncode as _; + + let pubkey = parse_pubkey(&duty.pubkey)?; + let validator = duty.validator_index; + let checked = crate::slashing_protection::blocking(&self.protection, move |protection| { + Ok(protection.check_block(&pubkey, slot)) + }) + .await; + refuse_unless_safe(slot, validator, "consult", checked)?; + + let pq_error = |err: crate::pq::PqSignError| Error::Signing { + pubkey, + reason: err.to_string(), + }; + let randao_reveal = crate::pq::vrf_proof(&pubkey, slot).map_err(pq_error)?; + let graffiti = self.settings.graffiti(&pubkey); + let contents = before( + deadline, + slot, + self.beacon_node + .produce_pq_block(slot, &randao_reveal, Some(graffiti)), + ) + .await?; + if contents.block.slot != slot || contents.block.proposer_index != validator { + return Err(Error::InconsistentResponse(format!( + "asked for validator {validator}'s PQ block at slot {slot}, node built one for \ + validator {} at slot {}", + contents.block.proposer_index, contents.block.slot + ))); + } + let block_root = contents.block.hash_tree_root(); + + let recorded = crate::slashing_protection::blocking(&self.protection, move |protection| { + protection.check_and_record_block(&pubkey, slot) + }) + .await; + refuse_unless_safe(slot, validator, "record", recorded)?; + + let signing_root = self.context.block_signing_root(block_root, slot); + let leaf = crate::pq::slot_leaf(slot, crate::pq::LEAF_OFFSET_BEACON_PROPOSER); + let signature = crate::pq::sign(&pubkey, leaf, signing_root).map_err(pq_error)?; + let signed = pq::SignedBeaconBlock { + message: contents.block, + signature, + }; + let published = before( + deadline, + slot, + self.beacon_node.publish_block( + ethlambda_types::beacon::fork::ForkName::Pq, + &signed.to_ssz(), + ), + ) + .await?; + info!(%slot, validator, "Published a PQ block"); + + let envelope = contents.execution_payload_envelope; + let domain = self.context.domain( + ethlambda_types::beacon::constants::DOMAIN_BEACON_BUILDER, + compute_epoch_at_slot(slot), + ); + let envelope_root = ethlambda_types::beacon::signing::compute_signing_root( + envelope.hash_tree_root(), + domain, + ); + let leaf = crate::pq::slot_leaf(slot, crate::pq::LEAF_OFFSET_EXECUTION_PAYLOAD_ENVELOPE); + match crate::pq::sign(&pubkey, leaf, envelope_root) { + Ok(signature) => { + let signed = pq::SignedExecutionPayloadEnvelope { + message: envelope, + signature, + }; + match self + .beacon_node + .publish_pq_execution_payload_envelope(&signed.to_ssz()) + .await + { + Ok(()) => info!(%slot, validator, "Published a PQ execution payload envelope"), + Err(err) => warn!(%slot, validator, %err, "Failed to publish the PQ envelope"), + } + } + Err(err) => warn!(%slot, validator, %err, "Failed to sign the PQ envelope"), + } + Ok(Proposed::Full(published)) + } + /// Everything up to and including the block signature: the slashing /// protection checks, the RANDAO reveal, the block request and the checks /// on what came back. diff --git a/crates/validator/src/subscriptions.rs b/crates/validator/src/subscriptions.rs index 5de975781..2a705e5cc 100644 --- a/crates/validator/src/subscriptions.rs +++ b/crates/validator/src/subscriptions.rs @@ -56,8 +56,14 @@ pub async fn subscribe( duties .iter() .map(|duty| { - let is_aggregator = match selection_for(context, &store, duty) { - Ok(selection) => selection.is_some(), + // At a PQ slot the node names the aggregators (the + // aggregation committee); there is no selection proof. + let pq_aggregator = duty.is_aggregator; + let is_aggregator = match pq_aggregator.map_or_else( + || selection_for(context, &store, duty).map(|proof| proof.is_some()), + Ok, + ) { + Ok(selected) => selected, Err(err) => { warn!( slot = duty.slot, @@ -160,6 +166,7 @@ mod tests { committees_at_slot: 64, validator_committee_index: 7, slot, + is_aggregator: None, } } diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index ba5d16acf..da5ce005b 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -17,6 +17,7 @@ - [Beacon Chain State Transition](./beacon_stf.md) - [The mainnet wire](./beacon_wire.md) - [The execution layer pairing](./beacon_engine.md) +- [The PQ fork](./beacon_pq.md) # Operations diff --git a/docs/beacon_pq.md b/docs/beacon_pq.md new file mode 100644 index 000000000..627c1ffc1 --- /dev/null +++ b/docs/beacon_pq.md @@ -0,0 +1,342 @@ +# The PQ fork + +consensus-specs' `pq` branch (pinned here at +`a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`) specifies an emergency fork that +replaces BLS with leanXMSS, a hash-based signature scheme believed secure +against a quantum adversary. It comes as two feature forks after gloas: + +1. **EIP-8483** (`specs/_features/eip8483`): validators and builders register + post-quantum material through two new execution-layer requests. A validator + registers 64 bytes (an XMSS key and a VRF root), a builder 32 (an XMSS key). + Nothing else changes. +2. **PQ** (`specs/_features/pq`): every validator and builder adopts the key it + registered, BLS signatures become XMSS ones, sync committees and the light + client protocol go, RANDAO reveals become VRF proofs, attestations lose their + signatures to a block-wide STARK proof, voluntary exits move to the execution + layer, and deposits carry a VRF root instead of a signature, guarded by + preregistrations (EIP-8205). + +Both are `ForkName`s (`Eip8483`, `Pq`), between `Gloas` and `Lean`, on-disk +selectors 8 and 9. Neither is scheduled on a public network. **The node +follows PQ** (`ForkName::is_followed`): it imports PQ blocks and envelopes +from gossip and req/resp, applies PQ aggregates and payload attestations to +fork choice, and asks its execution client about PQ payloads. It was run +next to Lighthouse's PQ client on a PQ-at-genesis devnet +([`tooling/pq-devnet`](../tooling/pq-devnet/README.md)). Its validator +client runs every PQ duty (attest, aggregate, the PTC vote, propose and +reveal the payload), and the beacon node builds PQ blocks with their +attestations' proofs merged: see [Validator duties](#validator-duties). It +does not follow EIP-8483 yet: see [What remains](#what-remains). + +Each builds on the fork before it, like every other fork. EIP-8483 is gloas +plus its two requests and PQ is EIP-8483 with BLS replaced, so a rule gated +`fork >= ForkName::Gloas` holds at both and one gated `>= ForkName::Eip8483` +holds at PQ. Each upgrade takes only its predecessor's state, so +`process_slots` reaches PQ through EIP-8483 even when the two share an epoch. +The same holds in the Beacon API wherever the question is which rules apply +at a slot: an attestation's payload flag, the head's payload status, the PTC +window, a state's builders. Where the question is which container a version +names, the two stay apart from gloas. PQ replaces every signed container and +EIP-8483 the block and envelope, but EIP-8483's attestations, aggregates, +attester slashings and bids are gloas's own: gossip decodes the first three +as gloas's, while the Beacon API refuses the `eip8483` version header, since +the node does not follow the fork. Gossip's bid rules judge a gloas bid +against a gloas state and a PQ bid against a PQ state, and leave EIP-8483's +states out, because they read the parent's envelope, whose shape EIP-8483 +changes. + +## What is implemented + +| Layer | Where | Status | +| --- | --- | --- | +| leanXMSS sign/verify | `ethlambda_crypto::lean_xmss` | KAT-pinned against consensus-specs and leanVM | +| Attestation proofs | `ethlambda_crypto::stark` | the specification's reference concatenation by default; leanVM proofs through a `leanvm-sidecar` with `--pq-prover-socket` (see [Proofs](#proofs)) | +| Containers | `ethlambda_types::beacon::containers::{eip8483, pq}` | SSZ round trips, gaps per the spec; the maximal aggregate, attester slashing and bid encode to `pq.yaml`'s size bounds | +| Fork upgrades | `upgrade::{upgrade_to_eip8483, upgrade_to_pq}` | unit-tested | +| Block processing | `stf::{eip8483, pq}` | unit-tested with real XMSS keys | +| Epoch processing | `stf::epoch::pq` | unit-tested | +| Fork choice | `fork_choice` (`Attestation::Pq`, `AttesterSlashing::Pq`, `on_pq_payload_attestation_message`, no `notify_ptc_messages` for PQ blocks, `on_execution_payload_envelope` over any fork's envelope) | run on the devnet; no unit tests of its own | +| Envelopes | `containers::SignedExecutionPayloadEnvelope` (gloas, EIP-8483, PQ), `stf::gloas::verify_execution_payload_envelope` | each fork's signature and requests root; storage, req/resp, gossip, the chain actor and the engine take the fork-tagged envelope | +| Spec-fixture runners | `tests/beacon_spec` | wired, but no fixtures exist yet | +| PQ gossip | `gossip::pq`, `ethlambda_p2p::beacon::pq` | every reshaped topic's rules, XMSS signatures and STARK proofs unit-tested with real keys; an accepted aggregate's vote, a payload attestation and an envelope reach the chain actor, every Lighthouse message on the devnet was accepted | +| Engine | `beacon_engine::gloas_new_payload_request` | each fork's typed requests list to `engine_newPayloadV5`, `engine_forkchoiceUpdatedV4` for a PQ head | +| Genesis | `genesis_anchor_block` | the body carries the state's bid from gloas on, as the specification's genesis helper and ethpandaops' generator build it | +| EIP-8483 gossip | `gossip`, `ethlambda_p2p::beacon::decode` | attestations, aggregates and attester slashings decode and validate as gloas's; blocks, envelopes and columns are ignored | +| Block production | `pq_block_production`, `rpc::beacon::pq_proposal` | self-build; attestations packed from validated aggregates, their proofs merged under a deadline; no blobs. Every block it built on the devnet was canonical on Lighthouse | +| Validator client | `ethlambda_validator::pq` and the duty services | keys derived from each BLS key as ethereum-package's generator and Lighthouse derive them; attestations, aggregates, PTC votes, proposals and envelopes, each at its leaf; run on the devnet | +| Aggregation | `rpc::beacon::pq_aggregation`, `pools::{PqSignaturePool, PqAggregatePool}` | the node collects its aggregators' committees' signatures and proves ahead on the sidecar | +| Beacon API | `rpc::beacon::{pool, pq_proposal, ptc, proposal}` | PQ attestations, aggregates (in and out), payload attestations, blocks, envelopes and `POST /eth/v1/validator/pq_blocks/{slot}`; the other endpoints refuse PQ's signed containers | + +## Gossip + +A PQ digest's topics are `pq/p2p-interface.md`'s: gloas's set without +`voluntary_exit`, `bls_to_execution_change` and the sync committee topics +(`topics::PQ_TOPIC_KINDS`), and `ATTESTATION_SUBNET_COUNT_PQ` (16) attestation +subnets, of which the node backbones its own set +(`subnets::compute_subscribed_subnets_pq`, the specification's function over +16 subnets). `attnets` advertises that set while the current digest is PQ's. +The aggregator and sync committee subnets a validator client asks for come +from BLS duties, so they are never joined under a PQ digest. + +Every reshaped topic decodes by its topic's digest into PQ's container +(`decode::decode_pq_gossip`, with the `MAX_*_SIZE_PQ` bounds) and is judged by +`state_transition::beacon::gossip::pq`: + +| Topic | What PQ changes | +| --- | --- | +| `beacon_aggregate_and_proof` | the aggregator's XMSS signature at its slot's aggregate leaf; aggregators are the aggregation committee, not a selection proof; the aggregate's STARK proof proves its attesters' signatures | +| `beacon_attestation_{subnet_id}` | the XMSS signature at the target epoch's attestation leaf; the subnet is one of 16 | +| `proposer_slashing`, `attester_slashing` | XMSS header signatures; each indexed attestation's STARK proof | +| `payload_attestation_message` | the XMSS signature at the slot's PTC leaf | +| `execution_payload_bid`, `proposer_preferences` | gloas's rules over the PQ builder market, with XMSS signatures and the builder-exit key compared on its last 32 bytes | +| `execution_payload` | PQ's execution request limits and the envelope's XMSS signature | +| `beacon_block` | PQ's operation and request limits (`gossip::pq::block_cheap_checks`), behind the `is_followed` gate | + +The rules are gloas's with only the `[Modified in PQ]` and `[New in PQ]` lines +applied; see `docs/spec_deviations.md`. A rule that needs a PQ state (every +stateful one) ignores the message when the state it reads is of another fork, +which on a live node is always: the node imports no PQ block. So the cheap +rules judge what arrives on a PQ digest a network schedules, and the rest end +in an ignore. An accepted message is relayed and recorded in the seen caches +the other forks share (bids and preferences in a PQ builder market of their +own, which block production never reads), and goes no further: no pool, event +stream or chain actor takes one. A message that does not decode as PQ's +container is ignored, not rejected, as on every topic of a fork the node does +not follow. + +Blocks, data column sidecars and the topic set the node holds keep the shared +path. A block of a fork the node does not follow is ignored before any rule. + +## The XMSS scheme + +The specification's `xmss` module is EthLib's leanXMSS: Keccak-256, two layers +of height-16 Merkle trees over WOTS+C one-time keys, a 32-byte public key (the +top root, then the public parameter) and a 2404-byte signature that starts with +its leaf index. It is **not** the XMSS lean consensus signs with +(`ethlambda_crypto::signature`), which is leanVM's own BLAKE2s scheme with +1208-byte signatures. + +`ethlambda_crypto::lean_xmss` is a port of leanVM's Rust implementation of the +same note (`crates/signatures/src/lean_xmss`, leanVM `4af2cddd`, the commit +EthLib's README cites). Its tests pin: + +- consensus-specs' `xmss_kat.json`: the signature verifies, the four negative + cases of `test_xmss.py` do not, and the seed `00..1f` regenerates the + vector's key and signature byte for byte; +- leanVM's own known answer, the vector EthLib's README pins too. + +Full key generation takes about a second and a half in `release-fast` across +cores, so both full-key tests run in `make test`. `range_key` builds a key real +over a few leaves only, for tests. + +### Proofs + +`ethlambda_crypto::stark` proves, merges and verifies the fork's attestation +proofs through one of two backends, chosen once per process: + +- **Reference** (the default): the specification's stand-in for a recursive + STARK. A proof is the signatures concatenated in statement order, and + verification checks each, plus the exact length. It is capped by + `MAX_STARK_PROOF_SIZE` (1 MiB), about 436 signatures per block. The + specification's fixtures use it. +- **Sidecar** (`--pq-prover-socket`, `stark::install_sidecar`): real leanVM + proofs, made and checked by a `leanvm-sidecar` process over a Unix socket, + through leanVM's own client crate at `283686575c` (`sidecar::LEANVM_REV`), + the revision Lighthouse's PQ branch pins. A statement list maps to leanVM + claims as Lighthouse maps it: the sorted, deduplicated + `(public key, message)` pairs, the empty proof for no claims, and merges + folded sixteen children at a time. `bin/ethlambda/pq/ethlambda-pq` starts a + sidecar and passes its socket. + +The encodings are not interchangeable, so every node of a network must use +the same backend; the PQ devnet's Lighthouse accepts only the sidecar. A +leanVM proof does not attest the leaf a signature was made at, so the sidecar +backend checks that each key signed the message at *some* leaf +(`docs/spec_deviations.md`). Lighthouse's blocks replay through ethlambda's +state transition with every proof verified this way, state root for state +root. + +### Leaves + +XMSS is stateful: signing two messages at one leaf breaks the key. The fork +assigns every message a leaf: + +| Leaves | Used by | +| --- | --- | +| `0 .. 2^15` (reserved) | voluntary exits, preregistrations (signed for the EL contracts) | +| `2^15 + epoch` | the attestation for target `epoch` | +| the next `2^30` | builder bids, any leaf once | +| `... + 8 * slot + offset` | per slot: proposal (0), aggregate (1), PTC vote (2), proposer preferences (3), envelope (4) | + +A signature carries its leaf index, and every check requires the leaf its +message may use (`compute_xmss_leaf_index`, `is_builder_bid_leaf_index`). + +## How the state holds XMSS keys + +A PQ `Validator` has a 32-byte `XMSSPubkey` and a `vrf_root` where every +earlier fork's has a 48-byte BLS key. Every registry accessor on `BeaconState` +hands out the shared `Validator` record, and most of the state transition +reads only its balances, epochs and credentials. So the PQ registry element, +`pq::Validator`, holds the shared record with its key in the 48-byte form an +execution-layer request names an XMSS key by (16 zero bytes, then the key: +`XmssPubkey::to_request_key`) and the VRF root beside it. Its SSZ encoding and +merkleization are written by hand through the specification's nine-field +`ValidatorRecord`, so its bytes and roots are the PQ container's exactly. + +That representation is also the one the specification compares request keys +in: after a request is checked not to be legacy (`is_legacy_pubkey`), the +specification's `XMSSPubkey(request.pubkey[16:]) in validator_pubkeys` is the +48-byte comparison electra's request processing already makes. That is why +the PQ withdrawal and consolidation requests are electra's, behind the legacy +check. + +The builder registry gets the same treatment (`pq::Builders`, gloas's records +in memory, `BuilderRecord` on the wire). + +## The gloas-family view + +Gloas, EIP-8483 and PQ states are separate structs, since each derives its own +SSZ shape (EIP-7495 progressive containers, with the fields PQ removes left as +gaps in the merkle tree via `active_fields`). They share every field the +EIP-7732 machinery reads, at the same types. `GloasFamily` and +`GloasFamilyMut` (`containers/gloas_family.rs`) borrow those fields from any of +the three, and `helpers::gloas::gloas_state(_ref)` return them, so the gloas +state transition (withdrawals, bids, payments, the PTC window, attestation +participation) runs on all three without a copy per fork. The view carries +`fork`, for the few rules that differ. + +## A newer gloas underneath + +The `pq` branch is based on a consensus-specs `master` newer than this +repository's pin (`v1.7.0-beta.2`). Three gloas state-transition rules changed +in between, and the PQ text depends on them, so they apply from EIP-8483 on +while gloas keeps its pinned rules. Four fork-choice rules changed for every +fork too, and those are not applied to any fork yet, the PQ forks included. See +`docs/spec_deviations.md` for both lists. + +## Committees + +The PQ fork caps a slot's committees at `MAX_COMMITTEES_PER_SLOT_PQ` (16 on +mainnet, 4 on minimal), for every epoch a PQ state is asked about, including a +previous epoch that ran before the fork. The committee cache's `ShufflingKey` +carries the cap, since a state either side of the fork names the same epoch and +deciding block but a different shuffling. + +Aggregators are no longer self-selected: each beacon committee has an +aggregation committee of up to `AGGREGATORS_PER_COMMITTEE` validators sampled +by effective balance, kept in the state's `aggregation_committee_window` for the +previous, current and lookahead epochs. + +## Inclusion delay + +A PQ block includes attestations from at least +`MIN_ATTESTATION_INCLUSION_DELAY_PQ` (2) slots before it, which gives the +proposer a slot to receive aggregates and merge their proofs into the block's. +The timely-head flag needs exactly that delay, and since the attested block can +then be the grandparent, its payload vote is checked against the availability +bit at its own slot (`get_latest_block_slot`) rather than the parent's. Gloas's +`process_attestation` and `get_attestation_participation_flag_indices` serve +PQ, and take the delay from the state's fork +(`helpers::pq::min_attestation_inclusion_delay`). + +## Validator duties + +The validator client keeps its BLS keystores and derives each validator's PQ +material from the BLS signing key at startup, the way ethereum-package's +genesis generator and Lighthouse's PQ branch do: the leanXMSS key from +`sha256("leanxmss-seed" || sk)` and the VRF tree from `sha256("vrf-seed" || +sk)` (`ethlambda_crypto::pq_keys`). The registry names a validator by its +key's 48-byte request form, which the client looks its validators up by and +maps back to the BLS key it holds, so a validator whose derived key is not +the registered one resolves to no index and gets no duties. Key generation +builds two bottom trees per key, which takes under a minute for 16 +validators. + +Each duty signs at the leaf the fork assigns it (see [Leaves](#leaves)): the +attestation at its target epoch's, the others at the slot's own offset. The +client refuses to sign two different roots at one leaf. That record is in +memory; across a restart, slashing protection still guards attestations and +proposals, the two that slashing punishes. + +- **Attesting**: a `SingleAttestation` signed at the target epoch's leaf, + published on one of PQ's 16 subnets. +- **The PTC vote**: as gloas's, signed at the slot's PTC leaf. +- **Proposing**: the client sends its VRF proof for the slot (the RANDAO + reveal) to `POST /eth/v1/validator/pq_blocks/{slot}`, signs the block and + the envelope revealing its self-built payload at their leaves, and publishes + both. The beacon node builds the block while the execution client builds + the payload, and meanwhile packs attestations: it picks, from the + aggregates it validated, what `process_attestation` accepts at the slot and + credits someone new, as `compute_on_chain_aggregate` shapes them, and merges + their proofs in `process_stark`'s order on the sidecar. A merge must end + three quarters of the way to the attestation deadline (Lighthouse's bound). + One child needs no merge; when more do not finish in time, the block packs + the most valuable single-committee attestation alone, whose proof is the + block's as it is. On the devnet's VM a two-child merge takes about four + seconds, for Lighthouse too, so blocks there carry one attestation each. +- **Aggregating**: a PQ aggregator is a member of its committee's + aggregation committee, which the attester duties report as + `is_aggregator`. Proving takes seconds, so the node proves ahead, as + Lighthouse's PQ branch does: the client's committee subscriptions join the + committee's subnet and register it, the node keeps the validated votes' + signatures for registered committees, and halfway between the attestation + and aggregate deadlines seals the slot and proves each committee before the + aggregate deadline. `GET aggregate_attestation` then answers the proved + aggregate and its proof, and the client signs the `AggregateAndProof` at the + slot's aggregate leaf. + +## Spec fixtures + +No consensus-specs release ships fixtures for either fork. The pyspec on the +`pq` branch generates them: + +```sh +git clone https://github.com/ethereum/consensus-specs && cd consensus-specs +git checkout a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c +make test reftests=true fork=eip8483 preset=minimal +make test reftests=true fork=pq preset=minimal +``` + +`reftests/` then holds `tests/minimal/{eip8483,pq}/...`, which unpack beside +the release's own forks under `consensus-spec-tests/tests/minimal/`. The +generator runs with XMSS inactive (key generation takes minutes in EthLib's +reference implementation), so the fixtures carry stub signatures and empty +proofs; the spec-test binary calls `helpers::pq::disable_xmss_verification` +for that reason, which accepts every XMSS signature and STARK proof the way +the pyspec's inactive stubs do. `GetLeafIndex` and the self-build empty +signature are still read from the bytes, as the pyspec's are. + +The runners route both forks' cases already (operations, epoch processing, +fork choice, rewards; the new `validator_pq_registration_request`, +`builder_pq_registration_request`, `validator_preregistration_request`, +`builder_preregistration_request`, `voluntary_exit_request` and `randao` +handlers, and the `aggregation_committee_window` and `preregistration_expiry` +epoch steps), but have never run against fixtures: expect fixes once they are +generated. + +## What remains + +- Generate and run the fixtures above, and add the PQ containers to the + `ssz_static` runner. +- The operation pool and the event stream for PQ gossip: slashings are + relayed but never packed into a block, and accepted PQ messages raise no + events. Attestations are pooled (see [Validator duties](#validator-duties)). +- Following EIP-8483 (`ForkName::is_followed` is false for it): its block's + cheap gossip rules, its `execution_payload` topic's decode and rules (its + envelope is its own container), its fetched envelopes' checks, and its + execution request limits. The envelope, engine and storage paths already + take its envelope. +- The ENR's `attnets` keeps the pre-PQ backbone across the PQ boundary: ethrex's + `DiscoveryServer` cannot replace the served record at runtime (the same gap + as its `eth2` entry, see `CLAUDE.md`). +- Blobs in PQ block production: a payload with blobs is refused, since the + envelope's column sidecars are not built for PQ yet. +- Merging ahead of a proposal: the aggregates a slot-`N` block may pack are + all known a slot earlier, so a node could merge during slot `N - 1` for a + proposer it serves, where today a slow merge costs every attestation but + one. +- A persistent leaf record for the aggregate, PTC and envelope leaves, which + slashing protection does not cover. +- The pubkey index cache keys a pubkey by bytes 1 to 9, all zero for a PQ + key's request form: still correct (every candidate is verified, and a miss + scans), but every PQ lookup scans. diff --git a/docs/cli.md b/docs/cli.md index 47b6085be..4cb26bbe8 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -159,6 +159,7 @@ something specific here. | Flag | Default | Meaning | |---|---|---| | `--network` | `mainnet` | A built-in network name (`mainnet`, `sepolia`, `hoodi` or `plataberget`), or a path to a directory of published network files. A value containing a slash is always a path, so `mainnet` is the built-in and `./mainnet` is a directory. The directory must hold `config.yaml` and `genesis.ssz`, and may hold `bootstrap_nodes.yaml` or `bootstrap_nodes.txt` | +| `--pq-prover-socket` | none | The Unix socket of a `leanvm-sidecar` built from leanVM `283686575c`, which makes and checks the PQ fork's attestation proofs as leanVM STARKs, the encoding the PQ devnet's clients use. Without it the node uses the specification's reference encoding (the signatures concatenated), which only another node on the reference accepts. Read only on a network scheduling PQ. `ethlambda-pq` (`bin/ethlambda/pq/`) starts a sidecar and passes this flag | `genesis_validators_root` and `genesis_time`, which the fork digest that keys every gossip topic, the ENR `eth2` entry and discv5 admission is computed from, diff --git a/docs/rpc.md b/docs/rpc.md index e291c7db5..c530c9062 100644 --- a/docs/rpc.md +++ b/docs/rpc.md @@ -280,9 +280,23 @@ surface rather than sitting beside it; a `/lean/v0` path on a beacon node is a | `POST` | `/eth/v1/validator/proposer_preferences` | *(status only)* | Validate, cache and gossip `SignedProposerPreferences` (gloas; JSON or SSZ body) | | `POST` | `/eth/v1/beacon/states/{state_id}/builders` | JSON | The gloas builder registry, filtered by id and status | | `POST` | `/eth/v1/validator/prepare_beacon_proposer` | *(status only)* | Record each validator's fee recipient for block production (see below) | +| `POST` | `/eth/v1/validator/pq_blocks/{slot}` | SSZ | An unsigned PQ block and the envelope of its self-built payload, `pq::BlockContents`, given the proposer's VRF proof (ethlambda's own; see [PQ](#pq)) | | `POST` | `/eth/v1/validator/register_validator` | *(status only)* | Forwards builder registrations of active or pending validators to `--builder`, and sets their fee recipients | | `GET` | `/eth/v1/events` | SSE | Live stream of chain events (see below) | +### PQ + +At a PQ slot (`docs/beacon_pq.md`) the endpoints a validator client needs +take PQ's containers under `Eth-Consensus-Version: pq`: attestations, payload +attestations and aggregates in the pool endpoints, blocks and envelopes in +the publish endpoints. The Beacon API has no PQ block production yet, so +ethlambda's validator client asks `POST /eth/v1/validator/pq_blocks/{slot}` +with `{"randao_reveal": [], "graffiti"?}`. +Attester duties at a PQ epoch add `is_aggregator`, the validator's seat in +its committee's aggregation committee, and `GET aggregate_attestation` at a +PQ slot answers `{"aggregate", "proof"}`, the aggregate this node proved for +its aggregators' committee (Lighthouse's PQ branch's `PqAggregate`), or 404. + ### Validator endpoints These are what `ethlambda validator` needs to attest through this node. Every diff --git a/docs/spec_deviations.md b/docs/spec_deviations.md index 2479dd7b5..3497e7720 100644 --- a/docs/spec_deviations.md +++ b/docs/spec_deviations.md @@ -867,3 +867,144 @@ signature, whichever way the node answered. Three departures: and gossip does not replay them, so a node that restarts would otherwise self-build without them, and ignore builders' bids, until the next dependent root. Same cadence as `prepare_beacon_proposer`, which a restart loses too. + +## EIP-8483 and PQ run on a newer gloas than the gloas fork does + +consensus-specs' `pq` branch (`a4d16c5be5e8bf8c5a3fb323ee1cb0293efecb1c`) is +based on a `master` newer than this repository's pin (`v1.7.0-beta.2`). Across +every fork's `beacon-chain.md`, `fork.md` and `fork-choice.md`, the rules that +changed in between are the three state-transition rules below, the four +fork-choice rules in the next section, and two that only touch +`upgrade_to_gloas` (which deposits `onboard_builders_from_pending_deposits` +keeps queued, and clearing the fork slot's `execution_payload_availability` +bit); everything else is renaming and type hints. The two upgrade rules do not +reach EIP-8483 or PQ, which upgrade from a gloas state. + +The EIP-8483 and PQ texts assume the newer state-transition rules, so they +apply from EIP-8483 on, and gloas keeps the pinned ones until its own fixtures +move: + +- `apply_parent_execution_payload` settles the parent's builder payment + *before* processing the parent's execution requests, so a builder exit + request is rejected while its payment is pending + (`stf::eip8483::apply_parent_execution_payload`, + `stf::pq::apply_parent_execution_payload`). +- The builders sweep pays a builder's balance net of the withdrawals the same + sweep already took from it (`get_builder_balance_after_withdrawals`, gated in + `stf::gloas::get_builders_sweep_withdrawals`). +- `slash_validator` clears every pending builder payment whose proposer is the + slashed validator, whatever the slashing (`helpers::mutators::slash_validator`). + The PQ `process_proposer_slashing` relies on it and no longer clears the + payment itself; the EIP-8483 one still runs gloas's narrower clearing first, + which the wider one subsumes. + +## EIP-8483 and PQ fork choice keeps the pinned release's rules + +The same newer `master` changed four fork-choice rules for every fork. PQ's +`on_block` calls `get_head`, so they reach PQ, but this repository applies none +of them to any fork yet, EIP-8483 and PQ included: + +- `calculate_committee_fraction` (proposer boost, and the weak-head and + strong-parent reorg thresholds) sums the *unslashed* active balance; here it + is still `get_total_active_balance`, which counts slashed validators that + are still active. +- `get_slot_component_duration_ms` rounds to the nearest millisecond; here it + truncates. On mainnet that moves only `ATTESTATION_DUE_BPS` (3333 bps of + 12 s), from 3999 to 4000 ms. +- `get_head` filters a node tree instead of a block tree (`filter_node_tree`). + Under gloas a leaf is a FULL or EMPTY node, so a block whose EMPTY branch + leads only to non-viable children can still be viable through its childless + FULL node, where the block-tree filter drops it. `get_head` also returns the + justified root's EMPTY node when nothing is viable. +- `on_tick` takes the time in milliseconds (`store.time_ms`, + `compute_slot_at_time_ms`). The store here keeps milliseconds too, but + `on_tick` and `on_tick_per_slot` still take whole seconds, so a tick cannot + land between two seconds. + +Fork-choice fixtures generated from the `pq` branch may exercise any of these, +so expect failures there until the rules move. Moving them changes every +fork's fork choice, not only the PQ forks'. + +## `upgrade_to_pq` reads a builder only for a payment with an amount + +`compute_builder_pending_withdrawals` and `compute_builder_pending_payments` +read `pre.builders[payment.withdrawal.builder_index]` for every pending +payment, an empty one (builder index zero) included, before checking the +amount. On a registry without builders that index raises, so the +specification's fork cannot happen on such a chain. Here the builder is read +only when the payment has a nonzero amount, the one case its key is used in, +which differs from the specification only where the specification has no +outcome to match. + +## PQ fixtures run with XMSS verification off + +consensus-specs generates its EIP-8483 and PQ fixtures with XMSS inactive, so +their signatures are stubs and their proofs empty, which the pyspec's inactive +`xmss.Verify` and `stark.VerifyAggregates` accept. The spec-test binary turns +this crate's verification off the same way +(`helpers::pq::disable_xmss_verification`, compiled only with the +`beacon-spec-tests` feature), so no PQ fixture checks a real signature. The +real verifier is covered by its own known-answer tests instead +(`ethlambda_crypto::lean_xmss`) and by the PQ unit tests, which sign with real +keys. + +## PQ gossip is gloas's rules with the PQ lines applied + +consensus-specs' `pq` branch restates every gossip rule it touches +(`specs/_features/pq/p2p-interface.md`) on a `master` newer than this +repository's pin, as `validate_*_gossip` functions that read the head's state +and order their checks differently. `gossip::pq` applies only the lines that +branch marks `[Modified in PQ]` or `[New in PQ]` to the gloas rules the other +gossip modules implement, so their deliberate departures carry over: + +- **The vote block's own post-state**, not the head's, for aggregates and + attestations, advanced to the attestation's epoch when its aggregation + committee window does not reach it (the branch's own `[New in PQ]` + `process_slots`, applied to this state rather than the head's), and only + by `MAX_VOTE_STATE_ADVANCE_EPOCHS` (2): advancing runs every slot between, + on any peer's say-so, and a node that does not follow PQ has every head + far behind a PQ slot. A vote block further behind is ignored. PQ proposer + preferences bound the advance of their dependent block's state the same + way. +- **The pubkey-only XMSS signature before any committee derivation.** +- **"Passes validation" is "post-state is cached".** +- **The newer master's bid rules are not taken**: the builder bid limit + (`get_builder_bid_limit`) and the per-builder bid counts. PQ's bid rule is + this repository's gloas bid rule over a PQ builder market, with XMSS + signatures and `[Modified in PQ]`'s builder-exit key comparison. +- **The proposer preferences domain is the one the specification names** + (`compute_fork_version(proposal_epoch)`), where gloas's rule accepts any of + three candidate domains for other clients' sake. + +Two more: + +- **A rule whose state is not a PQ one ignores the message.** The XMSS keys a + rule reads exist only in a PQ state, so a message whose block state is + another fork's (around the fork boundary) cannot be judged. Rejecting would + score down an honest peer. +- **A message that does not decode as PQ's container is ignored, not + rejected**: the containers are modelled from a moving specification branch. + +## PQ attestation proofs are leanVM proofs, which attest no leaf index + +The specification's `stark.VerifyAggregates` checks each signature at its +statement's leaf index, and its reference implementation does (a proof is +the signatures concatenated). The PQ devnet's clients prove with leanVM +instead, through a `leanvm-sidecar` (`--pq-prover-socket`, +`ethlambda_crypto::stark::sidecar`), and a leanVM proof attests only +`(public key, message)` claims (leanEthereum/leanVM#787). So under the +sidecar backend a proof shows each key signed the message at *some* leaf, +not at the statement's. Proving still refuses a signature made at another +leaf, since it sees the signatures, which is Lighthouse's behavior too. The +two encodings are not interchangeable: a node verifies only the backend it +runs, so a network picks one, and the reference stays the default for the +specification's fixtures. + +## The genesis block's body carries the state's bid from gloas on + +The specification defines the genesis block only in its test helpers +(`get_genesis_block`), which from gloas on put the state's +`latest_execution_payload_bid` in the body's bid; ethpandaops' generator +builds the genesis header the same way. `genesis_anchor_block` builds that +body, and falls back to the fully empty body when the state's header names +that one instead, for a generator built against an older gloas release. diff --git a/tooling/pq-devnet/Dockerfile b/tooling/pq-devnet/Dockerfile new file mode 100644 index 000000000..96fe13652 --- /dev/null +++ b/tooling/pq-devnet/Dockerfile @@ -0,0 +1,120 @@ +# syntax=docker.io/docker/dockerfile:1.7-labs + +# The ethlambda image for a network on the consensus-specs pq fork: the node, +# plus leanVM's prover sidecar and `ethlambda-pq`, which starts the sidecar +# before the node. Build from the repository root: +# +# docker build -f tooling/pq-devnet/Dockerfile -t ethlambda:pq-local . +# +# Release LTO is off, so the image builds in minutes; do not use +# `release-fast`, which enables debug assertions. + +# leanVM's sidecar, from the revision ethlambda's client crate and +# Lighthouse's PQ branch pin (`ethlambda_crypto::stark::sidecar::LEANVM_REV`): +# a proof only verifies with a compatible verifier. +FROM rust:1.97-bookworm AS sidecar +ARG LEANVM_REV=283686575c4e81790591d2c6e3ed21c69c0e3bbe +RUN git clone -q https://github.com/leanEthereum/leanVM.git /leanVM \ + && cd /leanVM \ + && git checkout -q ${LEANVM_REV} \ + && cargo build --release -p leanvm-sidecar --features experimental_prover \ + && cp target/release/leanvm-sidecar /leanvm-sidecar + +FROM rust:1.97-bookworm AS chef +WORKDIR /app + +# Install cargo-chef and system dependencies +RUN cargo install cargo-chef +RUN apt-get update && apt-get -y upgrade && apt-get install -y libclang-dev pkg-config + +# Builds a cargo-chef plan +FROM chef AS planner +COPY --exclude=.git --exclude=target . . +RUN cargo chef prepare --recipe-path recipe.json + +FROM chef AS builder +ENV CARGO_PROFILE_RELEASE_LTO=false +ENV CARGO_PROFILE_RELEASE_CODEGEN_UNITS=16 +COPY --from=planner /app/recipe.json recipe.json + +# Include .cargo/ so [target.*.rustflags] (e.g. -Ctarget-cpu=x86-64-v3) applies to +# cargo-chef's dep build as well as the final cargo build. Without this the +# cached deps are compiled with default flags and cargo reuses them. +COPY .cargo/ .cargo/ + +# Build profile, release by default +ARG BUILD_PROFILE=release +ENV BUILD_PROFILE=$BUILD_PROFILE + +# Extra Cargo features +ARG FEATURES="" +ENV FEATURES=$FEATURES + +# Disable default features (e.g. for `shadow-integration`, which must drop the +# default jemalloc allocator). Set to "--no-default-features" to enable. +ARG NO_DEFAULT_FEATURES="" +ENV NO_DEFAULT_FEATURES=$NO_DEFAULT_FEATURES + +# Cargo --locked by default. The Shadow build injects a [patch] (see below) that +# is absent from the committed lockfile, so it must build unlocked: set LOCKED= . +ARG LOCKED="--locked" +ENV LOCKED=$LOCKED + +RUN cargo chef cook --profile $BUILD_PROFILE $NO_DEFAULT_FEATURES --features "$FEATURES" $LOCKED --recipe-path recipe.json + +# Build application +# Include .git so vergen-git2 can extract version info (branch, commit SHA) +COPY --exclude=target . . + +# Shadow builds inject the quinn-udp [patch] into the workspace manifest, since a +# Cargo patch cannot be feature-gated and is therefore not committed. Set SHADOW=1. +ARG SHADOW="" +RUN if [ -n "$SHADOW" ]; then cat shadow/cargo-patch.toml >> Cargo.toml; fi + +# Channel reported in the client version. vergen reads the branch from .git, but a +# release-candidate tag is checked out detached (it would report "HEAD"), so CI +# passes the tag's `rc.N` suffix here. Left empty, vergen reads .git as before. +ARG GIT_BRANCH="" + +RUN if [ -n "$GIT_BRANCH" ]; then export VERGEN_GIT_BRANCH="$GIT_BRANCH"; fi; \ + cargo build --profile $BUILD_PROFILE $NO_DEFAULT_FEATURES --features "$FEATURES" $LOCKED --bin ethlambda + +# ARG is not resolved in COPY so we have to hack around it by copying the +# binary to a temporary location +RUN cp /app/target/$BUILD_PROFILE/ethlambda /app/ethlambda + +# Use Ubuntu as the release image +FROM ubuntu AS runtime +WORKDIR /app + +LABEL org.opencontainers.image.source=https://github.com/lambdaclass/ethlambda +LABEL org.opencontainers.image.description="A minimalist and fast Beacon Chain and Lean Chain client written in Rust by LambdaClass" +LABEL org.opencontainers.image.licenses="MIT" + +ARG GIT_COMMIT=unknown +ARG GIT_BRANCH=unknown +ARG VERSION=dev + +LABEL org.opencontainers.image.revision=$GIT_COMMIT +LABEL org.opencontainers.image.ref.name=$GIT_BRANCH +LABEL org.opencontainers.image.version=$VERSION + +# Copy ethlambda over from the build stage +COPY --from=builder /app/ethlambda /usr/local/bin +COPY --from=sidecar /leanvm-sidecar /usr/local/bin/leanvm-sidecar +COPY --chmod=755 bin/ethlambda/pq/ethlambda-pq /usr/local/bin/ethlambda-pq + +# Copy licenses +COPY LICENSE ./ + +# 9000/tcp, 9000/udp - P2P networking (discv5: always on for beacon, --discovery.enable for node; --discovery.port) +# 9001/udp - libp2p QUIC connections +# 9001/tcp - libp2p TCP (noise + yamux) connections, the fallback transport +# 5052 - API RPC +# 5054 - Prometheus metrics +# +# The swarm binds both protocols on 9001 and the ENR advertises both, so a `tcp` +# entry with no mapping behind it is worse than none: every peer racing the +# address list burns a connect timeout on a black-holed port. +EXPOSE 9000/tcp 9000/udp 9001/udp 9001/tcp 5052 5054 +ENTRYPOINT ["/usr/local/bin/ethlambda"] diff --git a/tooling/pq-devnet/README.md b/tooling/pq-devnet/README.md new file mode 100644 index 000000000..5ead90875 --- /dev/null +++ b/tooling/pq-devnet/README.md @@ -0,0 +1,93 @@ +# PQ devnet + +A local kurtosis devnet on the consensus-specs `pq` fork (pinned at +`a4d16c5b`), active from genesis, with ethlambda next to Lighthouse's PQ +client. `network_params.yaml` runs an ethlambda beacon node that only +follows; `network_params_validators.yaml` gives it 16 of the 64 validators +behind ethlambda's own validator client. See +[`docs/beacon_pq.md`](../../docs/beacon_pq.md) for what ethlambda implements +of the fork. + +## What runs + +- **Genesis**: `ethpandaops/ethereum-genesis-generator:pq` (eth-beacon-genesis + `284025a0`). Fulu, gloas and PQ at epoch 0, mainnet preset, 12 s slots. + Every validator's leanXMSS key is `KeyGen(sha256("leanxmss-seed" || sk))` + and its VRF seed `sha256("vrf-seed" || sk)`, `sk` its EIP-2333 signing key; + ethereum-package ships the first 1000 keys of its default mnemonic. The + EIP-7002, EIP-7251 and EIP-8282 request contracts are inert stubs. +- **Lighthouse**: `ethdreamer/lighthouse:pq`, which proves attestation + aggregates with leanVM through a `leanvm-sidecar` (`lighthouse-pq-bn`). +- **ethlambda**: this repository's image with `leanvm-sidecar` added, started + through `ethlambda-pq` (`bin/ethlambda/pq/ethlambda-pq`), which starts the + sidecar and passes its socket as `--pq-prover-socket`. Without the sidecar + ethlambda checks proofs with the specification's reference encoding, which + rejects every Lighthouse block that carries attestations. +- **Execution**: geth (`ethereum/client-go:latest`). +- **Dora**: `ethpandaops/dora:pq`. + +## Run it + +```sh +# 1. The image, from the repository root (about 15 minutes). +docker build -f tooling/pq-devnet/Dockerfile -t ethlambda:pq-local . + +# 2. The package: lambdaclass/ethereum-package's `add-ethlambda` (the +# ethlambda launcher) merged with ethpandaops/ethereum-package's `pq` (the +# PQ genesis and Lighthouse wiring), plus the launcher patch here. +git clone https://github.com/lambdaclass/ethereum-package.git ethereum-package +cd ethereum-package +git checkout 8159202d71cca5a1de086d8152d512c5b95465ca # add-ethlambda +git fetch https://github.com/ethpandaops/ethereum-package.git \ + 7fafe056d997b07ec70656b16e54389fa02ec506 # pq +git merge --no-edit FETCH_HEAD +git apply /tooling/pq-devnet/ethlambda-launcher.patch +cd .. + +# 3. The devnet. kurtosis wants the package as an absolute path. +kurtosis run --enclave pq "$PWD/ethereum-package" \ + --args-file /tooling/pq-devnet/network_params.yaml +``` + +Lighthouse's validator client derives its validators' leanXMSS keys at +startup, about two minutes for 64 on an 11-core machine, so the first slots +are missed. Lighthouse needs a supernode or 128 validators for PeerDAS, which +the params give it. + +With `network_params_validators.yaml`, ethlambda's validator client derives +its 16 keys in under a minute. Lighthouse keeps 48 (75%), enough to finalize +alone, so a fault on ethlambda's side shows as missed duties rather than a +stalled chain. + +## Known limits + +- **Lighthouse does not serve PQ envelopes over req/resp.** Its `pq` branch + answers `execution_payload_envelopes_by_{range,root}` with a gloas-shaped + envelope under the PQ digest ("Pq payload envelope RPC decoding is not + implemented" in its codec), which ethlambda refuses: it cannot decode as + PQ's container and carries no leanXMSS signature. So an ethlambda node that + falls behind cannot catch up from Lighthouse alone, since every FULL parent's + envelope is fetched that way; it can from another ethlambda node. Gossip is + unaffected. +- **Memory.** Every `leanvm-sidecar` warms up by proving twice and stays + large. In Docker Desktop's default 8 GB VM, starting a second sidecar next + to Lighthouse's has had the kernel kill Lighthouse's. Set `LEANVM_THREADS` + (through `cl_extra_env_vars`) on ethlambda nodes that only verify. + +## What to check + +- Both beacon APIs report the same head root, and ethlambda's + `lean_beacon_gossip_validation_total` is all `accept`: blocks, envelopes, + aggregates (their leanVM proofs checked through the sidecar), attestations, + payload attestations and proposer preferences. +- `lean_beacon_aggregate_total{outcome="applied"}` rises: PQ aggregates reach + fork choice. +- ethlambda's execution client answers `VALID` for every envelope + (`Execution payload envelope verified ... status=Valid`). +- Finality matches Lighthouse's. +- With ethlambda validators: every block ethlambda's validator client + publishes (`Published a PQ block`) is canonical on Lighthouse, the ones + after the first few carry attestations (`Packed PQ attestations`), and + Lighthouse's `gossipsub_rejected_messages_per_topic_total` stays empty for + ethlambda's attestations, PTC votes, blocks, envelopes and aggregates + (`Proved a PQ aggregate`, `Published PQ aggregates`). diff --git a/tooling/pq-devnet/ethlambda-launcher.patch b/tooling/pq-devnet/ethlambda-launcher.patch new file mode 100644 index 000000000..e3b1cfcdd --- /dev/null +++ b/tooling/pq-devnet/ethlambda-launcher.patch @@ -0,0 +1,17 @@ +diff --git a/src/cl/ethlambda/ethlambda_launcher.star b/src/cl/ethlambda/ethlambda_launcher.star +index 477e74f..158d344 100644 +--- a/src/cl/ethlambda/ethlambda_launcher.star ++++ b/src/cl/ethlambda/ethlambda_launcher.star +@@ -215,6 +215,12 @@ def get_beacon_config( + ) + ) + ++ # A network on the consensus-specs pq fork needs the leanVM prover sidecar ++ # its other clients prove with: `ethlambda-pq` starts one and passes its ++ # socket, as lighthouse-pq-bn does for Lighthouse. ++ if network_params.pq_fork_epoch != constants.FAR_FUTURE_EPOCH: ++ cmd[0] = "ethlambda-pq" ++ + cmd_str = " && ".join(setup_cmds + ["exec " + " ".join(cmd)]) + + env_vars = {LOG_LEVEL_ENV_VAR: log_level} diff --git a/tooling/pq-devnet/network_params.yaml b/tooling/pq-devnet/network_params.yaml new file mode 100644 index 000000000..835436b56 --- /dev/null +++ b/tooling/pq-devnet/network_params.yaml @@ -0,0 +1,21 @@ +# A PQ-at-genesis devnet: Lighthouse (pq) proposes and validates, the ethlambda +# beacon node follows. See README.md. +participants: + - el_type: geth + cl_type: lighthouse + cl_image: ethdreamer/lighthouse:pq + vc_image: ethdreamer/lighthouse:pq + validator_count: 64 + supernode: true + - el_type: geth + cl_type: ethlambda + cl_image: ethlambda:pq-local + validator_count: 0 + cl_log_level: info +network_params: + preset: mainnet + fulu_fork_epoch: 0 + gloas_fork_epoch: 0 + pq_fork_epoch: 0 +additional_services: + - dora diff --git a/tooling/pq-devnet/network_params_validators.yaml b/tooling/pq-devnet/network_params_validators.yaml new file mode 100644 index 000000000..6dd924000 --- /dev/null +++ b/tooling/pq-devnet/network_params_validators.yaml @@ -0,0 +1,28 @@ +# A PQ-at-genesis devnet where ethlambda validates too: Lighthouse (pq) with 48 +# validators (75%, enough to finalize alone), an ethlambda beacon node and +# validator client with 16. See README.md. +participants: + - el_type: geth + cl_type: lighthouse + cl_image: ethdreamer/lighthouse:pq + vc_image: ethdreamer/lighthouse:pq + validator_count: 48 + supernode: true + - el_type: geth + cl_type: ethlambda + cl_image: ethlambda:pq-local + # The sidecar proves aggregates and merges block proofs; two threads keep + # it next to Lighthouse's in Docker Desktop's default 8 GB VM. + cl_extra_env_vars: + LEANVM_THREADS: "2" + vc_type: ethlambda + vc_image: ethlambda:pq-local + vc_extra_params: ["--init-slashing-protection"] + validator_count: 16 +network_params: + preset: mainnet + fulu_fork_epoch: 0 + gloas_fork_epoch: 0 + pq_fork_epoch: 0 +additional_services: + - dora