Skip to content

perf(beacon): verify attestations against committee key sums - #742

Open
MegaRedHand wants to merge 79 commits into
mainfrom
perf/committee-key-sums
Open

MegaRedHand wants to merge 79 commits into
mainfrom
perf/committee-key-sums

Conversation

@MegaRedHand

@MegaRedHand MegaRedHand commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Stack 3/4 of the BLS verification work: (a) batched key sums -> (b) per-block signature batch -> (c) committee key sums -> (d) index-keyed pubkey cache. Based on (b).

An attestation aggregate usually covers most of its committee, and its verification key is the sum of the attesters' keys. This PR precomputes each committee's key sum once per shuffling, off the import path, and derives an attestation's key as committee_sum - sum(non-attesters) when more than half the committee attested, and as sum(attesters) otherwise. The group element is the same either way, so the pairing check is unchanged; only the number of point additions differs (absentees instead of attesters).

What it changes on top of the signature batch

  • ethlambda-crypto
    • bls/committee_sums.rs: AggregatePubkey (sum, minus, plus, verify) and CommitteeKeySums. Negation uses blst's safe sub_aggregate, so there is no unsafe.
    • SignatureSet::from_aggregate(key, message, signature): builds a set from an already-summed key (None when the signature fails sig_validate; one affine conversion of the key). AggregatePubkey::verify delegates to it, so both checks share one path, and the batch's identity-key guard covers it.
    • One projective sum helper in bls.rs (sum_validated_projective); the batched-key-sum PR's sum_validated_points is now a thin wrapper that converts to affine once. Committee sums add and subtract projective points, so building them on the affine helper would cost an extra field inversion per committee, about 512 per block.
  • ethlambda-storage: each resident shuffling gets a second OnceLock<Arc<CommitteeKeySums>> beside its committees, so eviction and head pinning stay in one place and the sums die with their shuffling. resident_key_sums is lookup only; get_or_init_key_sums builds into a resident entry and never holds the entry-list lock while building.
  • ethlambda-state-transition
    • CommitteeCacheExt::key_sums (lookup only: import never builds) and warm_key_sums (one iter_validators() pass, then rayon across committees). A committee with an invalid member key gets no sum and falls back to the direct path.
    • attesters_aggregate_pubkey (helpers/electra.rs), with the majority rule, usable by gloas through the same is_set closure as attesting_indices_from_committee_bits. Committees derived from their sums are added one by one; the attesters of every committee summed directly go into one sum. With cold sums that is the single sum over every attester that perf(beacon): verify a block's signatures in one random-weight batch #738 makes, at its cost: one sum per committee would pay blst's pool dispatch once per committee for a few hundred keys each, which made the cold path slower than perf(beacon): verify a block's signatures in one random-weight batch #738's.
    • Electra, fulu and gloas process_attestation build their SignatureSet from it (electra_attestation_signature_set, gloas_attestation_signature_set). The indexed attestation is still built, for the participation flags, and the same structural checks are made on it (non-empty, sorted and unique; gloas's MAX_VALIDATORS_PER_SLOT bound). Same signing root, same push label. Phase0 through deneb are unchanged.
  • ethlambda-blockchain (epoch_precompute.rs): the precomputed state is sent to the actor first, then the sums are warmed in the same blocking task, so the epoch's first block never waits for them. The task warms the epoch's sums and the previous epoch's (for the attestations its blocks carry from it), so a restart mid-epoch or a boundary whose worker did not run is covered at the next one. When the boundary state is already cached (a gloas bid for the epoch's first slot advances it), no worker runs, so a warm-only task runs from that state instead. Warming is skipped before electra, whose attestations never read the sums. New histogram lean_beacon_committee_key_sums_warm_seconds (in docs/metrics.md), recorded only when sums are actually built and covering the sums alone.

Soundness of keying sums by ShufflingKey

The sums are filed under (epoch, decision root), so two states with one key must agree on each member's public key as well as on who the members are. Checked against this repo's spec code:

  • The registry is append-only (process_deposit, electra's pending-deposit path); a validator's index and pubkey are fixed at append and no code assigns pubkey afterwards.
  • A new validator is appended with activation_epoch = FAR_FUTURE_EPOCH. Only process_registry_updates (epoch processing at epoch N) assigns one, compute_activation_exit_epoch(N) = N + 1 + MAX_SEED_LOOKAHEAD.
  • A member of epoch E's committees is active at E, so it was activated at some N <= E - 1 - MAX_SEED_LOOKAHEAD, and appended no later. That precedes the end of E - 2, so the decision-root block already has it at that index with that key, and every descendant inherits it.

The argument is in the CommitteeCacheExt::key_sums doc comment.

Measured

On a heavily loaded M3, blst 0.3.16, single-threaded, block shape of 11 signature sets and 33k signers:

Phase per block Ratio
key aggregation, sequential (before the batched sums) 1.4-2.5x the pairing
key aggregation, 5% missed only 20-40x less (realistic ~10x)

The single-threaded ratio overstates the wall-time gain on a multi-core host, where #738's one large sum already runs on blst's pool. In CPU time, profiling a Hoodi block replay on a Ryzen 9700X (the whole stack, warm sums) puts key summation at 12.3 ms CPU per block before the stack and 1.5 ms after.

Behavior notes

  • Accepted and rejected inputs are unchanged. Cold sums, an evicted shuffling or an invalid member key all take the direct sum of the attesters; a bad signature is rejected under the same label, warm or cold.
  • The sync aggregate is not covered: its keys come from current_sync_committee.pubkeys, which has no committee sums.
  • Warming builds the epoch's committees if they are not resident, and copies every validator's 48-byte key once: roughly 115 MB transient at 2.4M validators, on the blocking worker. It runs on rayon's global pool (and blst's, for each committee's sum), so it shares cores with import; the first warm-up after a start also validates every active key.
  • Import never builds sums. With cold sums (while syncing, when no precompute runs, or before the first warm-up after a start) attestations take the direct path, at perf(beacon): verify a block's signatures in one random-weight batch #738's cost.

Tests

New tests:

  • crypto: compressed-byte equality of sum/minus/plus against blst's direct sum (including duplicate keys, and minus over 40 random masks of a 64-key committee); verify against bls::fast_aggregate_verify with real signatures; the identity aggregate rejected; the empty and invalid-key cases.
  • storage: key_sums never builds or inserts; sums are built once into a resident shuffling only; sums are evicted with their shuffling.
  • state-transition: random masks (empty, one bit, below half, exactly half, half plus one, all, random) over one and two committees, cold and warm, checked by verifying the aggregate of the attesters' real signatures; invalid-member fallback; a block-level test with warm sums where a good attestation gives the same post-state root warm and cold and a bad signature is rejected with the attestation label either way.
  • spec runners: every electra-or-later operations/*/attestation case and every block run (sanity, finality, random) now runs twice, cold and after warm_key_sums (previous, current and next epoch of the state each block applies to); both runs must reach the same verdict and post-state.

Run (--profile release-fast):

  • cargo test -p ethlambda-crypto --lib: 35 passed, 14 ignored
  • cargo test -p ethlambda-storage --lib: 213 passed, 1 ignored
  • cargo test -p ethlambda-state-transition --lib --features beacon-spec-tests: 658 passed
  • beacon_spec_tests -- /attestation/: 356 passed; -- sanity/: 595 passed; -- random/: 1969 passed, 175 ignored
  • full beacon_spec_tests: 7493 passed, 977 ignored
  • cargo fmt --all; clippy -D warnings --all-targets on crypto, storage, state-transition, blockchain: clean
  • ethlambda-blockchain --lib (before the merge): 291 passed, 43 ignored

On this branch after its review fixes: crypto --lib 40 passed; storage 213; state-transition --lib 658; blockchain 291; beacon_spec_tests operations/ 2141, sanity/ 595, finality/ 41, fork_choice/ 206, random/ 1969 (175 ignored), transition/ 172, sync/ 6 passed.

Rerun after the review fixes, on the stack's tip (all four PRs, --profile release-fast): ethlambda-crypto --lib 47 passed, 14 ignored; ethlambda-state-transition --lib --features beacon-spec-tests (with the BLS fixture vectors) 658 passed; storage 213, blockchain 291, p2p 362, rpc 335 passed; full beacon_spec_tests 7493 passed, 977 ignored, 0 failed; make lint clean, also on a trial merge of current main.

… addition

A committee's key sum is the largest cost left in verifying an attestation
once its keys are cached. Folding the keys one by one through
AggregatePublicKey::aggregate pays a projective addition per key;
blst_p1s_add (reached through the safe MultiPoint::add) shares one field
inversion across the batch and runs 1.6-2.3x faster in a single-threaded
measurement. From 384 points up blst also spreads the work over its own
thread pool.

fast_aggregate_verify and eth_aggregate_pubkeys now sum through one private
helper, sum_validated_points, which guards the empty slice (MultiPoint::add
indexes the first element). The helper returns the same group element as the
sequential sum, including for repeated keys (sync committees are drawn with
replacement, so P + P occurs), which the new tests check at sizes around the
384-point threshold. Accepted and rejected inputs are unchanged.
An attestation aggregate usually covers most of its committee, so its
verification key is cheaper to derive as the committee's key sum minus the
non-attesters than as the sum of the attesters. The result is the same group
element, so the pairing check is unchanged; only the number of point
additions drops.

Each resident shuffling in the committee cache gets a second slot for its
committee key sums, so eviction and head pinning stay in one place. Block
import only looks the sums up: building them costs a pass over the whole
active set, which is more than they save for committees attested once. The
epoch precompute worker builds them instead, after it has handed the
advanced state to the actor, so the epoch's first block never waits.

Sums are keyed by the shuffling key. That is sound because the registry is
append-only, a pubkey is never reassigned, and a validator active at epoch E
was activated at least MAX_SEED_LOOKAHEAD + 1 epochs earlier, so it is
already in the history the decision root commits to.

Nothing in block processing calls the new key builder yet.
Block import paid one pairing per signature: the proposer's, the RANDAO
reveal, every attestation, the sync aggregate and each slashing, exit and
BLS change. A pairing is dominated by its final exponentiation, which a
batch pays once, so a block with a dozen signature sets checks in a
fraction of the time.

ethlambda-crypto gains SignatureSet and verify_signature_sets: resolved
checks combined with unpredictable non-zero 64-bit weights from the
OS-seeded CSPRNG (without them two invalid signatures can cancel). The
state transition builds each set where the specification verifies it and
pushes it into a SignatureBatch that the block driver verifies once
processing ends. A failing batch re-checks sets one by one so the error
still names the first bad signature with its usual label.

Signatures whose failure rejects the block are deferred; deposit
signatures, whose failure only skips the deposit, stay immediate, as does
epoch processing. process_block keeps its meaning and signature, apply_block
adds the proposer's signature to the same batch. The operation packer in
block production uses an immediate batch so a rejected candidate still
writes nothing. The attestation validity helpers are split into a set
builder plus the existing bool function, so gossip and fork choice are
untouched.
SignatureSet::fast_aggregate now sums its keys with part a's batched
sum_validated_points instead of a sequential AggregatePublicKey::aggregate,
and the identity-key check in verify_signature_sets documents that blst
rejects the same input.
Brings in the per-block signature batch and the batched key sums.

The two parts each added a point-sum helper. The batched one now lives in
bls.rs as a single projective sum, with sum_validated_points a thin wrapper
that converts to affine once, so committee key sums (which add and subtract
projective points) do not pay a field inversion per conversion. The
committee-sum module uses that helper instead of its own, and
AggregatePubkey::verify shares its pairing path with the new
SignatureSet::from_aggregate.
Electra, fulu and gloas process_attestation now build the attestation's
signature set from attesters_aggregate_pubkey, which derives the key as the
warmed committee sum minus the non-attesters when most of the committee
attested, and sums the attesters otherwise. The key is the same group
element either way, so the verdict is unchanged; only the point additions
differ. Import still never builds the sums: with them cold, the builder
sums the attesters directly.

The indexed attestation is still built for the participation flags, and the
structural checks on it (non-empty, sorted and unique, gloas's length bound)
and the push label are the ones the indexed path made.

The spec runners use a fresh committee cache, which only ever runs the cold
branch, so electra-and-later attestation cases and every block run now run
a second time with the sums warmed. Both runs must reach the same verdict
and post-state.
The replay drives import_block directly with no tick loop, so the epoch
precompute worker never runs and committee key sums stay cold; the
committee-sum saving is invisible in a replay although a live node gets
it. The flag warms each epoch's sums before its first block, outside
every measured span, so the warm-up is in no phase and no wall_seconds.
It is recorded in the report params and the human header.
@MegaRedHand
MegaRedHand added this pull request to stack #744 October 30, 2026 16:08
MegaRedHand added a commit that referenced this pull request Oct 30, 2026
perf(beacon): committee key sums, warmed off the import path

An attestation aggregate usually covers most of its committee, so its
verification key is cheaper to derive as the committee's key sum minus the
non-attesters than as the sum of the attesters. The result is the same group
element, so the pairing check is unchanged; only the number of point
additions drops.

Each resident shuffling in the committee cache gets a second slot for its
committee key sums, so eviction and head pinning stay in one place. Block
import only looks the sums up: building them costs a pass over the whole
active set, which is more than they save for committees attested once. The
epoch precompute worker builds them instead, after it has handed the
advanced state to the actor, so the epoch's first block never waits.

Sums are keyed by the shuffling key. That is sound because the registry is
append-only, a pubkey is never reassigned, and a validator active at epoch E
was activated at least MAX_SEED_LOOKAHEAD + 1 epochs earlier, so it is
already in the history the decision root commits to.

Nothing in block processing calls the new key builder yet.

perf(beacon): verify attestations against committee key sums

Electra, fulu and gloas process_attestation now build the attestation's
signature set from attesters_aggregate_pubkey, which derives the key as the
warmed committee sum minus the non-attesters when most of the committee
attested, and sums the attesters otherwise. The key is the same group
element either way, so the verdict is unchanged; only the point additions
differ. Import still never builds the sums: with them cold, the builder
sums the attesters directly.

The indexed attestation is still built for the participation flags, and the
structural checks on it (non-empty, sorted and unique, gloas's length bound)
and the push label are the ones the indexed path made.

The spec runners use a fresh committee cache, which only ever runs the cold
branch, so electra-and-later attestation cases and every block run now run
a second time with the sums warmed. Both runs must reach the same verdict
and post-state.

feat(benchmark): add --warm-committee-sums to the import replay

The replay drives import_block directly with no tick loop, so the epoch
precompute worker never runs and committee key sums stay cold; the
committee-sum saving is invisible in a replay although a live node gets
it. The flag warms each epoch's sums before its first block, outside
every measured span, so the warm-up is in no phase and no wall_seconds.
It is recorded in the report params and the human header.

Cherry-picked from perf/committee-key-sums at 508c43e.
A key and its negation are both valid, so a registry can hold both and an
aggregate can name both. Pin that the batched sum still matches the sequential
one when keys cancel, that such a pair aggregates to the identity, and that the
identity never verifies. Also state the CPU-count condition on blst's pool.
…tch unverified

A block fetched by request reaches the state transition without gossip's
precheck, and with the proposer's signature in the body's batch a block its
proposer never signed was only rejected after the whole body ran. Check that
one signature on its own again, before the body, as before the batch existed.

A deferring SignatureBatch could also be built and dropped without its sets
ever being checked, turning any function that takes one into "valid except for
its signatures". Outside the crate, a deferring batch now only comes from
SignatureBatch::deferred, which verifies once its closure returns.

Also cover an infinity signature inside a batch and keys that cancel.
# Conflicts:
#	crates/blockchain/state_transition/tests/beacon_spec/operations.rs
…arm missed epochs

With the sums cold, every committee was summed on its own, and each sum of a
few hundred keys paid blst's pool dispatch separately where the plain path
made one sum over all attesters: a cold multi-committee aggregate got slower
than before this change. Every directly summed committee now goes into one sum,
so the cold path is that plain sum again.

Warming only ran for the epoch a precompute worker advanced into. The worker
now also warms the previous epoch (after a restart mid-epoch, or a boundary
whose worker did not run), and a boundary state someone else already cached
(a gloas bid for the epoch's first slot) gets a warm-only task instead of no
warm-up at all. Pre-electra states skip it, since nothing reads the sums there,
and the warm-up histogram now times the sums alone, only when they are built.
@MegaRedHand
MegaRedHand marked this pull request as ready for review October 30, 2026 21:43
@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review

Review of PR 742: warm committee key sums and use them in electra and gloas attestation verification

I read the diff but did not build it or run the tests, so everything below comes from reading the code.

Overall: the design is sound and carefully argued. Warming happens off the import path. The cold path stays a single flat sum. The sum-minus-absentees result is the same group element as summing the attesters, so the pairing check can't tell them apart. The correctness argument for sharing sums on the ShufflingKey is stated in full on key_sums. I found no consensus-affecting bugs.

Checked and fine

  • attesters_aggregate_pubkey (helpers/electra.rs):
    • Empty attester sets return None through attestation_signature_set's ??, so they are rejected without a pairing, as before.
    • A committee with None in its sum slot falls back to the direct path. A bad key then only matters if that member actually attested. a_committee_with_an_invalid_member_key_falls_back covers this.
    • The strict-majority gate (attester_count * 2 > committee.len()) and the committee_offset accounting match attesting_indices_from_committee_bits.
  • attestation_signature_set keeps the sorted-and-unique and non-empty checks from the indexed path, and it uses the same domain and signing root.
  • The gloas wrapper keeps the MAX_VALIDATORS_PER_SLOT bound ahead of the shared builder.
  • The tests are good. The randomized masks compare cold and warm caches against an independent aggregate verify. The post-state and verdict equivalence test also covers a bad signature.

Minor points

  1. epoch_precompute.rs, the new decide(...)-with-already_cached: false branch: it can spawn a spawn_blocking warm task on every head or timer trigger while the epoch's state stays cached and the sums are still being built. The resident check makes a task cheap once the sums are filed. Until then, several tasks may overlap and each derive committees and walk the registry. This is only safe if get_or_init_key_sums blocks other callers (OnceCell-like). Please confirm that, or gate the branch the way in_flight gates the worker.
  2. warm_key_sums calls self.committees(state, epoch) outside the get_or_init_key_sums closure. It also builds the full committees for an epoch that may never be attested, and it does so on every call that isn't resident. That is fine for background use. A short doc note that the committee derivation counts as warming cost would help, since the metric deliberately excludes it.
  3. build_committee_key_sums: pubkeys copies every validator's 48-byte key, about 48 MB at roughly 1M validators. It is transient and off the hot path, and par_iter already runs inside spawn_blocking. This is acceptable. Do check that rayon's global pool isn't also the one the BLS and leanVM work relies on, so warming can't starve block import.
  4. replay.rs, warm_committee_sums: when previous_epoch is Some(p) and epoch > p + 1 (a gap in the corpus), only epoch is warmed, not epoch - 1. The live worker does warm both. That is a small benchmark-fidelity difference, and one the first-block case already handles.
  5. tests/beacon_spec/mod.rs, warm_key_sums_around: it warms current + 1, which shuffling_key may legitimately fail to key near genesis. That is harmless because warm_key_sums returns early on None.

Approve, with point 1 worth confirming before merge.


Automated review by Claude (Anthropic) · sonnet · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Codex Code Review

Overall this looks solid. I didn’t spot a consensus/safety bug in the changed logic, and the fallback behavior is conservative: when precomputed sums are missing or unusable, the code falls back to direct key summation rather than accepting anything it shouldn’t.

  • crates/blockchain/src/epoch_precompute.rs:199
    Minor maintainability nit: the branch condition recomputes decide() with already_cached: false purely to distinguish “cached state prevented worker start” from other None cases. That’s correct, but a bit opaque. Consider factoring this into a small helper / named boolean so the “warm only when the worker would otherwise have run” rule is easier to audit later.

  • crates/blockchain/src/epoch_precompute.rs:211
    Potential resource issue, not correctness: tokio::task::spawn_blocking is fired and ignored for cache-warm-only work. On a busy node with repeated cached-state hits around epoch boundaries, this can queue redundant warmers. Because warm_key_sums() is idempotent this should stay correct, but you may still want lightweight deduplication or an in-flight guard for warm-only jobs to avoid avoidable blocking-pool pressure.

  • crates/blockchain/state_transition/src/beacon/helpers/electra.rs:603
    Nice choice to gate the subtraction path on attester_count * 2 > committee.len(). That avoids regressing the cold path and preserves a predictable upper bound on additions. Also good that any invalid/missing pubkey in the absentee list returns Ok(None) and therefore fails verification rather than producing a partial aggregate.

  • crates/blockchain/state_transition/src/beacon/helpers/accessors.rs:369
    Good consensus hygiene here: warming is explicitly disabled pre-Electra, keyed by shuffling_key, and lookup-only on import. That keeps this optimization out of phase0/altair/deneb semantics and avoids accidental cache construction on the hot path.

  • crates/common/crypto/src/bls/committee_sums.rs:53
    AggregatePubkey::minus() looks sound from a memory-safety and crypto-API perspective: validated points only, no unsafe, and identity results are deferred to verification where they are rejected. The tests around duplicates and random masks are especially valuable here.

Looks good to me aside from the small blocking-pool/perf concern in epoch_precompute.rs:211.


Automated review by OpenAI Codex · gpt-5.4 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Kimi Code Review

I'll review this PR which introduces committee key sum caching to optimize attestation verification in Electra and later forks. Let me analyze the changes systematically.

Overall Assessment

This is a well-structured optimization PR that caches BLS public key sums per committee to speed up attestation verification. The core insight is sound: for committees with high attestation rates, subtracting absentee keys from a precomputed sum is cheaper than summing all attesters from scratch.


Critical Issues

1. Potential Race Condition in warm_committee_sums — bin/ethlambda/src/benchmark/import/replay.rs:217-248

fn warm_committee_sums(
    store: &Store,           // parameter name says `store`
    block: &SignedBeaconBlock,
    previous_epoch: Option<u64>,
) -> eyre::Result<()> {

Bug: The function signature declares store: &Store but the call sites pass &clock (a SystemClock). The body uses store.get_state(...) and store.config() etc., but SystemClock doesn't have these methods. This won't compile.

Lines 133-136 and 150-153:

if options.warm_committee_sums {
    warm_committee_sums(&clock, &block, previous_epoch)?;  // `clock` is SystemClock, not Store
}

Fix: Change call sites to pass server.store() or equivalent, and verify the parameter type matches.


2. Typo in Comment — crates/blockchain/src/epoch_precompute.rs:200

// Someone else advanced the state already (a gloas bid for the
// epoch's first slot does), so no worker runs: warm the sums from

"gloas bid" → "gloas block" or "block" (context suggests this refers to a block proposal, not a bid).


3. Logic Error in decide Re-evaluation — crates/blockchain/src/epoch_precompute.rs:198-210

if let Some(state) = cached
    && decide(&DecisionInputs {
        already_cached: false,
        ..inputs
    })
    .is_some()

Issue: The re-evaluation of decide with already_cached: false will always return Some(key) (assuming other conditions hold), because the original decide returned None precisely because already_cached was true. This means the branch always fires when cached.is_some().

However, the comment says "Someone else advanced the state already... so no worker runs: warm the sums from that state instead". The logic is correct but the decide re-call is misleading—it could just check !inputs.already_cached directly or use a clearer condition. The current code works but is confusing.

More critically: The tokio::task::spawn_blocking call at line 206 doesn't use the state from the cached variable correctly. Looking more carefully—the cached is Option<Arc<BeaconState>> from cached_state, and state is bound by if let Some(state) = cached. This seems correct.

Actually, re-reading: the issue is that inputs is consumed by the first decide call (via let Some(key) = decide(&inputs)), but inputs is used again in the if let block. This works because DecisionInputs derives Copy or is reborrowed—need to verify. The ..inputs suggests it's not moved. This seems okay.


Security Considerations

4. BLS Identity Key Handling — crates/common/crypto/src/bls/committee_sums.rs:85-89

pub fn verify(&self, message: Root, signature: &BlsSignature) -> bool {
    SignatureSet::from_aggregate(*self, message, signature).is_some_and(|set| set.verify())
}

The identity element (all keys subtracted away) is handled by SignatureSet::from_aggregate converting to PublicKey and then SignatureSet::verify rejecting it. This matches the comment and is consistent with existing behavior. Verified correct.

5. Invalid Key Fallback — crates/blockchain/state_transition/src/beacon/helpers/electra.rs:598-609

let sum = key_sums.as_ref().and_then(|sums| sums.get(number as usize));
match sum {
    Some(sum) if attester_count * 2 > committee.len() => {
        // subtract absentees
    }
    _ => {
        // direct sum of attesters
    }
}

Correct: When sum is None (invalid member key), falls back to direct summation. The attester_count * 2 > committee.len() majority rule properly guards the subtraction path—subtraction only used when minority absent.

However, line 600: attester_count * 2 > committee.len() can overflow if committee.len() is large and attester_count * 2 exceeds usize::MAX. For mainnet parameters this is impossible (committees are bounded), but the code should use checked_mul or saturating_mul for defense-in-depth, or document the safety assumption.


Performance Issues

6. Unnecessary Clone in build_committee_key_sums — crates/blockchain/state_transition/src/beacon/helpers/accessors.rs:404

let pubkeys: Vec<BlsPubkey> = state.iter_validators().map(|v| v.pubkey).collect();

This clones every validator's 48-byte public key into a flat vector. For 1M validators, that's ~48MB allocation per epoch warmed. The pubkeys vector is then indexed by ValidatorIndex.

Consider: Using state.validator(index) directly in the parallel map, or caching this differently. However, the comment notes state.validator(i) is a tree descent, so the flat copy is a trade-off. Given this runs off the import path, acceptable, but monitor memory pressure.

7. Double Hashing in warm_key_sums Test Path — crates/blockchain/state_transition/tests/beacon_spec/operations.rs:706-716

let warm = apply(&handler, case, &mut warm_state, &config, &committees);
if warm != outcome {

apply calls run which calls electra_stf::process_attestation which builds SignatureBatch and verifies. The comparison warm != outcome compares Result types. For the success case, this is fine. For error cases, this compares String error messages, which may differ in formatting between runs. Low risk but could cause flaky tests if error formatting changes.


Rust Best Practices

8. unwrap() in Test Helper — crates/blockchain/state_transition/src/beacon/helpers/electra.rs:701

let key = attesters_aggregate_pubkey(
    &state,
    slot,
    &bits,
    |bit| mask.get(bit).copied().unwrap_or(false),
    cache,
)
.unwrap();

In test code, acceptable. But unwrap() after ? in production paths should be audited. The production path attestation_signature_set uses ok()?? which is correct.

9. #[cfg(test)] Method on Public Struct — crates/common/crypto/src/bls/committee_sums.rs:97-100

#[cfg(test)]
fn to_bytes(self) -> [u8; 48] {
    self.0.to_public_key().to_bytes()
}

Good practice to keep test-only helpers gated.

10. Missing Debug Implementation — crates/storage/src/committee_cache.rs:126-130

#[derive(Debug, Default)]
struct Resident {
    committees: OnceLock<Arc<EpochCommittees>>,
    key_sums: OnceLock<Arc<CommitteeKeySums>>,
}

OnceLock implements Debug showing its state. Good.


Consensus Correctness

11. Committee Number Indexing — crates/blockchain/state_transition/src/beacon/helpers/accessors.rs:420-421

let first_slot = compute_start_slot_at_epoch(epoch);
// ...
.flat_map(|offset| (0..per_slot).map(move |index| (first_slot + offset, index)))

The committee number formula slot_offset * committees_per_slot + committee_index must match committee_number() in electra.rs. Verified: The flat_map produces exactly this ordering. Correct.

12. Shuffling Key Stability Proof — crates/blockchain/state_transition/src/beacon/helpers/accessors.rs:288-320

The extensive comment proving why key sums can be shared on shuffling key is excellent consensus documentation. The three bullet points cover:

  • Append-only registry with fixed pubkeys
  • Activation epoch lookahead ensuring registry membership at decision block
  • Inheritance through state descent

This is critical consensus infrastructure and the proof appears correct. I'd suggest moving this to a standalone .md document or module-level doc comment for discoverability.


Minor Issues

13. Inconsistent Parameter Naming — bin/ethlambda/src/benchmark/import/replay.rs

fn warm_committee_sums(
    store: &Store,  // actually called with `&clock` at lines 133, 150

As noted in Item 1, the parameter name doesn't match usage.

14. Unused Import — crates/blockchain/state_transition/src/beacon/helpers/electra.rs

use crate::beacon::bls::{AggregatePubkey, SignatureSet};

SignatureSet is used in attestation_signature_set, but verify this is the first import of AggregatePubkey in this module—it appears newly added.

15. Metrics Bucket Choice — crates/blockchain/state_transition/src/metrics.rs:151

vec![0.05, 0.1, 0.25, 0.5, 1.0, 2.0, 4.0, 8.0]

For key sum warming with parallelization over committees, sub-100ms is expected for typical validator sets. The 8.0 upper bound may be insufficient for worst-case (full registry, single-threaded fallback). Consider adding 16.0.


Documentation

16. docs/benchmarking.md — Good Addition

The --warm-committee-sums documentation is clear about:

  • What it reproduces (only sum warming, not state advance)
  • When it runs (epoch boundaries)
  • That it's outside measured spans

Approved.


Summary

Priority Item File Line
Critical Wrong parameter type in warm_committee_sums calls replay.rs 133, 150
High Overflow risk in majority check electra.rs 600
Medium Typo in comment epoch_precompute.rs 200
Low Metrics bucket upper bound metrics.rs 151
Low decide re-evaluation clarity epoch_precompute.rs 198-210

The core cryptography and consensus logic is well-designed and correct. The main blocker is the compilation error in replay.rs.


Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt

MegaRedHand added a commit that referenced this pull request Oct 30, 2026
…review fixes

perf(beacon): keep cold committee sums at the plain path's cost and warm missed epochs

With the sums cold, every committee was summed on its own, and each sum of a
few hundred keys paid blst's pool dispatch separately where the plain path
made one sum over all attesters: a cold multi-committee aggregate got slower
than before this change. Every directly summed committee now goes into one sum,
so the cold path is that plain sum again.

Warming only ran for the epoch a precompute worker advanced into. The worker
now also warms the previous epoch (after a restart mid-epoch, or a boundary
whose worker did not run), and a boundary state someone else already cached
(a gloas bid for the epoch's first slot) gets a warm-only task instead of no
warm-up at all. Pre-electra states skip it, since nothing reads the sums there,
and the warm-up histogram now times the sums alone, only when they are built.

Updates nightly's #742 from perf/committee-key-sums at 508c43e to 562b369.
@MegaRedHand

Copy link
Copy Markdown
Collaborator Author

Block import benchmark (offline replay of mainnet blocks, steady state)

leg STF per block vs #738 (parent) vs main
main + #737 + #738 145.9 ms -7.5%
+ #742, sums warmed (--warm-committee-sums) 144.1 ms -1.8 ms, CI [-2.4, -1.3] -8.6%
+ #742, sums cold (no flag) 155.6 ms +9.7 ms, CI [+9.1, +10.3] -1.3%

Open question: is the cold path expected to be slower than #738's direct sum? If #742 merges together with #743 the gap does not show; if #742 lands alone, it is worth a look.

Method

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

Overall, this is a very well-designed and thoroughly tested PR. The invariant justifying why committee key sums can share ShufflingKey, the fallback handling for invalid member pubkeys, and running the Electra+ spec fixtures against both cold and warm caches are all solid.

A couple of minor observations:

  1. Redundant / overlapping spawn_blocking tasks in maybe_start_epoch_precompute (crates/blockchain/src/epoch_precompute.rs:215-261)
    In the else branch of let Some(key) = decide(&inputs), a warm-only spawn_blocking task is spawned whenever cached is Some, only_the_cache_stopped_the_worker is true, and key_sums_warming is false. Two situations can trigger unnecessary blocking tasks here:

    • In the normal precompute worker (lines 257–260), actor.send(EpochPrecomputed { key, result }) happens before warm_epoch_key_sums, and the worker does not set key_sums_warming. Once the actor processes EpochPrecomputed, epoch_precompute_in_flight is cleared and CheckpointState is cached while the worker thread is still executing warm_epoch_key_sums. If a timer or head trigger fires during that window, it will enter the else branch, see key_sums_warming == false, and spawn a second blocking thread that blocks on the OnceLock.
    • Even after warm_epoch_key_sums completes and ClearOnDrop resets key_sums_warming to false, any subsequent trigger during the precompute window will see cached.is_some() and key_sums_warming == false, spawning a spawn_blocking task just to find the sums already resident and return.

    Checking whether the sums still need warming on the actor thread before spawning (e.g. checking committee_cache.key_sums(&state, candidate_epoch).is_none() || committee_cache.key_sums(&state, candidate_epoch.saturating_sub(1)).is_none() when state.fork_name() >= ForkName::Electra), and having the main precompute worker also hold key_sums_warming while it warms the sums, would prevent both cases.

  2. Single-point fast path in sum_validated_projective (crates/common/crypto/src/bls.rs:295-300)
    When a committee has all but one member attesting (a common case on a healthy network), AggregatePubkey::minus calls sum_validated_projective with a 1-element slice. Matching [single] => Some(AggregatePublicKey::from_public_key(single)) (similar to sum_validated_points right below it) would avoid going through MultiPoint::add (blst_p1s_add and its scratch allocation) for a single point.


Automated review by Gemini · gemini-4-argon · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Kimi Code Review

No blocking correctness or security issues found. The cold/warm paths preserve attestation semantics, including invalid-key fallback, empty and identity aggregate rejection, sorted/unique index checks, and the gloas length bound. Key sums are also correctly tied to shuffling-cache eviction, and the warm/cold spec-test runs provide strong consensus-equivalence coverage.

Non-blocking performance note:

  • crates/blockchain/state_transition/src/beacon/helpers/electra.rs, around lines 590–603 (attesters_aggregate_pubkey): the import path allocates a Vec<bool> for every committee and then traverses it again. This overhead also applies when sums are cold. Consider counting attesters and later querying is_set directly during the absent/attester pass, avoiding the allocation. This is minor compared with the BLS work, but relevant given the PR’s optimization goal.

Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Grok Code Review

The warm-sum design is sound: sums stay off the import path, a miss falls back to the spec sum, and the electra/gloas spec runners comparing cold vs warm post-states are the right check. A few places can still reject a valid attestation or miss the warm-up the benchmark claims to measure.

  1. attesters_aggregate_pubkey in crates/blockchain/state_transition/src/beacon/helpers/electra.rs (~614) turns a minus failure into Ok(None), which attestation_signature_set treats as an invalid signature. The cold path never looks at absentee keys. If an absentee key fails validation, is out of range, or differs from the key that was summed, this node rejects a block other clients accept. Fall through to the direct attester sum instead (same as a None committee sum).

  2. Same function, attestation_signature_set: attesters_aggregate_pubkey(...).ok()?? collapses a real Err (committee() / committee_number()) into that same invalid-signature None. Those should stay state-transition errors. None should mean only empty, unsorted, or cryptographically invalid.

  3. key_sums_warming does not cover the path that actually warms. In crates/blockchain/src/epoch_precompute.rs the worker sends EpochPrecomputed and only then calls warm_epoch_key_sums (~256). The actor can cache the checkpoint and a later decide can spawn the warm-only task while that worker is still inside get_or_init_key_sums. The flag was added to avoid a second blocking thread waiting on the OnceLock; set it for the worker’s warm as well, or warm before the send. Also, a warm-only attempt that loses the swap is dropped, not retried, so a single overlapping tick leaves the epoch cold until something calls decide again.

  4. Replay warming looks easy to point at the wrong handle. warm_committee_sums(&clock, ...) (bin/ethlambda/src/benchmark/import/replay.rs ~133) reads parent post-states and fills clock’s committee cache, while import goes through BlockChainServer::for_replay(store, ...). There is no test that import then sees resident_key_sums. If those handles do not share the DB and CommitteeCache, the flag either errors on the second block (no post-state for parent) or warms a cache the timed import never reads. On a contiguous epoch step the code also warms only the new epoch and assumes the previous epoch’s sums are still cached; an eviction silently goes cold and the report still says warm_committee_sums=true.

  5. warm_key_sums_change_no_verdict_and_no_post_state never advances the helper state, despite the comment that it is one epoch in. If that helper is at slot 0, the attestation at SLOTS_PER_EPOCH - 1 is in the future and process_attestation should not succeed. Separately, enumerate().skip(2) is not a strict majority when the committee length is 4 (minimal preset, 1024 validators: 2 > 4/2 is false), so that test never takes the subtract path there. The random-mask builder test does cover it.

The shuffling-key write-up (append-only registry, activation lookahead, immutable pubkeys) is the consensus-critical claim, and it is argued carefully. Subtraction matches the attester sum only while a validator cannot appear twice in one slot’s committees: the warm path is a multiset of bit positions, the indexed path is a set. Worth a debug assert if that assignment rule is ever relaxed. Crypto tests for minus / identity / invalid keys look right; keep failing closed on a bad member key.


Automated review by Grok (xAI) · grok-4.7 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 DeepSeek Code Review

⚠️ Review failed: DeepSeek returned an empty content field (finish_reason=length, reasoning_content present=yes). Raise max_tokens or lower reasoning_effort ('none' disables thinking).


Automated review by DeepSeek

… addition

A committee's key sum is the largest cost left in verifying an attestation
once its keys are cached. Folding the keys one by one through
AggregatePublicKey::aggregate pays a projective addition per key;
blst_p1s_add (reached through the safe MultiPoint::add) shares one field
inversion across the batch and runs 1.6-2.3x faster in a single-threaded
measurement. From 384 points up blst also spreads the work over its own
thread pool.

fast_aggregate_verify and eth_aggregate_pubkeys now sum through one private
helper, sum_validated_points, which guards the empty slice (MultiPoint::add
indexes the first element). The helper returns the same group element as the
sequential sum, including for repeated keys (sync committees are drawn with
replacement, so P + P occurs), which the new tests check at sizes around the
384-point threshold. Accepted and rejected inputs are unchanged.
A key and its negation are both valid, so a registry can hold both and an
aggregate can name both. Pin that the batched sum still matches the sequential
one when keys cancel, that such a pair aggregates to the identity, and that the
identity never verifies. Also state the CPU-count condition on blst's pool.
…mes from

The comment credited the sum's existence to `pubkeys` being non-empty
without saying what guarantees it. It is the early return at the top of
the function, not `validated_pubkeys`, which maps an empty input to an
empty `Some`.
Nothing pinned the early return that keeps an empty list away from the
batched sum. The identity signature is used because it is the one input
`eth_fast_aggregate_verify` accepts for an empty list, so the test also
shows the two functions part ways there.
From the second automated review round:

- sum_validated_points returns a one-key slice's key as is, skipping blst_p1s_add and its conversion back to affine coordinates for an aggregate with one signer.
- MultiPoint::add is called by its trait name, so it does not read like std::ops::Add.
- The thread-pool paragraph is tied to blst 0.3.16 and shortened, since its 384-point threshold is that version's.
- The size test's comment says plainly which paths a host exercises: one CPU stays on the calling thread and takes two 1024-point scratch passes at 2000 points; two or more split the work into chunks of about 256.
…e RANDAO mix changes

A deferred signature let its operation write before the signature was
known to be good, and every later operation ran on that write. That is
now confined to votes:

1. Proposer and attester slashings, voluntary exits and BLS-to-execution
   changes call SignatureBatch::verify_pending before they write, so their
   own sets and every set still pending are checked first. No operation
   that slashes, exits or redirects a validator writes on a forged
   signature or runs on a state an earlier forged signature wrote to. A
   forged one is refused at its own operation.
2. The RANDAO reveal is checked the same way before it is mixed in, the
   specification's order. In gloas the builder's bid, pushed just before
   it, is checked in the same batch.
3. A failed batch is bisected to its first failing set instead of
   re-checking every set on its own: fewer set checks than the batch had,
   at one final exponentiation per halving, rather than a full pairing per
   set on top of the batch.

Attestations, the sync aggregate and gloas's payload attestations still
go into one batch, checked at the next of those points or at the end.
transition_block returned the partly applied copy alongside the error,
so nothing but convention kept a caller from reading a state that a
rejected block had written to. It now returns Result<BeaconState>, and
SpeculativeTransition holds that Result, so a failed transition has no
state to read.
Its one caller was verify_block_signature, and as public API it invited
batching the proposer's signature again, which would undo the
one-pairing rejection of an unsigned block apply_block relies on.
verify_block_signature now calls verify_proposer_signature directly.
…ecks

Phase0, electra and gloas each had four functions for one check:
is_valid_indexed_attestation, its _with_domain variant, and the two
matching signature-set builders, with the structural rules copied into
each fork's file.

The rules now live once, in attestation::attesters_signature_set, which
takes the attester list, data and signature of any fork's container and
an optional domain. Each fork keeps is_valid_indexed_attestation and
indexed_attestation_signature_set, both thin over it: six functions where
there were twelve. The _with_domain pair is gone; gossip's aggregate
validation, its only caller, calls attesters_signature_set with the
schedule's domain.

The shared check includes gloas's MAX_VALIDATORS_PER_SLOT bound. Every
earlier fork's attester list is bounded at or below it by its own type,
so the check changes no earlier fork's answer.
key_is_identity changed no behavior, by its own documentation: blst
refuses an infinite public key wherever it adds one to a pairing
(BLST_PK_IS_INFINITY in blst/src/aggregate.c), on the batch path and the
single path alike, and the single-set arm returned before the check
anyway. The tests that pin it stay: [P, -P] fails alone and inside a
batch, with the identity signature and with a real one.

Also states what the random weights' soundness rests on. rand::rng() is
a ChaCha12 generator seeded from the operating system; rand 0.10.1, the
locked version, still reseeds it every 64 KiB, but the argument does not
need that.
bls::verify and bls::fast_aggregate_verify each repeated the validate,
sum and pair sequence SignatureSet holds, so only tests kept gossip and
fork choice deciding a signature the way block import does. They are now
SignatureSet::single and SignatureSet::fast_aggregate followed by its
verify.

eth_fast_aggregate_verify's empty-participants rule (valid exactly with
G2_POINT_AT_INFINITY) was written twice: in bls, which block
production's sync-aggregate check calls, and inline in altair's
process_sync_aggregate. It now lives in SignatureSet::eth_fast_aggregate,
which answers Valid, Invalid, or the set whose pairing decides.
bls::eth_fast_aggregate_verify is that answer checked now, and the sync
aggregate pushes the set or rejects from it.
…t module scope

The state transition's metrics are module-scoped LazyLock statics, as
CLAUDE.md prefers; this was the one function-scoped. Its help text and
docs/metrics.md now say what it counts since the failing set is found by
bisection: a failed batch whose narrowed-to set verifies alone.
… writes

Pins the ordering for one of the operations that change a validator's
record: through a batch that defers, a forged BLS-to-execution change is
refused by the operation itself, and a genuine one is refused under the
earlier set's label while a forged set is pending, with the credentials
untouched both times. With the signature deferred to the end of the
block, the first call returned Ok and rewrote them.
…check

The signatures module doc said deferral was confined to votes, but
process_execution_payload_bid records the builder's payment and the bid
before its signature is checked. It is checked right after, with the
RANDAO reveal, before any operation runs, whenever the reveal is
verified (always on import; block production skips it for its own
block). The doc now says so.

Also corrects the cost wording: a bisection costs about what the batch
did, not less, and a lone check is a pairing with a Miller loop of its
own for the signature, which is what a batch saves along with the final
exponentiation.
Base automatically changed from perf/block-signature-batch to main November 20, 2026 01:29

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

beacon Ethereum Beacon Chain client performance Performance improvements or possible performance improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants