Skip to content

feat(beacon): rebuild held blocks' data columns from the execution client's blob pool - #769

Open
MegaRedHand wants to merge 15 commits into
mainfrom
feat/engine-get-blobs-v2
Open

MegaRedHand wants to merge 15 commits into
mainfrom
feat/engine-get-blobs-v2

Conversation

@MegaRedHand

@MegaRedHand MegaRedHand commented Nov 3, 2026 •

Copy link
Copy Markdown
Collaborator

🗒️ Description / Motivation

A fulu block whose custody data columns have not all arrived is held until gossip (or a per-slot peer ask) delivers them. The blob transactions such a block commits to usually passed through the local execution client's pool first, so the node can often build those columns itself, sooner. Fulu's p2p-interface.md ("Distributed blob publishing using blobs retrieved from local execution-layer client") asks honest nodes to query engine_getBlobsV2 for exactly this, and Lighthouse and the other clients already do. This adds it to ethlambda beacon.

hold_block_for_columns (first hold of a fulu block with blob commitments, engine configured,
                        block at most MAX_ASK_AGE_SLOTS (4) behind the clock, proposer signature prechecked)
  └─ spawn task: engine_getBlobsV2(versioned hashes)        one attempt, 1 s
       null / error ─► count, give up (gossip and peer asks carry on)
       every blob ──► spawn_blocking:
                        imported meanwhile? / nothing missing? ─► stop
                        build the missing custody columns (no KZG verification)
                      check_data_column_sidecars(sidecars, SidecarOrigin::ExecutionLayer)
p2p (ExecutionLayer origin):
  record in seen_columns ─► publish on subnet (custody = subscribed) ─► new_data_column_sidecars
chain actor: store ─► release_block_if_columns_complete

What Changed

Piece Where
engine_getBlobsV2, BlobAndProofV2, wire-shape checks, capability list crates/net/engine
unverified_sidecars: builds only the requested columns, sharing code with verified_sidecars crates/blockchain/state_transition/src/beacon/data_columns.rs
Trigger and fetch task crates/blockchain/src/beacon_blobs.rs, hook in hold_block_for_columns
SidecarOrigin { Parked, ExecutionLayer } on the existing check_data_column_sidecars crates/net/api
Seen cache, publish, forward for ExecutionLayer sidecars crates/net/p2p/src/beacon/engine_columns.rs
Docs and metrics docs/beacon_engine.md (new section), docs/metrics.md, CLAUDE.md

Design decisions:

  • No new BlockChainToP2P method. The existing check_data_column_sidecars gains a SidecarOrigin. Parked keeps today's behavior (chain checks); ExecutionLayer skips the checks and does what the spec requires instead. The seen cache and publishing live in p2p, so that is where they happen.
  • One attempt, 1 s (osaka.md's timeout), outside the engine retry ladder: the call races gossip delivery of the same columns, so a retry after the ladder's backoff would land after that race is decided.
  • V2, not V3. V3 answers partially, but a column needs one cell from every blob; without partial-column assembly a partial blob set builds no complete column.
  • No KZG verification of the execution client's proofs, as the spec recommends ("SHOULD skip verification") and Lighthouse does (from_execution_verified).
  • Publish iff subscribed: only custody columns are built and published, since this node's column subnets are its custody columns.
  • No CLI flag: on whenever --execution-endpoint is set.

Not covered (follow-ups): asking on a data_column_sidecar's arrival (the spec's second trigger); gloas, where availability moves to the payload envelope; partial columns (V3).

Open questions for review:

  • Age gate (resolved in review round one). A hold asks only for a block at most MAX_ASK_AGE_SLOTS (4) behind the store's clock: a block gossip delivered in its slot or late, or a by-root answer for a recently missed parent. The age is checked before the proposer precheck, so range catch-up skips both and its blocks' columns come from peers. There is still no concurrency cap, but the gate bounds the asks to blocks near the tip.
  • Supernodes publish in one burst. A node custodying every column publishes all of them at once on a hit. Lighthouse spreads this out (publish_data_columns_gradually) to avoid a bandwidth spike.

New metrics: lean_engine_get_blobs_total{outcome}, lean_engine_get_blobs_duration_seconds, lean_engine_blobs_to_columns_seconds, lean_data_columns_from_execution_layer_total{result}.

Correctness / Behavior Guarantees

  • Nothing changes without --execution-endpoint: the hook needs an engine client.
  • Parked sidecars take exactly the old path.
  • An ExecutionLayer sidecar is recorded in seen_columns before it is published, so a later gossip copy of the same (slot, proposer, index) is ignored, and a column gossip already accepted is neither republished nor re-forwarded.
  • The fetch runs off the chain actor (async task plus spawn_blocking for the cells) and re-checks, just before building, that the block is still unimported and which custody columns are still missing.
  • Trust trade-off: an execution client that returns wrong proofs makes this node store and publish columns peers reject. Debug builds re-run the chain checks on every stored sidecar, so tests would catch a malformed build.

Tests Added / Run

  • ethlambda-engine: wire round trips for a null answer (and the request's method and params), a full answer in order, a wrong proof count, a length mismatch; unit tests for a wrong blob size and a null element.
  • data_columns: requested columns equal the matching entries of verified_sidecars, ascending and deduplicated; empty inputs; shape errors; a tampered proof still builds (nothing is verified).
  • engine_columns: newly seen custody columns are recorded and forwarded in one batch; an already seen column is skipped; a non-custody column is dropped; nothing new sends nothing.
  • beacon_blobs: against a mock execution client serving real KZG blobs and proofs: a full answer hands over exactly the custody columns; a stored column is not handed over again; a null answer hands over nothing; no engine asks nothing; a second hold does not ask again.
cargo test -p ethlambda-engine --profile release-fast
cargo test -p ethlambda-state-transition --lib --profile release-fast beacon::data_columns
cargo test -p ethlambda-p2p --lib --profile release-fast beacon::engine_columns
cargo test -p ethlambda-blockchain --lib --profile release-fast beacon_blobs
cargo test -p ethlambda-blockchain --lib --profile release-fast hold
make lint

Not yet run against a live execution client.

Related Issues / PRs

  • None

✅ Verification Checklist

  • Ran make fmt — clean
  • Ran make lint (clippy with -D warnings) — clean
  • Ran make test (test-consensus plus test-node, at release-fast) — not run; the per-crate tests above were run instead

…ient's blob pool

A fulu block whose custody columns have not all arrived waits on gossip and
on per-slot peer asks. The blob transactions it commits to usually passed
through the local execution client's pool first, so the node can often build
those columns itself, sooner. Fulu's p2p-interface asks honest nodes to do so
with engine_getBlobsV2, as Lighthouse and the other clients already do.

On a fulu block's first data-availability hold, with an execution client
configured, the chain actor spawns a task that asks engine_getBlobsV2 once
(1 s, the spec's timeout, outside the retry ladder: the call races gossip),
builds the custody columns still missing without verifying the KZG proofs
(the spec says to skip verification for local-EL blobs), and hands them to
p2p through the existing check_data_column_sidecars, now tagged with a
SidecarOrigin. For ExecutionLayer sidecars p2p runs no checks; it records
them in the seen cache, publishes them on their subnets, and forwards them
to the chain actor, which stores them and releases the block. Those two
steps are the spec's MUSTs for importing local-EL blobs, and p2p owns both.

V2 only: V3 answers partially, and a column needs a cell from every blob.
Gloas is not covered, since availability moves to the payload envelope.
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

🤖 Claude Code Review

Review of PR 769: execution-client blob fetching (engine_getBlobsV2)

The structure is clean. Splitting out column_sidecar, check_shape and compute_all_cells lets the verified and unverified builders share one assembly path, and the tests are thorough. The main concerns are about when the fetch fires, because the hit path publishes columns and fills the anti-equivocation cache without running any checks.

Correctness / security

  1. The fetch fires for blocks whose proposer signature may not have been checked yet (lib.rs, hold_block_for_columns hunk). The spec trigger is a valid beacon_block or data_column_sidecar from gossip. Gossip blocks have passed proposer-signature validation by the time they are held. Sync (req/resp) blocks have not; the signature is only checked at state transition. On a hit, engine_columns::publish_and_forward:

    • records (slot, proposer_index, index) in seen_columns;
    • publishes sidecars whose signed_block_header.signature is the unverified block signature.

    So a peer serving a forged or invalid block during sync can trigger two problems:

    • Publishing invalid columns. Peers will REJECT them, which costs this node peer score.
    • Seen-cache poisoning. The real block's valid columns for the same (slot, proposer, index) are then IGNOREd on gossip.

    Suggest gating on the block having passed gossip validation, or verifying the proposer signature first. The chain actor already has the parent state when it holds for DA, so that check is cheap.

  2. Fetch concurrency is unbounded, and the fetch is not limited to recent slots (beacon_blobs.rs, spawn_fetch; docs/beacon_engine.md says "Any slot and any source qualifies").

    • During range sync, each held fulu block spawns its own getBlobsV2 call against the EL.
    • Each hit spawns a spawn_blocking job that does rayon par_iter cell computation, so they compete for CPU.
    • For old blocks the EL pool will almost always miss, so this is mostly wasted EL load.
    • Columns that do get built for old slots are mostly IGNOREd by peers anyway.

    Suggest two limits:

    • only ask when slot is within a small window of the wall-clock slot;
    • cap in-flight fetches with a tokio::sync::Semaphore.
  3. Race with in-flight gossip validation (engine_columns.rs, seen_columns.record). If gossip received a copy whose verdict is still pending (queued or on the spawn_blocking checker) and does not record into seen_columns until it is accepted:

    • our publish succeeds, or hits gossipsub's duplicate cache;
    • the peer's copy is then IGNOREd, which denies that peer P3 credit.

    This is minor. Please confirm whether seen_columns is recorded on accept or on receipt, and document the behaviour.

  4. The "Built data columns" info! is logged even when the hand-over failed (beacon_blobs.rs, end of fetch_and_publish). Return early on Err from check_data_column_sidecars, or move the log above the call.

Rust / maintainability

  1. Inconsistent path for KzgProof (types.rs). KzgProof is imported at the top, but BlobAndProofV2::proofs spells out the full path ethlambda_types::beacon::primitives::KzgProof. Use the import.

  2. types.rs now depends on building.rs for parse_data and parse_fixed, which only became pub(crate) in this PR. These are generic hex/DATA helpers. Consider moving them to a small hex/wire module, or into types.rs next to data(), so the type definitions do not depend on the payload-building module.

  3. blob_count = entries.len() is captured before entries is moved. That is fine. The blobs.len() != commitments.len() check in check_shape is redundant with decode_blobs_v2's length check, which is harmless defence in depth.

  4. Possibly quadratic present.contains(index) (beacon_blobs.rs). If data_column_indices_for returns a Vec, the lookup is O(n·m). That is negligible at 128 columns, but a HashSet would make the intent clearer.

Tests

  1. Several beacon_blobs tests rely on fixed sleeps (200–300 ms) to assert that something did not happen:

    • a_null_answer_hands_nothing_over
    • without_an_engine_nothing_is_asked_or_handed_over
    • a_second_hold_of_the_same_root_does_not_ask_again

    These are acceptable, but can be flaky on loaded CI. Waiting on the request counter first, as the null test already partly does, and then yielding is more robust.

  2. Missing test: a hit where has_state(block_root) is already true, i.e. the block was imported while the EL answered. It should hand nothing over; this path is untested.

Docs

The docs are accurate and the "Trust" section is a good addition. Once Item 1 or Item 2 is addressed, update the "Any slot and any source" sentence in beacon_engine.md to match.


Automated review by Claude · claude-opus-5-5 · custom prompt

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

🤖 Codex Code Review

  • [P1] Validate the block header before publishing columns or updating the seen cache — engine_columns.rs:63–66. Sync blocks reach hold_block_for_columns before signature validation or state transition. A malicious peer can supply a forged block referencing blobs already in the local execution client; this path then publishes sidecars carrying its invalid signature and records (slot, proposer, column) as seen. This can suppress legitimate columns and damage this node’s peer score. Verify the header signature and expected proposer before recording or publishing; trusting local KZG proofs does not validate a peer-supplied header.

Tests could not run because the required Rust toolchain attempted to write to a read-only directory.


Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

🤖 Grok Code Review

The execution-layer blob path is well structured, and the V2-only, one-shot, 1s timeout matches the spec. A few consensus and liveness issues should be fixed before this is relied on.

  1. Wrong blobs still complete data availability. unverified_sidecars (data_columns.rs ~275) only checks counts and column indices. fetch_and_publish never checks that each blob’s KZG commitment (or versioned hash) matches blob_kzg_commitments. SidecarOrigin::ExecutionLayer then skips every column check and the chain stores what it gets. A buggy or compromised EL can make this node treat DA as satisfied, import the block, and attest to data peers will reject. Peer-score loss, as docs/beacon_engine.md describes, is the mild outcome. Compare commitments in the blocking task and, on mismatch, do not hand the sidecars to p2p so gossip and peer asks stay open. Spec “SHOULD skip verification” is not a reason to skip the binding between the block and the blobs.

  2. Seen-cache update is not tied to a successful store. In engine_columns.rs (~64–72) seen_columns.record runs before publish and before new_data_column_sidecars (~118–131). If the chain ref is None, the send fails, or the actor drops the batch, that (slot, proposer, index) is poisoned: a later gossip copy is ignored, and this node may never store the column. Record only after the chain accepts the batch, or roll the seen key back on send failure. Log the blockchain == None path; it currently drops the batch silently.

  3. Every historical hold hits the EL. hold_block_for_columns (lib.rs ~3180) spawns an unbounded task for any fulu block with commitments, including sync, and each task clones the full block across a 1s RPC. The blob pool will miss almost all of those, but the calls still occupy EngineClient. If call_once shares a connection or inflight limit with newPayload / forkchoiceUpdated, backfill can stall fork choice for a second per held block, and memory tracks every in-flight payload. Gate on a recent slot (blob-pool retention / gossip window) and cap inflight fetches. The new comment that this call “races nobody” is also wrong; the module docs correctly say it races gossip.

  4. No test covers the bytes actually published. publish_and_forward publishes sidecar.to_ssz() on the p2p actor (engine_columns.rs ~86). That must be the inner fulu gossip SSZ (then compress_message), not an enum encoding, and only when that subnet topic is actually subscribed. Custody is only a proxy for subscription and can lag a custody update. A unit test that decodes the published payload the same way gossip does would lock this down.

Smaller notes: lean_engine_get_blobs_total{hit} is incremented before the blobs are checked against the block, so shape failures look like hits. Debug-only chain_checks after new_data_column_sidecars will not roll back a block already released from the hold; do not treat that as a safety net.


Automated review by Grok (xAI) · grok-4.7 · custom prompt

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

🤖 Gemini Code Review

The implementation of engine_getBlobsV2, unverified_sidecars, and the P2P hand-over is well-structured and cleanly tested. However, there is one important security/correctness issue regarding unvalidated blocks from Req/Resp sync reaching hold_block_for_columns.

Issues

  1. Unverified blocks from Req/Resp can trigger engine_getBlobsV2, poison seen_columns, and publish invalidly-signed sidecars to gossip (crates/blockchain/src/lib.rs:3182-3195, crates/blockchain/src/beacon_blobs.rs:134-184, crates/net/p2p/src/beacon/engine_columns.rs:53-94)

    hold_block_for_columns runs in process_or_pend_block before state_transition verifies the block's proposer index and signature. While blocks arriving via gossip have already had their proposer signature validated in p2p, blocks arriving via Req/Resp (BeaconBlocksByRange / BeaconBlocksByRoot) have not.

    Because spawn_fetch runs for any held block regardless of source and SidecarOrigin::ExecutionLayer skips all header checks in engine_columns::publish_and_forward:

    • A peer answering a Req/Resp request can send a block with a known parent_root, the current slot and honest proposer_index, an invalid signature, and blob_kzg_commitments taken from public blob transactions currently in the execution mempool.
    • engine_getBlobsV2 will succeed (since the blobs are in the local EL's pool), and unverified_sidecars will wrap them in the forged block's header.
    • engine_columns::publish_and_forward will then:
      1. Record (header.slot, header.proposer_index, index) in server.seen_columns, which is keyed by (slot, proposer_index, index) rather than block_root — causing the honest proposer's real columns on gossip to be dropped as already_seen.
      2. Publish the sidecars carrying the invalid block header signature to data_column_sidecar_{subnet_id}, causing peers on the mesh to reject them and penalize/ban this node.
      3. Forward the sidecars to new_data_column_sidecars, which in debug builds re-runs column::chain_checks and will panic on the invalid header signature.

    Suggestion: Verify the block header (slot > parent slot, expected proposer_index, and proposer BLS signature against the parent state, or only trigger spawn_fetch for gossip-validated blocks) before building/publishing sidecars as SidecarOrigin::ExecutionLayer.

Minor

  1. Pre-allocate proofs vector (crates/blockchain/src/beacon_blobs.rs:162)
    let mut proofs: Vec<KzgProof> = Vec::with_capacity(entries.len() * CELLS_PER_EXT_BLOB);
    Each entry has CELLS_PER_EXT_BLOB (128) proofs, so pre-allocating avoids repeated reallocations in proofs.extend(entry.proofs).

Automated review by Gemini · gemini-4-argon · custom prompt

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

🤖 DeepSeek Code Review

Overall this is a clean refactor (column_sidecar extraction, the SidecarOrigin split, and the wire decoding are all readable). The main concerns are around the trusted execution-layer path and resource bounds.

Findings:

  • crates/net/p2p/src/beacon/engine_columns.rs in the publish loop: server.swarm_handle.publish(...) ignores its Result, but published is incremented unconditionally. If publish returns Err (for example insufficient peers), the column is still marked seen and forwarded, but the metric/log claims it was published. Consider inspecting the result and counting publish failures separately.

  • crates/blockchain/src/beacon_blobs.rs in fetch_and_publish: spawn_fetch launches an unbounded task per first hold, and every engine hit then launches a spawn_blocking cell-computation task. During sync or a burst of held Fulu blocks this can create many concurrent 1 s engine calls plus CPU-heavy cell jobs. A semaphore or bounded worker/queue would be safer.

  • crates/blockchain/state_transition/src/beacon/data_columns.rs in unverified_sidecars: the execution-layer path skips KZG proof verification and also does not verify that the returned blobs correspond to block.message.body.blob_kzg_commitments / the requested versioned hashes. That is documented and spec-allowed for a trusted local EL, but a buggy or malicious EL can make the node store and gossip invalid sidecars and lose peer score before anything catches it. A cheap versioned-hash/commitment sanity check would reduce the blast radius without full proof verification.

  • crates/net/engine/src/client.rs in get_blobs_v2: the call is not gated by engine_exchangeCapabilities. If the EL does not advertise engine_getBlobsV2 or the network is not Osaka yet, every held Fulu block will produce an error/miss metric and a wasted attempt. Consider using negotiated capabilities to disable the path or warn once.

  • crates/net/p2p/src/beacon/engine_columns.rs in publish_and_forward: sidecars are recorded in seen_columns before the chain handover. If server.blockchain is None or new_data_column_sidecars fails, the column is already marked seen and will never be retried through this path. That is probably intentional for duplicate suppression, but the failure mode should be explicit/metric’d.

Minor:

  • crates/net/engine/src/types.rs now imports from building.rs, which likely already imports from types.rs; this intra-crate cycle works but is a bit tangled. Moving parse_data/parse_fixed to a small shared module would be cleaner.
  • In beacon_blobs.rs, proofs could be pre-allocated with Vec::with_capacity(entries.len() * preset::CELLS_PER_EXT_BLOB) rather than growing from zero.

Automated review by DeepSeek · deepseek-v4-pro · custom prompt

@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown

🤖 Kimi Code Review

Findings

  1. Unbounded fetch and reconstruction fan-out — crates/blockchain/src/lib.rs:3180-3195, crates/blockchain/src/beacon_blobs.rs:76-87,136
    Every distinct held root spawns an independent engine request, and every successful response can queue a spawn_blocking job that computes cells for all blobs. During sync, or when peers systematically withhold columns, many valid blocks can be held quickly, producing numerous concurrent engine requests and a large backlog of blocking tasks and blob-filled responses. The import re-check limits wasted computation once a job starts, but does not bound queued work or memory. Consider a bounded worker or semaphore acquired before spawning, with work dropped or coalesced when saturated.

  2. Potential test race when waiting for the hand-over — crates/blockchain/src/beacon_blobs.rs, wait_for_hand_over; crates/blockchain/src/lib.rs:4727-4731
    RecordingP2P pushes to checks, releases that mutex, and only then pushes to check_origins. wait_for_hand_over waits solely on checks, so a_full_answer_hands_over_exactly_the_custody_columns can theoretically observe the sidecars before the origin is recorded and fail intermittently. Record the origin before releasing the checks update, use one mutex for both vectors, or wait for both.

Otherwise, the implementation looks careful: strict wire-shape validation, correct V2 all-or-nothing handling, bounded/deduplicated column construction, and seen-cache recording before publication all align well with the intended Fulu flow.


Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt

MegaRedHand added a commit that referenced this pull request Nov 5, 2026
…ient's blob pool (#769)

A fulu block whose custody columns have not all arrived waits on gossip and
on per-slot peer asks. The blob transactions it commits to usually passed
through the local execution client's pool first, so the node can often build
those columns itself, sooner. Fulu's p2p-interface asks honest nodes to do so
with engine_getBlobsV2, as Lighthouse and the other clients already do.

On a fulu block's first data-availability hold, with an execution client
configured, the chain actor spawns a task that asks engine_getBlobsV2 once
(1 s, the spec's timeout, outside the retry ladder: the call races gossip),
builds the custody columns still missing without verifying the KZG proofs
(the spec says to skip verification for local-EL blobs), and hands them to
p2p through the existing check_data_column_sidecars, now tagged with a
SidecarOrigin. For ExecutionLayer sidecars p2p runs no checks; it records
them in the seen cache, publishes them on their subnets, and forwards them
to the chain actor, which stores them and releases the block. Those two
steps are the spec's MUSTs for importing local-EL blobs, and p2p owns both.

V2 only: V3 answers partially, and a column needs a cell from every blob.
Gloas is not covered, since availability moves to the payload envelope.

On nightly, a Parked recheck keeps #765's ColumnArrival::Recheck, and the
CLAUDE.md peer-scoring line keeps #739's slow-peer text. Columns built from
the execution client write no block_events telemetry rows: #765's column
rows cover gossip-accepted and fetched columns only.

Cherry-picked from feat/engine-get-blobs-v2 at 39d5f07 (e0f69b3 plus a
merge of main).

Claude-Session: https://claude.ai/code/session_011vghNmndKPkZinHRkVTwVB
…proposer signed

A held fulu block's blob fetch builds columns that carry the block's own
signed header, and p2p records and publishes those with no check
(SidecarOrigin::ExecutionLayer). A block reaches its hold unverified when it
comes from range sync or a by-root answer, since only gossip checks the
proposer signature before data availability. A peer serving a forged copy whose
commitments name blobs in the local pool therefore had this node publish
columns its peers reject, and take the seen-cache key of the real block's
columns from the same proposer and slot.

The first hold now runs precheck_block against the parent's post-state
(expected proposer and signature) before asking. The tests sign their blocks
over a keyed parent state, and a new one checks that an altered copy is not
asked about.
# Conflicts:
#	CLAUDE.md
#	crates/net/p2p/src/lib.rs
parse_data and parse_fixed lived in building.rs and became pub(crate)
when decode_blobs_v2 in types.rs needed them, which made the type
definitions depend on the payload-building module. They now sit beside
data() in types.rs, and building.rs imports them from there like it
already imports data and quantity. BlobAndProofV2::proofs uses the
KzgProof the module imports instead of spelling out its path.
publish_and_forward's tests covered the seen cache and the forward to
the chain but not the publish. A new test captures the swarm commands
and checks that only the newly recorded custody column is published, on
its own subnet topic under the digest held for its slot, and that its
bytes decompress and decode as the fulu container gossip validation
reads.

The module doc now also says that gossip records a column only on
Accept, so a gossip copy still being validated when the execution
layer's column is recorded is ignored like any second valid copy.
Every first hold of a fulu block with commitments asked engine_getBlobsV2,
whatever its age. Range sync holds blocks by the batch, every one older
than the slot it arrives in, so a catch-up paid a parent-state read and a
BLS verification on the chain actor (the proposer precheck), an engine
round trip and, on a hit, a cell computation for each, with nothing
bounding how many ran at once. Those blocks' columns are already asked of
peers as soon as they are held.

A hold now asks only for a block at most MAX_ASK_AGE_SLOTS (4) behind the
store's clock, which covers a block gossip delivered in its slot or late
and a by-root answer for a recently missed parent. The age is checked
before the precheck, so range sync skips both. A new test holds one block
just past the window and one at its edge and checks that only the second
is asked about.

The hold's doc said the call "races nobody" and was "as useful for a
block at the tip as for an old one", which contradicted the module doc;
it now says it races gossip and why the peer-ask timing does not apply.
docs/beacon_engine.md, docs/metrics.md and CLAUDE.md drop "any slot"
and "normal for old blocks". The Trust paragraph now also says the
blobs are not checked against the block's commitments, and that wrong
ones release the held block as available.
fetch_and_publish logged "Built data columns from the execution
client's blobs" even when check_data_column_sidecars had just failed;
it now returns after logging the failure. The proofs vector is sized for
CELLS_PER_EXT_BLOB proofs per blob up front.

A new test covers the re-check before the build: a hit for a block whose
state is already stored (gossip completed it while the execution client
answered) builds and hands over nothing. It awaits fetch_and_publish
directly, so it waits on no timer. RecordingP2P now records a hand-over's
origin before its sidecars, so a test that waits on the sidecars always
finds the origin.
@MegaRedHand

MegaRedHand commented Nov 19, 2026 •

Copy link
Copy Markdown
Collaborator Author

Round one (the automated review of e0f69b3). Fixed in 3ef0e53 (pushed before this reply), b4d66b4, a18cdc9, 434b867 and 18dd341. make fmt and make lint are clean. No tests were run locally; CI runs them.

Fixed

  • Claude 1, Codex P1, Gemini 1, a forged sync block publishing columns and taking the seen-cache key. Fixed by 3ef0e53, which landed after this review. A first hold now runs precheck_block against the parent's post-state (expected proposer and signature) before asking. Test: a_block_whose_signature_does_not_verify_is_not_asked_about.
  • Claude 2, Grok 3, DeepSeek 2, Kimi 1, unbounded fan-out. A hold now asks only for a block at most MAX_ASK_AGE_SLOTS (4) behind the store's clock. That covers a gossip block in its slot or a little late, and a by-root answer for a recently missed parent.
    • Range sync no longer asks. Every block it delivers is older than its arrival slot, and it holds them by the batch.
    • The age is checked before the precheck, so range sync also skips the parent-state read and the BLS verification on the chain actor.
    • Those blocks' columns are asked of peers as soon as they are held.
    • I didn't add a semaphore: the window bounds how many fetches run at once.
    • Test: only_a_recent_block_is_asked_about, which holds one block just past the window and one at its edge.
  • Grok 3, the "races nobody" comment. The hold's doc now says the call races gossip, and why the peer-ask timing does not apply to it. "Any slot and any source" and "normal for old blocks" are gone from docs/beacon_engine.md, docs/metrics.md and CLAUDE.md.
  • Claude 3, an in-flight gossip copy. Confirmed: gossip records a column only on Accept (Validated::record_seen in verdict.rs). A copy still being validated when the execution-layer column is recorded is therefore ignored, like any second valid copy. The engine_columns module doc now says so.
  • Claude 4, the success log after a failed hand-over. It now returns after logging the error.
  • Claude 10, a hit for a block imported meanwhile. New test a_hit_for_a_block_imported_meanwhile_hands_nothing_over. It awaits fetch_and_publish directly, so it waits on no timer.
  • Grok 4, what goes on the wire. New test a_new_custody_column_is_published_as_gossip_decodes_it. It captures the swarm commands, checks that only the new custody column is published, on its own subnet topic under its slot's digest, and checks that the bytes decompress and decode as the fulu container gossip validation reads.
  • Claude 5, Claude 6, DeepSeek minor, the engine crate's layout. parse_data/parse_fixed moved next to data() in types.rs, so types.rs no longer imports from building.rs. BlobAndProofV2::proofs uses the imported KzgProof.
  • Gemini 2, DeepSeek minor: proofs is pre-sized.
  • Kimi 2: RecordingP2P records the origin before the sidecars. It could not race before either: #[tokio::test] is single-threaded, and the two pushes have no await between them.

Not changed

  • Grok 1, DeepSeek 3, blobs not checked against the commitments. The spec says a node SHOULD skip verifying its own execution client's blobs, and checking a blob against its commitment costs a commitment computation per blob. The Trust paragraph in docs/beacon_engine.md now states the cost: wrong blobs or proofs are stored, published and release the held block, and only debug builds catch it.
  • Grok 2, DeepSeek 5, recording seen before the chain hand-over. This is the same order gossip uses (record on Accept, then forward). A column lost to a failed send is not lost for good: fetched columns go through column_checks, which never reads seen_columns, so the next peer ask still stores it.
  • Grok 4, custody standing in for subscription. custody_columns is set once at startup and never reassigned. build_swarm and the fork transition both subscribe from it.
  • Grok 3, sharing a connection with newPayload. EngineClient uses a pooled reqwest client with no in-flight cap, and each getBlobsV2 has its own 1 s timeout.
  • Grok, hit counted before the shape check. outcome counts the execution client's answer. A shape failure is logged as a warning.
  • DeepSeek 1, the ignored publish result. SwarmHandle::publish returns (): it queues to the swarm task, and gossipsub's own result is only known there, as for every other publish.
  • DeepSeek 4, the capability check. A fulu node needs an Osaka execution client, and Osaka defines engine_getBlobsV2. A client without it answers at once, and the answer counts as error.
  • Claude 7: nothing to change. Claude 8: contains on at most 128 columns, the same idiom as the hold itself.
  • Claude 9, sleep-based absence tests. The fixed sleeps only bound how long a test looks for something that should not happen, so load makes them less sensitive rather than flaky. The asserts that something did happen poll for it, for up to about a second. The new imported-meanwhile test awaits the fetch itself.

Every reviewer produced findings this round.

@MegaRedHand

Copy link
Copy Markdown
Collaborator Author

/review

@github-actions

Copy link
Copy Markdown

🤖 Claude Code Review

Review: engine_getBlobsV2 path for held fulu blocks

Overall this is a well-scoped, well-documented change. The gating is careful (first hold, recent, signed by the proposer), the SidecarOrigin split is clean, and the column_sidecar refactor guarantees the full set and a subset are built the same way. The test coverage is good. Points below, roughly by importance.

Correctness / consensus

  1. Gossip propagation can be lost when racing an in-flight gossip copy (p2p/src/beacon/engine_columns.rs, record-then-publish loop).

    • The race:
      • With validate_messages() on, a gossip copy of column i that is still awaiting its verdict is already in gossipsub's duplicate cache.
      • Our EL-built sidecar has the same SSZ bytes, so the same message-id.
      • swarm_handle.publish will therefore fail with Duplicate, and the error is ignored here.
      • We have already recorded (slot, proposer, i) in seen_columns, so the pending gossip copy then gets Ignore.
    • Result: this node neither publishes nor forwards that column. The module doc calls this "ignored, as any second valid copy is", but here it is the only copy the node would relay.
    • Possible fixes:
      • Surface the publish result and, on Duplicate, let the pending copy resolve as Accept.
      • Or have the gossip verdict treat "seen-cache key taken by our own identical EL copy" as Accept.
    • At minimum, add a metric for publish failures so this is visible.
  2. The EL answer is not checked against the block's commitments (beacon_blobs.rs, the spawn_blocking closure).

    • In release builds, a buggy or misconfigured EL causes three problems:
      • Columns that peers reject get published, costing this node peer score.
      • The block is released as data-available when it is not.
      • The seen-cache keys for the real columns get burned.
    • This is documented and spec-permitted. Still, blob_to_kzg_commitment per blob, compared with blob_kzg_commitments[i], is cheap next to compute_cells (one MSM vs. 128-cell FFT and extension). It would close most of the gap without the full cell-proof batch verification.
    • I'd suggest doing it at the top of the build.
  3. has_state(&block_root) is the only "still relevant?" check (beacon_blobs.rs, spawn_blocking closure).

    • A block that was dropped from the hold in the meantime (e.g. it failed import, or was evicted at finality) still gets its columns built, published and forwarded.
    • Low impact, but consider also checking that the root is still held. That needs a round-trip to the actor, so it is acceptable to leave as-is if documented.

Performance

  1. Synchronous parent-state read plus BLS verification on the chain actor (lib.rs hold_block_for_columns → signed_by_its_proposer).

    • store.get_state(&parent_root) may deserialize a full beacon state if it isn't cached. On mainnet that can be tens of MB, and it happens on the actor's thread per qualifying hold.
    • The recency gate bounds the count, but not the per-call cost.
    • Options:
      • Move the precheck into the spawned task (it already holds a Store clone).
      • Or confirm the parent state is served from a hot cache here.
  2. The spawn_blocking cell computation is unbounded.

    • Gossip blocks are deduped per (slot, proposer), but by-root answers are not. An equivocating proposer can produce many distinct validly-signed blocks for a recent slot whose commitments name public mempool blobs.
    • Each one triggers an engine call plus a full cell computation for up to MAX_BLOBS blobs, on the shared rayon pool via par_iter.
    • Consider a small semaphore, analogous to column_check_permits, and skipping the ask when none is available.

Minor

  1. signed_by_its_proposer logs precheck failures at warn!. A peer serving forged copies can drive this log; debug! seems more appropriate, matching the "no post-state" branch.
  2. .expect("DB read should succeed") in signed_by_its_proposer panics the chain actor on a transient DB error. Returning false with an error! loses nothing, since the gossip and peer paths carry on.
  3. The size check in deneb::Blob::try_from(entry.blob) is redundant after decode_blobs_v2. That's fine, but the warn! text there is unreachable in practice; a debug_assert! or a comment would clarify.
  4. metrics.rs (p2p) doc comment is missing a space: so nothing was published),`"already_seen"`.
  5. Tests rely on sleep-polling (300ms, 100×50ms). They are fine locally but can be flaky under CI load:
    • a_null_answer_hands_nothing_over
    • a_block_whose_signature_does_not_verify_is_not_asked_about
    • For negative assertions, consider awaiting a deterministic signal, e.g. joining the spawned handle by returning it from spawn_fetch under cfg(test).

Looks good

  • decode_blobs_v2 correctly rejects length mismatch, null elements, wrong blob size and wrong proof count.
  • The one-attempt, 1 s call_once matches the Osaka timeout.
  • unverified_sidecars sorts and dedups indices, bound-checks against NUMBER_OF_COLUMNS, and is tested against verified_sidecars output.
  • The trust model and the reasons for "not gloas" and "not V3" are clearly documented in docs/beacon_engine.md.

Automated review by Claude · claude-opus-5-5 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Gemini Code Review

This PR looks great. The engine_getBlobsV2 integration, precheck gating before building unverified sidecars, off-actor cell computation, and P2P hand-over for seen-cache recording and subnet publication are well-designed and thoroughly tested.

Key highlights:

  • Running precheck_block against the parent state in signed_by_its_proposer (gated behind recent_enough_to_ask and first-hold deduplication) prevents unverified by-root blocks from polluting the P2P seen cache or broadcasting invalid sidecars when KZG verification is skipped.
  • Re-checking block import status and already-stored column indices inside the blocking task before computing cells avoids redundant work when gossip completes some or all columns while the engine call is in flight.
  • publish_and_forward in crates/net/p2p/src/beacon/engine_columns.rs properly records (slot, proposer_index, index) in seen_columns prior to publishing and forwarding, matching the Fulu P2P specification requirements.

Automated review by Gemini · gemini-4-argon · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Codex Code Review

  • [P2] Validate all header rules before publishing unchecked columns — beacon_blobs.rs:139. precheck_block skips the expected-proposer check when the slot falls outside the parent’s lookahead, and never checks finalized ancestry. A validator can sign a current-slot block referencing an old stored parent and commitments available in the local EL, pass this precheck, and cause the node to publish sidecars that fail normal header validation. This also consumes seen-cache entries and expensive cell computations. Enforce the sidecar header rules before taking the unchecked publication path; skipping KZG verification for local EL data does not justify skipping these checks.

Review based on static inspection; tests were not run.


Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Grok Code Review

The EL blob path is well bounded: recent blocks only, one shot, V2 all-or-nothing, and the proposer precheck closes the forged-header seen-cache attack. A few correctness gaps remain around that same seen cache and the actor hot path.

  1. Seen-cache write is not tied to a successful import. In crates/net/p2p/src/beacon/engine_columns.rs (publish_and_forward, the seen_columns.record loop and the later new_data_column_sidecars call), a column is recorded and published before the chain handoff, and a failed new_data_column_sidecars is only logged. Gossip keys are (slot, proposer, index) with no block root, and a later valid copy is ignored once the key is taken. If the mailbox send fails, or debug chain_checks_for then rejects the unverified sidecar, those custody columns can never be accepted from peers and the hold never completes. Record and publish only after the handoff succeeds, or roll the seen key back on failure.

  2. A wrong EL answer is treated as data availability. unverified_sidecars copies the block’s commitments and inclusion proof but never checks that the blobs match them (beacon_blobs.rs build path; documented in docs/beacon_engine.md). Release builds store whatever new_data_column_sidecars delivers and can release the held block. Combined with Item 1, a buggy or compromised EL (JWT-local, but still the only check skipped) both publishes columns peers will reject and blocks the real columns. Spec “SHOULD skip” KZG verification is not a MUST NOT. At least reject blobs whose blob_to_kzg_commitment does not match the block before building sidecars; verifying cell proofs before the seen-cache write would close the rest.

  3. Proposer precheck runs on the chain actor, ahead of the peer ask, and can panic. hold_block_for_columns calls signed_by_its_proposer synchronously (lib.rs around the new call; beacon_blobs.rs ~119). That loads the parent post-state and runs BLS before the block is parked and before the peer column request the comment says starts “below”. Every recent blob block pays that latency even when the EL misses. get_state(...).expect("DB read should succeed") also kills the actor on a DB error; the fetch path already fails closed. Move the precheck into the spawned task (still before engine_getBlobsV2) and return false on a read error.

  4. Cell reconstruction is an unbounded spawn_blocking. fetch_and_publish (~196) has no permit, unlike gossip’s column_check_permits. A burst of multi-blob holds can fill the shared blocking pool and stall gossip validation. Cap it the same way.

Smaller notes: negative tests that sleep 200–300 ms and assert “no RPC” (a_block_whose_signature_does_not_verify, without_an_engine) can pass on a slow runner even if a request is late. Startup still does not warn when the EL omits engine_getBlobsV2 from capabilities; the call will just count as error.


Automated review by Grok (xAI) · grok-4.7 · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 DeepSeek Code Review

The PR is well-scoped, documented, and has good test coverage around the new execution-client blob path. I didn’t see anything concerning in the fork-choice / attestation / justification / SSZ paths.

A few issues worth addressing:

  • crates/blockchain/src/beacon_blobs.rs — signed_by_its_proposer
    store.get_state(&block.parent_root()).expect("DB read should succeed") will panic the chain actor on a store read error. This path runs synchronously in hold_block_for_columns, so a transient/corruption DB error crashes the node rather than just skipping the EL ask. I’d handle the Err case explicitly: log error! and return false.

  • crates/net/p2p/src/beacon/engine_columns.rs — publish_and_forward
    Columns are inserted into seen_columns before the chain hand-over via new_data_column_sidecars. If that hand-over fails (server.blockchain is None, or the send returns an error), the sidecars are dropped but remain marked as seen. A later valid gossip copy would then be ignored as already_seen, so the held block could stay incomplete. Consider compensating by removing/rolling back the seen entry on forward failure, or otherwise ensure a failed forward cannot poison the seen cache.

  • crates/net/p2p/src/beacon/engine_columns.rs — publish_and_forward
    Sidecars whose index is not in beacon.custody_columns are dropped before both seen_columns.record and chain forwarding. spawn_fetch builds exactly the chain actor’s self.custody_columns at hold time. If those two custody sets can diverge (custody rotation/update timing, or p2p/chain updating separately), the chain actor could remain permanently stuck waiting for a column p2p silently discarded. If the invariant is guaranteed, I’d add a debug assertion/comment; otherwise forward all ExecutionLayer sidecars to the chain actor and let it ignore the ones it no longer needs.

  • crates/blockchain/state_transition/src/beacon/data_columns.rs — unverified_sidecars
    This is a pub unverified sidecar builder whose safety depends entirely on the caller using it only for SidecarOrigin::ExecutionLayer. That’s documented, but the type system doesn’t enforce it. Worth tightening visibility where possible, or adding a private/feature-gated API path so it can’t be accidentally used for network- or peer-sourced blobs later.


Automated review by DeepSeek · deepseek-v4-pro · custom prompt

@github-actions

Copy link
Copy Markdown

🤖 Kimi Code Review

Overall, this looks solid; I found no blocking correctness or consensus issues. The proposer-signature gate before unchecked publishing, first-hold/age limits, strict V2 response decoding, off-runtime cell computation, and seen-cache-before-publish ordering address the main risks well.

Two non-blocking observations:

  • crates/blockchain/src/beacon_blobs.rs:136-140: signed_by_its_proposer uses .expect("DB read should succeed") for the parent-state read. Unlike the later store errors in fetch_and_publish, this can panic the chain-actor task on a database failure. Consider logging and returning false instead.

  • crates/net/engine/src/types.rs:278-291: BlobAndProofV2 documents a shape invariant—correct blob length and exactly CELLS_PER_EXT_BLOB proofs—but its public fields allow any downstream crate to construct values violating that invariant. The current caller defensively revalidates, so this is not presently exploitable; private fields with accessors or a validating constructor would make the API match its documentation.

The decision to trust blob contents and proofs from the local execution client is explicitly documented and consistent with Fulu’s skip-verification allowance. The accompanying warning about invalid execution-client responses is important, since such responses could release data availability incorrectly and populate the seen cache with unusable columns.


Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt

… client's blobs

Columns built from an engine_getBlobsV2 answer went to p2p as
SidecarOrigin::ExecutionLayer with no check at all, then into the seen
cache, onto the wire and into DataColumns. Two things went wrong with that.

The proposer precheck before the call cannot name the slot's proposer past
the parent state's lookahead, so it only checks the signature against the
key of whichever validator the block names. A block on an old parent,
signed by any registered validator and committing to blobs from the public
pool, had this node publish columns under a header from the wrong proposer,
which peers reject.

And a wrong blob or proof from the execution client was stored as if
verified. A stored column keeps every other copy of it out (gossip and the
fetch checks answer AlreadyStored) and nothing deletes it, so the block's
import failed at its availability check every time; once import trusts
stored columns, the same answer would import the block on unverified data.

The blocking task now runs the built sidecars through the chain checks a
fetched column gets, in the same two stages (ChainPrechecks, then one
kzg_verdicts batch), and hands p2p only the ones that pass. A refusal is
counted in lean_engine_blob_columns_refused_total by rule and logged; a
column another route stored meanwhile is not a refusal. The spec lets a node
skip verifying its own execution client's blobs; docs/spec_deviations.md
says why this one does not, at one KZG batch per hit.

Two tests: an answer whose proofs belong to other blobs hands nothing over,
and a block past the lookahead from a validator that is not the slot's
proposer hands nothing over while the real proposer's block does. Both fail
without the check.
@pablodeymo

Copy link
Copy Markdown
Collaborator

/review

@pablodeymo

Copy link
Copy Markdown
Collaborator

I pushed a commit onto this branch (1a310ba) for the two problems I found reviewing it, so you don't have to chase them. Revert or rework it if you'd rather go another way.

What it does. The columns built from getBlobsV2 now go through the same checks a fetched column gets before anything is published, marked seen or stored: ChainPrechecks, then one kzg_verdicts batch, in the existing blocking task. Only the ones that pass go to p2p. Refusals are logged and counted in a new lean_engine_blob_columns_refused_total{reason}.

Why:

  • Wrong proposer past the lookahead. signed_by_its_proposer can't name the slot's proposer when the slot is past the parent state's lookahead; there it only checks the signature against whichever validator the block names. A block on an old parent (the finalized block, say), signed by any registered validator and committing to blobs from the public pool, got its columns published under a wrong-proposer header. The age gate doesn't cover it: that block is at the current slot, only its parent is old. The chain checks advance the parent's state and refuse it.
  • A wrong answer from the execution client. A bad blob or proof was stored as if verified. A stored column keeps every other copy out (AlreadyStored in gossip and the fetch checks) and nothing deletes it, so the block failed its availability check at every import. Once perf(beacon): stop re-verifying stored custody columns at block import #760 lands, import trusts stored columns, so the same answer would instead import the block on unverified data. With this commit every stored column is verified, which is what perf(beacon): stop re-verifying stored custody columns at block import #760 relies on.

Cost. One KZG batch over the custody columns per hit, before publishing. The spec says a node SHOULD skip this, so there's a new entry in docs/spec_deviations.md saying why we don't. If the latency matters for the race with gossip, an alternative would be to publish first and verify before storing, but then we'd still put unverified columns on the wire.

Tests. Two new ones: an answer carrying another blob's proofs hands nothing over, and a wrong-proposer block past the lookahead hands nothing over while the real proposer's block does. Both fail with the check removed. The blockchain lib tests, the p2p engine_columns tests and the data_columns builder tests pass, and fmt and lint are clean. I didn't run make test or try a live execution client.

Left for you, smaller:

  • a block that fails the precheck is still held and asked about every slot until finality;
  • published in lean_data_columns_from_execution_layer_total counts publish attempts, not successes;
  • when a gossip copy is still in validation, our publish can fail as a duplicate and neither copy gets relayed.

@github-actions

Copy link
Copy Markdown

AI security review: 3 finding(s) to check

Reviewed by Opus, GPT, Kimi, GLM. A judge model checked every one of the findings against the code.
DeepSeek (error, see the run log) did not finish; Opus, GPT, Kimi, GLM reviewed the whole PR. Comment /review continue to retry, with a new budget; reviewers that finished are reused.
Malicious-code check (tests, build and install scripts, hooks, editor tasks, dependency sources): nothing found.

# Finding Severity Status Found by Where
M3 Forward recovered columns even when another root owns the gossip key medium Bug Opus, GPT (2/4) crates/net/p2p/src/beacon/engine_columns.rs:66
M1 Bound execution-layer reconstruction jobs independently of block age medium Bug GPT (1/4) crates/blockchain/src/beacon_blobs.rs:228
M2 Peer-supplied held blocks can force unbounded BLS verification on the chain actor medium Bug Kimi (1/4) crates/blockchain/src/lib.rs:3330
Details

M3. Forward recovered columns even when another root owns the gossip key (medium, logic)

  • Trigger: A proposer signs two headers A and B for the same slot. A's valid sidecars arrive through gossip first, then block B is held for missing custody columns. The execution client has B's blobs, and beacon_blobs::check_built successfully validates the reconstructed B sidecars before passing them to publish_and_forward.
  • Mechanism: SeenColumns is keyed only by (slot, proposer, index), not block root. Consequently, A's accepted sidecars make record return false for B, and this continue suppresses forwarding to the chain as well as gossip publication. However, check_built's AlreadyStored check and the availability gate are root-specific: storing A's columns does not supply any columns for B. The existing fetched-column path correctly bypasses this gossip seen-cache restriction and forwards checked columns for either root.
  • Impact: The node discards valid, locally recovered columns for B and leaves B held despite possessing all required blob data. If peers do not supply B's columns—for example, the equivocating proposer withheld them—the block cannot be imported through this recovery path. The gossip key should suppress duplicate publication, not storage of checked columns for a different root.
  • Judge: The equivocation/storage claim holds: A and B share the gossip cache key, but availability and stored-column checks distinguish their roots. The early continue therefore discards B's checked columns before chain forwarding. The additional report describes a different relay-only race: the local seen-cache suppression is confirmed, but the external libp2p duplicate-publication behavior could not be

M1. Bound execution-layer reconstruction jobs independently of block age (medium, DoS/resource)

  • Trigger: A validator-controlled peer supplies many distinct, recent Fulu blocks through handle_beacon_blocks_by_range_response or requested by-root replies in crates/net/p2p/src/req_resp/handlers.rs. The blocks reference a retained parent outside its proposer lookahead and blobs available in the local execution client's pool; changing graffiti gives each block a different root.
  • Mechanism: The age gate bounds slot values, not the number of roots. Past the parent's lookahead, precheck_block accepts the signature of whichever registered validator the block names, without establishing that it is the expected proposer. Req/resp admission does not apply gossip's first-valid-block cache, and blocks_awaiting_columns only deduplicates identical roots. Every successful engine answer therefore starts another blocking job that computes cells and then runs check_built, including cloning and advancing the parent state before rejecting a wrong proposer. Unlike the existing fetched/parked column path, these jobs never acquire column_check_permits or another admission limit. The one-second Engine timeout does not bound this subsequent CPU work.
  • Impact: An adversary can accumulate expensive reconstruction and state-advance jobs, consuming CPU, blocking-pool capacity, and memory and delaying normal block import and validator work. A bounded admission mechanism is needed for the whole fetch/rebuild operation; the four-slot age gate does not provide one.
  • Judge: Distinct recent roots can each launch an unrestricted reconstruction task. Beyond the retained parent's lookahead, an attacker-controlled registered validator can pass the signature precheck without being the expected proposer. Each Engine hit then computes cells and can clone and advance the parent state before rejection. Neither root deduplication nor the age window bounds concurrent work.

M2. Peer-supplied held blocks can force unbounded BLS verification on the chain actor (medium, DoS/resource)

  • Trigger: On a node with an execution endpoint and p2p configured, a peer supplies many distinct Fulu blocks near the current slot, each with blob commitments and a bad proposer signature, through BlocksByRoot. The peer can induce the requests by gossiping children naming those unknown parent roots; an unknown proposer index passes queue_unless_forged's Reference::Recent check. handle_blocks_by_root_response then forwards each root-matching answer directly to the chain actor without gossip validation.
  • Mechanism: The new guard deduplicates only by block root and restricts slot age, not the number of distinct blocks per slot or proposer. Every distinct root reaches signed_by_its_proposer, which calls precheck_block with Reference::Parent and performs bls::verify at precheck.rs:128 on the single chain-actor thread before rejecting the bad signature. The BlocksByRoot path does not apply the gossip seen-block or signature checks, and MAX_ASK_AGE_SLOTS does not bound attacker-controlled block count within its five-slot window.
  • Impact: An unauthenticated peer can make the chain actor perform an arbitrary number of expensive BLS verifications, delaying block imports, column storage, and held-block release so the node can fall behind and stop keeping up with sync.
  • Judge: The new hold guard performs signature verification synchronously on the chain actor for every distinct qualifying root. Unauthenticated gossip children with unknown proposer indices can induce parent requests; root-matching responses bypass gossip validation. Responses naming a known expected proposer but carrying an invalid signature reach BLS verification, with no per-slot or work-admission limi
Leads from a script (not checked): 2

Found without a model, from the lines this PR adds. Some can be wrong, e.g. a path created at runtime or a name used through reflection.

  • p2p-interface.md does not exist in the repository (referenced at crates/blockchain/src/beacon_blobs.rs:29, crates/net/api/src/lib.rs:68, docs/spec_deviations.md:692)
  • osaka.md does not exist in the repository (referenced at crates/net/engine/src/client.rs:30, docs/beacon_engine.md:355)

Cost of this review: ~$4.52 of a $30 budget (estimated from token counts).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants