Repository navigation
feat(beacon): rebuild held blocks' data columns from the execution client's blob pool - #769
MegaRedHand wants to merge 15 commits into
Conversation
…ient's blob pool A fulu block whose custody columns have not all arrived waits on gossip and on per-slot peer asks. The blob transactions it commits to usually passed through the local execution client's pool first, so the node can often build those columns itself, sooner. Fulu's p2p-interface asks honest nodes to do so with engine_getBlobsV2, as Lighthouse and the other clients already do. On a fulu block's first data-availability hold, with an execution client configured, the chain actor spawns a task that asks engine_getBlobsV2 once (1 s, the spec's timeout, outside the retry ladder: the call races gossip), builds the custody columns still missing without verifying the KZG proofs (the spec says to skip verification for local-EL blobs), and hands them to p2p through the existing check_data_column_sidecars, now tagged with a SidecarOrigin. For ExecutionLayer sidecars p2p runs no checks; it records them in the seen cache, publishes them on their subnets, and forwards them to the chain actor, which stores them and releases the block. Those two steps are the spec's MUSTs for importing local-EL blobs, and p2p owns both. V2 only: V3 answers partially, and a column needs a cell from every blob. Gloas is not covered, since availability moves to the payload envelope.
🤖 Claude Code ReviewReview of PR 769: execution-client blob fetching (
|
🤖 Codex Code Review
Tests could not run because the required Rust toolchain attempted to write to a read-only directory. Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt |
🤖 Grok Code ReviewThe execution-layer blob path is well structured, and the V2-only, one-shot, 1s timeout matches the spec. A few consensus and liveness issues should be fixed before this is relied on.
Smaller notes: Automated review by Grok (xAI) · grok-4.7 · custom prompt |
🤖 Gemini Code ReviewThe implementation of Issues
Minor
Automated review by Gemini · gemini-4-argon · custom prompt |
🤖 DeepSeek Code ReviewOverall this is a clean refactor ( Findings:
Minor:
Automated review by DeepSeek · deepseek-v4-pro · custom prompt |
🤖 Kimi Code ReviewFindings
Otherwise, the implementation looks careful: strict wire-shape validation, correct V2 all-or-nothing handling, bounded/deduplicated column construction, and seen-cache recording before publication all align well with the intended Fulu flow. Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt |
…ient's blob pool (#769) A fulu block whose custody columns have not all arrived waits on gossip and on per-slot peer asks. The blob transactions it commits to usually passed through the local execution client's pool first, so the node can often build those columns itself, sooner. Fulu's p2p-interface asks honest nodes to do so with engine_getBlobsV2, as Lighthouse and the other clients already do. On a fulu block's first data-availability hold, with an execution client configured, the chain actor spawns a task that asks engine_getBlobsV2 once (1 s, the spec's timeout, outside the retry ladder: the call races gossip), builds the custody columns still missing without verifying the KZG proofs (the spec says to skip verification for local-EL blobs), and hands them to p2p through the existing check_data_column_sidecars, now tagged with a SidecarOrigin. For ExecutionLayer sidecars p2p runs no checks; it records them in the seen cache, publishes them on their subnets, and forwards them to the chain actor, which stores them and releases the block. Those two steps are the spec's MUSTs for importing local-EL blobs, and p2p owns both. V2 only: V3 answers partially, and a column needs a cell from every blob. Gloas is not covered, since availability moves to the payload envelope. On nightly, a Parked recheck keeps #765's ColumnArrival::Recheck, and the CLAUDE.md peer-scoring line keeps #739's slow-peer text. Columns built from the execution client write no block_events telemetry rows: #765's column rows cover gossip-accepted and fetched columns only. Cherry-picked from feat/engine-get-blobs-v2 at 39d5f07 (e0f69b3 plus a merge of main). Claude-Session: https://claude.ai/code/session_011vghNmndKPkZinHRkVTwVB
…proposer signed A held fulu block's blob fetch builds columns that carry the block's own signed header, and p2p records and publishes those with no check (SidecarOrigin::ExecutionLayer). A block reaches its hold unverified when it comes from range sync or a by-root answer, since only gossip checks the proposer signature before data availability. A peer serving a forged copy whose commitments name blobs in the local pool therefore had this node publish columns its peers reject, and take the seen-cache key of the real block's columns from the same proposer and slot. The first hold now runs precheck_block against the parent's post-state (expected proposer and signature) before asking. The tests sign their blocks over a keyed parent state, and a new one checks that an altered copy is not asked about.
# Conflicts: # CLAUDE.md # crates/net/p2p/src/lib.rs
parse_data and parse_fixed lived in building.rs and became pub(crate) when decode_blobs_v2 in types.rs needed them, which made the type definitions depend on the payload-building module. They now sit beside data() in types.rs, and building.rs imports them from there like it already imports data and quantity. BlobAndProofV2::proofs uses the KzgProof the module imports instead of spelling out its path.
publish_and_forward's tests covered the seen cache and the forward to the chain but not the publish. A new test captures the swarm commands and checks that only the newly recorded custody column is published, on its own subnet topic under the digest held for its slot, and that its bytes decompress and decode as the fulu container gossip validation reads. The module doc now also says that gossip records a column only on Accept, so a gossip copy still being validated when the execution layer's column is recorded is ignored like any second valid copy.
Every first hold of a fulu block with commitments asked engine_getBlobsV2, whatever its age. Range sync holds blocks by the batch, every one older than the slot it arrives in, so a catch-up paid a parent-state read and a BLS verification on the chain actor (the proposer precheck), an engine round trip and, on a hit, a cell computation for each, with nothing bounding how many ran at once. Those blocks' columns are already asked of peers as soon as they are held. A hold now asks only for a block at most MAX_ASK_AGE_SLOTS (4) behind the store's clock, which covers a block gossip delivered in its slot or late and a by-root answer for a recently missed parent. The age is checked before the precheck, so range sync skips both. A new test holds one block just past the window and one at its edge and checks that only the second is asked about. The hold's doc said the call "races nobody" and was "as useful for a block at the tip as for an old one", which contradicted the module doc; it now says it races gossip and why the peer-ask timing does not apply. docs/beacon_engine.md, docs/metrics.md and CLAUDE.md drop "any slot" and "normal for old blocks". The Trust paragraph now also says the blobs are not checked against the block's commitments, and that wrong ones release the held block as available.
fetch_and_publish logged "Built data columns from the execution client's blobs" even when check_data_column_sidecars had just failed; it now returns after logging the failure. The proofs vector is sized for CELLS_PER_EXT_BLOB proofs per blob up front. A new test covers the re-check before the build: a hit for a block whose state is already stored (gossip completed it while the execution client answered) builds and hands over nothing. It awaits fetch_and_publish directly, so it waits on no timer. RecordingP2P now records a hand-over's origin before its sidecars, so a test that waits on the sidecars always finds the origin.
|
Round one (the automated review of e0f69b3). Fixed in 3ef0e53 (pushed before this reply), b4d66b4, a18cdc9, 434b867 and 18dd341. Fixed
Not changed
Every reviewer produced findings this round. |
|
/review |
🤖 Claude Code ReviewReview:
|
🤖 Gemini Code ReviewThis PR looks great. The Key highlights:
Automated review by Gemini · gemini-4-argon · custom prompt |
🤖 Codex Code Review
Review based on static inspection; tests were not run. Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt |
🤖 Grok Code ReviewThe EL blob path is well bounded: recent blocks only, one shot, V2 all-or-nothing, and the proposer precheck closes the forged-header seen-cache attack. A few correctness gaps remain around that same seen cache and the actor hot path.
Smaller notes: negative tests that sleep 200–300 ms and assert “no RPC” ( Automated review by Grok (xAI) · grok-4.7 · custom prompt |
🤖 DeepSeek Code ReviewThe PR is well-scoped, documented, and has good test coverage around the new execution-client blob path. I didn’t see anything concerning in the fork-choice / attestation / justification / SSZ paths. A few issues worth addressing:
Automated review by DeepSeek · deepseek-v4-pro · custom prompt |
🤖 Kimi Code ReviewOverall, this looks solid; I found no blocking correctness or consensus issues. The proposer-signature gate before unchecked publishing, first-hold/age limits, strict V2 response decoding, off-runtime cell computation, and seen-cache-before-publish ordering address the main risks well. Two non-blocking observations:
The decision to trust blob contents and proofs from the local execution client is explicitly documented and consistent with Fulu’s skip-verification allowance. The accompanying warning about invalid execution-client responses is important, since such responses could release data availability incorrectly and populate the seen cache with unusable columns. Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt |
… client's blobs Columns built from an engine_getBlobsV2 answer went to p2p as SidecarOrigin::ExecutionLayer with no check at all, then into the seen cache, onto the wire and into DataColumns. Two things went wrong with that. The proposer precheck before the call cannot name the slot's proposer past the parent state's lookahead, so it only checks the signature against the key of whichever validator the block names. A block on an old parent, signed by any registered validator and committing to blobs from the public pool, had this node publish columns under a header from the wrong proposer, which peers reject. And a wrong blob or proof from the execution client was stored as if verified. A stored column keeps every other copy of it out (gossip and the fetch checks answer AlreadyStored) and nothing deletes it, so the block's import failed at its availability check every time; once import trusts stored columns, the same answer would import the block on unverified data. The blocking task now runs the built sidecars through the chain checks a fetched column gets, in the same two stages (ChainPrechecks, then one kzg_verdicts batch), and hands p2p only the ones that pass. A refusal is counted in lean_engine_blob_columns_refused_total by rule and logged; a column another route stored meanwhile is not a refusal. The spec lets a node skip verifying its own execution client's blobs; docs/spec_deviations.md says why this one does not, at one KZG batch per hit. Two tests: an answer whose proofs belong to other blobs hands nothing over, and a block past the lookahead from a validator that is not the slot's proposer hands nothing over while the real proposer's block does. Both fail without the check.
|
/review |
|
I pushed a commit onto this branch (1a310ba) for the two problems I found reviewing it, so you don't have to chase them. Revert or rework it if you'd rather go another way. What it does. The columns built from Why:
Cost. One KZG batch over the custody columns per hit, before publishing. The spec says a node SHOULD skip this, so there's a new entry in Tests. Two new ones: an answer carrying another blob's proofs hands nothing over, and a wrong-proposer block past the lookahead hands nothing over while the real proposer's block does. Both fail with the check removed. The blockchain lib tests, the p2p Left for you, smaller:
|
AI security review: 3 finding(s) to checkReviewed by Opus, GPT, Kimi, GLM. A judge model checked every one of the findings against the code.
DetailsM3. Forward recovered columns even when another root owns the gossip key (medium, logic)
M1. Bound execution-layer reconstruction jobs independently of block age (medium, DoS/resource)
M2. Peer-supplied held blocks can force unbounded BLS verification on the chain actor (medium, DoS/resource)
Leads from a script (not checked): 2Found without a model, from the lines this PR adds. Some can be wrong, e.g. a path created at runtime or a name used through reflection.
Cost of this review: ~$4.52 of a $30 budget (estimated from token counts). |
🗒️ Description / Motivation
A fulu block whose custody data columns have not all arrived is held until gossip (or a per-slot peer ask) delivers them. The blob transactions such a block commits to usually passed through the local execution client's pool first, so the node can often build those columns itself, sooner. Fulu's
p2p-interface.md("Distributed blob publishing using blobs retrieved from local execution-layer client") asks honest nodes to queryengine_getBlobsV2for exactly this, and Lighthouse and the other clients already do. This adds it toethlambda beacon.What Changed
engine_getBlobsV2,BlobAndProofV2, wire-shape checks, capability listcrates/net/engineunverified_sidecars: builds only the requested columns, sharing code withverified_sidecarscrates/blockchain/state_transition/src/beacon/data_columns.rscrates/blockchain/src/beacon_blobs.rs, hook inhold_block_for_columnsSidecarOrigin { Parked, ExecutionLayer }on the existingcheck_data_column_sidecarscrates/net/apiExecutionLayersidecarscrates/net/p2p/src/beacon/engine_columns.rsdocs/beacon_engine.md(new section),docs/metrics.md,CLAUDE.mdDesign decisions:
BlockChainToP2Pmethod. The existingcheck_data_column_sidecarsgains aSidecarOrigin.Parkedkeeps today's behavior (chain checks);ExecutionLayerskips the checks and does what the spec requires instead. The seen cache and publishing live in p2p, so that is where they happen.osaka.md's timeout), outside the engine retry ladder: the call races gossip delivery of the same columns, so a retry after the ladder's backoff would land after that race is decided.from_execution_verified).--execution-endpointis set.Not covered (follow-ups): asking on a
data_column_sidecar's arrival (the spec's second trigger); gloas, where availability moves to the payload envelope; partial columns (V3).Open questions for review:
MAX_ASK_AGE_SLOTS(4) behind the store's clock: a block gossip delivered in its slot or late, or a by-root answer for a recently missed parent. The age is checked before the proposer precheck, so range catch-up skips both and its blocks' columns come from peers. There is still no concurrency cap, but the gate bounds the asks to blocks near the tip.publish_data_columns_gradually) to avoid a bandwidth spike.New metrics:
lean_engine_get_blobs_total{outcome},lean_engine_get_blobs_duration_seconds,lean_engine_blobs_to_columns_seconds,lean_data_columns_from_execution_layer_total{result}.Correctness / Behavior Guarantees
--execution-endpoint: the hook needs an engine client.Parkedsidecars take exactly the old path.ExecutionLayersidecar is recorded inseen_columnsbefore it is published, so a later gossip copy of the same(slot, proposer, index)is ignored, and a column gossip already accepted is neither republished nor re-forwarded.spawn_blockingfor the cells) and re-checks, just before building, that the block is still unimported and which custody columns are still missing.Tests Added / Run
ethlambda-engine: wire round trips for anullanswer (and the request's method and params), a full answer in order, a wrong proof count, a length mismatch; unit tests for a wrong blob size and anullelement.data_columns: requested columns equal the matching entries ofverified_sidecars, ascending and deduplicated; empty inputs; shape errors; a tampered proof still builds (nothing is verified).engine_columns: newly seen custody columns are recorded and forwarded in one batch; an already seen column is skipped; a non-custody column is dropped; nothing new sends nothing.beacon_blobs: against a mock execution client serving real KZG blobs and proofs: a full answer hands over exactly the custody columns; a stored column is not handed over again; anullanswer hands over nothing; no engine asks nothing; a second hold does not ask again.Not yet run against a live execution client.
Related Issues / PRs
✅ Verification Checklist
make fmt— cleanmake lint(clippy with-D warnings) — cleanmake test(test-consensusplustest-node, atrelease-fast) — not run; the per-crate tests above were run instead