Repository navigation
perf(p2p): verify one KZG batch per block for fetched data columns - #754
Conversation
A range or by-root answer, or a batch of parked columns replayed after their parent imports, carries many columns of the same block. Each column was checked on its own blocking task with its own verify_cell_kzg_proof_batch call, and the chain actor heard nothing until every sidecar of the whole call was done, so data availability waited on the slowest block of the batch. check_and_forward now groups the sidecars by block (the signed header's root for fulu, beacon_block_root for gloas). A group takes one column_check_permits permit, runs every sidecar's rules that do not read the cells, then verifies the proofs of all sidecars still standing in one batch. A batch that fails is re-run sidecar by sidecar, so exactly the sidecars rejected before are rejected. The group's Keep and AwaitParent sidecars go to the chain actor as soon as the group finishes. Verdicts do not change: the proofs are the last rule on the chain path (the lookahead rule after them is the gossip path's alone), so chain_checks is now chain_precheck followed by a KZG step, with gossip's stateful_checks and stateful_checks_gloas keeping their order through the same split halves. On this machine a batch over 9-blob columns costs 3.3x less than separate calls at 8 columns and 4.8x less at 32.
Every fulu sidecar of a block carries the same signed header, so the group added in the previous commit repeated one BLS verification per sidecar, now one after another under a single permit. SignatureMemo remembers header_signature_is_valid's answer for the group, keyed on the header root, the signature bytes and (for the head-state check made before parking) the head it read, so a forged signature over a shared header is judged on its own. Only that answer is remembered; every other rule runs per sidecar in its order, so verdicts are unchanged. The memo lives in one group's spawn_blocking closure; gossip and the single-sidecar wrappers use a fresh one per call.
🤖 Claude Code ReviewReview: PR 754, one KZG batch per block for fetched data columnsI read the diff only. I did not build it or run the tests. Overall this looks good, and I found nothing blocking. Correctness
Things to consider (non-blocking)
Tests and docs
Verdict: approve. Item 1 is the only point I would act on, and it is just a comment tweak. Automated review by Claude (Anthropic) · sonnet · custom prompt |
🤖 Codex Code ReviewI found one substantive issue; otherwise the batching approach looks sound and preserves per-sidecar verdicts.
Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Kimi Code ReviewI'll review this PR which introduces KZG batch verification for data column sidecars grouped by block. Let me analyze the changes systematically. Overall AssessmentThis is a well-structured optimization that batches KZG proof verification across multiple data column sidecars of the same block, reducing cryptographic overhead. The design is sound but I have several concerns ranging from correctness edge cases to API safety. Critical Issues1.
|
| Priority | Count | Categories |
|---|---|---|
| Critical | 3 | Signature hashing correctness, usize overflow, unwind safety |
| Security | 1 | Unbounded memo memory |
| Correctness | 3 | Lifetime fragility, HashMap ordering, length validation |
| Performance | 2 | Unnecessary allocations, eager conversion |
| Style | 5 | Various minor improvements |
The PR is well-designed and the batching optimization is valuable. The critical issues should be addressed before merge, particularly the BlsSignature hashing correctness and AssertUnwindSafe soundness claims.
Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt
) A range or by-root answer, or a batch of parked columns replayed after their parent imports, carries many columns of the same block. Each column was checked on its own blocking task with its own verify_cell_kzg_proof_batch call, and the chain actor heard nothing until every sidecar of the whole call was done, so data availability waited on the slowest block of the batch. check_and_forward now groups the sidecars by block (the signed header's root for fulu, beacon_block_root for gloas). A group takes one column_check_permits permit, runs every sidecar's rules that do not read the cells, then verifies the proofs of all sidecars still standing in one batch. A batch that fails is re-run sidecar by sidecar, so exactly the sidecars rejected before are rejected. The group's Keep and AwaitParent sidecars go to the chain actor as soon as the group finishes. Verdicts do not change: the proofs are the last rule on the chain path (the lookahead rule after them is the gossip path's alone), so chain_checks is now chain_precheck followed by a KZG step, with gossip's stateful_checks and stateful_checks_gloas keeping their order through the same split halves. On this machine a batch over 9-blob columns costs 3.3x less than separate calls at 8 columns and 4.8x less at 32. The branch's second commit verifies a block group's shared header signature once: every fulu sidecar of a block carries the same signed header, so the group repeated one BLS verification per sidecar, one after another under a single permit. SignatureMemo remembers header_signature_is_valid's answer for the group, keyed on the header root, the signature bytes and (for the head-state check made before parking) the head it read, so a forged signature over a shared header is judged on its own. Only that answer is remembered; every other rule runs per sidecar in its order, so verdicts are unchanged. The memo lives in one group's spawn_blocking closure; gossip and the single-sidecar wrappers use a fresh one per call. Adapted onto nightly's #752: its column_check in-use gauge and blocking-task timer now wrap one block group's closure instead of one sidecar's, and docs/metrics.md says a column_check permit and task cover a block's group. Cherry-picked from perf/column-kzg-batch-per-block at abe310a.
Block import benchmark (offline replay of mainnet blocks, steady state)The replay feeds blocks only: no data columns are fetched or verified, so this PR's code path does not run there. The leg served as a noise control.
Measuring this PR needs a live node or a replay that also fetches columns. Method
|
…events layout (#765) Opt-in per-block telemetry: with --telemetry.clickhouse-url (or ETHLAMBDA_TELEMETRY_CLICKHOUSE_URL), a node writes one ClickHouse row per point in each block's life, and per data column arrival, in Silver's block_events table layout, so both clients' timings can be compared block by block from one table. Off by default: without the flag, emit builds nothing. See docs/telemetry.md. Cherry-picked from feat/clickhouse-block-events at 119e70e. Adapted to three nightly-only changes the PR's base (main 4992d8d) lacks: - #755 sends engine_newPayload from a task while the state transition runs speculatively, so the `engine` pair is only the wait left afterwards. The el_sent and el_verdict rows now come from new engine_sent/engine_answered instants that bracket the request itself, and the speculative transition writes stf_spec_start/stf_spec_end. docs/telemetry.md describes the overlap. - #760 answers a block whose stored columns are complete with DataAvailability::ColumnsVerified, which also counts as columns required, so such a block writes da_available rather than da_check_end. - #754 checks a block's sidecars as one group with a batched KZG, so the column rows are written from GroupVerdicts::record, once each sidecar's verdict is known.
pablodeymo
left a comment
There was a problem hiding this comment.
I found no wrong-verdict bug: rule order is preserved on every path, the memo key covers what the signature answer depends on, and a failed batch falls back to per-sidecar verdicts.
The comments are about what moving to one thread and one permit per block costs, then tests and cleanup. Read from the diff; I did not build or run it.
The ones I would act on before merging:
- A group's size is unbounded, so one peer answer can force a very large batch and a long serial fallback (
group_by_block). - The header-level work other than the signature (advanced proposer, finalized ancestry, gloas block decode) now runs serially per sidecar (
judge_before_kzg). - One blocking thread and one long-held permit per block, on a pool shared with envelope checks and gossip-queued sidecars (
check_and_forward).
The rest are smaller: panic isolation in the KZG stage, the per-column length check in verify_column_cells_batch, two stale comments outside the diff, test gaps, and a few simplifications.
| } | ||
|
|
||
| /// Group `sidecars` by block, in the order each block first appears. | ||
| fn group_by_block(sidecars: Vec<DataColumnSidecar>) -> Vec<Vec<DataColumnSidecar>> { |
There was a problem hiding this comment.
A group has no size bound. The column response reader accepts up to max_request_data_column_sidecars() = 16,384 chunks, retain_requested_columns filters on root and index only, and the by-range handler on slot only, so neither removes a repeated (block, index). has_data_column cannot catch a repeat inside a group either, since nothing of the group is stored until the group finishes.
A peer can answer with 16,384 sidecars of one block: a real header, commitments and inclusion proof, with low-entropy cells that compress to very little on the wire. They all land in one group. verify_column_cells_batch then copies every cell into one Vec (about 700 MB at 21 blobs, on top of the decoded response), the batch fails, and the fallback re-verifies all 16,384 serially on one thread before any honest column of that block is forwarded. Before, at most 16 sidecars' cells were copied at a time.
Suggest capping a batch at a bounded size (NUMBER_OF_COLUMNS at most) and handling a repeated (block, index) within one answer explicitly.
There was a problem hiding this comment.
Fixed in 0ec7ceb and e775465. group_by_block now keeps the first sidecar of each (block, index) and counts the rest under already_seen in lean_data_columns_rejected_total. The prechecks refuse an index past NUMBER_OF_COLUMNS before the KZG stage, so at most 128 sidecars of one block reach it, and those are verified in sub-batches of at most 16 (KZG_BATCH_COLUMNS). That bounds both the cell copy into one batch and a failed batch's one-by-one re-check at 16 columns. Keeping the first copy is the one verdict this changes: a valid copy that follows an invalid copy of the same column in the same answer used to be kept and is now dropped unjudged. Only a faulty peer sends a repeat, and the chain actor's parking already dedups the same way (first wins). a_repeated_block_and_index_is_kept_once covers it with 1,000 copies.
| @@ -287,38 +532,49 @@ fn judge_against_parent( | |||
| (None, PastLookahead::Queue) => None, | |||
| (None, PastLookahead::Advance) => { | |||
| let Some(expected) = advanced_proposer(parent_state, header.slot, &config) else { | |||
There was a problem hiding this comment.
Only the signature answer is memoized, but everything judge_before_kzg decides up to the inclusion proof depends on the signed header alone, which the whole group shares. When the slot is outside the parent state's proposer lookahead, each sidecar clones the parent state and runs process_slots here. finalized_ancestry below repeats its LiveChain scan per sidecar, and on the gloas side gloas_before_kzg decodes the full block (get_signed_block) once per sidecar.
All of that was per sidecar before too, but it ran up to 16 wide. It is now serial on the group's one thread, so 8 custody columns wait for 8 state advances before the block's first column reaches the chain actor (128 for a supernode), and that happens exactly when the chain has gaps.
Judging the header once per group (parent state, NotAfterParent, signature, expected proposer, ancestry) and leaving structure, AlreadyStored, inclusion proof and KZG per sidecar would fix this. It would also remove the SignatureMemo parameter from the six functions it is threaded through, including the gossip path where it can never hit.
There was a problem hiding this comment.
Fixed in b3f541b. ChainPrechecks now judges everything a fulu signed header decides once per group: the parent's post-state (or the head-state check before parking), then NotAfterParent, signature, proposer including advanced_proposer, and finalized ancestry. It keys that on the header's root and signature, so a forged signature over the same header is still judged on its own. A gloas block is decoded once per root. Structure, clock, finality, AlreadyStored, the inclusion proof and the gloas slot and shape rules stay per sidecar, in the same order. The shared answer is only worked out once a sidecar needs it, so a group that is all already stored reads no state. SignatureMemo is gone from all six functions, gossip included. Tests:
a_fulu_group_gets_the_verdicts_each_sidecar_gets_alonechecks that a mixed group gets exactly whatchain_checks_forgives each sidecar alone, with the header judged once per signature.a_group_that_stops_early_never_judges_its_headercovers the lazy part.
| for group in group_by_block(sidecars) { | ||
| // The semaphore is never closed, so this only returns once a | ||
| // permit is free. | ||
| let Ok(permit) = permits.clone().acquire_owned().await else { |
There was a problem hiding this comment.
A permit is now held for a whole block's prechecks, batch and fallback instead of one sidecar, and the pool is shared. Envelope checks take one permit per envelope from it (envelope_checks.rs:100), and range sync holds its next batch until those are forwarded. Every gossip Queue/Overloaded sidecar also comes through here as a group of one (verdict.rs:358).
Permits used to turn over in a few milliseconds. With 16 large groups in flight, or one group in its serial fallback, each waiter now waits for a whole group to finish. A separate pool for envelopes might be worth it, or at least a measurement of how long a permit is held during a supernode range batch.
COLUMN_CHECK_PERMITS' doc in lib.rs still says "How many data column sidecars may be in the chain checks at once ... A sidecar arriving with none free waits for one", which is no longer what 16 bounds.
There was a problem hiding this comment.
Partly addressed in e775465. The proofs now hold a permit for one sub-batch of at most 16 columns, where they used to hold one for a whole block plus its fallback. One sub-batch measures 12.6 ms on one thread at 9 blobs. The prechecks still run a whole group in turn on the group's permit, and I haven't measured how long that holds it on a live node. Nothing waits for a permit while holding one, and tokio's semaphore serves waiters in order, so an envelope waits behind only what was queued before it. I rewrote COLUMN_CHECK_PERMITS' doc to say what a permit covers now. I didn't split out an envelope pool: the proofs' share of a hold is bounded, and I'd rather decide on numbers than guess. lean_data_column_kzg_verify_seconds now times exactly one sub-batch, which is the hold for every sub-batch after a block's first. A supernode follower (or #752's contention metrics) would show whether the precheck hold makes a separate pool worth it.
| continue; | ||
| let blockchain = blockchain.clone(); | ||
| tokio::spawn(async move { | ||
| let checked = tokio::task::spawn_blocking(move || { |
There was a problem hiding this comment.
One blocking thread per block, however many permits are free. By the description's own measurement a batch is 3.3x cheaper at 8 columns and 4.8x at 32, but the per-sidecar checks it replaces ran up to 16 wide.
For a single-block call with idle permits (a by-root answer, a parked-column replay), 8 columns took about one verification of wall-clock and now take 8 / 3.3, about 2.4. 32 columns go from 2 to about 6.7, and 128 from 8 to about 27 if the 4.8x holds, plus the serial prechecks.
Total CPU is lower, but these are the tip-following paths where a held block or envelope is waiting on the columns. Splitting a large group into a few sub-batches across the free permits would keep most of both gains. If the wall-clock at 8 columns is not a concern this is fine to leave, but the trade is worth stating in the description.
There was a problem hiding this comment.
Done in e775465 for blocks with more than 16 pending columns. They're split into near-equal sub-batches of at most 16, the first on the group's permit and each further one on its own. Measured on one thread at 9 blobs, a batch costs about 3 ms plus 0.55 ms per column (3.2 ms for one column, 7.9 ms for 8, 12.6 ms for 16, 73 ms for 128). Computed from those, assuming free permits and cores, 128 columns now take about 13 ms of wall-clock against 26 ms before the PR and 73 ms as one batch, for 101 ms of CPU against 414 ms and 73 ms. I kept the trade at the small end: 16 columns or fewer stay one batch, so a node's 8 sampled columns take 7.9 ms where separate checks would take 3.2 ms, for a third of the CPU. That's stated in the description and in beacon_wire.md. The bench now covers 1 to 128 columns (6805df6).
| .iter() | ||
| .map(|(sidecar, kzg)| (sidecar, kzg)) | ||
| .collect(); | ||
| let kzg_verdicts = column::kzg_verdicts(&items); |
There was a problem hiding this comment.
kzg_verdicts runs outside the catch_unwind, so a panic here unwinds through verdicts and takes the whole group with it. That includes sidecars already settled as AwaitParent, which are then never parked, and the drop outcomes, which are never counted.
The comment on the JoinError arm below says losing the group "is what dropping it would have done anyway", which is not true of those. It is unlikely to fire, but either wrap this call too and forward what was already settled, or say so in the comment.
There was a problem hiding this comment.
Fixed in e775465. The KZG proofs now run on their own spawn_blocking per sub-batch. By then, the prechecks' parked and dropped sidecars have already been forwarded and counted. A panicking sub-batch loses only its own sidecars, uncounted, and the group's other sub-batches are unaffected. The comments on both JoinError arms now say exactly what each one loses.
|
|
||
| /// Run the chain checks on `sidecars`, then send the chain actor the ones that | ||
| /// passed as one batch and the ones waiting on a block as another. | ||
| /// passed as one batch per block and the ones waiting on a block as another. |
There was a problem hiding this comment.
Two comments outside the diff still describe the old contract.
handle_data_column_sidecars_range_response in handlers.rs justifies its check_and_forward call with "One batch through the checks, which forward what passes as one message ... a message per sidecar would put that many mailbox hops between the answer and the imports waiting on it". A 128-slot range answer is now up to 256 chain-actor messages, one keep batch and one awaiting-parent batch per block.
CLAUDE.md's data-columns bullet still says these sidecars go through column::chain_checks, with no mention of the per-block grouping or the precheck/batch split.
There was a problem hiding this comment.
Both fixed in 5f8c9c6. The range handler's comment now describes forwarding per block, one message for what passes and one for what waits on a block, and says why that is still the point. The CLAUDE.md bullet now covers the grouping, ChainPrechecks, the sub-batches and the verdict guarantee. I also rewrote this PR's own beacon_wire.md paragraph and the metrics.md KZG row, which the rework had made stale.
| ChainVerdict::Keep, | ||
| ] | ||
| ); | ||
| let one_by_one: Vec<_> = items |
There was a problem hiding this comment.
This compares kzg_verdicts with kzg_verdict, which is also verify_column_cells_batch (over one column), so it shows the fallback agrees with itself rather than with the old check. Comparing against fork_choice::verify_data_column_sidecar_kzg_proofs, which gossip still uses, would pin "verdicts do not change".
Two more gaps:
- No p2p test reaches
Forwarded::Checked: everycheck_and_forwardtest ends inAwaitingParentor a drop. Sojudge_group's pairing of pending sidecars with the batch's verdicts is unexercised. - These tests build
PendingKzgby hand, so nothing runschain_precheck's ownPendingoutput into the batch, and there is no gloas batch case.
There was a problem hiding this comment.
- The fallback test now compares against
fork_choice::verify_data_column_sidecar_kzg_proofs(and the gloas one), not againstkzg_verdict. a_fulu_group_gets_the_verdicts_each_sidecar_gets_alonebuilds a real block's sidecars (verified_sidecars, signed by the expected proposer, finalized ancestry passing). It mixes in malformed, already stored, forged-signature, bad-inclusion-proof and bad-cell columns, runs them throughChainPrechecksand onekzg_verdictsbatch, and asserts the result equalschain_checks_forone by one, with the good onesKeep.a_gloas_group_reads_its_block_once_and_keeps_what_verifiesis the gloas batch case.- In p2p,
a_blocks_columns_that_verify_are_forwarded_together_across_sub_batchesreachesForwarded::Checked: 18 gloas columns in two sub-batches, a forged column in the second, and every other column back in one batch, in order.
|
|
||
| /// The block a sidecar belongs to: a fulu sidecar's signed header's root, a | ||
| /// gloas sidecar's `beacon_block_root`. | ||
| fn block_of(sidecar: &DataColumnSidecar) -> Root { |
There was a problem hiding this comment.
DataColumnSidecar::block_root() already does this with the same match, and retain_requested_columns uses it. Calling it here avoids a second definition of which block a sidecar belongs to.
There was a problem hiding this comment.
Done in 0ec7ceb. group_by_block uses block_root() and the local helper is gone.
|
|
||
| /// What [`chain_precheck`] found out without reading a cell. | ||
| #[derive(Debug, Clone, PartialEq, Eq)] | ||
| pub enum ChainPrecheck { |
There was a problem hiding this comment.
Nit: this is Result<PendingKzg, ChainVerdict> under another name. The module already writes "verdict or carry on" as a Result with ? (judge_before_kzg, gloas_before_kzg). A Result here would let chain_precheck_with and chain_precheck_gloas use ? instead of repeating return ChainPrecheck::Settled(ChainVerdict::Drop(Outcome::...)) at every early return, and would remove settle's match.
There was a problem hiding this comment.
Done in b3f541b. ChainPrechecks::check returns Result<PendingKzg, ChainVerdict>, the early returns are plain Errs, and settle is gone.
| } | ||
| // [IGNORE] The sidecar's slot is outside the parent state's proposer | ||
| // lookahead window (MAY queue). | ||
| if past_lookahead == PastLookahead::Queue |
There was a problem hiding this comment.
After the split, judge_against_parent has one caller, stateful_checks, which always passes PastLookahead::Queue (chain_precheck_with calls judge_before_kzg directly). So this guard is always true and the parameter is dead in this function.
The doc above still says the function is "shared by stateful_checks and chain_checks". Dropping the parameter, or inlining this tail into stateful_checks, would make the gossip-only path read as gossip-only.
There was a problem hiding this comment.
Done in b3f541b. judge_against_parent is gone and its tail (inclusion proof, KZG, lookahead) is inlined into stateful_checks, which now reads as gossip-only. The rules both paths share up to the inclusion proof are now judge_header_against_parent, where PastLookahead is live: gossip passes Queue, the chain path Advance. Its doc names its two callers.
…ells_batch Concatenating the columns left only the totals to compare, so a column with two cells and one commitment beside one with one cell and two commitments paired a cell with its neighbour's commitment and could verify. Each entry's cells, commitments and proofs must now agree, an Err otherwise, as c-kzg answers for one column on its own. verify_column_cells now delegates to the batch function over one column, so gossip and the chain checks share one copy of the cell conversion.
The sub-batch size the chain checks split a block's columns into is read off this curve, so it now covers every count a block can have, and reports the fastest of 15 runs rather than the median of 5, which a loaded machine skewed by more than the differences being measured.
…group Only the header signature was remembered across a block's sidecars; everything else that reads nothing but the signed header ran for every sidecar, serially on the group's one thread: the parent state read, the head-state check before parking, NotAfterParent, the proposer (a process_slots on a state clone past the lookahead) and the finalized ancestry walk. On the gloas side every sidecar decoded the whole block. ChainPrechecks now runs those once per signed header (root and signature, so a forged signature over a block's header is still judged on its own) and the gloas block lookup once per block root, worked out the first time a sidecar needs them, so a group that stops earlier (all already stored, say) reads no state. The structure, clock, finality, AlreadyStored, inclusion proof and gloas slot/shape rules stay per sidecar, in their place. Its check returns Result<PendingKzg, ChainVerdict>, replacing the ChainPrecheck enum and settle. SignatureMemo is gone, and with it the parameter threaded through the gossip path, where it could never hit. judge_against_parent is folded into stateful_checks, which is its only caller, so the gossip tail (inclusion proof, KZG, lookahead) reads as gossip's; the header rules both paths share are judge_header_against_parent, where PastLookahead still picks between them. Tests: a mixed fulu group (good, malformed, stored, forged signature, bad inclusion proof, bad cell) judged together equals chain_checks_for one by one, through a real block's sidecars and a KZG batch that keeps the good ones; a gloas group reads its block once and keeps what verifies; a group that stops early judges no header; and the fallback test now compares against verify_data_column_sidecar_kzg_proofs, the check gossip runs.
A group had no size bound. The column response reader accepts up to 16,384 chunks, and neither the by-root filter (root and index) nor the by-range one (slot) drops a repeated (block, index), so one answer could put thousands of copies of one block's column into one group: all of their cells copied into one KZG batch, and on failure all re-verified one by one before any honest column of the block was forwarded. group_by_block now keeps the first sidecar of each (block, index) and reports how many repeats it left out, counted under already_seen in lean_data_columns_rejected_total. A block has NUMBER_OF_COLUMNS columns and the prechecks refuse an index past that, so no group sends more than that many sidecars to the KZG stage. The first copy is kept, as the chain actor's own parking does: an honest answer never repeats a column, so the only verdict this changes is a faulty peer's later copy. Groups by DataColumnSidecar::block_root() instead of a second definition of which block a sidecar belongs to.
One blocking thread and one permit per block meant a block's columns were verified serially however many permits were free, and a permit, on a pool the envelope checks share, was held for a whole block's prechecks, batch and fallback. Measured locally (9 blobs, fastest of 15, one thread), one column costs 3.2 ms, a batch of 8 7.9 ms, of 16 12.6 ms and of 128 73 ms: about 3 ms plus 0.55 ms per column. A group is now checked in two stages. The prechecks run on the permit the group took, and the sidecars they settle (parked or dropped) are forwarded at once. The KZG proofs of the rest are split into near-equal sub-batches of at most KZG_BATCH_COLUMNS (16): the first runs on the group's permit, each further one waits for a permit of its own, and the group holds none while it waits, so the pool cannot deadlock. The kept sidecars still reach the chain actor as one batch per block. What this trades, computed from those timings assuming free permits and cores: a block of 16 columns or fewer is still one batch, so a node's 8 sampled columns take 7.9 ms of wall-clock where 8 separate checks 16 wide would take 3.2 ms, for a third of the CPU. 128 columns are 8 sub-batches: about 13 ms against 73 ms as one batch and 26 ms as separate checks, at 101 ms of CPU against 73 ms and 414 ms. A block's proofs no longer hold one permit from first column to last, and a failed batch's one-by-one re-check covers 16 columns at most. The prechecks still run a group's sidecars in turn on one permit; how long that holds it on a live node is not measured. A panicking KZG sub-batch now loses only its own sidecars: what the prechecks settled has already been forwarded and counted, and the group's other sub-batches are unaffected. The comments say exactly what each panic arm loses. COLUMN_CHECK_PERMITS' doc says what a permit now covers. The p2p crate takes ethlambda-state-transition's test-utils as a dev-dependency, as rpc does, for a stored post-state in the new test, which runs a gloas block's columns through two sub-batches with a forged one in the second and expects every other column back in one Checked batch.
The range response handler justified its check_and_forward call with the old contract, one message for the whole answer; it is now up to two per block, and the comment says why that is still the point. CLAUDE.md's data-columns bullet named only column::chain_checks, with no grouping, no precheck/batch split and no sub-batches. beacon_wire.md's paragraph on checking fetched columns now covers the (block, index) repeats, the shared rules judged once per group, the sub-batches and what their size trades, with the measured figures; metrics.md's KZG histogram row says it times one sub-batch, which for every sub-batch after a block's first is how long it holds its permit.
|
Going through the review body in order:
The smaller points are answered in their threads: panic isolation in the KZG stage (e775465), the per-column length check (ab21359), the stale comments (5f8c9c6), the tests, and the three simplifications. |
|
/review |
🤖 Codex Code ReviewNo actionable findings in the diff. The shared checks retain signature-specific validation, and failed KZG batches fall back to individual verification. Tests could not run because rustup attempted to write to the read-only Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt |
🤖 Claude Code ReviewReview: batched data-column chain checks (PR 754)This PR is careful and well tested. The verdict-equivalence tests ( Things I checked and found correct:
Below are a few issues worth considering, then nits. Issues
Nits
Automated review by Claude · claude-opus-5-5 · custom prompt |
🤖 Gemini Code ReviewThe PR is well-structured and thoroughly tested. Splitting the chain checks into per-block One minor edge case to consider:
Automated review by Gemini · gemini-4-argon · custom prompt |
🤖 Kimi Code ReviewNo blocking correctness or security issues found. The batching design preserves the per-sidecar verdicts: per-column length checks prevent cross-column pairing, fulu header caching includes the signature to avoid cache poisoning, and failed KZG batches fall back to individual verification. Permit ownership in Minor, non-blocking:
The new tests provide good coverage of mixed verdicts, forged signatures, duplicate suppression, malformed batch entries, and KZG fallback behavior. Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt |
🤖 Grok Code ReviewThe batching is sound: per-column length checks block the cross-pairing trick, a failed batch falls back to the same per-sidecar verdicts, and header/block work is cached only after it finishes. A few availability and observability gaps are worth fixing.
Automated review by Grok (xAI) · grok-4.7 · custom prompt |
🤖 DeepSeek Code ReviewOverall this is a careful refactor. The per-sidecar verdict parity between A few things I’d consider addressing:
No consensus-critical issues found in the KZG batch verification, inclusion-proof ordering, or the fulu/gloas logic. Automated review by DeepSeek · deepseek-v4-pro · custom prompt |
sub_batches took the ceiling of the per-batch share as every batch's size, so the last batch took whatever was left: 100 columns came out as six batches of 15 and one of 10, not the "as near equal as that allows" its doc promised. It now gives the first n % count batches one column more than the rest, so 100 columns are two of 15 and five of 14. The count and the bound of 16 per batch are unchanged. The sub-batch test now pins the 100-column split.
Three paths in the column checks lost sidecars with no more than an error log: a sidecar whose own prechecks panicked, a whole group whose precheck task panicked, and a KZG sub-batch whose task panicked. A fourth, the sub-batch loop's break when a permit cannot be had, lost the group's remaining unverified sidecars without even a log. That cannot happen while the semaphore is never closed, but it would be silent if it ever were. All four now count what they drop under internal in lean_data_columns_rejected_total, and the three task-level ones log how many sidecars they lost. The repeats of a (block, index) are counted in one increment rather than one per repeat, since a hostile answer can carry thousands: inc_data_column_rejected takes a count. Also: group_by_block's doc says outright that a repeat is not judged even when the first copy fails, and that every caller hands over one source's sidecars; the COLUMN_CHECK_PERMITS doc no longer runs past 100 columns; and the blank line before the tests module is back.
…itment copy kzg_verdicts with no items fell through to an empty verify_cell_kzg_proof_batch call, which still added an observation to lean_data_column_kzg_verify_seconds. It now returns at once. No caller in the node passes an empty slice (sub-batches are never empty), but the function is public. gloas_before_kzg returned an owned copy of the bid's commitments, which gloas_block had already copied out of the block, so gossip's stateful_checks_gloas paid two copies per sidecar where main paid one. It now borrows them from the looked-up block: gossip checks against the borrow, and only ChainPrechecks copies them into the PendingKzg. ChainPrechecks' doc said it remembers the store as it stood when the group was judged; each answer is the store as it stood when that answer was first worked out. The doc now says so, and that a parent whose post-state lands mid-group costs the rest of the group a round trip through park_data_columns, which sends a judgeable sidecar straight back, not the sidecars.
beacon_wire.md put the batch's 7.9 ms of wall-clock beside the 3.2 ms eight separate checks take side by side, then said "for a third of the CPU" with no CPU figure, which read as eight separate checks costing 3.2 ms in all. It now gives both figures for both shapes: the batch is 7.9 ms of wall-clock and of CPU, the separate checks 3.2 ms of wall-clock and 25.8 ms of CPU.
|
Round three (the automated review of 5f8c9c6). Fixed in 87c266a, c17eeed, 11b312e and e335380. Fixed
Not changed
Codex found no actionable issues; its tests did not run (read-only rustup directory on the runner). |
|
/review |
🤖 Codex Code ReviewNo actionable findings. The batching preserves per-sidecar validation, isolates invalid KZG proofs through individual fallback checks, and avoids permit deadlocks. Tests could not run because Rustup attempted to write to a read-only directory. Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt |
🤖 Gemini Code ReviewThis PR is well-structured and clean. Grouping column sidecars by A few highlights that look particularly good:
LGTM! Automated review by Gemini · gemini-4-argon · custom prompt |
🤖 Claude Code ReviewReview: PR 754, batched KZG verification for chain-checked data columnsThe PR is careful and well tested. Verdict equivalence with Correctness
Suggestions
Nits
Overall the PR is approve-worthy. Item 1 and Item 2 are the ones I'd address before merging. Automated review by Claude · claude-opus-5-5 · custom prompt |
🤖 Grok Code ReviewThe batching split is sound: per-column length checks, signature-keyed header cache, and the one-by-one fallback on a failed proof batch match the solo verdicts. A few correctness holes remain.
Otherwise the permit protocol is consistent (no acquire while holding a permit, FIFO, first sub-batch reuses the group permit), and Automated review by Grok (xAI) · grok-4.7 · custom prompt |
🤖 Kimi Code ReviewNo blocking correctness or security issues found. The KZG batching preserves the important invariants: lengths are checked per column before flattening, cells/commitments/proofs remain paired correctly, and a failed batch falls back to per-column verification so valid columns are not rejected alongside a forged one. The fulu cache key also correctly includes the signature, preventing a forged signature from poisoning verdicts for the valid signed header. Permit handoff in Two non-blocking concerns:
The regression tests are particularly strong: mixed-verdict groups are compared against standalone checks, forged-cell attribution is tested, and multi-sub-batch forwarding is covered. Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt |
🤖 DeepSeek Code ReviewAutomated review by DeepSeek |
🗒️ Description / Motivation
A range or by-root answer, or a batch of parked columns replayed after their parent imports, carries many columns of the same block. Each column was checked on its own blocking task with its own
verify_cell_kzg_proof_batchcall, and the chain actor heard nothing until every sidecar of the whole call was done, so data availability waited on the slowest block of the batch. Every rule that reads only the block's shared signed header (BLS signature, proposer, finalized ancestry, a state advance past the lookahead) also ran once per sidecar.What Changed
check_and_forward(p2p/src/beacon/column_checks.rs) groups the sidecars of one call by block (the signed header's root for fulu,beacon_block_rootfor gloas), keeping one sidecar per(block, index). It then checks each group in two stages oncolumn_check_permitspermits:Prechecks, with the shared rules run once (
state_transition/src/beacon/gossip/column.rs). Every rule before the KZG proofs runs sidecar by sidecar through oneChainPrechecks, which judges what the block's sidecars share once per group:NotAfterParent, the signature, the proposer (including a state advance past the lookahead) and the finalized ancestry, keyed on the header's root and signature;The shared answer is worked out only when a sidecar first needs it. The sidecars this stage settles (parked or dropped) are forwarded at once.
KZG proofs in sub-batches. The proofs of the sidecars still standing are verified in near-equal sub-batches of at most 16 columns. Each sub-batch is one
verify_cell_kzg_proof_batchcall (fork_choice::verify_column_cells_batch) on a permit of its own; the first uses the group's permit. A failed batch re-verifies its sidecars one by one. The block's kept sidecars go to the chain actor as one batch.Gossip's
stateful_checks/stateful_checks_gloaskeep their order and are not batched.verify_column_cells_batchchecks each column's lengths, and the single-column path delegates to it.Correctness / Behavior Guarantees
chain_checks_forone sidecar at a time.(block, index)already in the same call is dropped unjudged and counted asalready_seen. Only a faulty peer sends one, and dropping it is what bounds a group atNUMBER_OF_COLUMNSsidecars.Trade-off: wall-clock against CPU
Measured locally, one thread, 9 blobs, fastest of 15 runs: one column takes 3.2 ms, a batch of 8 columns 7.9 ms, of 16 columns 12.6 ms, and of 128 columns 73 ms.
Computed from those, assuming free permits and cores (not measured concurrently):
Tests Added / Run
chain_checks_forone by one (fulu, real block sidecars).verify_data_column_sidecar_kzg_proofs.Checkedbatch across two sub-batches.cargo fmt --all -- --check: clean.cargo clippy --workspace --all-targets -- -D warnings: clean.cargo test --profile release-fast -p ethlambda-state-transition -p ethlambda-p2p: state-transition 647 passed (16 ignored), p2p 375 passed (1 ignored).make testspec suites; CI covers them.Related Issues / PRs
Part of a beacon import perf batch, each PR independent and based on
main: #752 (validation contention metrics, also editscolumn_checks.rsanddocs/metrics.md), #753 (merkle layer hashing). The engine/STF overlap PR also editsfork_choice.rs, in a different region.✅ Verification Checklist
make fmt— cleanmake lint(clippy with-D warnings) — cleanmake test(test-consensusplustest-node, atrelease-fast) — not run locally (spec suites); unit tests of the touched crates pass