Repository navigation
fix(storage): grow the beacon vote table exactly instead of doubling - #740
Conversation
🤖 Claude Code ReviewReview of PR 740: grow the latest-message table exactly rather than by doubling The change looks correct and I found nothing blocking. Correctness
Performance
Tests
Style
Approve. Automated review by Claude (Anthropic) · sonnet · custom prompt |
🤖 Codex Code ReviewI found one correctness concern.
No fork-choice / attestation / justification / SSZ logic changes are in this PR; the production change is a localized memory-usage optimization and otherwise looks reasonable. Automated review by OpenAI Codex · gpt-5.4 · custom prompt |
🤖 Kimi Code ReviewI'll review this PR which optimizes memory growth for the Overall AssessmentThis is a well-scoped optimization PR that addresses a real memory inefficiency. The logic is mostly sound, but I have several concerns ranging from edge cases to code clarity. Detailed Findings1. Potential Panic on Integer Overflow (Lines 4288-4290)if messages.capacity() <= slot {
messages.reserve_exact(slot + 1 + LATEST_MESSAGES_HEADROOM - messages.len());
}Bug: Fix: Use let new_cap = slot
.checked_add(1 + LATEST_MESSAGES_HEADROOM)
.expect("validator index + headroom fits in usize");
messages.reserve_exact(new_cap.saturating_sub(messages.len()));2.
|
| Aspect | Status |
|---|---|
latest_message read path unchanged |
✓ No diff |
None initialization preserves "no vote" semantics |
✓ |
slot bounds check (messages.len() <= slot) preserved |
✓ |
MAX_PLAUSIBLE_VALIDATOR_INDEX still enforced elsewhere |
✓ (unchanged) |
Recommendations Summary
| Priority | Item | Location |
|---|---|---|
| Medium | Fix potential usize overflow in capacity calculation |
store.rs:4289 |
| Low | Clarify reserve_exact + resize interaction with comment |
store.rs:4289-4291 |
| Low | Rename test for accuracy | store.rs:9132 |
| Low | Strengthen test assertions against fragility | store.rs:9154-9159 |
| Trivial | Fix comment grammar ("name" → "reference") | store.rs:4285 |
The core optimization is sound and the test coverage is appreciated. Address the overflow check (Item 1) before merge.
Automated review by Kimi (Moonshot AI) · kimi-k2.6 · custom prompt
…740) Store::latest_messages is a dense table indexed by validator index, and set_latest_message grew it with resize, which at least doubles the capacity when an index lands past it. A registry that keeps gaining validators crosses that edge soon after the table is first sized, so on the mainnet follower the table had room for about twice the registry and roughly half of its 256 MB was never used. Growth now reserves the new index plus a fixed headroom, so a registry that gains a few validators an epoch still reallocates rarely, but the table stays within the headroom of the registry's size. That frees about 120 MB on the mainnet follower. Cherry-picked from fix/storage-latest-messages-exact-growth at 83aabfc.
|
@MegaRedHand could you address the Kimi review before this goes in? The items
Items 5 and 7 in Kimi's list (index bound, mutex held across the allocation) |
|
@pablodeymo all five are in, one commit each:
Kimi's items 5 and 7 stay out, as you said. One more from a second review pass, |
get_head tallied every validator's latest message on every call, about 1M on mainnet, and a live import pays it twice for the first timely block of a slot (on_block's boost-gate walk, then the actor's get_head). That tally was most of get_head's ~17 ms in the eth-5 replay. The store now keeps the vote weights in a node array (Lighthouse unstable's proto-array layout: one node per block, carrying its PENDING, EMPTY and FULL weights). One attestation is one vote batch: Store::record_latest_messages records the messages and moves the attesters' weight under one lock, and the next read refreshes the subtree totals in one reverse pass over the blocks. The head walk reads them through cached_node_weights. The proposer boost, gloas's previous-slot rule and the walk bound depend on the clock, so they stay read-time. There are no best-child pointers: under gloas the choice moves without any vote, which is why Lighthouse stopped maintaining them. What the cache cannot follow incrementally (finality pruning, a live-window removal, a restart, a new justified balances snapshot, the gloas rule switch, a block stored without its node) marks it stale, and the next read rebuilds it from the latest messages at the old tally's cost, about once per epoch. A rebuild rescans LiveChain under the lock, so an API read that scanned before a removal cannot rebuild the removed blocks back in. compute_node_weights stays, as the reference. Equivalence is checked by a proptest against a naive tally (with the incremental state also equal to a fresh rebuild after every operation), on the random trees at every step of every seed, by a new test that feeds votes, blocks and equivocations through update_latest_messages, on_block's insertion order and insert_equivocating_index, and at every head check of every fork-choice fixture. Local benchmark (1M validators, 73 blocks, laptop): the tally 27 ms; a warm cached read 7 us; a read after a 15.6k-vote batch 49 us; the batch itself 1.3 ms; a rebuild 17 ms. Cherry-picked from perf/beacon-proto-array at b4adbaf. Adapted onto nightly: both vote writers (set_latest_message and the new record_latest_messages) grow the dense vote table through one grow_latest_messages helper, which keeps #740's exact growth plus headroom; the storage and fork-choice imports are merged with #759's and #761's caches. Replaces #767, reverted in the previous commit.
…740): review fixes - fix(storage): reserve the vote table once per attestation, not per index. An aggregate lists its attesters in ascending order, so growing the table one index at a time copied it whenever an index passed the headroom: hundreds of copies of a registry-sized table for one aggregate spanning it. Store::reserve_latest_messages makes room up to the highest index first. - refactor(storage): check the vote table's capacity arithmetic (checked_add), and say why the resize after it cannot reallocate. - test(storage): assert the growth contract (index plus headroom) rather than a ratio, name the test after it, and check that a reserved batch of ascending votes never moves the table. On nightly the table has a second writer, #775's Store::record_latest_messages, which records a whole attestation under one lock and is the only one on the production path. It reserves for the batch's highest index itself, under that lock, so #740's call from fork_choice::update_latest_messages is not needed here; both writers share the slot and reservation helpers. Updates nightly's #740 from fix/storage-latest-messages-exact-growth at 83aabfc to be37485 (the branch head, bfb4697, adds only a merge of main). Claude-Session: https://claude.ai/code/session_011vghNmndKPkZinHRkVTwVB
|
/review |
🤖 Claude Code ReviewReview: PR 740 (dense vote table growth strategy)Overall: The change is correct and well-reasoned. The exact-growth-plus-headroom strategy avoids leaving up to half of a registry-sized table unused. That is roughly 56 B × N entries, which is tens of MB at mainnet scale. The arithmetic in 1. The fork-choice comment misattributes the cost (
|
🤖 Codex Code ReviewNo actionable findings. Batch reservation preserves vote-update semantics, and the allocation arithmetic correctly accounts for Tests could not run because rustup attempted to write to a read-only directory. Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt |
🤖 Grok Code ReviewI'll inspect the surrounding fork-choice and store code so the review is grounded in how the vote table is locked and grown. Automated review by Grok (xAI) · grok-4.7 · custom prompt |
🤖 Gemini Code ReviewThe changes look good. Pre-reserving capacity for batches in Automated review by Gemini · gemini-4-argon · custom prompt |
🤖 DeepSeek Code ReviewThe optimization is reasonable: reserving once for a batch avoids repeated A few issues worth addressing:
Production correctness of the Automated review by DeepSeek · deepseek-v4-pro · custom prompt |
🤖 Kimi Code ReviewNo blocking issues found. The allocation logic is sound:
Non-blocking security note: Automated review by Kimi (Moonshot AI) · kimi-k3.1-pro · custom prompt |
|
/review |
🤖 Kimi Code ReviewAutomated review by Kimi (Moonshot AI) |
🤖 Gemini Code ReviewThe changes look good. The rationale for combining Automated review by Gemini · gemini-4-argon · custom prompt |
🤖 Claude Code ReviewReview of PR 740: dense vote table growthOverall the change is small, correct, and well documented. The arithmetic in 1. Dropping amortized growth makes non-batched callers potentially quadratic (
|
🤖 Codex Code ReviewNo actionable findings. The reservation arithmetic is correct, attesting indices come from validated paths, and vote selection semantics remain unchanged. Tests could not run because rustup attempted to write outside the permitted workspace. Automated review by OpenAI Codex · gpt-6.1-sol · custom prompt |
🤖 DeepSeek Code ReviewOverall the allocation math is correct: A few notes:
No correctness issues found in the reserve/resize logic or the fork-choice ordering. Automated review by DeepSeek · deepseek-v4-pro · custom prompt |
🤖 Grok Code ReviewThe allocation change is correct and the batch reserve is the right fix.
Automated review by Grok (xAI) · grok-4.7 · custom prompt |
Store::latest_messages is a dense table indexed by validator index, and set_latest_message grew it with resize, which at least doubles the capacity when an index lands past it. A registry that keeps gaining validators crosses that edge soon after the table is first sized, so on the mainnet follower the table had room for about twice the registry and roughly half of its 256 MB was never used. Growth now reserves the new index plus a fixed headroom, so a registry that gains a few validators an epoch still reallocates rarely, but the table stays within the headroom of the registry's size. That frees about 120 MB on the mainnet follower.
The sum was safe only because a debug assertion bounds the index, so the line could not be read as correct on its own. Name the target and check the addition.
The reserve above it is what makes the growth exact; without the note, the resize reads as if it could still double the table.
…ises reserve_exact guarantees at least the requested capacity, not exactly it, so the old name claimed more than the code does.
`grown < 2 * sized` held only because of the indices the test picked. Check what the code promises instead: the highest index inside the headroom fits without a reallocation, and growth past it lands at the new index plus the headroom, as an upper bound.
An indexed attestation's attesters are sorted ascending. With exact growth, each index past the headroom reallocated and copied the whole table, so the first aggregate spanning the registry after a start copied a registry-sized table hundreds of times while holding the beacon lock. update_latest_messages now reserves room for the batch's highest index first, so the batch reallocates once at most. Indices are taken by max, not by position, so an unsorted caller is still covered.
From the second automated review round: - update_latest_messages's comment credited the repeated copies to growing one index at a time; they come from this PR's exact growth, which replaced resize's doubling. Said so, so the reservation does not read as a micro-optimization someone could drop. - The headroom constant says the table never shrinks, and why that is fine. - The growth test says why its upper bounds stay although reserve_exact promises only at least: they are what tell exact growth from doubling, and std's Vec takes exactly what it asks for. update_latest_messages is the only production caller of set_latest_message, so the batch reservation needs no API of its own yet.
81de754 to
2ba572f
Compare
…s equivocators From the third automated review round: - reserve_latest_messages states set_latest_message's precondition (an index from a validated attestation, since the table is allocated up to it at once) and that update_latest_messages is its one caller and the only production path recording votes, the table being rebuilt through it after a restart. Four reviewers asked for this or for a release-mode bound; the bound stays out of this PR, as agreed in review. - update_latest_messages says why equivocators count towards the reserved index, and why max rather than last.
🗒️ Description / Motivation
Store::latest_messages(the beacon fork choice's latest vote per validator) is a dense table indexed by validator index.set_latest_messagegrew it withresize, which at least doubles the capacity whenever an index lands past it. A registry that keeps gaining validators crosses that edge soon after the table is first sized. So on our mainnet follower the table had room for about twice the registry, and roughly half of its 256 MB (heap profile) was never used.Exact growth has its own trap: an indexed attestation's attesters are sorted ascending. Growing the table one index at a time, each index past the headroom reallocated and copied the whole table, so the first aggregate spanning the registry after a start copied a registry-sized table hundreds of times while holding the beacon lock.
What Changed
crates/storage/src/store.rs:LATEST_MESSAGES_HEADROOM(4096) spare entries withreserve_exact, thenresizes as before. The capacity arithmetic is checked, and the growth lives in one helper,reserve_latest_message_slot.Store::reserve_latest_messages(highest): makes room for every index up tohighestin one reallocation at most.crates/blockchain/state_transition/src/beacon/fork_choice.rs:update_latest_messagesreserves for the batch's highest index before recording it. The highest is taken bymax, not by position, so an unsorted caller is still covered.Correctness / Behavior Guarantees
Tests Added / Run
the_latest_message_table_grows_with_headroom_rather_than_doubling:a_reserved_batch_of_ascending_votes_never_moves_the_table: indices spaced wider than the headroom, recorded in ascending order after one reservation, never move the table's buffer or change its capacity.cargo test --profile release-fast -p ethlambda-storage --lib: 212 passed, 1 ignored.cargo test --profile release-fast -p ethlambda-state-transition --lib fork_choice: 93 passed.Related Issues / PRs
✅ Verification Checklist
make fmt— cleanmake lint(clippy with-D warnings) — clean. Rancargo clippy --locked -p ethlambda-storage -p ethlambda-state-transition --all-targets -- -D warnings(clean), not the full workspace targetmake test(test-consensusplustest-node, atrelease-fast) — all passing. Ran only the tests above