Skip to content

Prevent slow chain polls from starving fee updates - #1146

Open
benthecarman wants to merge 3 commits into
lightningdevkit:mainfrom
benthecarman:fix-audit-ldk-node-issue-159
Open

benthecarman wants to merge 3 commits into
lightningdevkit:mainfrom
benthecarman:fix-audit-ldk-node-issue-159

Conversation

@benthecarman

Copy link
Copy Markdown
Contributor

The bitcoind polling loop used a biased select over two separate two-second intervals, with the chain polling tick prioritized over the fee rate update tick. If each chain poll took longer than the polling interval, the chain tick was already ready every time we returned to the select, so the fee rate update branch was never taken and fee estimates could go stale indefinitely.

Since both intervals fire at the same rate anyway, we now drop the separate fee rate interval and update fee rate estimates right after each chain poll whenever the best block changed. Every step still races against the stop signal, so shutdown remains prompt.

This was found by Project Loupe.

@ldk-reviews-bot

ldk-reviews-bot commented Oct 9, 2026 •

Copy link
Copy Markdown

I've assigned @tnull as a reviewer!
I'll wait for their review and will help manage the review process.
Once they submit their review, I'll check if a second reviewer would be helpful.

@ldk-reviews-bot
ldk-reviews-bot requested a review from tnull October 9, 2026 18:30
@hash-money

Copy link
Copy Markdown

Field data point from a mainnet deployment, in case it helps prioritise review.

Setup: ldk-node at 0cea341 (0.8.0+git) as an LSP, mainnet, chain source ChainSource::BitcoindRpc against our own Bitcoin Core 31.1 over a VPC. Mempool at the time: ~75k transactions.

Symptom: after switching this node from Esplora to bitcoind RPC, NodeStatus::latest_fee_rate_cache_update_timestamp stayed at the start-up value for 3.5 h, while latest_lightning_wallet_sync_timestamp / latest_onchain_wallet_sync_timestamp advanced every 3–5 s and the node tracked the tip. Nothing was logged: no Failed to retrieve fee rate estimates, no timeout, and estimatesmartfee / getmempoolinfo answered in ~20 ms when run by hand. On the Esplora source the same node had refreshed its fee cache every 30 min all day.

Why the poll exceeds the 2 s tick here: sampling NodeStatus showed the wallet sync timestamps advancing in 3–5 s steps, i.e. each poll_and_update_listeners takes 3–5 s. We have not profiled the poll itself, but update_mempool_entries_cache (getrawmempool plus one sequential getmempoolentry per txid not yet cached) is the obvious candidate: the first poll after start-up, which fetches every entry, took about six minutes here (the sync timestamps stayed at the start-up value until then). So every poll outlasts the 2 s tick, the chain tick is always ready again when the loop returns to the select!, and with biased; the fee branch is never reached, exactly as described above.

With this PR cherry-picked onto 0cea341 (clean pick, branch here): restarted 04:33:22Z (2026-10-18); the first continuous poll finished at 04:39:42Z and the fee cache refreshed right behind it (first movement of latest_fee_rate_cache_update_timestamp on this source other than start-up); the next block, 971946, refreshed it again at 04:42:17Z, within seconds of the block. Steady-state wallet sync timestamps advance every 2–3 s, same as before the patch. So the fix works as described on a node where the unpatched loop had starved the fee update for hours.

Happy to test a revised version on this node if useful.

@tnull tnull left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a rebase by now.

Comment thread src/chain/bitcoind.rs Outdated
}

let current_best_block_hash = channel_manager.current_best_block().block_hash;
if last_best_block_hash != Some(current_best_block_hash) {

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we sure we want this? Core added mempool-based fee estimation with 32.0 / bitcoin/bitcoin#34075, so only updating after each block would lose that inter-block resolution, no?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now also refresh if 30s have passed since the last successful update, so we pick up Core's mempool-based estimates between blocks without doing ~10 RPCs every 2s poll tick.

The bitcoind polling loop used a biased select over two separate
two-second intervals, with the chain polling tick prioritized over the
fee rate update tick. If each chain poll took longer than the polling
interval, the chain tick was already ready every time we returned to
the select, so the fee rate update branch was never taken and fee
estimates could go stale indefinitely.

Since both intervals fire at the same rate anyway, we now drop the
separate fee rate interval and update fee rate estimates right after
each chain poll whenever the best block changed. Every step still
races against the stop signal, so shutdown remains prompt.

This was found by Project Loupe.

This change was made with the help of Claude Code.
@benthecarman
benthecarman force-pushed the fix-audit-ldk-node-issue-159 branch from 418dc30 to 7a2e99e Compare November 17, 2026 17:05

@tnull tnull left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely against the approach, but wouldn't the more trivial fix be to move the _ = fee_rate_update_interval.tick() => { arm up to the second position?

Comment thread src/chain/bitcoind.rs Outdated
// polling interval to starve the fee rate updates.
let fee_rate_refresh_interval = Duration::from_secs(FEE_RATE_REFRESH_INTERVAL_SECS);
let mut last_best_block_hash = None;
let mut last_fee_rate_update: Option<Instant> = None;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, kind of a regression that on startup we immediately re-retrieve the fee rate even though we just did (hence the .reset() above). Can we try to skip the initial update again?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants