Skip to content

Poll queue settings as a fallback when a notifier is configured - #1582

Merged
brandur merged 2 commits into
riverqueue:masterfrom
yash-bitla:queue-settings-poll-fallback
Nov 5, 2026
Merged

brandur merged 2 commits into
riverqueue:masterfrom
yash-bitla:queue-settings-poll-fallback

Conversation

@yash-bitla

@yash-bitla yash-bitla commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1579.

A queue pause or resume reaches running clients only by NOTIFY. If the notification is sent while a client's notifier is reconnecting, it's lost, and the client keeps working the queue until it restarts. pollForSettingChanges already handles this in poll-only mode, but it only ran when Notifier == nil.

yash-bitla and others added 2 commits November 5, 2026 01:52
A queue pause or resume reaches running clients only by NOTIFY. If the
notification is lost, like while the notifier is reconnecting, the
client keeps working the queue until it restarts, because
pollForSettingChanges only ran in poll-only mode. Run it with a
notifier too, once a minute, as riverqueue#1564 did for job cancellation.

Also give the notifier's health check ping a timeout. A connection that
goes silent without an error blocked the ping, so the notifier never
reconnected.

Fixes riverqueue#1579.
…reconnect

Apply a few more fixes on top of the original changes in riverqueue#1582:

* More reliable Go state reconciliation: Polling compares database
  settings against producer's applied state. Fixes a received pause
  followed by a lost resume leaving the producer paused indefinitely.
  Metadata changes followed by reversions are also fixed.

* Metadata dedup: Polling skips metadata already applied through a
  notification, avoiding duplicate callbacks.

* Shared fallback interval: Queue settings and cancellation polling use
  `NotificationPollInterval`, defaulting to one minute. Rust uses the same.

* Cleaner shutdown: Queue settings polling stops when fetching stops.
  Cancellation polling continues while running jobs drain.

* Rust recover: Adds periodic queue settings refresh when notifier
  exists, supplementing existing refresh on reconnect Reuses existing
  timer.

* Expanded tests: Extend Yash's polling test to cover metadata. Add
  regression for notification /poll interactions and duplicate
  callbacks. Add shared Go driver and Rust Postgres/SQLite coverage.

* Docs and small fixes: Documents fallback polling, updates changeslogs,
  and fixes the Go notification test helper to encode metadata as JSON
  rather than base64.
@brandur
brandur merged commit 43e6f22 into riverqueue:master Nov 5, 2026
60 checks passed
@brandur

brandur commented Nov 5, 2026

Copy link
Copy Markdown
Contributor

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Queue pause/resume is lost if sent while the notifier is reconnecting

2 participants