Skip to content

Add a Where escape hatch to JobDeleteManyParams - #1570

Merged
brandur merged 2 commits into
riverqueue:masterfrom
mokevnin:delete-many-where
Nov 1, 2026
Merged

brandur merged 2 commits into
riverqueue:masterfrom
mokevnin:delete-many-where

Conversation

@mokevnin

Copy link
Copy Markdown
Contributor

JobListParams has Where(sql, NamedArgs) for predicates its filters can't express, but JobDeleteManyParams doesn't. We need to delete queued jobs by a value in their args: when erasing a customer's personal data (GDPR), every non-running job whose args name that customer has to go, in the same transaction as the erasure. Today that means listing the jobs first and deleting them by ID, or going around River to the river_job table, which we'd rather not do.

This adds JobDeleteManyParams.Where(sql string, namedArgs ...NamedArgs), mirroring JobListParams.Where and built on the same dblist path, so it works with every driver:

client.JobDeleteManyTx(ctx, tx, river.NewJobDeleteManyParams().
    Kinds("send_email").
    Where("(args->>'customer_id')::bigint = any(@ids)", river.NamedArgs{"ids": ids}))

Two details:

  • A Where predicate counts as a filter, so params with only a Where don't need UnsafeAll (and calling UnsafeAll after Where panics, like after any other filter).
  • Unlike the list version, each delete predicate is wrapped in parentheses. JobDeleteMany appends its state != 'running' guard with AND, so an unparenthesized OR would widen the deletion past the other filters and to running jobs. The closing parenthesis goes on its own line so a trailing -- comment in the predicate can't comment it out.

Named args are merged into a fresh map, so the caller's maps aren't modified.

The other ports have no raw predicate on their list either, so this is Go-only, like JobListParams.Where.

Tests

  • delete_many_params_test.go: Where counts for filtersEmpty and makes UnsafeAll panic; the generated where clause wraps each predicate (including one with a trailing comment); named args merge without mutating the caller's maps; the receiver isn't modified.
  • client_test.go (Test_Client_JobDeleteMany): deleting by args->>'customer_id' together with Kinds; a Where alone without UnsafeAll; and an OR predicate that must not delete a running job or a job of another kind. That last test fails with the parentheses removed (it then deletes all three jobs).
  • The driver suite in riverdriver/riverdrivertest/job_delete.go already covers JobDeleteMany with a raw WhereClause and named args on every driver.

🤖 Generated with Claude Code

JobListParams has Where for predicates its filters can't express, but
JobDeleteManyParams doesn't, so deleting queued jobs by a value in their
args (for example every job naming a customer whose data is erased) meant
listing the jobs first and deleting them by ID, or going around River.

JobDeleteManyParams.Where mirrors JobListParams.Where and goes through the
same dblist path, so it works with every driver. A Where predicate counts
as a filter, so params with only a Where don't need UnsafeAll. Each
predicate is wrapped in parentheses: JobDeleteMany appends its
`state != 'running'` guard with AND, so an unparenthesized OR would widen
the deletion past the other filters and to running jobs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@brandur

brandur commented Oct 31, 2026

Copy link
Copy Markdown
Contributor

Thanks @mokevnin! Do you think you could sign our https://github.com/riverqueue/rivercla? I'll take a look at this in a bit.

@mokevnin

Copy link
Copy Markdown
Contributor Author

Thanks @brandur! Signed the CLA in riverqueue/rivercla#41.

A few tweaks for riverqueue#1570:

* Add a set of shared driver tests to verify wide compatibility for
  predicates, `OR` safety, trailing comments, and skipped locked jobs.

* Docs clarifying batch limits and skipped lock jobs.

* Some method and test ordering cleanup (largely for alphabetical order
  which is our internal convention for organizing things).
@brandur

brandur commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Ty!

Pushed one more commit here with:

  • A few more delete-related tests that gets some edges like predicates, OR safety, and trailing comments.
  • Docs that mention batch limits and skipped locked jobs.
  • Alphabetized some functions and test cases according to project convention.

@brandur
brandur merged commit 737146d into riverqueue:master Nov 1, 2026
15 checks passed
@brandur

brandur commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Released v0.51.0

mokevnin added a commit to getsphericon/sphericon that referenced this pull request Nov 1, 2026
…, drop the ent mapping of river_job

River v0.51.0 adds JobDeleteManyParams.Where (riverqueue/river#1570), so Erasure's purge deletes
the jobs that name an erased Contact through River on the erasure transaction instead of an ent
entity over River's table. River deletes a batch at a time, so the purge repeats until a batch
comes back short. A job a fetch holds locked is skipped as a running one is: its worker re-checks
that what it refers to still exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
mokevnin added a commit to getsphericon/sphericon that referenced this pull request Nov 1, 2026
… on River v0.51.0 (#905)

* chore(deps): update River to v0.51.0

Picks up the fixes for a lost cancellation notification, the notifier run loop
that could block Client.Stop forever, and InsertManyFast/QueueUpdate without a
listener. The release carries no database migration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(erasure): delete queued jobs through River's JobDeleteManyTx, drop the ent mapping of river_job

River v0.51.0 adds JobDeleteManyParams.Where (riverqueue/river#1570), so Erasure's purge deletes
the jobs that name an erased Contact through River on the erasure transaction instead of an ent
entity over River's table. River deletes a batch at a time, so the purge repeats until a batch
comes back short. A job a fetch holds locked is skipped as a running one is: its worker re-checks
that what it refers to still exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants