Skip to content

feat: SentTaskHandle and ReceivedTaskHandle - #2636

Open
nicolas-grekas wants to merge 27 commits into
php:mainfrom
nicolas-grekas:bgworker-tasks
Open

nicolas-grekas wants to merge 27 commits into
php:mainfrom
nicolas-grekas:bgworker-tasks

Conversation

@nicolas-grekas

@nicolas-grekas nicolas-grekas commented Sep 7, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #2617 and #2635: review the commits on top of them.

Tasks for background workers, the task half of #2319. A worker declares the namespaces it receives, and a sender names the namespace:

worker {
	file images.php
	background
	num 4
	tasks image.resize image.thumbnail
}
// anywhere
$task = new FrankenPHP\SentTaskHandle('image.resize', ['file' => 'photo.jpg']);
while (null !== $update = $task->read()) {
}

// the background worker, in its tick() loop
while ($task = $handle->receive()) {
    $task->update(['progress' => 50]);
    $task->complete(resize($task->getNamespace(), $task->getPayload()));
}
  1. Constructing a SentTaskHandle waits for a thread to pick the task up and throws on timeout, so a busy worker pushes back instead of queueing without bounds.
  2. A pool shares one queue for all its namespaces, and each task wakes one parked thread through its WorkerHandle, so the loop of feat: background workers and FrankenPHP\WorkerHandle #2617 doesn't change.
  3. Both task handles are Io\Poll\Handle with a getStream(), like the worker's. abandon(), or dropping the handle, gives up on the task, which the worker learns from update() returning false; a worker letting go of a task without complete() makes the sender's next read() throw.
  4. Each side waits on its own descriptor, eventfd on Linux, kqueue on macOS, a socket pair on Windows, pooled between tasks: 8 senders on 8 threads went from 2k to 32k tasks/s along the way.

The metrics follow: frankenphp_busy_workers and frankenphp_worker_queue_depth count tasks, plus frankenphp_worker_task_count{outcome} and frankenphp_worker_task_time for a Metrics implementation that also implements TaskMetrics.

Left out compared to #2319: a queue ahead of pickup, and cancelling a task before it.

@nicolas-grekas

Copy link
Copy Markdown
Contributor Author

Pushed 23d3094 after benchmarking the task path in the Docker builder image (harness kept out of the PR). Three changes: a send wakes one parked thread instead of every thread of the pool; the sender waits for the pickup in the kernel, on the task's socket, instead of in a Go select (waking a thread parked inside a cgo callback costs the scheduler a P hand-off); and the queue mutex is no longer held across a syscall nor taken twice per wake-up. One thread: 549 to 285µs per task. A pool of 8: 1745 to 320µs. 8 senders on 8 threads: 2k to 32k tasks/s. Absolute numbers are that VM's, the ratios are the point.

@nicolas-grekas
nicolas-grekas force-pushed the bgworker-tasks branch 5 times, most recently from 4c5af56 to 2b77a5b Compare September 9, 2026 17:53
@nicolas-grekas
nicolas-grekas force-pushed the bgworker-tasks branch 5 times, most recently from e0be4ac to fd177fc Compare September 14, 2026 06:16
@nicolas-grekas
nicolas-grekas force-pushed the bgworker-tasks branch 11 times, most recently from a8f1a7f to a9d72df Compare September 18, 2026 16:58
@nicolas-grekas nicolas-grekas changed the title feat: frankenphp_send_task(), frankenphp_receive_task() and friends feat: SentTaskHandle and ReceivedTaskHandle Sep 18, 2026
FRANKENPHP_WORKER_BACKGROUND is now a flag set to 1, like FRANKENPHP_WORKER,
since the name comes from the handle. The threads endpoint names background
threads after their worker rather than their script, which several workers
may share.
zend_enum_get_case_cstr() only asserts that the case exists, so a snapshot
or payload written against another version of an enum dereferenced NULL in
release builds, or took a plain constant of that name for an object. The
case is now looked up in the constants table and checked, and a mismatch
throws the existing LogicException.
… handle

A persistent copy is malloc memory outside the memory_limit of the request
making it, and every occurrence of a shared value gets its own copy: a
hundred references to one megabyte took a hundred megabytes. The check now
counts the bytes of the copy and refuses one past memory_limit, which also
bounds the walk of a value shared many times over, and says which rule a
value breaks.

frankenphp_get_vars() created a cgo.Handle and copied the namespace on
every read, about 250ns and three allocations, and throughput fell as
readers were added (global handle map, GC). The snapshots being copied now
sit on a stack of the thread instead, which also pairs the nested read an
autoloader can make.

Conflicting declarations name both workers, a worker reading its own vars
before its first tick is told so, and an unknown namespace says where it
was looked for.
A pickup crossing an abort no longer leaves a second wake-up for the sender's next wait.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant