Repository navigation
Execute asynchronous tasks in fresh worker processes #22
Description
Problem
The current implementation runs tasks inline or forks the scheduler process. Forked tasks inherit open connections, sockets, containers, and other parent state. The documentation warns about inherited resources and suppresses normal child shutdown to avoid running cleanup against copied parent resources.
A database task should not need to know which connections the planner opened before launch. Restarting the planner also loses its in-memory child registry, even when a child is still running. Fresh-process isolation and durable recovery are complementary: merely replacing fork() does not solve recovery.
Proposed improvement
Make a fresh PHP process the standard asynchronous execution path, using the same configured application executable/entrypoint as the planner. This may be a standalone script or a framework entrypoint; a separate worker binary must not be required.
- Use a bounded scalar-only worker envelope to identify the execution, its authority, the admitted configuration source/revision, and task policy. Validate its shape, size, required fields, and identity before application bootstrap; verify durable authority before constructing task services, then load the admitted configuration and rebuild the command and dependencies locally.
- Rebuild the command, container, and runtime dependencies in the worker; never inherit the planner's PHP object graph.
- Retain a managed handle that can observe the worker and request graceful or hard termination within a documented owned process scope.
- Pin the admitted source revision and configuration identity. The same executable path alone does not ensure the same deployed code.
- Keep explicit synchronous execution available for short local work, with its blocking and supervision limitations documented.
If worker lifecycle/startup limits are configured, measure them from authoritative worker launch, independently of command execution limits. Restart, shutdown, and reattachment must not reset an existing lifecycle deadline. A separate shutdown budget starts at the shutdown request. A bounded wait or hard signal is not proof that every descendant closed.
Acceptance criteria
- An asynchronous task starts with its own PHP runtime and newly constructed application dependencies.
- Worker invocation works through the configured application entrypoint without requiring a framework console.
- A bounded scalar-only worker envelope is validated for shape, size, identity, authority, source/revision, and policy before task services are constructed; invalid input, a rejected claim, or a missing admitted revision cannot invoke user code or fall back to the current revision.
- Managed workers can be observed and stopped within the declared platform/scope boundary; escaped descendants are not claimed as contained.
- Operators can identify workers by task/execution without treating a display name as ownership proof.
- Worker-launch and command-start clocks are distinct, and deadline anchors survive reattachment.
- Integration tests cover independent task resources, worker startup failure, and parent death.
Related work and evidence
This ticket owns worker input validation/reconstruction and fresh-process execution. It depends on #20. #21 owns restart decisions; #24 owns reattachment identity; #33 and #28 own their respective stop policies.
The existing fork and child lifecycle and resource warnings establish the current behavior. #12 is a fresh-worker implementation reference, not proof that every lifecycle improvement here is complete.
Activity
samdark commentedon Nov 1, 2026
This one seems to be very important. We can't use forks reliably because of lack of resource isolation, as noted in "Related work and evidence" so we need to decide about how it runs tasks overall.
@KalimeroMK what do you think?
KalimeroMK commentedon Nov 2, 2026
I agree a fork cannot give a task its own connections, and the README warning only works around that. Three things I measured on master first.
The launch command has to be configured rather than derived.
PHP_BINARYplusargv[0]does not reproduce the scheduler's own runtime: started through a wrapper withphp -d memory_limit=512M, a relaunch of the same script comes up with the ini default instead. Pointing at the same script does not reproduce the flags it was started with.Cost, this package alone and no container: 85.9 ms per fresh process against 18.0 ms per fork, of which 20 ms is autoload and 3 ms is rebuilding a 50-task schedule. That is cheap at cron granularity and worth writing into the ticket, so that a task triggering every few seconds is not moved onto this path by default.
Task identity needs settling before any of it. A closure task is identified by
closure@file:line, so an edit above it re-keys the task and a worker holding the old id finds nothing. Two closures differing only in bound state share an id today, as do two instances of one class constructed with different arguments.withId()already exists and #12 requires explicit ids; I would make that a prerequisite of this ticket instead of part of it.On scope: #19 claims a run before it is handed out, records its outcome, and reclaims a claim whose owner is gone or whose window has passed. The problem statements in #20 and #24 are worth re-reading against master once it lands, and PID reuse there is bounded by
reclaimAfter().samdark commentedon Nov 2, 2026
How often closures are actually useful as tasks?
samdark commentedon Nov 2, 2026
So do you agree that we should switch to processes?
KalimeroMK commentedon Nov 2, 2026
yes to fresh processes. One costs 85.9 ms against 18.0 ms for a fork, measured on this package alone with no container. That buys every task its own connections instead of a copy of the scheduler's, and at cron granularity it is a small price.
Closures keep working on the synchronous path, so fork does not survive as a second mode to maintain.
One thing to settle with it: the launch command has to be configured.
PHP_BINARYplusargv[0]loses the flags the scheduler was started with, so through a wrapper withphp -d memory_limit=512Ma relaunch comes back at the ini default.