Skip to content

Keep a checkpoint record per task instead of one for the whole schedule - #17

Merged
KalimeroMK merged 3 commits into
masterfrom
feat/per-task-checkpoint
Nov 2, 2026
Merged

KalimeroMK merged 3 commits into
masterfrom
feat/per-task-checkpoint

Conversation

@KalimeroMK

@KalimeroMK KalimeroMK commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #16 — review that one first. The base moves to master once it lands.

#16 made the checkpoint record which tasks had already run, by id. That left one record holding every run emitted at a due time, so its size grew with the tasks sharing that time, and persist() rewrote the whole list after every single run.

Measured over one due minute, with every task due at it:

tasks largest stored value written over the minute
10 341 B → 42 B 2.1 kB → 292 B
100 2.9 kB → 42 B 148 kB → 2.5 kB
1000 29 kB → 42 B 14.3 MB → 25 kB

Each task now keeps the time of its own last emitted run under a key of its own, and the schedule keeps the time it was last evaluated to the end. Writing is one value per run instead of a growing list, and a stored value is the same size whatever the schedule, so no backend limit on the size of a value is in reach.

The position of the schedule moves only once a tick has drained. A process that died partway leaves it where it was, so the runs it never started stay due — which is what lets a task be seeded strictly after its own record, and drops the pass that had to tell a re-discovered run from a fresh one.

A checkpoint written by either earlier format is read for its time alone, so an upgrade resumes after it without replaying.

Summary by Sourcery

Keep checkpoint state per task so schedule persistence remains bounded and interrupted runs resume correctly.

Enhancements:

  • Store each task's last emitted run independently while retaining a separate schedule evaluation position, keeping checkpoint values bounded and reducing persistence to one value per run.
  • Resume interrupted and upgraded schedules without replaying completed runs, while preserving runs left due after a partial tick.
  • Use bounded hashed cache keys for per-task checkpoint records and validate the new checkpoint payloads.

Tests:

  • Add coverage for per-task checkpoint storage, interrupted first ticks, malformed task records, cache key limits, and checkpoint write behavior.

@sourcery-ai

sourcery-ai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Reviewer's Guide

Replaces the growing, shared checkpoint payload with independently persisted last-run timestamps per task plus a schedule-level drained position, allowing crash-safe resumption, bounded value sizes, one write per run, and backward-compatible upgrades.

Sequence diagram for per-task checkpoint persistence

sequenceDiagram
    participant Generator as TaskGenerator
    participant Checkpoint
    participant Cache
    participant Task

    Generator->>Checkpoint: lastRun(taskId)
    Checkpoint->>Cache: getMultiple(scheduleKey, taskKeys)
    Cache-->>Checkpoint: schedule position and task timestamps
    Checkpoint-->>Generator: task last-run time
    Generator->>Task: getNextRunDate(seedTime)
    Generator-->>Task: yield TaskContext
    Generator->>Checkpoint: save(taskId, runTime)
    Checkpoint->>Cache: set(taskKey(taskId), runTime)
    Generator->>Checkpoint: markTick(now)
    Checkpoint->>Cache: set(cacheKey(), schedule position)
Loading

Entity relationship diagram for checkpoint records

erDiagram
    SCHEDULE_CHECKPOINT {
        string schedule_key PK
        timestamp last_evaluated
    }
    TASK_CHECKPOINT {
        string task_key PK
        string task_id
        timestamp last_run
    }
    SCHEDULE_CHECKPOINT ||--o{ TASK_CHECKPOINT : owns
Loading

File-Level Changes

Change Details Files
Split checkpoint persistence into one schedule position and one last-run record per task.
  • Store task run times under hashed, PSR-16-safe keys.
  • Persist each task run independently and persist the schedule’s drained-through position separately.
  • Load all current task records in bulk while retaining compatibility with earlier checkpoint formats.
src/Generator/Checkpoint.php
Resume and generate runs from each task’s individual checkpoint, advancing the schedule position only after a tick drains.
  • Seed each task strictly after its own last run, or from the schedule position for tasks without a record.
  • Remove shared-time emitted-run filtering and always mark a successfully drained tick.
  • Pass the schedule’s task identifiers into checkpoint loading.
src/Generator/TaskGenerator.php
Validate and document the task collection as an ordered list.
  • Update the task return type annotation to list semantics.
src/Schedule.php
Expand coverage for per-task records, migration behavior, malformed task records, and updated write counts.
  • Verify task-specific checkpoint keys and constant-size schedule state.
  • Verify invalid per-task payloads are rejected.
  • Update expectations for the separate task and schedule writes while preserving legacy checkpoint upgrade coverage.
tests/Generator/TaskGeneratorTest.php
Document the new checkpoint storage and write-efficiency behavior.
  • Add the per-task checkpoint change and its scaling characteristics to the unreleased changelog.
CHANGELOG.md

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@KalimeroMK
KalimeroMK added this pull request to stack #18 October 31, 2026 15:55

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Needs a human reviewer. If a per-task checkpoint is wrong or fails to advance as intended, the scheduler can run that task again after a restart; any side effects the task already performed would remain after reverting this change. The checkpoint records can be cleared or rebuilt, but effects from a duplicate run may not be reversible.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

@KalimeroMK
KalimeroMK force-pushed the feat/per-task-checkpoint branch from 0d01fb4 to b069fc7 Compare October 31, 2026 16:03
@KalimeroMK
KalimeroMK requested a review from a team October 31, 2026 16:06
@KalimeroMK
KalimeroMK force-pushed the feat/per-task-checkpoint branch from b069fc7 to 96c35d8 Compare October 31, 2026 18:41
Base automatically changed from fix/checkpoint-task-identity to master November 1, 2026 10:48
One record held every run emitted at a due time, so its size grew with the
tasks sharing that time and every run rewrote all of them: a thousand tasks due
at the same minute wrote 14.3 MB over that minute, and a backend with a limit
on the size of a stored value would eventually refuse the write.

Each task now keeps the time of its own last emitted run, and the schedule
keeps the time it was last evaluated to the end. A stored value is 42 bytes
whatever the schedule, and the same thousand tasks write 25 kB.

The position of the schedule moves only once a tick has drained, so a process
that died partway leaves the runs it never started still due; a task picked up
after its own record no longer has to be told from one that was already run.
@KalimeroMK
KalimeroMK force-pushed the feat/per-task-checkpoint branch from 96c35d8 to 37687a7 Compare November 1, 2026 10:48
@KalimeroMK
KalimeroMK removed this pull request from stack #18 November 1, 2026 13:00
@KalimeroMK
KalimeroMK added this pull request to stack #39 November 1, 2026 13:00

@samdark samdark left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two regressions need correction before merge:

  1. In TaskGenerator.php:149, a new stateful scheduler can lose unstarted runs after its first due tick is interrupted. Query nextRunDate() at 00:00, start the due tick at 00:01, and interrupt it after the first task completes. No drained schedule position exists yet, so a fresh scheduler seeds the remaining task from 00:01 and skips its pending run. The base resumes that task. Persist the initial seed before processing can be interrupted and cover recovery without a prior idle tick.

  2. In Checkpoint.php:244, appending the task hash breaks previously valid names on portable cache backends. daily_subscription_invoices used a 51-character key but now requests 68-character task keys, which a 64-character-limited cache rejects during scheduler construction. PSR-16 guarantees support only through 64 characters. Derive a bounded task key from the schedule/task identity and add a 64-character-limited cache regression.

The position of the schedule moves only once a tick has drained, which is what
leaves the runs of a process that died partway still due. Until the first tick
drains there is no position at all, and a task with no record of its own was
then seeded from the now of whichever process read the schedule next.

A daemon that asks for the next run date at 00:00, wakes for the tick at 00:01
and is killed after its first task therefore lost the rest of that tick: the
replacement process found no position, seeded the untouched tasks from 00:01
and stepped over the runs they never started.

The point the runs are computed from is now written down as they are computed,
before any of them is emitted, so an interrupted first tick leaves its unstarted
runs behind a position rather than behind nothing.
A task key spelled the schedule name out and appended the hash of the task id,
so it ran 41 characters longer than that name. A schedule called
daily_subscription_invoices kept its own position under a key of 51 characters,
which every backend took, and asked for task keys of 68, which a backend
limited to 64 refuses — while reading the records back, so the schedule could
not even be constructed.

Hashing both halves holds the key at 57 characters whatever the schedule and
the tasks are called. The tasks of one schedule still share a prefix, so they
can be told apart from another schedule's when the cache is inspected.
@KalimeroMK
KalimeroMK merged commit 2e8e268 into master Nov 2, 2026
27 checks passed
@KalimeroMK
KalimeroMK deleted the feat/per-task-checkpoint branch November 2, 2026 10:57
KalimeroMK added a commit that referenced this pull request Nov 2, 2026
#17 landed while this was open. The checkpoint it adds and the identifiers this changes do not
touch the same code; the changelog is the only file both write to.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants