Skip to content

fix(mothership): retry a job dispatch whose usage read failed instead of skipping the run - #9366

Merged
waleedlatif1 merged 2 commits into
stagingfrom
fix/scheduled-jobs-review-round-2
Oct 24, 2026
Merged

waleedlatif1 merged 2 commits into
stagingfrom
fix/scheduled-jobs-review-round-2

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

Fixes the review findings on the v0.9.33 release PR (#9339) that hold up in the scheduled-jobs lane.

  • Usage outage: checkAttributedUsageLimits also refuses with reason: 'usage_unavailable' when the payer's usage can't be read. Dispatch closed such an occurrence permanently as skipped_budget ("The usage limit was reached."). It now counts it as a failed dispatch attempt through the existing bounded retry (retryDispatch: the reconciler retries, and the last allowed attempt ends the run as failed), the way workflow admission treats the same reason as retryable.
  • Billing refusals: a run skipped for billing (a blocked account, a payer or member limit) records billing's own message as its summary.
  • Test teardown: the job lane suite closes open connections before stopping its fake worker, so a test that fails before ending its stream can't hang afterAll.

Type of Change

  • Bug fix

Testing

bun run test:integration lane.integration (real Postgres and Redis): 20 pass. The new test (an outage leaves the occurrence queued with one counted failure; a blocked account records billing's message) fails on the previous dispatch code. tsc --noEmit is clean; job unit tests pass.

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

… of skipping the run

- a usage outage (`usage_unavailable`) is a failed dispatch attempt, retried and bounded like any other, not a spent limit
- a run skipped for billing records billing's own reason
- the job lane suite closes open connections before stopping its fake worker
@vercel

vercel Bot commented Oct 24, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Oct 24, 2026 6:45pm UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no blocking issues remain.

Summary

Keeps scheduled runs queued when usage cannot be read, then retries them through the existing five-attempt limit. Billing refusals retain billing’s message.

  • Scheduled jobs retry when hosted usage cannot be checked.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Check usage before dispatch] --> B{Usage result}
  B -->|Allowed| C[Start run]
  B -->|Billing refusal| D[Skip with billing message]
  B -->|Usage unavailable| E[Count failed dispatch]
  E --> F{Five attempts reached?}
  F -->|No| G[Keep queued]
  G --> H[Reconciler retries]
  H --> A
  F -->|Yes| I[Fail without starting a turn]
Loading

Reviews (2) · Last reviewed commit: "test(mothership): cover a job dispatch r..." · Reviewed by Greptile

Comment thread apps/sim/lib/mothership/jobs/lane.integration.ts Outdated
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@cubic-dev-ai review this PR

@cubic-dev-ai

cubic-dev-ai Bot commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

@cubic-dev-ai review this PR

@waleedlatif1 I have started the AI code review. It will take a few minutes to complete.

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Heads up: you’ve reached your flex budget. Increase your flex budget or wait for usage to reset.

Turn on auto-fix | Re-trigger cubic

@waleedlatif1
waleedlatif1 merged commit 420dc7a into staging Oct 24, 2026
68 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/scheduled-jobs-review-round-2 branch October 24, 2026 19:04

This branch was previously deployed

1 inactive deployment
Preview — 9e203397 Deployed Oct 24, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant