Skip to content

Add runner heartbeat lease recovery and watchdog#80

Open
syifan wants to merge 1 commit into
mainfrom
runner-heartbeat-watchdog
Open

Add runner heartbeat lease recovery and watchdog#80
syifan wants to merge 1 commit into
mainfrom
runner-heartbeat-watchdog

Conversation

@syifan

@syifan syifan commented Feb 26, 2026

Copy link
Copy Markdown
Owner

Summary

  • add job heartbeat tracking and lease-based stale-job recovery to the runner claim path
  • add Supabase migration to store heartbeat_at and update claim_next_job to requeue stale running jobs
  • update runner process to emit heartbeats while executing jobs and pass lease timeout to claim RPC
  • add scheduled runner-watchdog GitHub Action to auto-restart production PM2 runner when offline
  • document new heartbeat/lease environment settings

Testing

  • cd runner && npm ci
  • cd runner && npm test

@vercel

vercel Bot commented Feb 26, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cgra-flow-ui Ready Ready Preview, Comment Feb 26, 2026 1:30am

Request Review

@github-actions

github-actions Bot commented Feb 26, 2026

Copy link
Copy Markdown

Preview Environment Ready

Service Status
Supabase Branch https://rhgkbbwtfsrxowbxznji.supabase.co
Runner Deployed via SSH (runner-80)
Docker Image cgra-flow:pr-80

The runner is connected to this PR's Supabase branch and will process jobs using the PR-specific Docker image.

@supabase

supabase Bot commented Feb 26, 2026

Copy link
Copy Markdown

Updates to Preview Branch (runner-heartbeat-watchdog) ↗︎

Deployments Status Updated
Database Thu, 26 Feb 2026 01:30:16 UTC
Services Thu, 26 Feb 2026 01:30:16 UTC
APIs Thu, 26 Feb 2026 01:30:16 UTC

Tasks are run on every commit but only new migration files are pushed.
Close and reopen this PR if you want to apply changes from existing seed or migration files.

Tasks Status Updated
Configurations Thu, 26 Feb 2026 01:30:27 UTC
Migrations Thu, 26 Feb 2026 01:30:29 UTC
Seeding Thu, 26 Feb 2026 01:30:29 UTC
Edge Functions Thu, 26 Feb 2026 01:30:29 UTC

View logs for this Workflow Run ↗︎.
Learn more about Supabase for Git ↗︎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant