Design checkpoint schema for pipeline execution state

[snipcodeit]

Description

Define a checkpoint schema stored in .mgw/active/.json that records: current pipeline step (triage/plan/execute/verify/pr), step-specific progress (e.g., which GSD phase is executing), last successful agent output path, accumulated artifacts, and resume instructions. Schema must be forward-compatible -- new pipeline steps can be added without breaking existing checkpoints.

Acceptance Criteria

• Design checkpoint schema for pipeline execution state

Milestone Context

• Milestone: v8 — Agent Reliability & Failure Recovery
• Phase: Phase 46: Checkpoint & Resume for Long-Running Pipelines
• GSD Route: plan-phase

Dependencies

none

[snipcodeit]

MGW · work-started · 2026-03-06T07:32:29Z
Milestone: v8 | Phase 46: Checkpoint & Resume

Work Started

Issue	#235 — Design checkpoint schema for pipeline execution state
Route	plan-phase
Phase	46 — Checkpoint & Resume
Progress	6/9 complete

[snipcodeit]

MGW · work-starting · 2026-03-06T07:34:46.009Z
Milestone: v8 — Agent Reliability & Failure Recovery | Phase 46: Checkpoint & Resume for Long-Running Pipelines

Work Starting

Route	plan-phase — schema design requires structured planning for forward-compatibility
Scope	medium — 4 files across lib, workflows, templates
Conflicts	none — extends existing active issue state schema with new checkpoint field

Work begins on branch issue/235-design-checkpoint-schema-for-pipeline-ex.

Affected Files

• lib/state.cjs (modify — add checkpoint migration)
• workflows/state.md (create — checkpoint schema docs)
• templates/schema.json (modify — add checkpoint field)
• lib/pipeline.cjs (reference)

[snipcodeit]

---

phase: quick-8
plan: 1
type: quick-full
wave: 1
depends_on: []
files_modified:

• workflows/state.md
• lib/state.cjs
• templates/schema.json
• commands/run/execute.md
• commands/run/triage.md
  autonomous: true
  requirements:
• Design checkpoint schema stored in .mgw/active/.json
• Record current pipeline step (triage/plan/execute/verify/pr)
• Record step-specific progress (e.g., which GSD phase is executing)
• Record last successful agent output path
• Record accumulated artifacts
• Record resume instructions
• Forward-compatible: new pipeline steps can be added without breaking existing checkpoints
  must_haves:
  truths:
  • Checkpoint schema is a new "checkpoint" field on existing issue state JSON
  • Schema includes pipeline_step, step_progress, last_agent_output, artifacts, and resume fields
  • migrateProjectState() in lib/state.cjs handles migration of existing state files (adds checkpoint with defaults)
  • workflows/state.md documents the checkpoint schema alongside existing Issue State Schema
  • commands/run/execute.md references checkpoint updates at key pipeline stages
  • commands/run/triage.md references checkpoint initialization at triage time
  • Schema is forward-compatible via additionalProperties pattern
    artifacts:
  • workflows/state.md (modified — checkpoint schema documentation added)
  • lib/state.cjs (modified — migration adds checkpoint field defaults)
  • templates/schema.json (modified — checkpoint field added to template schema if applicable)
  • commands/run/execute.md (modified — checkpoint update pseudocode at key stages)
  • commands/run/triage.md (modified — checkpoint initialization pseudocode)
    key_links:
  • lib/state.cjs
  • workflows/state.md
  • lib/pipeline.cjs
  • commands/run/execute.md
  • commands/run/triage.md

---

Plan: Design checkpoint schema for pipeline execution state

Objective

Design and implement a checkpoint schema that extends the existing .mgw/active/<issue>.json issue state format. The checkpoint tracks pipeline execution progress at a granular level, enabling resume after failures, context switches, or multi-session execution.

Context

The existing issue state schema (defined in workflows/state.md) tracks high-level pipeline_stage but lacks fine-grained execution progress. When a pipeline fails mid-execution, there is no record of which GSD phase was running, what artifacts were produced, or how to resume. This issue adds a checkpoint field to the existing state object to fill that gap.

Tasks

Task 1: Define checkpoint schema and document in workflows/state.md

• files: commands/workflows/state.md
• action: Add a new "## Checkpoint Schema" section to workflows/state.md documenting the checkpoint field structure. The checkpoint field is a nested object within the existing issue state JSON. Document each sub-field with types, defaults, and usage notes. Include a "Forward Compatibility" subsection explaining the extensibility contract.
• verify: The new section exists in state.md with complete field documentation.
• done: [ ]

Task 2: Add checkpoint migration to lib/state.cjs

• files: lib/state.cjs
• action: Extend migrateProjectState() to add a checkpoint field with sensible defaults to active issue state files that lack it. The default checkpoint should be null (checkpoint is only populated when pipeline execution begins). Add a helper function updateCheckpoint(issueNumber, checkpointData) that merges checkpoint data into an active issue state file (partial updates, preserves existing fields).
• verify: Run node -e "const {migrateProjectState}=require('./lib/state.cjs'); migrateProjectState();" and verify existing state files get the checkpoint field. Test updateCheckpoint() with a simple merge.
• done: [ ]

Task 3: Add checkpoint update pseudocode to pipeline command files

• files: commands/run/execute.md, commands/run/triage.md
• action: Add checkpoint initialization at triage (step validate_and_load) and checkpoint update calls at key pipeline stages in execute.md (after planner, after executor, after verifier). These are pseudocode annotations showing where updateCheckpoint() should be called and what data to record. Do NOT change actual executable logic — these are documentation annotations for future implementation.
• verify: The pseudocode blocks exist at the correct locations in the command files.
• done: [ ]

Verification

• checkpoint field is documented in workflows/state.md with all sub-fields
• migrateProjectState() adds checkpoint field to existing active issue files
• updateCheckpoint() function exists in lib/state.cjs
• Forward-compatibility contract is documented
• Pipeline command files reference checkpoint updates at appropriate stages

Success Criteria

• The checkpoint schema is fully defined and documented
• Existing state files are migrated cleanly (no breaking changes)
• The schema design supports adding new pipeline steps without breaking existing checkpoints
• Pipeline commands show where checkpoints should be updated

Output

• Modified: workflows/state.md, lib/state.cjs, commands/run/execute.md, commands/run/triage.md

[snipcodeit]

Summary: Design checkpoint schema for pipeline execution state

One-Liner

Added a forward-compatible checkpoint schema to the MGW issue state format that tracks fine-grained pipeline progress, accumulated artifacts, and resume instructions for failure recovery.

Changes Made

1. Checkpoint Schema Documentation (workflows/state.md)

• Added comprehensive "Checkpoint Schema" section documenting the new checkpoint field
• Defined all sub-fields: schema_version, pipeline_step, step_progress, last_agent_output, artifacts, resume, started_at, updated_at, step_history
• Documented step-specific step_progress shapes for each pipeline step (triage, plan, execute, verify, pr)
• Established Forward Compatibility Contract (5 rules: unknown-field preservation, new step extensibility, schema_version bump criteria, append-only arrays, opaque resume.context)
• Added checkpoint lifecycle diagram and update pattern example
• Added consumer reference table showing which commands read/write checkpoints

2. Checkpoint Migration & API (lib/state.cjs)

• Extended migrateProjectState() to add checkpoint: null to active issue files lacking the field (idempotent migration)
• Added initCheckpoint(pipelineStep) — creates a fresh checkpoint object with correct defaults and schema_version
• Added updateCheckpoint(issueNumber, data) — partial merge updater that:
  • Shallow-merges scalar fields (pipeline_step, last_agent_output)
  • Shallow-merges step_progress (preserves existing keys)
  • Replaces resume entirely (per opaque context contract)
  • Appends to artifacts and step_history arrays (never replaces)
  • Auto-initializes checkpoint if absent
  • Always updates the updated_at timestamp
• Exported CHECKPOINT_SCHEMA_VERSION, initCheckpoint, updateCheckpoint

3. Pipeline Command Annotations (commands/run/triage.md, commands/run/execute.md)

• Added checkpoint initialization pseudocode in triage.md (validate_and_load step)
• Added checkpoint update calls at three key pipeline stages in execute.md:
  • After planner agent completes (step 4 — records plan path and sets resume to plan-checker/executor)
  • After executor agent completes (step 8 — records summary and sets resume to verifier/PR)
  • After verifier agent completes (step 10 — records verification and sets resume to PR creation)

Key Files

• commands/workflows/state.md — 172 lines added (schema docs, lifecycle, consumers)
• lib/state.cjs — 135 lines added (migration, initCheckpoint, updateCheckpoint)
• commands/run/triage.md — 16 lines added (checkpoint init pseudocode)
• commands/run/execute.md — 64 lines added (checkpoint update pseudocode at 3 stages)

Technical Decisions

• checkpoint: null default — checkpoint is only populated when pipeline execution begins, keeping triage-only state files lightweight
• schema_version field — enables future migration without parsing ambiguity
• Append-only arrays — artifacts and step_history never lose data, supporting audit trails
• Opaque resume.context — step-specific resume data evolves independently without cross-step coupling
• Shallow merge in step_progress — allows incremental updates without requiring full progress state on every call

Verification

• migrateProjectState() adds checkpoint field to existing active issue files
• initCheckpoint() creates valid checkpoint structure with schema_version=1
• updateCheckpoint() correctly merges partial data (tested: scalar merge, append-only arrays, step_progress merge)
• Forward-compatibility contract documented with 5 explicit rules
• Pipeline command files reference checkpoint updates at appropriate stages

[snipcodeit]

Verification: Design checkpoint schema for pipeline execution state

VERIFICATION PASSED

Must-Haves Check

#	Must-Have	Status	Evidence
1	Checkpoint schema is a new "checkpoint" field on existing issue state JSON	PASS	workflows/state.md Issue State Schema now includes "checkpoint": null
2	Schema includes pipeline_step, step_progress, last_agent_output, artifacts, and resume fields	PASS	All 10 fields documented in Checkpoint Fields table with types and defaults
3	migrateProjectState() handles migration of existing state files	PASS	lib/state.cjs line adds checkpoint: null to active issue files lacking the field
4	workflows/state.md documents the checkpoint schema	PASS	172-line Checkpoint Schema section added with fields, shapes, lifecycle, and consumers
5	commands/run/execute.md references checkpoint updates at key stages	PASS	Three checkpoint update blocks added (after planner, executor, verifier)
6	commands/run/triage.md references checkpoint initialization	PASS	Checkpoint init block added in validate_and_load step
7	Schema is forward-compatible via additionalProperties pattern	PASS	5-rule Forward Compatibility Contract documented

Functional Verification

Test	Result	Detail
initCheckpoint() creates valid structure	PASS	Returns object with schema_version=1, all required fields
updateCheckpoint() merges partial data	PASS	Shallow merge preserves existing keys, appends to arrays
updateCheckpoint() append-only arrays	PASS	Second update with artifacts appended (count: 1 → 2)
migrateProjectState() adds checkpoint	PASS	All 7 active issue files gained checkpoint field
Schema version exported	PASS	CHECKPOINT_SCHEMA_VERSION=1 accessible from module

Forward Compatibility Verification

Rule	Verified
Unknown fields preserved on read-modify-write	YES — updateCheckpoint uses Object.assign with existing as base
New pipeline_step values tolerated	YES — no validation against fixed set
schema_version bump criteria documented	YES — "only for breaking structural changes"
artifacts and step_history append-only	YES — concat, never replace
resume.context treated as opaque	YES — entire resume object replaced, not merged

No Breaking Changes

• Existing state files continue to work (checkpoint defaults to null)
• No changes to pipeline_stage, retry_count, dead_letter, or triage fields
• No changes to cross-refs.json schema
• No changes to project.json schema
• All existing lib/state.cjs exports preserved

[snipcodeit]

MGW · pr-ready · 2026-03-06T07:43:16.626Z
Milestone: v8 — Agent Reliability & Failure Recovery | Phase 46: Checkpoint & Resume

PR Ready

PR #244 — #244

Added a forward-compatible checkpoint schema to the MGW issue state format that tracks fine-grained pipeline progress, accumulated artifacts, and resume instructions for failure recovery.

Testing procedures posted on the PR.
This issue will auto-close when the PR is merged.

Pipeline Summary

Stage	Status
Triage	done
Planning	done
Execution	done
Verification	done
PR Creation	done