Merge pull request #449 from DeepL/docs/pipeline-dedupe-open-prs

fix(pipeline): don't open a PR for docs that already have one

Author: shirgoldbird

.github/workflows/docs-pipeline.yml

@@ -5,6 +5,11 @@ name: Docs Pipeline
 # pipeline detects gaps, drafts + reviews the missing content, and opens a PR.
 # There is deliberately no schedule/cron trigger — spec change is the only
 # automatic entry point. (workflow_dispatch is a manual escape hatch.)
+#
+# Gaps stay open until their PR merges, so consecutive runs see the same ones.
+# The pipeline skips gaps an open PR already covers and won't open a duplicate
+# PR (pipeline/open_prs.py) — a run with nothing new to say exits cleanly having
+# done nothing. GH_TOKEN below is what lets it read the open PRs.
 on:
   push:
     branches: [main]
@@ -62,7 +67,8 @@ jobs:
           # Required — add this secret in repo Settings → Secrets → Actions.
           # The pipeline can't call Claude without it.
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
-          # ship.py / post_review.py use the gh CLI, which reads GH_TOKEN.
+          # ship.py / post_review.py and the open-PR duplicate check all use the
+          # gh CLI, which reads GH_TOKEN.
           GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
         run: |
           set -euo pipefail

.github/workflows/test-scripts.yml

@@ -4,11 +4,13 @@ on:
   pull_request:
     paths:
       - "scripts/**"
+      - "pipeline/**"
   push:
     branches:
       - main
     paths:
       - "scripts/**"
+      - "pipeline/**"
 
 jobs:
   test:
@@ -19,3 +21,14 @@ jobs:
         with:
           node-version: 22
       - run: node --test scripts/*.test.mjs
+
+  pipeline:
+    name: Test pipeline
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v5
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      # Stdlib only — these tests stub out gh and never call the network.
+      - run: python -m unittest discover -s pipeline -p "test_*.py" -v

pipeline/README.md

@@ -30,6 +30,46 @@ pip install anthropic pyyaml
 export ANTHROPIC_API_KEY=...
 ```
 
+## Duplicate PRs
+
+A gap stays open until its PR **merges**, so every run re-detects the gaps that are
+already waiting for review. Left alone, the pipeline opens one PR per run for the
+same missing pages.
+
+`pipeline/open_prs.py` is the check that prevents that. It runs twice:
+
+- **`generate.py`** drops gaps an open PR already covers, before any model call, so a
+  duplicate run costs nothing.
+- **`ship.py`** refuses to open a PR when an open one already covers the whole run.
+
+Both exit with code **3** ("nothing to do") instead of 0 or 1, and `run.py` stops the
+chain cleanly when it sees it.
+
+A gap is "already in flight" if any open PR matches it on:
+
+| Signal | Source | Catches |
+| --- | --- | --- |
+| Gap key | `<!-- docs-pipeline-gaps: [...] -->` marker in the PR body, written by `ship.py` | Any gap, including ones whose filename the model invents (two runs of the same gap produce different slugs) |
+| File path | The PR's changed files | Gaps with a predictable target path, including PRs the pipeline didn't open |
+| `covers:` frontmatter | The PR's diff | `missing_group_coverage` gaps, whose filename is unpredictable but whose group name is in the diff |
+
+Overrides, for when you want the PR anyway:
+
+```bash
+python pipeline/generate.py --ignore-open-prs   # draft claimed gaps too
+python pipeline/ship.py --latest --force-new-pr  # open a second PR
+```
+
+If `gh` can't be reached the check is skipped with a warning rather than blocking the
+run, so a missing CLI never stops the pipeline (it just allows a duplicate).
+
+The logic has tests (stdlib only, no gh, no network), run in CI on any `pipeline/**`
+change:
+
+```bash
+python pipeline/test_open_prs.py
+```
+
 ## Pipeline Steps
 
 The pipeline runs as a sequence of independent scripts. Each step reads the output of the previous one. You can run any step standalone, or use `run.py` to chain them all.
@@ -64,8 +104,12 @@ python pipeline/generate.py --section admin              # generate for one fami
 python pipeline/generate.py --type missing_orientation   # generate one gap type
 python pipeline/generate.py --force                      # regenerate even if files exist
 python pipeline/generate.py --section admin --force      # regenerate one section
+python pipeline/generate.py --ignore-open-prs            # include gaps an open PR covers
 ```
 
+Gaps that an open PR already covers are skipped before any model call. See
+[Duplicate PRs](#duplicate-prs).
+
 Each run creates a timestamped folder under `pipeline/drafts/` with the generated `.mdx` files and a `report.json` with metadata.
 
 ### 2b. Rework Existing Pages (Phase 2 alternative to Generate)
@@ -137,10 +181,15 @@ python pipeline/ship.py --latest --dry-run              # preview branch, commit
 python pipeline/ship.py --latest                        # create branch, commit, prompt before push
 python pipeline/ship.py --latest --yes                  # skip push confirmation
 python pipeline/ship.py --latest --branch docs/my-branch  # custom branch name
+python pipeline/ship.py --latest --force-new-pr           # ship even if an open PR covers it
 ```
 
 Requires `gh` CLI authenticated. Stages only `docs/**/*.mdx` and `docs.json`. Never force-pushes.
 
+Exits 3 without creating a branch when an open PR already covers the whole run, and
+stamps every PR body with the gap keys it covers so later runs recognize it. See
+[Duplicate PRs](#duplicate-prs).
+
 ### 7. Post Review (PR Suggestions)
 
 Posts "consider" findings from the review report as GitHub PR review comments with `suggestion` blocks. The reviewer can click "Apply suggestion" to accept changes directly in the PR.
@@ -306,5 +355,5 @@ The live site structure (which page sits in which tab) is read from `docs.json`
 - Python 3.8+
 - `anthropic` (for generate.py, rework.py, review.py, post_review.py)
 - `pyyaml` (for all scripts)
-- `gh` CLI (for ship.py, post_review.py)
+- `gh` CLI (for ship.py, post_review.py, and the open-PR check in generate.py)
 - `mint` / `npx` (optional, for broken-links check in promote.py)

pipeline/generate.py

@@ -12,6 +12,11 @@
     python pipeline/generate.py --dry-run                 # show what would be generated
     python pipeline/generate.py --force                   # regenerate even if files exist
     python pipeline/generate.py --section admin --force   # regenerate one section
+    python pipeline/generate.py --ignore-open-prs         # draft gaps even if a PR is open for them
+
+Gaps that an open PR already covers are skipped before any model call — a gap is
+only closed when its PR merges, so every run would otherwise redraft and reship
+the same pages. See pipeline/open_prs.py.
 """
 
 import argparse
@@ -33,6 +38,7 @@
 
 
 from util import build_authoring_system_prompt
+from open_prs import EXIT_NOTHING_TO_DO, fetch_open_prs, split_claimed_gaps
 
 REPO_ROOT = Path(__file__).resolve().parent.parent
 OPENAPI_PATH = REPO_ROOT / "api-reference" / "openapi.yaml"
@@ -292,6 +298,29 @@ def determine_output_path(gap, family_name, content=None):
     return None  # missing_description handled inline; non-generative types skipped
 
 
+def _predicted_path(gap):
+    """The repo-relative path a gap would write to, or None if unpredictable.
+
+    Used only for open-PR matching. For missing_howto / missing_group_coverage the
+    filename comes from the title the model invents, so there's nothing to predict
+    and those gaps are matched on their gap key instead.
+    """
+    out_path = determine_output_path(gap, gap.get("family", "unknown"))
+    if not out_path:
+        return None
+    try:
+        return str(out_path.relative_to(REPO_ROOT))
+    except ValueError:
+        return None
+
+
+def _pr_review_hint(claimed):
+    """One line pointing at the PRs worth reviewing, for the skip summary."""
+    numbers = sorted({pr.get("number") for _, pr, _ in claimed if pr.get("number")})
+    listed = ", ".join(f"#{n}" for n in numbers)
+    return f"Review or close {listed} to let the pipeline redraft these."
+
+
 def apply_description(gap, description):
     """Insert a frontmatter description into an existing page."""
     page_path = REPO_ROOT / gap["path"]
@@ -332,6 +361,10 @@ def main():
     parser.add_argument("--dry-run", action="store_true", help="Show what would be generated")
     parser.add_argument("--force", action="store_true", help="Regenerate even when files already exist")
     parser.add_argument("--gap-report", help="Path to existing gap report JSON (skips re-running detection)")
+    parser.add_argument(
+        "--ignore-open-prs", action="store_true",
+        help="Draft every gap, even ones an open PR already covers (default: skip those)",
+    )
     args = parser.parse_args()
 
     if args.gap_report:
@@ -348,7 +381,33 @@ def main():
 
     if not gaps:
         print("No gaps to generate for.")
-        return 0
+        return EXIT_NOTHING_TO_DO
+
+    # Drop gaps that an open PR already covers. Done here, before any model call,
+    # so a duplicate run costs nothing instead of a full generate → review cycle
+    # that ends in a PR nobody wants.
+    if args.ignore_open_prs:
+        print("Skipping the open-PR check (--ignore-open-prs).")
+    else:
+        prs, err = fetch_open_prs()
+        if prs is None:
+            print(f"Warning: could not check open PRs ({err}).")
+            print("  Proceeding without deduplication — this run may duplicate an open PR.")
+        else:
+            gaps, claimed = split_claimed_gaps(
+                gaps, prs,
+                path_for_gap=_predicted_path,
+            )
+            if claimed:
+                print(f"Skipping {len(claimed)} gap(s) already covered by an open PR:")
+                for gap, pr, reason in claimed:
+                    print(f"  - {gap['type']} ({gap.get('family', 'site-wide')}): {reason}")
+                print(f"  {_pr_review_hint(claimed)}")
+
+    if not gaps:
+        print("\nEvery detected gap is already covered by an open PR. Nothing to do.")
+        print("Merge or close those PRs, then re-run the pipeline.")
+        return EXIT_NOTHING_TO_DO
 
     standards = yaml.safe_load(open(STANDARDS_PATH))
     families = standards.get("families", {})