Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@ Safest local verification sequence after non-trivial changes:

- `ci_timeout` is an idle timeout, not an absolute deadline: only `timeoutAnchor` re-arms when the upstream default-branch tip advances, `started` stays fixed for poll pacing, and re-arm only ever extends the deadline (fail-safe on transient base-tip failures). Value semantics (`0` unset, negative unlimited sentinel, keyword parsing) live in `config.go`; keep `config.DefaultCITimeout` and `defaultConfigYAML` in sync (`TestDefaultConfigYAML_MatchesGoDefaults`). User-facing semantics are owned by `docs/src/content/docs/reference/global-config.md`.
- GitHub readiness is the union of the exact current PR head commit's check rollup and every Actions workflow run returned by the Actions API for that same SHA. A workflow rejected before creating jobs/check-runs is absent from the commit rollup but still present in that API; run discovery errors and unknown run states fail closed instead of certifying a green rollup. Regressions: `TestGetChecksIncludesFailedWorkflowRunMissingFromPRRollup`, `TestGetChecksBindsRollupAcrossABAHeadMovement`, `TestCIStep_FailedHeadWorkflowRunPreventsChecksPassed`.
- When the GitHub status-check rollup itself is unreadable with the configured credential (a Checks-API capability failure, which fine-grained tokens hit while still being allowed to read Actions), `internal/scm/github/actions_fallback.go` derives evidence from the Actions REST API bound to the exact repository, PR, and published head. Only that classification opens the fallback; every other rollup failure keeps surfacing as before. It is fail-closed by construction: Actions cannot see other apps' check runs, so green is certified only when branch protection's required-check set is readable and every required identity maps to exactly one exact-head latest-attempt Actions result; non-green evidence needs no certification and is returned as-is. Every failure mode wraps `scm.ErrChecksUnavailable`, and the CI step parks with an ask-user outcome after `maxConsecutiveCheckReadFailures` consecutive evidence-free polls rather than stalling to `ci_timeout` (a successful read resets the budget). Regressions: `internal/scm/github/actions_fallback_test.go`, `TestCIStep_RepeatedEvidenceFailuresParkForADecision`, `TestCIStep_SuccessfulReadResetsTheEvidenceFailureBudget`.
- CI readiness never treats an unproven empty forge check list as green. Ready requires observed all-green checks, or trusted default-branch `no_ci: true` with zero registered checks (`internal/pipeline/steps/ci.go` decides whether the declaration applies; `internal/cimonitor` owns the agent-facing log vocabulary and Ready/DeclaredNoCI parse). Delayed registration, pending checks, failures, errors, unknowns, and stale-head evidence stay not-ready; registered checks on a declared no-CI repo are still honored. Regressions: `TestChecksPassed_PR607RealLogSequence`, `TestCIStep_EmptyChecksWithoutNoCIStaysNotReadyPastOldGracePeriod`, `TestCIStep_EmptyChecksWithTrustedNoCIBecomesReady`, `TestCIStep_DelayedCheckRegistrationStaysNotReadyUntilGreen`, `TestCIStep_DeclaredNoCIWithUnexpectedChecksHonorsThem`, `TestEffectiveRepoConfig_NoCITrustedOnly`.
- Reap an orphaned monitor from outside its worktree with `no-mistakes axi abort --run <id>`; it needs only `NM_HOME` and never starts a stopped daemon. A known run succeeds only with durable terminal truth, a recorded nonterminal run fails unconfirmed, and only an unknown id is an idempotent no-op. Bare `axi abort` stays worktree/branch-scoped.
- A merged or closed PR observation transactionally completes an active run and its CI step; PR lifecycle state is monotonic, so duplicate or delayed observations cannot reactivate or regress a terminal run. Startup reconciles legacy `pending` or `running` rows that already hold terminal PR state before parked-run planning and generic crash recovery. Regressions: `TestUpdateRunPRStateFinalizesActiveTerminalOutcomes`, `TestUpdateRunPRStateIgnoresDuplicateAndDelayedRegressions`, `TestReconcileTerminalPRRunsFinalizesLegacyActiveRows`, `TestRecoverOnStartup_FinalizesLegacyTerminalPRRun`, e2e `TestTerminalPRRunDisappearsFromActiveListing`.
Expand Down
2 changes: 2 additions & 0 deletions docs/src/content/docs/reference/pipeline-steps.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,6 +279,8 @@ Monitors PR health after creation and auto-fixes CI failures. Mergeability polli
- The [`ci_timeout` reference](/no-mistakes/reference/global-config/#ci_timeout) owns idle re-arming, unlimited monitoring, and fail-closed reconciliation while that gate is parked
- On GitHub, GitLab, Forgejo, and Azure DevOps, polls provider mergeability alongside CI checks while the PR remains open
- On GitHub, combines the exact current PR head commit's check rollup with Actions workflow runs for that same commit, so a workflow rejected during validation before it creates a job or check-run still blocks readiness
- On GitHub, if that check rollup itself cannot be read with the configured credential - the Checks API answers a permission failure, which some fine-grained tokens receive while still being allowed to read Actions - derives evidence from the Actions REST API for the same repository, pull request, and published head commit instead of producing nothing. That fallback reports failing and still-running jobs as usual, but certifies the commit as green only when the base branch's required-check definition is readable and every required check maps to exactly one Actions result from the current head's latest attempt. A run on another commit, a job from a superseded attempt, an incomplete listing, a required check with no unique Actions result, an unprotected base branch, and an unreadable required-check definition all stay unavailable evidence, never green
- If consecutive polls produce no CI evidence at all - the rollup is unreadable and the fallback cannot complete either - pauses for user approval after a small number of attempts with a finding naming the last provider failure, rather than repeating the same warning until the idle timeout expires. A successful read resets that budget, so ordinary transient failures are absorbed
- While the PR stays open, the TUI and terminal title show `Checks passed` once CI readiness is established and known mergeability is clear, and `no-mistakes axi` returns `outcome: checks-passed` with successful-output reporting instructions so agents can summarize the run, ask the user to review and merge, and list any pipeline fixes instead of waiting
- An empty forge check list is never treated as green unless the trusted default-branch config declares [`no_ci: true`](/no-mistakes/reference/repo-config/#no_ci). That declaration is positive durable evidence the repository intentionally has no CI; absence means CI is expected and delayed registration stays not-ready. If checks still appear on a declared no-CI repo, their actual states are honored
- If the [PR base branch](/no-mistakes/reference/repo-config/#prbase_branch) moves after `checks-passed`, keeps watching the same PR; a clean behind PR needs no action, while an actual GitHub, GitLab, Forgejo, or Azure DevOps merge conflict is auto-fixed by rebasing onto the PR base branch and re-pushing through the force-push safety guard
Expand Down
21 changes: 20 additions & 1 deletion internal/pipeline/steps/ci.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,16 @@ import (
const (
defaultBaseBranchTipResolveWindow = 30 * time.Second
defaultPublishedHeadResolveWindow = 30 * time.Second
// maxConsecutiveCheckReadFailures bounds how many polls in a row may end
// with no CI evidence at all before the step parks for a decision.
// A read that keeps failing is not the same as CI that keeps running: it
// produces nothing the user or an agent can act on, and letting it repeat
// until the idle timeout turns a credential or provider problem into an
// hours-long silent stall whose only artifact is a repeated warning. The
// budget is generous enough that an ordinary transient failure - a rate
// limit, a dropped connection, a provider blip - is absorbed by the next
// successful poll, which resets it.
maxConsecutiveCheckReadFailures = 5
)

// CI monitoring status messages. These are surfaced to the user and parsed by
Expand Down Expand Up @@ -244,6 +254,10 @@ func (s *CIStep) Execute(sctx *pipeline.StepContext) (*pipeline.StepOutcome, err
timeoutFailingChecks := []string{}
timeoutMergeConflict := false
lastMonitorLog := ""
// checkReadFailures counts consecutive polls that produced no CI evidence.
// A successful read resets it, so only an uninterrupted run of failures can
// reach the bound.
checkReadFailures := 0
timeoutOutcome := func() (*pipeline.StepOutcome, error) {
sctx.Log("CI timeout reached")
if len(timeoutFailingChecks) > 0 || timeoutMergeConflict {
Expand Down Expand Up @@ -347,8 +361,13 @@ func (s *CIStep) Execute(sctx *pipeline.StepContext) (*pipeline.StepOutcome, err
if err != nil {
clearCIMonitorReady(sctx)
lastMonitorLog = ""
sctx.Log(fmt.Sprintf("warning: could not check CI: %v", err))
checkReadFailures++
sctx.Log(fmt.Sprintf("warning: could not check CI (attempt %d/%d): %v", checkReadFailures, maxConsecutiveCheckReadFailures, err))
if checkReadFailures >= maxConsecutiveCheckReadFailures {
return ciEvidenceUnavailableOutcome(err, checkReadFailures), nil
}
} else {
checkReadFailures = 0
// checksPending is the narrow execution state: only checks that are
// actively running or queued block a rerun or issue escalation. A
// provider-cancelled check is terminal enough to enter the transient
Expand Down
25 changes: 25 additions & 0 deletions internal/pipeline/steps/ci_checks.go
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,31 @@ func ciMergeabilityOutcome(summary, description string) *pipeline.StepOutcome {
}
}

// ciEvidenceUnavailableOutcome parks the run when consecutive polls produced no
// CI evidence at all. It is deliberately not a failure verdict on the code: the
// step never learned whether the checks passed, so the finding says exactly
// that and asks for a decision, instead of leaving the run to expire against
// its idle timeout with nothing but repeated warnings in the log.
func ciEvidenceUnavailableOutcome(err error, attempts int) *pipeline.StepOutcome {
reason := "the provider reported no reason"
if err != nil {
reason = err.Error()
}
findings := Findings{
Summary: fmt.Sprintf("CI evidence could not be read on %d consecutive attempts", attempts),
Items: []Finding{{
Severity: "warning",
Description: fmt.Sprintf("CI check evidence is unavailable, so this run cannot tell whether the checks passed: %s", reason),
Action: types.ActionAskUser,
}},
}
findingsJSON, _ := json.Marshal(findings)
return &pipeline.StepOutcome{
NeedsApproval: true,
Findings: string(findingsJSON),
}
}

func ciMonitoringTimeoutOutcome() *pipeline.StepOutcome {
findings := Findings{
Summary: "CI monitoring timed out before PR was merged or closed",
Expand Down
105 changes: 105 additions & 0 deletions internal/pipeline/steps/ci_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2672,3 +2672,108 @@ func TestCIStep_DelayedSameNameCheckRetainsLegacyNameBehavior(t *testing.T) {
t.Fatal("new conclusive link did not retire the rerun record")
}
}

// A check read that keeps failing produces no evidence at all, so the step must
// stop at a bounded, actionable outcome instead of repeating the same warning
// until the idle timeout expires.
func TestCIStep_RepeatedEvidenceFailuresParkForADecision(t *testing.T) {
t.Parallel()
dir, baseSHA, headSHA := setupGitRepo(t)

env := fakeCIGHChecksError(t, "OPEN", "MERGEABLE", "gh: Resource not accessible by personal access token (HTTP 403)")

prURL := "https://github.com/test/repo/pull/42"
ag := &mockAgent{name: "test"}
sctx := newTestContextWithDBRecords(t, ag, dir, baseSHA, headSHA, config.Commands{})
sctx.Env = env
sctx.Run.PRURL = &prURL
sctx.Config.CITimeout = 24 * time.Hour

var logs []string
sctx.Log = func(s string) { logs = append(logs, s) }

current := time.Date(2026, time.January, 1, 12, 0, 0, 0, time.UTC)
polls := 0
step := &CIStep{
pollIntervalOverride: 30 * time.Second,
now: func() time.Time { return current },
waitForNextPoll: func(ctx context.Context, interval time.Duration) error {
polls++
if polls > maxConsecutiveCheckReadFailures {
t.Fatalf("step polled %d times without parking, logs: %v", polls, logs)
}
current = current.Add(interval)
return nil
},
}

outcome, err := step.Execute(sctx)
if err != nil {
t.Fatalf("Execute() error = %v", err)
}
if outcome == nil || !outcome.NeedsApproval {
t.Fatalf("outcome = %+v, want a parked decision", outcome)
}
var findings Findings
if err := json.Unmarshal([]byte(outcome.Findings), &findings); err != nil {
t.Fatalf("unmarshal findings: %v", err)
}
if !strings.Contains(findings.Summary, "could not be read") {
t.Fatalf("findings.Summary = %q, want the unavailable evidence named", findings.Summary)
}
if len(findings.Items) != 1 || findings.Items[0].Action != types.ActionAskUser {
t.Fatalf("findings.Items = %+v, want one ask-user finding", findings.Items)
}
if !strings.Contains(findings.Items[0].Description, "unavailable") {
t.Fatalf("finding = %q, want it to say the evidence was unavailable", findings.Items[0].Description)
}
}

// A poll that succeeds resets the budget, so intermittent read failures never
// accumulate into a park while CI is still reporting.
func TestCIStep_SuccessfulReadResetsTheEvidenceFailureBudget(t *testing.T) {
t.Parallel()
dir, baseSHA, headSHA := setupGitRepo(t)

// Alternate: a failed read, then a pending check, repeatedly.
checks := []string{}
for i := 0; i < maxConsecutiveCheckReadFailures*3; i++ {
if i%2 == 0 {
checks = append(checks, fakeChecksReadFailure)
} else {
checks = append(checks, `[{"name":"build","state":"IN_PROGRESS","bucket":"pending"}]`)
}
}
env := fakeCIGHSequence(t, "OPEN", checks)

prURL := "https://github.com/test/repo/pull/42"
ag := &mockAgent{name: "test"}
sctx := newTestContextWithDBRecords(t, ag, dir, baseSHA, headSHA, config.Commands{})
sctx.Env = env
sctx.Run.PRURL = &prURL
sctx.Config.CITimeout = 24 * time.Hour

current := time.Date(2026, time.January, 1, 12, 0, 0, 0, time.UTC)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
sctx.Ctx = ctx

polls := 0
step := &CIStep{
pollIntervalOverride: 30 * time.Second,
now: func() time.Time { return current },
waitForNextPoll: func(ctx context.Context, interval time.Duration) error {
polls++
current = current.Add(interval)
if polls >= maxConsecutiveCheckReadFailures*2 {
cancel()
return ctx.Err()
}
return nil
},
}

if _, err := step.Execute(sctx); !errors.Is(err, context.Canceled) {
t.Fatalf("Execute() error = %v, want the monitor still polling", err)
}
}
14 changes: 14 additions & 0 deletions internal/pipeline/steps/steps_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,10 @@ func fakeCIGHRerun() {
os.Exit(0)
}

// fakeChecksReadFailure is the sequence entry that makes the fake gh answer a
// check read with a provider failure instead of a check list.
const fakeChecksReadFailure = "ERROR"

func fakeCIGHSequenceHandler(args []string) {
state := os.Getenv("FAKE_CLI_STATE")
checksPath := os.Getenv("FAKE_CLI_CHECKS_PATH")
Expand Down Expand Up @@ -532,6 +536,10 @@ func fakeCIGHSequenceHandler(args []string) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
if entry := entries[index]; entry == fakeChecksReadFailure {
fmt.Fprintln(os.Stderr, "gh: Resource not accessible by personal access token (HTTP 403)")
os.Exit(1)
}
fmt.Println(entries[index])
os.Exit(0)
}
Expand Down Expand Up @@ -559,6 +567,12 @@ func fakeCIGHSequenceHandler(args []string) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
if entry := entries[index]; entry == fakeChecksReadFailure {
// A sequence entry can ask for a failed read, so a test can drive
// the monitor's bounded no-evidence handling.
fmt.Fprintln(os.Stderr, "gh: Resource not accessible by personal access token (HTTP 403)")
os.Exit(1)
}
printFakeCommitChecks(entries[index], args)
os.Exit(0)
}
Expand Down
Loading
Loading