Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 15 additions & 5 deletions .github/workflows/no-mistakes-required.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ on:

permissions:
contents: read
pull-requests: read

# GitHub concurrency groups retain at most one pending run, replacing older
# pending runs even when cancel-in-progress is false. Give body-bearing events
Expand All @@ -40,16 +41,25 @@ jobs:
steps:
- name: Verify no-mistakes signature in PR body
env:
PR_BODY: ${{ github.event.pull_request.body }}
GH_TOKEN: ${{ github.token }}
PR_AUTHOR: ${{ github.event.pull_request.user.login }}
PR_NUMBER: ${{ github.event.pull_request.number }}
run: |
set -eu
marker='Updates from [git push no-mistakes](https://github.com/kunchenguid/no-mistakes)'
if printf '%s' "${PR_BODY:-}" | grep -qF -- "$marker"; then
echo "Found no-mistakes signature in PR #${PR_NUMBER} body."
exit 0
fi
max_attempts=5
attempt=1
while [ "$attempt" -le "$max_attempts" ]; do
pr_body="$(gh api "repos/${GITHUB_REPOSITORY}/pulls/${PR_NUMBER}" --jq '.body // ""')"
if printf '%s' "$pr_body" | grep -qF -- "$marker"; then
echo "Found no-mistakes signature in PR #${PR_NUMBER} body."
exit 0
fi
if [ "$attempt" -lt "$max_attempts" ]; then
sleep 2
fi
attempt=$((attempt + 1))
done
{
echo "::error::This PR was not raised through no-mistakes."
echo
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,7 +151,7 @@ After installing from npm, the skill is available under the installed package di
```

- **Incremental commits** - each successful iteration is a separate unsigned git commit, so you can cherry-pick or revert individual changes without GPG or SSH signing prompts blocking the run; if `git commit` fails, gnhf preserves the uncommitted work and asks the next agent iteration to repair it
- **Failure handling** - failed iterations are rolled back with `git reset --hard` except commit failures, which preserve uncommitted work for repair; agent-reported failures proceed to the next iteration immediately, retryable hard agent errors use exponential backoff, and permanent agent errors such as Claude low credit balance abort immediately and print the run log path. Complete no-op iterations are reported as failures and count toward the consecutive-failure abort limit. If the run exits with a pending commit failure, the exit summary warns that uncommitted changes were left for repair.
- **Failure handling** - failed iterations are rolled back with `git reset --hard` except commit failures, which preserve uncommitted work for repair; agent-reported failures proceed to the next iteration immediately, retryable hard agent errors use exponential backoff, and permanent agent errors such as Claude low credit balance abort immediately and print the run log path. When an agent completes a turn without a final answer, gnhf nudges it once inside the same session to continue before recording a failure, and records the nudge in the run log. Covered: OpenCode, Rovo Dev, and ACP targets reuse their live session, and `claude` and `codex` resume the exact session the empty turn used - they skip the nudge when that session cannot be identified or resumed (no session id reported, or agent args such as `--no-session-persistence` or `--ephemeral`). Not covered: `pi`, because gnhf runs it with `--no-session`, and `copilot`, because it has no verified exact-session resume contract ([#193](https://github.com/kunchenguid/gnhf/issues/193)). Whenever the nudge is skipped the failure names the reason, because a retry that cannot reach the original session could only invent a summary. Complete no-op iterations are reported as failures and count toward the consecutive-failure abort limit. If the run exits with a pending commit failure, the exit summary warns that uncommitted changes were left for repair.
- **Runtime caps** - `--max-iterations` stops before the next iteration begins, `--max-tokens` can abort mid-iteration once reported usage reaches the cap, and `--stop-when` ends the loop after an iteration whose agent output reports the natural-language condition is met unless a commit failure needs repair first; resumed runs reuse the saved stop condition unless you pass a new value, or `--stop-when ""` to clear it; pending commit-failure repair work is preserved and other uncommitted work is rolled back, and in the interactive TUI the final state remains visible until you press Ctrl+C to exit
- **Iteration finalization** - agents are expected to finish validation, stop any background processes they started, and only then emit the final JSON result for the iteration
- **Graceful interrupts** - in the interactive TUI, the first Ctrl+C requests a graceful stop and lets the current iteration finish (or ends backoff early), the second Ctrl+C force-stops immediately, and `SIGTERM` also force-stops immediately
Expand Down
112 changes: 112 additions & 0 deletions e2e/e2e.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,118 @@ describe("gnhf e2e", () => {
expect(debugEvents).toContain("run:complete");
}, 30_000);

it("recovers one completed empty turn in the same session and records the nudge", async () => {
const cwd = createRepo();
tempDirs.push(cwd);
const logDir = mkdtempSync(join(tmpdir(), "gnhf-e2e-logs-"));
tempDirs.push(logDir);
const mockLogPath = join(logDir, "mock-opencode.jsonl");

const result = await runCli(
cwd,
[
"recover the empty response",
"--agent",
"opencode",
"--max-iterations",
"1",
"--prevent-sleep",
"off",
],
{
env: {
...createTestEnv(mockLogPath, tempDirs),
GNHF_MOCK_OPENCODE_EMPTY_ONCE: "1",
},
},
);

expect(result.code).toBe(0);
expect(git(["rev-list", "--count", "HEAD"], cwd)).toBe("2");

const messages = readJsonLines(mockLogPath).filter(
(entry) => entry.event === "message:start",
);
expect(messages).toHaveLength(2);
expect(messages[1]?.sessionId).toBe(messages[0]?.sessionId);
expect(messages[1]?.prompt).toBe(
"You did not produce a final answer. Continue and provide your final summary now.",
);

const debugLogPath = findRunLogPath(cwd);
const continuationEvents = readJsonLines(debugLogPath).filter(
(entry) => entry.event === "opencode:output:continuation",
);
expect(continuationEvents).toEqual([
expect.objectContaining({
attempt: 1,
sessionId: messages[0]?.sessionId,
prompt:
"You did not produce a final answer. Continue and provide your final summary now.",
}),
]);
expect(
readFileSync(join(dirname(debugLogPath), "notes.md"), "utf-8"),
).toContain("**Summary:** mocked objective complete");
}, 30_000);

it("fails after one continuation when both completed turns are empty", async () => {
const cwd = createRepo();
tempDirs.push(cwd);
const logDir = mkdtempSync(join(tmpdir(), "gnhf-e2e-logs-"));
tempDirs.push(logDir);
const mockLogPath = join(logDir, "mock-opencode.jsonl");

const result = await runCli(
cwd,
[
"stop after one empty-response retry",
"--agent",
"opencode",
"--max-iterations",
"1",
"--prevent-sleep",
"off",
],
{
env: {
...createTestEnv(mockLogPath, tempDirs),
GNHF_MOCK_OPENCODE_ALWAYS_EMPTY: "1",
},
},
);

expect(result.code).toBe(0);
expect(git(["rev-list", "--count", "HEAD"], cwd)).toBe("1");

const messages = readJsonLines(mockLogPath).filter(
(entry) => entry.event === "message:start",
);
expect(messages).toHaveLength(2);
expect(messages[1]?.sessionId).toBe(messages[0]?.sessionId);
expect(messages[1]?.prompt).toBe(
"You did not produce a final answer. Continue and provide your final summary now.",
);

const debugLogPath = findRunLogPath(cwd);
const debugEntries = readJsonLines(debugLogPath);
expect(
debugEntries.filter(
(entry) => entry.event === "opencode:output:continuation",
),
).toHaveLength(1);
const iterationEnd = debugEntries.find(
(entry) => entry.event === "iteration:end",
);
expect(iterationEnd).toMatchObject({
success: false,
summary: "OpenCode produced no final answer",
});
expect(
readFileSync(join(dirname(debugLogPath), "notes.md"), "utf-8"),
).toContain("[ERROR] OpenCode produced no final answer");
}, 30_000);

it("runs on the current branch and pushes each successful iteration", async () => {
const cwd = createRepo();
tempDirs.push(cwd);
Expand Down
40 changes: 40 additions & 0 deletions e2e/fixtures/mock-opencode-server.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -157,6 +157,34 @@ function emitCompletedEvents(sessionId, summary) {
return output;
}

function emitEmptyCompletedEvents(sessionId) {
broadcast({
directory: "/repo",
payload: {
type: "message.updated",
properties: {
sessionID: sessionId,
info: {
id: "msg-empty-1",
role: "assistant",
tokens: {
input: 3,
output: 0,
cache: { read: 0, write: 0 },
},
},
},
},
});
broadcast({
directory: "/repo",
payload: {
type: "session.idle",
properties: { sessionID: sessionId },
},
});
}

function applyWorkspaceChange(sessionId) {
const session = sessions.get(sessionId);
if (!session?.directory) return;
Expand Down Expand Up @@ -282,6 +310,18 @@ const server = createServer(async (req, res) => {
return;
}

const shouldEmitEmpty =
process.env.GNHF_MOCK_OPENCODE_ALWAYS_EMPTY === "1" ||
(process.env.GNHF_MOCK_OPENCODE_EMPTY_ONCE === "1" &&
!session?.emittedEmptyTurn);
if (shouldEmitEmpty && session) {
session.emittedEmptyTurn = true;
emitEmptyCompletedEvents(sessionId);
res.writeHead(204);
res.end();
return;
}

applyWorkspaceChange(sessionId);
emitCompletedEvents(sessionId, "mocked objective complete");
res.writeHead(204);
Expand Down
163 changes: 163 additions & 0 deletions src/core/agents/acp.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -516,6 +516,169 @@ describe("AcpAgent", () => {
expect(result.output).toEqual(VALID_OUTPUT);
});

it("nudges the same session once when a completed turn produced no output text", async () => {
const { runtime, calls } = createFakeRuntime([
{ events: [], result: { status: "completed" } },
{
events: [textDelta(JSON.stringify(VALID_OUTPUT))],
result: { status: "completed" },
},
]);
const agent = makeAgent(runtime);
const onUsage = vi.fn();

const result = await agent.run("p", "/w", { onUsage });

expect(result.output).toEqual(VALID_OUTPUT);
expect(calls.startTurnInputs).toHaveLength(2);
expect(calls.ensureSessionInputs).toHaveLength(1);
expect(calls.startTurnInputs[1]?.text).toContain(
"You did not produce a final answer",
);
expect(calls.startTurnInputs[1]?.text).not.toContain(
"gnhf final output contract",
);
expect(onUsage).toHaveBeenLastCalledWith(result.usage);
});

it("nudges when a completed turn produced only whitespace output", async () => {
const { runtime, calls } = createFakeRuntime([
{
events: [textDelta(" \n\t")],
result: { status: "completed" },
},
{
events: [textDelta(JSON.stringify(VALID_OUTPUT))],
result: { status: "completed" },
},
]);
const agent = makeAgent(runtime);

const result = await agent.run("p", "/w");

expect(result.output).toEqual(VALID_OUTPUT);
expect(calls.startTurnInputs).toHaveLength(2);
expect(calls.startTurnInputs[1]?.text).toContain(
"You did not produce a final answer",
);
});

it("reports usage across both the empty turn and its continuation", async () => {
// The first turn streams only reasoning, so it completes with no output
// text while still burning output tokens that must survive into the total.
const firstTurnText = "thinking hard";
const secondTurnText = JSON.stringify(VALID_OUTPUT);
const { runtime } = createFakeRuntime([
{
events: [textDelta(firstTurnText, "thought")],
result: { status: "completed" },
},
{
events: [textDelta(secondTurnText)],
result: { status: "completed" },
},
]);
const agent = makeAgent(runtime);

const result = await agent.run("p", "/w");

expect(result.usage.outputTokens).toBe(
Math.ceil(firstTurnText.length / 4) +
Math.ceil(secondTurnText.length / 4),
);
});

it("does not add fallback input to a cumulative continuation update", async () => {
const { runtime } = createFakeRuntime([
{
events: [
{
type: "status",
text: "u",
tag: "usage_update",
used: 100,
size: 1000,
},
textDelta(JSON.stringify(VALID_OUTPUT)),
],
result: { status: "completed" },
},
{ events: [], result: { status: "completed" } },
{
events: [
{
type: "status",
text: "u",
tag: "usage_update",
used: 160,
size: 1000,
},
textDelta(JSON.stringify(VALID_OUTPUT)),
],
result: { status: "completed" },
},
]);
const agent = makeAgent(runtime);

await agent.run("warmup", "/w");
const onUsage = vi.fn();
const result = await agent.run("recover", "/w", { onUsage });

expect(result.usage.inputTokens).toBe(60);
expect(onUsage).toHaveBeenLastCalledWith(result.usage);
});

it("continues when tool work leaves no final output message", async () => {
const { runtime, calls } = createFakeRuntime([
{
events: [
textDelta("I will inspect the file."),
{ type: "tool_call", text: "Read file", toolCallId: "1" },
],
result: { status: "completed" },
},
{
events: [textDelta(JSON.stringify(VALID_OUTPUT))],
result: { status: "completed" },
},
]);
const agent = makeAgent(runtime);

const result = await agent.run("p", "/w");

expect(result.output).toEqual(VALID_OUTPUT);
expect(calls.startTurnInputs).toHaveLength(2);
});

it("rejects after the ACP continuation is also empty", async () => {
const { runtime, calls } = createFakeRuntime([
{ events: [], result: { status: "completed" } },
{ events: [], result: { status: "completed" } },
]);
const agent = makeAgent(runtime);

await expect(agent.run("p", "/w")).rejects.toThrow(
"ACP agent returned no output text",
);
expect(calls.startTurnInputs).toHaveLength(2);
});

it("does not nudge when the empty turn failed rather than completed", async () => {
const { runtime, calls } = createFakeRuntime([
{
events: [],
result: {
status: "failed",
error: { message: "transient", retryable: true },
},
},
]);
const agent = makeAgent(runtime);

await expect(agent.run("p", "/w")).rejects.toThrow("transient");
expect(calls.startTurnInputs).toHaveLength(1);
});

it("throws PermanentAgentError when the runtime reports a non-retryable failure", async () => {
const { runtime } = createFakeRuntime([
{
Expand Down
Loading
Loading