Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ Prompt clarification extension for [pi coding agent](https://github.com/earendil
- **Vague input detection** - Flags structurally empty input (blank, single-character, or pure punctuation) and lets the LLM judge ambiguity for everything else via an injected system-prompt guideline
- **`/clarify` toggle** - Enable or disable clarification with `/clarify on|off`
- **`~` bypass prefix** - Prefix prompts with `~` to skip clarification for one turn
- **Network/proxy issue handling** - When a tool call fails with a network, proxy, connectivity, or rate-limit error, the model stops and asks the user how to proceed instead of silently retrying or switching approaches

## Installation

Expand Down Expand Up @@ -68,6 +69,20 @@ Prefix your prompt with `~` to skip clarification for one turn:

> `~` is used instead of `!` because pi reserves `!`/`!!` as the built-in shell-command prefix.

### Network / proxy issue handling

When a tool call fails with a network, proxy, connectivity, or rate-limit error (timeout, `ECONNREFUSED`, `ENOTFOUND`, `ETIMEDOUT`, `ECONNRESET`, proxy error, `429`, `502/503/504`, quota exceeded, certificate issues, …), the extension:

1. Injects a `NETWORK_ISSUE_PROMPT` guideline into the system prompt telling the model to **not silently retry more than once** and to **not silently switch approaches**.
2. Detects network/proxy error signatures in failed tool results (`tool_result` with `isError: true`) and appends a reminder that nudges the model to call `clarify_prompt` with options like:
- "Retry the same request"
- "Switch to a different proxy / network"
- "Wait and try again later"
- "Use a fallback approach / different tool"
- "Skip this step and continue"

This behavior is governed by the same `/clarify` toggle and is disabled for RPC/print mode (no interactive UI).

## How vague input is detected

Clarification is driven by the LLM, not by keyword matching. When enabled, the extension appends a `CLARIFY_PROMPT` guideline to the system prompt instructing the model to call `clarify_prompt` whenever a request is ambiguous, has unclear outcomes/scope, admits multiple valid interpretations, or is missing constraints.
Expand Down
50 changes: 49 additions & 1 deletion clarify-utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,54 @@ INSTEAD: Call \`clarify_prompt\` with:

Wait for the tool result. You may call \`clarify_prompt\` multiple times for different unclear aspects.`;

/** Exported for testing: network/proxy issue instructions appended to system prompt */
export const NETWORK_ISSUE_PROMPT = `╔══════════════════════════════════════════════════════════════════════════════╗
║ NETWORK / PROXY ISSUE HANDLING ║
╚══════════════════════════════════════════════════════════════════════════════╝

When a tool call fails with a network, proxy, connectivity, or rate-limit error
(timeout, ECONNREFUSED, ENOTFOUND, ETIMEDOUT, ECONNRESET, proxy error, 429, 502,
503, 504, quota exceeded, certificate issues, etc.):

1. Do NOT silently retry more than once.
2. Do NOT silently switch to an alternative approach or tool.
3. STOP and call the \`clarify_prompt\` tool to ask the user how they want to
proceed. Provide concrete options such as:
- "Retry the same request"
- "Switch to a different proxy / network"
- "Wait and try again later"
- "Use a fallback approach / different tool"
- "Skip this step and continue"
4. Wait for the user's choice before continuing.`;

/** Exported for testing: regex matching network/proxy/rate-limit error signatures */
export const NETWORK_ERROR_PATTERN =
/(network|proxy|timeout|timed out|ECONNREFUSED|ECONNRESET|ENOTFOUND|ETIMEDOUT|EHOSTUNREACH|EAI_AGAIN|socket hang up|unreachable|bad gateway|rate\s*limit|quota|429|502|503|504|超时|代理|网络|连接被拒绝|curl:\s*\(\s*(7|28|35|56|60)\s*\)|SSL|certificate)/i;

/** Exported for testing: true when an errored tool result looks like a network/proxy/rate-limit failure */
export function isNetworkIssueResult(result: {
isError?: boolean;
content?: unknown;
}): boolean {
if (!result.isError) return false;
const text = JSON.stringify(result.content ?? "");
return NETWORK_ERROR_PATTERN.test(text);
}

const NETWORK_REMINDER_TEXT = `\n\n[NETWORK/PROXY ISSUE DETECTED] This tool failed due to a network, proxy, connectivity, or rate-limit problem. Do NOT keep retrying or switch approaches silently. Call the clarify_prompt tool and ask the user which remedy they prefer (retry / switch proxy or network / wait / fallback / skip).`;

/** Exported for testing: builds a tool_result patch that appends the network reminder */
export function buildNetworkReminderResult<T extends { content?: unknown[] }>(
event: T,
): { content: Array<{ type: "text"; text: string }> } {
return {
content: [
...((event.content ?? []) as Array<{ type: "text"; text: string }>),
{ type: "text", text: NETWORK_REMINDER_TEXT },
],
};
}

/** Exported for testing: tool guidelines that appear in system prompt when tool is active */
export const CLARIFY_GUIDELINES = [
"STOP: If the user prompt is vague, ambiguous, or unclear, you MUST use the clarify_prompt tool FIRST.",
Expand Down Expand Up @@ -81,7 +129,7 @@ export function buildClarifyAgentStartResult({
// Append after the base system prompt so critical base instructions keep
// primacy; prepending would displace them.
const result: ClarifyAgentStartResult = {
systemPrompt: `${systemPrompt}\n\n${CLARIFY_PROMPT}`,
systemPrompt: `${systemPrompt}\n\n${CLARIFY_PROMPT}\n\n${NETWORK_ISSUE_PROMPT}`,
};

if (isVague) {
Expand Down
115 changes: 115 additions & 0 deletions index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,10 @@
import {
CLARIFY_PROMPT,
CLARIFY_GUIDELINES,
NETWORK_ISSUE_PROMPT,
buildClarifyAgentStartResult,
buildNetworkReminderResult,
isNetworkIssueResult,
shouldBypassClarify,
stripClarifyBypassPrefix,
isVagueInput,
Expand Down Expand Up @@ -250,6 +253,118 @@ function runTests() {
}
},
},
{
name: "NETWORK_ISSUE_PROMPT contains key rules",
run: () => {
if (!NETWORK_ISSUE_PROMPT.includes("Do NOT silently retry")) {
throw new Error("Expected prompt to forbid silent retries");
}
if (!NETWORK_ISSUE_PROMPT.includes("clarify_prompt")) {
throw new Error("Expected prompt to mention clarify_prompt tool");
}
if (!NETWORK_ISSUE_PROMPT.includes("Switch to a different proxy")) {
throw new Error("Expected prompt to suggest proxy/network options");
}
},
},
{
name: "isNetworkIssueResult: detects common network/proxy/rate-limit errors",
run: () => {
const cases = [
"curl: (28) Operation timed out",
"fetch failed: connect ECONNREFUSED 127.0.0.1:8080",
"getaddrinfo ENOTFOUND api.example.com",
"Error: socket hang up",
"ProxyError: bad gateway",
"429 Too Many Requests",
"quota exceeded",
"请求超时",
"代理连接失败",
"SSL certificate verify failed",
"ETIMEDOUT",
"HTTP 502 Bad Gateway",
"503 Service Unavailable",
];
for (const text of cases) {
if (!isNetworkIssueResult({ isError: true, content: [{ type: "text", text }] })) {
throw new Error(`Expected '${text}' to be detected as network issue`);
}
}
},
},
{
name: "isNetworkIssueResult: ignores non-error results",
run: () => {
const result = isNetworkIssueResult({
isError: false,
content: [{ type: "text", text: "curl: (28) Operation timed out" }],
});
if (result) {
throw new Error("Expected non-error result NOT to be flagged");
}
},
},
{
name: "isNetworkIssueResult: ignores unrelated errors",
run: () => {
const cases = [
"SyntaxError: unexpected token",
"file not found: /tmp/missing.ts",
"Command failed: npm install (exit code 1)",
"TypeError: Cannot read properties of undefined",
];
for (const text of cases) {
if (isNetworkIssueResult({ isError: true, content: [{ type: "text", text }] })) {
throw new Error(`Expected '${text}' NOT to be flagged`);
}
}
},
},
{
name: "buildClarifyAgentStartResult: injects NETWORK_ISSUE_PROMPT alongside CLARIFY_PROMPT",
run: () => {
const result = buildClarifyAgentStartResult({
enabled: true,
bypassForThisTurn: false,
systemPrompt: "Base",
isVague: false,
});
if (!result) {
throw new Error("Expected result when enabled");
}
if (!result.systemPrompt.includes(CLARIFY_PROMPT)) {
throw new Error("Expected systemPrompt to include CLARIFY_PROMPT");
}
if (!result.systemPrompt.includes(NETWORK_ISSUE_PROMPT)) {
throw new Error("Expected systemPrompt to include NETWORK_ISSUE_PROMPT");
}
},
},
{
name: "buildNetworkReminderResult appends reminder to content",
run: () => {
const content = [{ type: "text", text: "original output" }];
const patch = buildNetworkReminderResult({ content });
if (patch.content.length !== 2) {
throw new Error("Expected reminder to be appended");
}
if (patch.content[0].text !== "original output") {
throw new Error("Expected original content to be preserved");
}
if (!patch.content[1].text.includes("NETWORK/PROXY ISSUE DETECTED")) {
throw new Error("Expected reminder text to be appended");
}
},
},
{
name: "buildNetworkReminderResult handles empty content",
run: () => {
const patch = buildNetworkReminderResult({ content: undefined });
if (patch.content.length !== 1) {
throw new Error("Expected exactly one reminder entry");
}
},
},
];

console.log("Running clarify extension tests...\n");
Expand Down
15 changes: 14 additions & 1 deletion index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,12 @@
* suggested clarifications plus an "Other" option for freeform input.
*
* Toggle: /clarify on|off
* Bypass: prefix prompt with `!` to skip clarification for one turn
* Bypass: prefix prompt with `~` to skip clarification for one turn
*
* Network/proxy handling: when a tool fails with a network, proxy,
* connectivity, or rate-limit error, the system prompt instructs the model
* to stop and ask the user how to proceed via clarify_prompt instead of
* silently retrying or switching approaches.
*/

import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
Expand All @@ -19,6 +24,8 @@ import {
stripClarifyBypassPrefix,
CLARIFY_GUIDELINES,
buildClarifyAgentStartResult,
isNetworkIssueResult,
buildNetworkReminderResult,
} from "./clarify-utils.js";

export { CLARIFY_PROMPT } from "./clarify-utils.js";
Expand Down Expand Up @@ -83,6 +90,12 @@ export default function (pi: ExtensionAPI) {
return { action: "continue" };
});

pi.on("tool_result", async (event) => {
if (!enabled) return;
if (!isNetworkIssueResult(event)) return;
return buildNetworkReminderResult(event);
});

pi.on("before_agent_start", async (event) => {
const result = buildClarifyAgentStartResult({
enabled,
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.