Repository navigation
Preserve useful context when filtering handoff narration and redirects #62
Description
Customer failure
A handoff reply can lose its useful problem acknowledgment or earlier-attempt context and leave only a detail question plus the application delivery notice.
Reproduced on deployed main and the ranking candidate
Two independently constructed synthetic examples reproduce separate pre-existing paths:
presentReviewedAnswercallsstripPingNarration/dropPingNarration. A sentence containingneeds a person who can lookis removed as a whole sentence. A useful symptom clause joined by a semicolon is removed with the narration.- After successful staff delivery,
escalateReplycallsstripSupportRedirect. A sentence containingcontact supportis removed as a whole sentence. Earlier-attempt context joined to that redirect by a semicolon is also removed.
Both leave only the separate intake question and confirmed-delivery footer. These examples are comfortably below the reply-length limit; clipping does not cause this synthetic loss.
The implementations of utils.js, answerPipeline.js, honesty.js, commandcode.js and reviewDecision.js are byte-identical between the deployed baseline and ranking candidate. Four synthetic diagnostic checks pass for both versions. No production behavior was modified and no protected input was read.
Evidence limit
The historical evaluation saves final replies, per-stage times, usage and escalation signals, but not draft/reviewer/pre-presentation text or stage fingerprints. Therefore the exact transition responsible for a historical short reply cannot be reconstructed from those saved logs. The reproductions prove pre-existing erasure paths, not which one ran in that historical occurrence. Retrieval can also change generated wording despite unchanged filters.
Required repair and verification
- Preserve useful acknowledgment and already-tried context when removing unsupported staff-notification narration or an unnecessary redirect; do not restore claims of unverified access, delivery, acceptance or resolution.
- Use independent synthetic compound-sentence, multilingual, citation, safety-warning and all-body-removed fixtures with observed red-before tests.
- Keep confirmed delivery notices application-owned; preserve honest failed-delivery and person-required paths.
- Add privacy-safe evaluation-only stage evidence sufficient to distinguish generation, review, safety/presentation and clipping without logging customer text or secrets.
- Retain the independent quality/release gate. Do not copy protected inputs or replies into tests or selectively rerun historical cases.
Tracked separately from the already reviewed ranking change and the planned non-handoff email-redirect correction.
Activity
aryanorastar commentedon Oct 24, 2026
Scope retained after closing #77 without merging: preserve reviewed symptoms, previous attempts and protective warnings while keeping their conditions; distinguish quoted customer actions from verifiable bot actions; use one presentation implementation; add text-free stage evidence on the actual index.js/live-eval.js path through review, presentation, escalation/redirect cleanup and clipping. The one general-clause rework passes its seven new synthetic examples but independently fails prerequisite preservation: removing a staff-confirmation clause can leave unconditional destructive advice. The narrowed published whitelist also does not solve general symptom/attempt loss. Preserve #77 branch copilot/preserve-context-filtering-narration at 789e5d2 and the rejected local experiment; no merge/deployment. Keep this issue open. The two same-case French cards observed in TEST are intentional skipDedupe behavior, not a demonstrated production-deduplication defect.
aryanorastar commentedon Oct 24, 2026
Status check — main
16ee8ca, 2026-10-24Half fixed. Checked with new synthetic sentences against main.
Done: the redirect path (
stripSupportRedirect);survive, support addresses are detected in any language, and Source lines and secret warnings are kept.Not done: the narration path (
dropPingNarrationviapresentReviewedAnswer)copilot/preserve-context-filtering-narrationat789e5d2is preserved.scripts/live-eval.jshas no per-stage fingerprints, so a shortened reply can't be traced to generation, review, presentation, escalation/redirect cleanup or clipping.Agreed next step: measure first. One PR adds text-free stage fingerprints (length, short hash, flags) on the real
index.js+live-eval.jspath, with a single presentation implementation and no filter change. Then one comparison shows how often narration removal deletes useful text. If it's rare, prefer a prompt rule that routing status goes in its own sentence. If it's common, design the filter change from that data, keeping conditions and prerequisites with their instructions and never keeping a first-person bot action claim.Related PRs: #67 (merged), #80 (merged), #77 (closed, branch kept).
aryanorastar commentedon Oct 25, 2026
October25 status audit: #67/#80's narration/redirect clause-preservation repairs are on main. The remaining evidence-only boundary is now implemented in PR #88, exact
c1fe9d6, ready for review, not merged or deployed.It observes the actual handler/reviewer/safety/presentation/clipping/handoff/send projection only during explicit test/evaluation collection, using fixed labels/lengths and ephemeral keyed fingerprints, not stored stage text or a parallel formatter. It cannot reconstruct stages missing from historical artifacts or claim a send attempt is a receipt. Nine new tests, full1174/no-model8, exact-head Node20/24/real PostgreSQL CI and independent35-check scoped review pass. No paid rerun, Discord post or customer-data use.
Keep this issue open until the remaining accepted implementation is on main and its full scope is verified. No historical causation, generalized accuracy or production fix is claimed by these diagnostics.
aryanorastar commentedon Oct 26, 2026
October26 scope verification: #88's evaluation-only real-path stage evidence is now merged as 884fb46; integrated main 928676b passes required Node20/24 and real PostgreSQL CI. This supplies future text-free diagnostics, not historical causation or a general filter repair.
Keep #62 open. The earlier October25 status line calling both narration/redirect paths repaired was too broad: #67/#80 repaired redirect handling, while dropPingNarration still drops the whole compound sentence. A new independent synthetic check on current main, “Your recording is still missing after the restart; this needs a person who can look. Which app version do you use?”, retains only the version question. No protected input was used. The symptom/attempt-preservation requirement remains, together with prerequisite/safety preservation and honest application-owned delivery. Use the merged stage evidence to measure/design that repair; no paid comparison is authorized by this update. Existing assignments are unchanged. No customer-resolution or deployment claim.