Skip to content

Keep customer corrections from becoming unrelated action requests - #75

Closed
aryanorastar wants to merge 9 commits into
mainfrom
fix/customer-intent-context
Closed

aryanorastar wants to merge 9 commits into
mainfrom
fix/customer-intent-context

Conversation

@aryanorastar

@aryanorastar aryanorastar commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Customer failure

A technical follow-up can receive an unrelated financial/privacy answer when a bot guess, another participant or a generated title is treated as the owner's request. A withdrawn action can also revive later, or correcting it can discard the separate technical problem.

Current checkpoint — October 25, 2026

Head bce466151aefa70536ea76e51f3048712fd7ce98; runtime reviewed at e0374a82decdccb7070cf250a359671edb5f327e (later heads change only documentation clarity/counting). Draft; no merge or release approval.

  • Use the existing bounded fetch with application-assigned actor roles/turn IDs and a chronological owner-intent projection before prompt clipping. Explicit withdrawals persist until owner reopening; independent positive requests remain protected.
  • Constrain rejected planner goals, facts, requirements, queries and acknowledgments together. Source IDs establish speaker provenance, not semantic truth; unknown personal intent keeps a conservative person path.
  • The same owner record accompanies planning, answer and review. Full answer/reviewer history stays available; generated titles/other speakers cannot establish owner action authority.
  • Follow-up repair: preserve the bounded sanitized case context alongside a valid planner's standalone question. Version/caption echoes must not erase earlier symptoms or already-read diagnostics.
  • Withdrawals reuse the correction recognizer's existing direct-object vocabulary as explicit refund/payment-adjustment/account-deletion/data-deletion scopes. Only the withdrawn scope is masked, not the entire turn or broad routing lane. Surviving owner signals are re-evaluated: withdrawing a refund cannot cancel a separate cancellation/charge; declining account deletion cannot erase separate data deletion. The shared intent record distinguishes scopes from broad lanes. No general clause parser or topic-specific rule is added.
  • Retained recognized scopes contribute through the same scope-to-lane map as withdrawals, alongside legacy router signals. An independently requested billing adjustment must not disappear simply because the legacy router lacks that phrase.

No new fetch, model call, private capability, retention, schema, delivery-fence or optional-feature change. Intent prompts change, including the scope/lane distinction, not the support voice. Bounded English/Spanish grammar is not a general multilingual classifier.

Verification

  • Four new regression tests fail on prior head 936b718, then pass after this repair: prior-symptom retrieval, attachment-diagnostic retrieval, mixed-turn actual retrieval and mixed-turn projection. Privacy withdrawal/reopening and an independent financial-action control pass too.
  • Three further original regression groups fail on intermediate abcd59c, then pass after the scope-specific follow-up: independent cancellation/duplicate-charge retention, separate account/data deletion, actual-handler financial handoff. The revised account/data control also reproduces against frozen abcd59c compiled in memory.
  • The next independent check found that recognized billing-adjustment scope being lost; two pure/actual-handler regressions fail on e444cc7, then pass after the shared-map repair.
  • Full npm test: 1007/1007; offline no-model scenarios: 8/8. Guarded isolated PGlite database integration passes at abcd59c; this evidence remains valid because the follow-ups change no SQL/schema/adapters. This is not a live-provider or real Discord test.
  • Earlier red-before role/title/chronology tests, direct-object denial controls, actual injected planner/answer/reviewer request assertions and memory-off snapshots remain passing.
  • git diff --check: PASS. Exact final-head hosted Node20/24/Postgres run 40171583971: all PASS. Earlier-head green CI is not reused as final-head proof.
  • Independent scoped code re-review at runtime e0374a8: PASS, 299 affected tests, plus a separate stronger configured-planner actual-handler control delivering one staff card with zero answer-model calls. The overlapping runs are not summed. The documented shared-object/unknown grammar is not comprehensively recognized and remains an unresolved quality/intent limit, not certified safe. Earlier HOLDs remain recorded as red-before evidence.

Still required

Evaluation-adapter parity, this PR's own final-code answer-quality comparison and appropriate affected-flow multi-turn verification, including the disclosed semantic/grammar limits. Scoped code review and green CI are not readiness approval. The separate saved C/#71 comparison does not test #75 and cannot approve it. No paid rerun without Aryan's explicit approval.

All historical evidence and unmet scope remain in docs/CUSTOMER_INTENT_ROUTING.md and the master plan. No customer text, protected input or result answers are published. No Discord posts, provider calls, settings changes, merge/deploy or memory enablement for this PR repair.

@aryanorastar

Copy link
Copy Markdown
Collaborator Author

Independent review at75d0c124: keep draft; changes required. P1 requestCorrections.js9–18: negation spans swallow unrelated subsequent positive refund/account-deletion requests, including a separate Spanish variant. P2 index.js709–713: a three-turn financial request, explicit technical correction/withdrawal, then version-only follow-up restores the withdrawn financial route. Add observed-red mixed negative/positive and chronological multi-turn regressions. Preserve #67 routing/source rules, #49/#74 privacy/cleanup and #70 recovery fences during main integration. Six existing focused tests pass but miss these failures. Current-head checks, affected multi-turn handler verification, adapter parity and this PR own independently graded generated-answer comparison remain required. No provider/production/customer/protected-input access occurred. A formal request-changes review cannot be submitted from this account because it owns the PR; this comment records the hold instead.

@aryanorastar

Copy link
Copy Markdown
Collaborator Author

Closing unmerged under the October 25 final-attempt decision, not as completed work. The single authorised simultaneous comparison completed 81 cases twice per arm with zero execution-invalid rows. Main: 70 pass-both, p95 49.051 s; this head bce4661: 72 pass-both, p95 51.122 s (+2.071 s). Numerical gates pass, but independent full reading of all 486 final and six earlier replies found four material candidate-only differences: scenario-inappropriate charging advice, a false feature-absence claim, an unverified customer-specific sync conclusion, and unrelated setup guidance. No actual patch causation is claimed from this bounded sample; the agreed accuracy condition is nevertheless not met. No new protected missed handoff was observed. Preserve this branch, tests and all private evidence. Customer-intent correction remains unfinished backlog scope, not a resolved product problem. No rework or paid rerun is authorised under this decision. No deployment or settings change is being made.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant