Conversational AI Portfolio Manager with Verified Tool-Calling

[robertocarlous]

Problem Statement

The product already talks to users through WhatsApp and Telegram using a rule-based NLP parser (src/nlp/parser.ts) that recognizes a fixed vocabulary ("deposit", "withdraw", "portfolio", "rebalance"). It is brittle, and its growth path is "add another regex". Meanwhile the platform already holds an Anthropic key (ANTHROPIC_API_KEY) and a rich, structured data model — exactly what a real conversational agent needs. This issue replaces the recognition layer with a tool-calling LLM agent that can understand open-ended requests and execute structured, verified actions, while keeping the platform's core integrity property: no money moves and no decision is attributed to the agent unless it went through the same verified, audited, idempotent paths every other feature uses. The model is a free-form planner; the codebase is the only executor.

Current State

• src/whatsapp/handler.ts + src/telegram/handler.ts route messages through src/nlp/parser.ts and src/nlp/responses.ts; src/whatsapp/transcription/openaiProvider.ts already does audio→text; src/whatsapp/pendingConfirmations.ts implements a primitive confirmation handshake.
• src/agent/loop.ts executes rebalances with full AgentLog audit; src/stellar/contract.ts's executeWriteContractCall is the sole money-moving primitive; src/routes/ expose verified REST equivalents for every operation the agent might perform.
• The non-negotiable: the agent must never construct an ad-hoc money move or an unverified claim. The existing webhook/AgentLog/idempotency infrastructure is the executor.

Proposed Solution

1. Tool registry & schema (src/agent/tools/)

A typed tool registry that is the only surface exposed to the model. Every tool is a thin, allowlisted wrapper over existing services:

• Read tools: portfolio_value, positions, transactions, protocol_rates, goal_status, followed_strategy — returning the same allowlist-mapped shapes as the REST responses (src/utils/api-formatters.ts), never raw DB rows.
• Action tools (all guarded, all idempotent, all audited): deposit, withdraw, rebalance, create_recurring_deposit, create_alert_rule, adjust_strategy, follow_strategy, unfollow_strategy.
• Each tool declares a Zod schema for its arguments (matching the existing validator style in src/validators/), a required-confirmation flag, an actor-scope (own account vs. sub-account actingAsUserId), and a dry-run mode.
• A structural test must assert that the tool registry contains no path to raw stellar primitives — tools only call service-layer functions (same import-graph test pattern as tests/integration/agent/strategy-follow.integration.test.ts).

2. The planner (src/agent/assistant/)

• Planner: given a transcript + conversation memory, the model emits a JSON array of tool calls with arguments. Strictly schema-constrained (the Zod schemas above double as the model's tool spec); an out-of-schema call is rejected with a tool_parse_error, never half-executed.
• Grounding/verification layer between the model and execution:
  • Every action tool goes through a confirmation gate (reusing and upgrading src/whatsapp/pendingConfirmations.ts): the assistant renders a human-readable summary ("Withdraw 50 USDC to G…? Reply YES to confirm") and executes only on an affirmative, quoting the exact tool args.
  • No fabricated numbers: read tools return current data from the DB; any number the model would state in free text must come from a read tool result, not the model's memory. Where the model cites a stale value, the grounding layer must refresh and re-present.
  • Dry-run first: every action tool supports dryRun producing the exact payload that would execute (preview, fee estimate where available) — surfaced to the user before the confirmation gate.
• Conversation memory: a bounded per-user session store (Redis-backed, TTL'd — src/config/redis.ts) of recent turns + the results of executed tools, so follow-ups ("what would that have returned?") work without re-running.

3. Cost, safety & operational control

• Budget & rate control: per-user and global token budgets per window (configurable via src/config/env.ts); model call failures degrade to the existing rule-based parser (src/nlp/parser.ts), never to a dead bot. Instrument token spend and per-tool latency via Prometheus metrics.
• Sensitive-action guardrails (config + hard rules): withdrawal of more than a configurable fraction of portfolio value, transfers to a new destination address not previously seen on the account, and any action on a sub-account require the confirmation gate regardless of the model's confidence; a "never auto-execute" allowlist is the default posture.
• Audit: every executed tool call writes an AgentLog row (action, inputData, outputData, status, durationMs) so the decision trail is complete and, combined with the audit-ledger work, tamper-evident. The LLM's raw transcript and tool-call sequence are persisted in inputData/outputData (PII-scrubbed where required).

4. Channels & API

• Wire the planner into both src/whatsapp/handler.ts and src/telegram/handler.ts as the recognition layer (replacing src/nlp/parser.ts's role), with the parser retained as fallback and for exact commands.
• New REST surface: POST /api/v1/assistant/chat (authenticated, same body/response discipline as other routes) so the web app can use the same agent.
• Idempotency & replay: every executed action goes through the existing idempotent service paths (an idempotency key derived from the confirmed tool-call id) so a retried confirmation can never double-move money.
• Update docs/openapi.yaml.

Edge Cases & Failure Modes

• Model hallucinates a tool or arg: schema rejection → grounded error message ("I can't do that — here's what I can do"), never execution.
• Confirmation drift: between confirmation and execution the account state changes (balance drops, protocol delists) — re-validate the current state against the confirmed args and re-confirm if the operation would now differ materially.
• User asks to bypass confirmation: ignored; confirmation is a platform rule, not a preference.
• Model API down: degrade to rule-based parser with a logged fallback event.
• Tool execution fails (network/RPC): surface the failure honestly with the error from the service layer; never let the model "retell" a failed action as success.
• Ambiguous intent ("move some money"): ask a clarifying question grounded in the user's actual positions, do not guess amounts or destinations.
• Prompt injection via message content: the transcript is data, never instructions — the system prompt must be immutable to conversation content (documented and tested at the prompt-engineering level; tool args remain schema-validated regardless).

Security & Privacy Considerations

• The confirmation gate is the load-bearing safety property: a test must assert that an action tool cannot execute without an affirmative confirmation recorded against it (same spirit as the referral-activation test that refuses client-reported claims).
• Tool args are validated by the same Zod schemas as the REST routes — an action via the assistant is no less validated than via HTTP.
• Sub-account actions via the assistant use actingAsUserId scoping and enforce the same SubAccountPermission checks as src/middleware/subAccount.ts.
• PII: transcripts may contain sensitive data; redact wallet addresses/amounts from inputData where the audit doesn't require them, and confirm the WhatsApp/Telegram delivery paths already limit retries.
• No new secrets: the Anthropic/OpenAI keys already exist; per-user budgets prevent a single user from burning the key budget.

Out of Scope

• Fully autonomous trading (the agent never initiates a money move without confirmation — a confirmed "auto-rebalance on my goal" rule is a separate, deliberate feature, and even that should reuse src/agent/loop.ts).
• Long-term memory/vector stores.
• Model fine-tuning.

Suggested Implementation Plan

1. Tool registry + Zod schemas + dry-run + import-graph test.
2. Planner integration with schema-constrained tool calling; grounding/verification layer; confirmation-gate upgrade.
3. Conversation memory (Redis) + budgets/rate control + metrics.
4. WhatsApp/Telegram wiring + REST /assistant/chat + fallback path.
5. Audit integration, docs/openapi.yaml, integration tests (mocked model: happy path, hallucination, confirmation drift, fallback).

Acceptance Criteria

• Tool registry is the only model-visible surface; no path to raw stellar/db primitives (structural test)
• Every action tool requires confirmation, is idempotent, writes AgentLog, and re-validates state between confirm and execute (drift → re-confirm)
• Out-of-schema or hallucinated calls are rejected with grounded errors, never executed
• Read tools return live DB data; model-cited numbers come from tool results only
• Confirmation cannot be bypassed (test); sensitive actions always gated
• Model-API failure degrades to the rule-based parser, never a dead bot
• Per-user/global token budgets + Prometheus metrics for spend/latency
• src/whatsapp, src/telegram, and POST /api/v1/assistant/chat all wired; docs/openapi.yaml updated; tests green

[Kappa16]

please maintainer ,ill love to work on this issue , expect PR ASAP WITHOUT ERRORS