Prototype Spike: Rogue Gateway substrate + tokenomics router

[bearmug]

Summary

Prototype a standalone Rogue Gateway feasibility spike focused on tokenomics-efficient routing, context compression, cache-aware processing, and substrate reuse.

The existing context/MCP spike is strong on context compression and ctx:// lookup, but not enough yet for the larger product claim. This ticket adds the explicit Substrate + Tokenomics Probe needed to test whether Rogue can be the efficient router/smartifier/cache processor without rebuilding commodity gateway infrastructure.

Problem statement

Rogue Gateway should prove that it can manage model/resource economics wisely while avoiding generic gateway rebuilds.

The spike should answer whether Rogue can sit above or beside existing gateway substrates such as direct OpenAI-compatible upstreams, LiteLLM Proxy, Portkey Gateway, or mocks, while keeping Rogue-specific value in:

• route planning,
• profile policy,
• tokenomics quotes,
• context shaping / compression,
• premium guardrails,
• cache-aware savings attribution,
• run explanations,
• event-ledger-backed observability.

Coverage target

Area	Target coverage
Model routing	Deterministic route planner with alternatives, not just one static profile target
Stats collection	Event ledger that can reconstruct decisions and savings
Boilerplate management	Explicit substrate comparison against LiteLLM/Portkey-style gateways
Avoid commodity gateway rebuilds	Small substrate adapter boundary; no provider dashboards/retry engines/budget systems in Rogue
Smart model routing	Policy + economics comparison; no smart-model call required for this spike
Context management/compression	Preserve existing typed-lens / ctx:// / MCP lookup proof
Tokenomics/spend accounting	Make economics the central demo metric

Hypothesis

Rogue should not implement generic provider plumbing, dashboards, virtual-key systems, retry engines, or provider cost maps from scratch.

Instead:

Rogue owns:
  route plan
  profile policy
  tokenomics quote
  context shape
  premium guardrails
  savings attribution
  run explanation

Substrate owns:
  provider adapters
  generic budgets
  provider retries/fallbacks
  virtual keys
  generic spend dashboards
  generic provider logs

Substrate can be:

direct OpenAI-compatible upstream
LiteLLM Proxy
Portkey Gateway
mock provider

The spike should prove Rogue remains useful when the boring gateway layer is external.

Deliverables

1. GatewaySubstrate interface

Add a small interface, even if only mock/direct implementations exist first.

export interface GatewaySubstrate {
  id: string;

  listModels(): Promise<SubstrateModel[]>;

  callChat(req: {
    model: string;
    messages: unknown[];
    stream?: boolean;
    max_tokens?: number;
    temperature?: number;
    metadata?: Record<string, unknown>;
  }): Promise<SubstrateChatResult>;

  estimateCost?(req: {
    model: string;
    inputTokensApprox: number;
    outputTokensApprox: number;
    cachedInputTokensApprox?: number;
  }): Promise<SubstrateCostQuote>;

  getUsage?(runId: string): Promise<SubstrateUsage | null>;
}

Implement at least:

substrate-mock
substrate-openai-compatible

Optional/manual:

substrate-litellm
substrate-portkey

Optional adapters can be simple HTTP forwarders. Do not embed LiteLLM/Portkey as hard dependencies.

2. Asset registry with resource classes

Add static asset registry to config.

Asset classes:

owned_capacity
metered_api
subscription_quota
composite_fusion
mock

Example shape:

{
  "assets": {
    "local.qwen35": {
      "class": "owned_capacity",
      "substrate": "openai-compatible.local",
      "model": "qwen3.6-35b-a3b-q4",
      "contextWindow": 128000,
      "observedTps": 31,
      "inputCostPerMTok": 0,
      "cachedInputCostPerMTok": 0,
      "outputCostPerMTok": 0,
      "privacy": "local",
      "qualityTier": "mid",
      "tags": ["coding", "debug", "cheap-broad-scan"]
    },
    "remote.cheap": {
      "class": "metered_api",
      "substrate": "mock",
      "model": "cheap-remote",
      "contextWindow": 128000,
      "observedTps": 80,
      "inputCostPerMTok": 0.2,
      "cachedInputCostPerMTok": 0.02,
      "outputCostPerMTok": 1.0,
      "privacy": "remote",
      "qualityTier": "cheap"
    },
    "remote.premium": {
      "class": "metered_api",
      "substrate": "mock",
      "model": "premium-frontier",
      "contextWindow": 256000,
      "observedTps": 45,
      "inputCostPerMTok": 5.0,
      "cachedInputCostPerMTok": 0.5,
      "outputCostPerMTok": 30.0,
      "privacy": "remote",
      "qualityTier": "frontier"
    },
    "subscription.smart": {
      "class": "subscription_quota",
      "substrate": "mock",
      "model": "subscription-smart",
      "contextWindow": 128000,
      "observedTps": 60,
      "quotaRemaining": 20,
      "quotaResetHours": 8,
      "privacy": "remote",
      "qualityTier": "smart"
    }
  }
}

For the spike, numbers can be fake/manual.

3. Tokenomics quote endpoint

Add:

POST /rogue/economics/quote

Input:

{
  "profile": "fast-but-not-crazy",
  "taskKind": "coding_debug",
  "rawInputTokensApprox": 82000,
  "forwardedInputTokensApprox": 2400,
  "expectedOutputTokensApprox": 900,
  "contextPolicy": "typed_lens",
  "candidateAssets": [
    "local.qwen35",
    "remote.cheap",
    "remote.premium",
    "subscription.smart"
  ]
}

Output should include:

• selected route/asset,
• at least 3 alternatives,
• raw-premium baseline,
• premium sealed baseline,
• explicit savings attribution,
• guards that block raw full-context premium by default.

Representative output:

{
  "selected": {
    "asset": "local.qwen35",
    "route": "local_first_typed_lens",
    "estimatedWallTimeMs": 77000,
    "estimatedMeteredUsd": 0,
    "estimatedQuotaCost": 0,
    "estimatedTotalTokenCostUsd": 0,
    "reason": "broad debugging; local capacity is free and premium is not justified before repeated failure"
  },
  "alternatives": [
    {
      "asset": "remote.cheap",
      "route": "cheap_remote_typed_lens",
      "estimatedMeteredUsd": 0.00138,
      "estimatedWallTimeMs": 30000,
      "reasonNotChosen": "profile prefers local first"
    },
    {
      "asset": "remote.premium",
      "route": "premium_sealed_packet",
      "estimatedMeteredUsd": 0.039,
      "estimatedWallTimeMs": 24000,
      "reasonNotChosen": "premium requires repeated failure or explicit profile"
    },
    {
      "asset": "remote.premium",
      "route": "premium_raw_oracle_eval_only",
      "estimatedMeteredUsd": 0.437,
      "estimatedWallTimeMs": 60000,
      "reasonNotChosen": "raw full context to premium is forbidden outside eval"
    }
  ],
  "savings": {
    "tokensAvoidedByContextLens": 79600,
    "premiumRawVsPremiumSealedUsdSaved": 0.398,
    "premiumRawVsSelectedUsdSaved": 0.437,
    "cacheSavingsEstimatedUsd": 0,
    "localFirstSavingsEstimatedUsd": 0.039
  },
  "guards": {
    "rawFullContextToPremium": false,
    "maxPremiumInputTokens": 5000,
    "premiumRequires": [
      "same_error_repeated",
      "user_explicit_quality",
      "large_diff_review",
      "architecture_deadlock"
    ]
  }
}

4. Deterministic smart route planner v0

No smart model call in this spike. The “smart” part is explicit policy + economics comparison.

Inputs:

profile
taskKind
asset registry
raw tokens
forwarded/lens tokens
loop/retry hints if available
budget caps
privacy policy

Routes:

local_first_typed_lens
cheap_remote_typed_lens
subscription_hard_call
premium_sealed_packet
premium_raw_oracle_eval_only

Initial rules:

if profile.privacy == local_only:
  choose local_first_typed_lens

if rawInputTokensApprox > threshold and contextPolicy == typed_lens:
  forbid premium_raw except eval

if profile == local-first-economy:
  choose local unless repeatedFailureHint >= 2

if profile == fast-but-not-crazy:
  choose local for broad/debug first pass
  choose subscription or cheap remote if latency preference high
  choose premium sealed only if repeatedFailureHint >= 2 or taskKind == high_value_review

if profile == premium-surgical:
  choose premium sealed
  never choose raw premium unless explicit eval flag

5. Event ledger, not only run ledger

In addition to one run record, append events:

.pi/rogue-gateway-spike/events.jsonl

Event types:

request_received
profile_resolved
artifact_detected
context_lens_created
economics_quoted
route_planned
upstream_call_started
upstream_call_finished
cache_estimated
context_lookup
response_returned

Example:

{
  "eventId": "evt_...",
  "runId": "run_...",
  "type": "economics_quoted",
  "timestamp": "...",
  "data": {
    "selectedAsset": "local.qwen35",
    "rawPremiumCostUsd": 0.437,
    "selectedCostUsd": 0,
    "premiumSealedCostUsd": 0.039,
    "tokensAvoidedByLens": 79600
  }
}

6. Commodity substrate comparison report

Add an explicit report section:

## Commodity Gateway Substrate Check

The report must answer:

1. Which parts would be duplicated if Rogue built everything from scratch?
2. Which parts can LiteLLM/Portkey own?
3. Can Rogue sit in front of a generic OpenAI-compatible substrate?
4. Does Rogue require substrate-specific code for the tokenomics/context logic?
5. What minimal substrate interface was enough?
6. What would be painful to outsource?

Optional manual checks, non-CI-blocking:

• Run LiteLLM Proxy locally and point substrate-openai-compatible at it.
• Run Portkey Gateway locally and point substrate-openai-compatible at it.

Experiments

Experiment A — Existing Context/MCP feasibility

Preserve the existing context-compression + visible-artifact + ctx:// + MCP companion lookup experiment from the original spike.

Pass criteria:

• Bare OpenAI-compatible traffic works.
• Visible artifact capture works.
• ctx:// handles are emitted/looked up.
• MCP companion lookup is demonstrated.

Experiment B — Routing/profile behavior

Question:

Does profile selection change resource usage without harness changes?

Run the same request with:

model = rogue/local
model = rogue/fast
model = rogue/premium-surgical

Expected:

rogue/local:
  selected local.qwen35

rogue/fast:
  selected local or cheap/subscription depending fake latency/cost config

rogue/premium-surgical:
  selected premium_sealed_packet, not raw premium

Pass criteria:

• Same input produces different route plans.
• Route explanation is readable.
• Premium raw context remains forbidden by default.

Experiment C — Tokenomics accounting

Question:

Can Rogue explain money/resource savings in a way a user cares about?

For a large-log request, report:

raw input tokens
forwarded tokens
tokens avoided
selected route cost
premium sealed cost
premium raw cost
cache-read estimate if configured
local opportunity time estimate
subscription shadow cost if configured

Pass criteria:

• Report shows selected route and at least 3 baselines:
  • local selected,
  • cheap remote,
  • premium sealed,
  • premium raw/oracle.
• Report includes explicit savings:
  • context lens saved X tokens,
  • selected-vs-premium saved $Y,
  • premium sealed-vs-raw saved $Z.

Experiment D — Boilerplate avoidance

Question:

Can Rogue avoid implementing commodity gateway functionality?

Run or simulate:

Rogue -> direct mock substrate
Rogue -> direct OpenAI-compatible substrate
Optional: Rogue -> LiteLLM/Portkey substrate

Pass criteria:

• Rogue route/economics/context logic is substrate-agnostic.
• Provider-specific retries/budgets/dashboards are not implemented in Rogue.
• Code has explicit TODO/adapter path for LiteLLM/Portkey.

Acceptance criteria

• /rogue/economics/quote works without a model call.
• Quote compares local, cheap remote, premium sealed, and premium raw/oracle baselines.
• Quote includes selected route, alternatives, raw-premium baseline, and explicit savings attribution.
• Route planner changes behavior based on selected Rogue profile.
• Run report includes selected cost, raw-premium baseline cost, premium-sealed baseline cost, and savings.
• Event ledger records request/profile/artifact/economics/route/upstream/response events.
• Demo run produces at least 6 event types.
• Events can reconstruct why the route happened.
• Events can compute a savings report.
• Substrate interface exists.
• At least mock + OpenAI-compatible substrate implementations work.
• Commodity substrate report explains what Rogue should not build from scratch.
• Demo report clearly answers whether tokenomics/economics is the main value or merely a side metric.
• Existing context/MCP proof still works: OpenAI-compatible traffic, visible artifact capture, ctx:// handles, and MCP companion lookup.

Final report questions

The spike report should answer:

1. Did route profiles produce meaningfully different plans?
2. Was tokenomics visible enough to make a resource decision before spending?
3. How much would raw premium have cost versus selected route?
4. How much did context lensing save separately from model routing?
5. Which gateway boilerplate should be delegated to LiteLLM/Portkey?
6. Which Rogue-specific logic cannot be delegated?
7. Was the substrate adapter boundary clean?
8. What is the next smallest experiment to prove wise resource use on a real local model?
9. Did the Context/MCP side remain useful after adding tokenomics routing?
10. Is the “Economics API is king” framing supported by demo evidence?

Killer demo target

The demo should produce a report like:

Raw premium baseline:
  82k input tokens, estimated $0.437

Premium sealed packet:
  2.4k input tokens, estimated $0.039

Selected local-first route:
  $0 metered cost, ~77s estimated local time

Savings:
  context lens avoided ~79.6k tokens
  premium sealed saved ~$0.398 vs raw premium
  local-first saved ~$0.039 vs premium sealed
  raw full context to premium was blocked by profile guard

Explicit non-goals

• Do not build provider dashboards.
• Do not build virtual-key systems.
• Do not build generic provider retry/fallback engines beyond minimal adapter behavior needed for the spike.
• Do not add LiteLLM or Portkey as hard dependencies.
• Do not make production cost claims from fake/manual numbers.
• Do not choose raw premium full-context routing unless an explicit eval flag enables the oracle baseline.

Notes / references

• LiteLLM already covers commodity proxy concerns such as OpenAI-compatible gatewaying, virtual keys, budgets, spend/cost tracking, and hooks: https://docs.litellm.ai/docs/
• Portkey Gateway configs cover routing rules, fallbacks, load balancing, retries, and caching: https://docs.portkey.ai/docs/product/ai-gateway/configs

[bearmug]

Progress update on #207: standalone spike package implemented in this branch. Commit: 6eeb1ef

Added packages/rogue-gateway with:

• substrate abstraction (GatewaySubstrate), mock + OpenAI-compatible adapters
• asset registry + route planner + savings model
• events ledger and /rogue/economics/quote + /health endpoints
• planner/server tests

Validation:

• npm run check --workspace @fiale-plus/pi-rogue-gateway
• npm run test --workspace @fiale-plus/pi-rogue-gateway
  Both pass.

[bearmug]

#207 update: PR opened for the standalone gateway spike.

PR: #210

It includes the committed package plus validation results:

• npm run check --workspace @fiale-plus/pi-rogue-gateway
• npm run test --workspace @fiale-plus/pi-rogue-gateway
• npm run check --workspaces --if-present
• npm run test --workspaces --if-present

[bearmug]

#207 checkpoint update (retry):

• Fixed CI lockfile sync failure by updating package-lock.json with npm@10.5.2 (PR branch feature/issue-207-rogue-gateway-spike).
• Squashed/amended commit: f0b3508
• Re-ran checks:
  • npm ci succeeds (with npm@10.5.2)
  • npm run check --workspaces --if-present ✅
  • npm run test --workspaces --if-present ✅
  • GitHub Actions PR check (feat: add rogue gateway tokenomics routing spike package #210) now passes.

PR: #210

[bearmug]

Rogue Gateway spike outcome

Status: validated spike / pilot setup, not a target-system architecture recommendation yet.

What is now in PR #210:

• deterministic tokenomics routing package for the Rogue Gateway spike
• commodity gateway substrate check documentation
• OpenAI-compatible substrate with /models + /v1/models and /chat/completions + /v1/chat/completions fallback support
• env-driven Portkey/OpenAI-compatible adapter with no bundled credentials
• Pi Rogue config model routing: pi-dedicated resolves to the configured upstream GPT/model target
• token measurement harness for:
  • raw_forward
  • typed_lens
  • lookup_compress

Local validation performed:

• npm run check --workspace @fiale-plus/pi-rogue-gateway --silent
• npm run test --workspace @fiale-plus/pi-rogue-gateway --silent
• npm run check --workspaces --if-present --silent
• npm run test --workspaces --if-present --silent

Manual Portkey experiment setup is prepared, but a live Portkey run was not executed because no Portkey endpoint/key is available in this environment.

Expected manual env setup:

PORTKEY_BASE_URL=...
PORTKEY_API_KEY=...
PORTKEY_AUTH_HEADER=x-portkey-api-key # optional; defaults are supported
PI_ROGUE_CONFIG_PATH=...
PI_ROGUE_ROUTER_CONFIG_PATH=...

Fugu-ultra review was attempted but not obtained: Sakana returned 429 Subscription window is exhausted. Current strategic advice available in-session came from the configured advisor model, not fugu-ultra.

Conclusion: this is good evidence for the pilot/spike. Before treating it as the target system direction, follow-ups should harden config schema validation, failure-mode tests, credential redaction, and observability.

[bearmug]

Final explicit spike status / manual experiment readiness

Outcome: PR #210 is a validated Rogue Gateway spike / pilot setup, not a recommendation to promote this as the target gateway architecture yet.

What is in the PR now:

• deterministic tokenomics route planner and event-led observability
• commodity gateway substrate check documentation
• OpenAI-compatible substrate with /models + /v1/models and /chat/completions + /v1/chat/completions fallback support
• env-driven Portkey/OpenAI-compatible adapter with no bundled credentials
• Pi Rogue config model routing that maps pi-dedicated to the configured upstream model target
• token-measurement harness for:
  • raw_forward
  • typed_lens
  • lookup_compress

Verification completed locally and in CI:

• PR CI is green.
• Local workspace validation passed:
  • npm run check --workspace @fiale-plus/pi-rogue-gateway --silent
  • npm run test --workspace @fiale-plus/pi-rogue-gateway --silent
  • npm run check --workspaces --if-present --silent
  • npm run test --workspaces --if-present --silent

Manual Portkey experiment readiness: prepared, but not live-run here because this environment does not have a real Portkey/OpenAI-compatible endpoint + key.

Required env:

export PORTKEY_BASE_URL="https://..."          # Portkey/OpenAI-compatible base URL
export PORTKEY_API_KEY="..."                  # secret; not stored in repo
export PORTKEY_AUTH_HEADER="x-portkey-api-key" # optional; default Authorization/Bearer also supported
export PI_ROGUE_CONFIG_PATH="$HOME/.pi/agent/pi-rogue/config.json"
export PI_ROGUE_ROUTER_CONFIG_PATH="$HOME/.pi/agent/pi-rogue/router/config.json"

One-shot manual measurement command from repo root:

cat > /tmp/rogue-portkey-measure.ts <<'TS'
import { measurePiDedicatedModes } from "./packages/rogue-gateway/src/measurement.js";

const report = await measurePiDedicatedModes({
  profile: process.env.PI_ROGUE_PROFILE ?? "local-smart",
  role: process.env.PI_ROGUE_ROLE ?? "smart",
  request: {
    profile: "local-first-economy",
    taskKind: "manual_portkey_measurement",
    rawInputTokensApprox: Number(process.env.RAW_INPUT_TOKENS ?? 82000),
    forwardedInputTokensApprox: Number(process.env.FORWARDED_INPUT_TOKENS ?? 2400),
    expectedOutputTokensApprox: Number(process.env.EXPECTED_OUTPUT_TOKENS ?? 900),
    contextPolicy: "typed_lens",
    candidateAssets: ["local.qwen35", "remote.cheap", "remote.premium", "subscription.smart"],
  },
});

console.log(JSON.stringify({
  profile: report.profile,
  role: report.role,
  requestedModel: report.requestedModel,
  upstreamModel: report.upstreamModel,
  modes: report.modes.map((mode) => ({
    mode: mode.mode,
    forwardedInputTokensApprox: mode.request.forwardedInputTokensApprox,
    selectedRoute: mode.quote.selected.route,
    selectedAsset: mode.quote.selected.asset,
    ok: mode.chat.ok,
    status: mode.chat.status,
    providerUsage: mode.chat.usage ?? null,
  })),
}, null, 2));
TS

# Run from repo root so the relative package import resolves:
npx tsx /tmp/rogue-portkey-measure.ts

Expected manual output: JSON showing pi-dedicated resolved to the configured upstream model and per-mode route/usage values for raw_forward, typed_lens, and lookup_compress.

Still missing for a live external result: real PORTKEY_BASE_URL + PORTKEY_API_KEY. Without those, only mock OpenAI-compatible tests can be validated in CI/local automation.

Fugu-ultra note: attempted but not obtained; Sakana returned 429 Subscription window is exhausted. Available strategic advice in-session was not fugu-ultra.

[bearmug]

Added a repo-local manual experiment artifact for the Portkey self-host path:

• packages/rogue-gateway/PORTKEY_SELF_HOST_EXPERIMENT.md
• packages/rogue-gateway/src/measure.ts
• npm run measure:portkey --workspace @fiale-plus/pi-rogue-gateway

This keeps the spike scope intact: the repo still cannot launch Portkey itself, but the exact env-driven manual run is now drafted and runnable once a real Portkey/OpenAI-compatible endpoint + API key are provided in the runtime environment.

[bearmug]

Drafted the tiny self-host Portkey experiment setup requested:

• packages/rogue-gateway/docker-compose.yml
• packages/rogue-gateway/.env.example
• updated packages/rogue-gateway/PORTKEY_SELF_HOST_EXPERIMENT.md
• added npm run measure:portkey --workspace @fiale-plus/pi-rogue-gateway

This keeps the experiment repo-local and env-driven. It does not launch Portkey from within this repo; it expects a real Portkey/OpenAI-compatible runtime to be started externally via Docker/infra and then pointed to with PORTKEY_BASE_URL.