Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 39 additions & 31 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 19 additions & 0 deletions packages/rogue-gateway/.env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Portkey Gateway OSS / OpenAI-compatible base URL
PORTKEY_BASE_URL=http://127.0.0.1:8787/v1

# Optional: only needed if your Portkey deployment requires auth.
PORTKEY_API_KEY=
PORTKEY_AUTH_HEADER=x-portkey-api-key
PORTKEY_AUTH_SCHEME=Bearer
PORTKEY_EXTRA_HEADERS_JSON={}

# Rogue config path defaults (override if you keep config elsewhere)
PI_ROGUE_CONFIG_PATH=/Users/pavel/.pi/agent/pi-rogue/config.json
PI_ROGUE_ROUTER_CONFIG_PATH=/Users/pavel/.pi/agent/pi-rogue/router/config.json

# Manual experiment knobs
PI_ROGUE_PROFILE=local-smart
PI_ROGUE_ROLE=smart
RAW_INPUT_TOKENS=82000
FORWARDED_INPUT_TOKENS=2400
EXPECTED_OUTPUT_TOKENS=900
36 changes: 36 additions & 0 deletions packages/rogue-gateway/COMMODITY_GATEWAY_SUBSTRATE_CHECK.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# Commodity Gateway Substrate Check (Issue #207)

## 1) Which parts would be duplicated if Rogue built everything from scratch?
- Auth/key tenanting + provider credential flows
- OpenAI-compatible request/response normalization and protocol compatibility matrix
- Retry/circuit-breaker/failover logic
- Rate limiting, budget enforcement, and usage/cost accounting logic
- Caching, retries, load balancing, and failover orchestration
- Operational observability layers (dashboards, hooks, request tracing)

## 2) Which parts can LiteLLM/Portkey own?
- Commodity gateway concerns above (key handling, retries, policy config, routing policies, budget/rate-limit enforcement, usage telemetry, failover/fallbacks, etc.)
- In practice, these are best delegated to avoid reimplementation in Rogue and keep this package focused.

## 3) Can Rogue sit in front of a generic OpenAI-compatible substrate?
**Yes.** `OpenAICompatibleSubstrate` intentionally implements a minimal adapter that calls `GET /models` and `POST /chat/completions` over a configurable base URL and optional API key, so Rogue can evaluate routing decisions without assuming a specific gateway vendor.

## 4) Does Rogue require substrate-specific code for the tokenomics/context logic?
**No.** Routing/tokenomics decisions are computed in `src/planner.ts` using normalized `GatewayAsset` metadata and context-profile + task heuristics (no vendor-specific branch logic).

## 5) What minimal substrate interface was enough?
The minimal interface was:
- `listModels()`
- `callChat(req)`
- optional `estimateCost(req)`
- optional `getUsage(runId)`

This has been sufficient for deterministic planning and explainable alternatives in the spike.

## 6) What would be painful to outsource?
- Rogue’s own policy semantics: local-first preference, context-lens scoring, deterministic alternatives/reasons, and explanation-ready savings calculations.
- If these are moved into external gateways, we lose the same decision traceability and control Rogue needs for policy-level policy evolution.

## Notes
- This spike does not include LiteLLM/Portkey as hard dependencies (matching the requested scope).
- Live LiteLLM/Portkey runtime checks are out-of-scope for this PR and remain optional follow-up validations.
66 changes: 66 additions & 0 deletions packages/rogue-gateway/PORTKEY_SELF_HOST_EXPERIMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Portkey self-host experiment for Rogue Gateway spike

This repository cannot launch a live Portkey instance by itself. The experiment is therefore prepared as a **repo-local, env-driven manual run** against an already-running Portkey/OpenAI-compatible endpoint.

## Outcome statement

This spike is a **validated spike / pilot setup**, not target architecture yet.

## What this experiment measures

- `raw_forward`
- `typed_lens`
- `lookup_compress`

For each mode, Rogue resolves `pi-dedicated` through the Pi Rogue router config to the configured upstream GPT target, then measures the routed request against the live endpoint.

## Required runtime configuration

Set these in your shell or runtime environment:

```sh
export PORTKEY_BASE_URL="http://127.0.0.1:8787/v1"
export PORTKEY_API_KEY="" # optional for self-host; set if your deployment requires auth
export PORTKEY_AUTH_HEADER="x-portkey-api-key" # optional
export PORTKEY_AUTH_SCHEME="Bearer" # optional
export PI_ROGUE_CONFIG_PATH="$HOME/.pi/agent/pi-rogue/config.json"
export PI_ROGUE_ROUTER_CONFIG_PATH="$HOME/.pi/agent/pi-rogue/router/config.json"
```

Optional knobs:

```sh
export PI_ROGUE_PROFILE="local-smart"
export PI_ROGUE_ROLE="smart"
export RAW_INPUT_TOKENS="82000"
export FORWARDED_INPUT_TOKENS="2400"
export EXPECTED_OUTPUT_TOKENS="900"
```

## Run command

From repo root:

```sh
cp packages/rogue-gateway/.env.example packages/rogue-gateway/.env
# optionally edit packages/rogue-gateway/.env to match your local paths / auth

docker compose --env-file packages/rogue-gateway/.env -f packages/rogue-gateway/docker-compose.yml up -d
npm run measure:portkey --workspace @fiale-plus/pi-rogue-gateway
```

## Expected output

A JSON report containing:

- resolved profile/role
- `pi-dedicated` requested model
- upstream model target from Pi config
- per-mode route decision
- upstream usage/tokens returned by the live endpoint

## What is still missing for live execution

- A real Portkey/OpenAI-compatible base URL
- A running Portkey self-host instance (or other compatible endpoint)
- Optional auth only if your deployment requires it
61 changes: 61 additions & 0 deletions packages/rogue-gateway/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
# @fiale-plus/pi-rogue-gateway

Spike package for the **Rogue Gateway** prototype.

This package is intentionally scoped to three concerns:

- gateway substrate boundary (`GatewaySubstrate`, `substrate-mock`, `substrate-openai-compatible`)
- tokenomics planner (`quoteRoute`, `RoutePlanChoice`, `QuoteResult`, route selection)
- event ledger hooks for run-level explainability (`events.ts`)

The code is intentionally conservative and deterministic, designed to prove that
Rogue can own **routing/policy/tokenomics** without reimplementing commodity
provider/gateway plumbing.

## Commodity Gateway Substrate Check

Issue #207 requires an explicit check against LiteLLM/Portkey-style commodities. This spike’s scope is intentionally narrow and the results are:

1. **Which parts would be duplicated if Rogue built everything from scratch?**
- Provider auth/key/tenant wiring, protocol drift handling (OpenAI-compatible endpoints/headers), retries, circuit-breaking, and fallbacks.
- Budget/rate-limit enforcement, spend/usage accounting, audit trails, and production-grade observability.
- Retry policy DSLs, multi-endpoint failover/load-balancing, and gateway caching/normalization layers.

2. **Which parts can LiteLLM/Portkey own?**
- Those commodity runtime concerns above (routing rules, retries, budgets, key management, provider fan-out/failover, cost metrics, dashboarding).
- In a full product shape, these can remain delegated so Rogue does not become another gateway implementation.

3. **Can Rogue sit in front of a generic OpenAI-compatible substrate?**
- **Yes (within this spike).**
- `OpenAICompatibleSubstrate` in `src/substrate-openai-compatible.ts` only assumes a base URL + optional API key and `/models`, `/v1/models`, and `/chat/completions`-style flows.
- Local smoke tests verify model listing and chat-call plumbing against a mock compatible endpoint.

4. **Does Rogue require substrate-specific code for tokenomics/context logic?**
- **No.**
- Routing/tokenomics policy is driven by the provided `GatewayAsset`/candidate metadata (`quoteRoute` in `src/planner.ts`) and is decoupled from any provider API details.
- The planner operates on normalized assets + route heuristics (`local_first`, raw/sealed/context-lens variants), not on vendor SDK semantics.

5. **What minimal substrate interface was enough?**
- `GatewaySubstrate` (`src/substrate.ts`) with `listModels`, `callChat`, optional `estimateCost`, and optional `getUsage`.
- This was sufficient to validate deterministic planning plus route observability without hard binding to any vendor.

6. **What is painful to outsource?**
- Anything tied to Rogue’s own product semantics: task/profile-specific scoring, local-first policy, context-lens savings accounting, and deterministic explainable route reasons.
- Those are explicitly implemented in Rogue to keep behavior explainable, while gateway plumbing remains external.

### Explicitly out-of-scope for this spike

- No hard dependency on LiteLLM or Portkey in-package (consistent with the issue request).
- No production gateway execution path (no claim to own full retry/key/dashboard/resilience stack).
- Optional non-blocking manual checks remain pending (e.g., live LiteLLM Proxy and Portkey runs) and were not a CI requirement for this PR.

### Config / measurement hooks

- `PI_ROGUE_CONFIG_PATH` and `PI_ROGUE_ROUTER_CONFIG_PATH` can point the spike at local Pi Rogue config files.
- `measurePiDedicatedModes()` in `src/measurement.ts` reports raw-forward, typed-lens, and lookup-compress token economics while resolving the pi-dedicated alias to the configured upstream GPT target.
- `npm run measure:portkey --workspace @fiale-plus/pi-rogue-gateway` runs the env-driven manual measurement wrapper in `src/measure.ts`.
- `docker-compose.yml` and `.env.example` provide the tiny self-host Portkey experiment draft.
- Self-host/manual experiment steps are documented in `PORTKEY_SELF_HOST_EXPERIMENT.md`.

Full written answers for the ticket’s required Commodity Gateway Substrate Check (including the 1..6 questions) are in:
`packages/rogue-gateway/COMMODITY_GATEWAY_SUBSTRATE_CHECK.md`.
9 changes: 9 additions & 0 deletions packages/rogue-gateway/docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
services:
portkey:
image: portkeyai/gateway:latest
container_name: rogue-portkey-gateway
ports:
- "8787:8787"
env_file:
- .env
restart: unless-stopped
22 changes: 22 additions & 0 deletions packages/rogue-gateway/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"name": "@fiale-plus/pi-rogue-gateway",
"version": "0.1.0",
"description": "Standalone Rogue tokenomics and substrate routing prototype module.",
"private": true,
"type": "module",
"license": "MIT",
"scripts": {
"check": "tsc -p ../../tsconfig.json --noEmit",
"test": "cd ../.. && vitest run packages/rogue-gateway/src/*.test.ts",
"measure:portkey": "tsx src/measure.ts"
},
"main": "./src/index.ts",
"exports": {
".": "./src/index.ts"
},
"files": [
"src",
"README.md",
"package.json"
]
}
Loading
Loading