diff --git a/README.md b/README.md index 6b9b483fc..612de7af0 100644 --- a/README.md +++ b/README.md @@ -72,7 +72,7 @@ provider over some other base URL need a rule for it, and a rule that names a port matches only that port: ```bash -pxpipe warp --route '127.0.0.1:8082/v1/*=http://127.0.0.1:47821' -- codex +pxpipe warp --route '127.0.0.1:9090/v1/*=http://127.0.0.1:47821' -- codex ``` ## Offline export (no proxy) @@ -175,6 +175,7 @@ is confabulations, so lower is better. | `grok-4.5` | **100/100** | **97/98** | 17/18 | **0/16** | 0/15 | native 14px/84 quality suite (live profile); [quality](eval/grok-density/QUALITY_RESULTS.md), [native-sweep](eval/grok-density/native-sweep/RESULTS.md) | | `grok-4.6` high | **100/100** | **97/98** | 17/18 | **0/16** | 0/15 | native 14px/84, reasoning high; [quality](eval/grok-profile/QUALITY_RESULTS.md) | | `moonshotai/kimi-k3` | 79/100 | 84/98 | 15/18 | 1/16 | 0/15 | generic GPT profile: [quality results](eval/sol-profile/KIMI_K3_QUALITY_RESULTS.md) | +| `qwen-3.8` (`@cf/qwen/qwen3.8-27b`) | 98/100 | 72/98 | 11/18 | **0/16** | 0/15 | prior 5×8 broad suite (0/15 hex); native 14px pilot: 8/8 exact, 0 inventions, 11/15 hex: [pilot & quality](eval/qwen-profile/QUALITY_RESULTS.md) | ### Native-profile cost check diff --git a/eval/gemini-profile/gemini-client.mjs b/eval/gemini-profile/gemini-client.mjs index e6cabf769..e29ef66f8 100644 --- a/eval/gemini-profile/gemini-client.mjs +++ b/eval/gemini-profile/gemini-client.mjs @@ -1,7 +1,8 @@ // Google AI Studio eval client through the local Cloudflare AI Gateway. function gatewayOrigin() { - const base = (process.env.OPENAI_BASE_URL || 'http://127.0.0.1:8082/v1').replace(/\/$/, ''); + const base = (process.env.OPENAI_BASE_URL ?? '').replace(/\/$/, ''); + if (!base) throw new Error('OPENAI_BASE_URL is required (point it at your gateway)'); return new URL(base).origin; } diff --git a/eval/gist-recall/OPUS_14PX.md b/eval/gist-recall/OPUS_14PX.md index f5cfa41b1..28fba3df2 100644 --- a/eval/gist-recall/OPUS_14PX.md +++ b/eval/gist-recall/OPUS_14PX.md @@ -20,7 +20,7 @@ density, matching how fable's matrix cells are scored at fable's `5x8`. pageWidthPx}`; `run*.py` **preflight-aborts before any API call** if the corpus manifest model ≠ the model under test (guards silent profile/model mismatch). -- Model reached via **upstream (`:8082`), pxpipe-bypassed** — so pxpipe's own +- Model reached via **the gateway upstream, pxpipe-bypassed** — so pxpipe's own text→image rendering does not contaminate the arms; the model still gets the images. Concurrency 3, `CCI_READY_TIMEOUT=120`, `CCI_TIMEOUT=210`. - Raw results: `work/results.opus-14px.jsonl`, `work2/…`, `work3/…`. diff --git a/eval/glyph-matrix/PLAN.md b/eval/glyph-matrix/PLAN.md index c5ae2c412..b715b5822 100644 --- a/eval/glyph-matrix/PLAN.md +++ b/eval/glyph-matrix/PLAN.md @@ -7,7 +7,7 @@ ## What this measures Per-character confusion matrix for 12-char hex IDs read back through the -pixelpipe render path, across render-style arms — the precision-tier +pxpipe render path, across render-style arms — the precision-tier failure mode that gist-recall (98/98) structurally cannot surface and needle-haystack only bracketed (~87% exact-match on worst-case hex at prod style). diff --git a/eval/grok-density/QUALITY_SUITE.md b/eval/grok-density/QUALITY_SUITE.md index 9aefabb21..0804aa05f 100644 --- a/eval/grok-density/QUALITY_SUITE.md +++ b/eval/grok-density/QUALITY_SUITE.md @@ -4,7 +4,7 @@ Grok is evaluated through the OpenAI-compatible Responses endpoint used by Codex. Fable and Opus use the Claude harnesses. ```bash -export OPENAI_BASE_URL=http://127.0.0.1:8082/v1 +export OPENAI_BASE_URL=http://127.0.0.1:/v1 export OPENAI_API_KEY=… export SOL_QUALITY_MODEL=grok-4.5 export SOL_QUALITY_LIVE=1 diff --git a/eval/grok-density/native-sweep/README.md b/eval/grok-density/native-sweep/README.md index 50eb2bacc..42dccdd74 100644 --- a/eval/grok-density/native-sweep/README.md +++ b/eval/grok-density/native-sweep/README.md @@ -10,7 +10,7 @@ atlas swap → rebuild → render, live ask locks answers before truth is scored ```bash node eval/grok-density/native-sweep/gen-fixture.mjs node eval/grok-density/native-sweep/render-ladder.mjs -OPENAI_BASE_URL=http://127.0.0.1:8082/v1 OPENAI_API_KEY=… \ +OPENAI_BASE_URL=http://127.0.0.1:/v1 OPENAI_API_KEY=… \ bash eval/grok-density/native-sweep/_ask_all.sh node eval/grok-density/native-sweep/score-all.mjs ``` diff --git a/eval/grok-density/native-sweep/RESULTS.md b/eval/grok-density/native-sweep/RESULTS.md index 67fbbdc06..cfa54e8a0 100644 --- a/eval/grok-density/native-sweep/RESULTS.md +++ b/eval/grok-density/native-sweep/RESULTS.md @@ -1,7 +1,7 @@ # Grok 4.5 native JetBrains Mono 8–16px blind sweep Date: 2026-07-23. Model: `grok-4.5` via direct Responses (`OPENAI_BASE_URL` -upstream `:8082`, pxpipe port 47821 rejected by the client). +upstream local gateway; pxpipe port 47821 rejected by the client). ## Protocol (same spirit as Opus / Sol native sweeps) @@ -58,7 +58,7 @@ evidence; this sweep is the native-size negative result. # atlases already built under eval/opus-density/native-sweep/atlases (symlinked) node eval/grok-density/native-sweep/gen-fixture.mjs # fresh truth (optional) node eval/grok-density/native-sweep/render-ladder.mjs # swaps atlas*, rebuilds, restores -OPENAI_BASE_URL=http://127.0.0.1:8082/v1 OPENAI_API_KEY=… \ +OPENAI_BASE_URL=http://127.0.0.1:/v1 OPENAI_API_KEY=… \ bash eval/grok-density/native-sweep/_ask_all.sh # locks answers-*.json node eval/grok-density/native-sweep/score-all.mjs ``` diff --git a/eval/grok-density/native-sweep/ask.mjs b/eval/grok-density/native-sweep/ask.mjs index 09e4cdc49..e207ed06a 100644 --- a/eval/grok-density/native-sweep/ask.mjs +++ b/eval/grok-density/native-sweep/ask.mjs @@ -1,6 +1,6 @@ // Live-call grok-4.5 on one rendered rung. Blind: never opens truth.json. // Writes answers-