teamchong · teamchong · Aug 18, 2026 · Aug 17, 2026
diff --git a/FINDINGS.md b/FINDINGS.md
@@ -3,7 +3,7 @@
 **Status:** ⚠️ **VERDICT REVERSED — see correction below.** Originally ruled "dead"; live measurement shows pxpipe is a working *lossy gist-compressor* saving ~68% on real (dense) Claude Code traffic, with a known verbatim-recall gap.
 **Date:** 2026-05-28 (original) · 2026-05-29 (correction) · 2026-06-09 (Fable 5 update) · 2026-06-10 (gist-recall A/B, SWE-bench pilot) · 2026-06-12 (field observation, n=1) · 2026-06-23 (reframe: correct baseline = /compact) · 2026-07-09 (GPT-5.6 Sol raw-recall pilot) · 2026-07-19 (K/H glyph-surgery model-level A/B)
 **Models tested:** `claude-opus-4-5` (original run), `claude-opus-4-8` (re-test after a model bump), `claude-fable-5` (2026-06-09), `gpt-5.6-sol` (2026-07-09 raw-image pilot), `claude-opus-5` (2026-07-20 default-scope evaluation)
-**Model scope (current):** Fable 5, **Opus 5**, and Gemini 3.6 Flash. Sol, GPT 5.5, and Grok remain explicit opt-ins.
+**Model scope (current):** Fable 5, Gemini 3.6 Flash, and Gemini 3.7 Flash. Opus 5, Sol, GPT 5.5, and Grok remain explicit opt-ins.
 **Harnesses:** Claude/Opus/Fable: `eval/needle-haystack/` (older receipts preserved from `/tmp/needle_eval`); Sol: `eval/sol-profile/` (raw responses and receipts committed)
 
 ---

diff --git a/README.md b/README.md
@@ -125,7 +125,7 @@ without running the proxy.
 - **`claude-opus-5`:** weaker recall than Fable 5 (verbatim **2/15 vs 13/15**), good
   enough otherwise (100/100 arithmetic, 0/16 never-stated), **~4.7×** context before
   `/compact`. Suggested effort: **medium**. Details: [FINDINGS.md](FINDINGS.md).
-- **Model scope:** default `PXPIPE_MODELS=claude-fable-5,gemini-3.6-flash`. Opus 5, Sol, GPT 5.5,
+- **Model scope:** default `PXPIPE_MODELS=claude-fable-5,gemini-3.6-flash,gemini-3.7-flash`. Opus 5, Sol, GPT 5.5,
   and **Grok** are opt-in only (dashboard chips or
   `PXPIPE_MODELS`). The exact Sol id still matters. Sibling variants such as
   `gpt-5.6-terra` do not
@@ -168,11 +168,12 @@ is confabulations, so lower is better.
 | model | arithmetic (N=100) | gist (N=98) | state (N=18) | never-stated (N=16) | dense hex (N=15) | profile provenance and receipts |
 | --- | ---: | ---: | ---: | ---: | ---: | --- |
 | `claude-fable-5` | **100/100** | **98/98** | **18/18** | **0/16** | 13/15 | June 2026 production profiles: [arithmetic + hex](FINDINGS.md), [gist/state/guards](eval/gist-recall/) |
-| `google/gemini-3.6-flash` | **100/100** | **98/98** | **18/18** | **0/16** | **14/15** | current shipped profile: [quality results](eval/gemini-profile/QUALITY_RESULTS.md) |
+| `google/gemini-3.6-flash`, `3.7-flash` | **100/100** | **98/98** | **18/18** | **0/16** | **14/15** | current shipped profile: [quality results](eval/gemini-profile/QUALITY_RESULTS.md) |
 | `claude-opus-5` | **100/100** | 94/98 | 17/18 | **0/16** | 2/15 | current profile: [arithmetic](eval/gsm8k/), [gist/state/guards](eval/gist-recall/), [dense hex](eval/verbatim-15/) |
 | `gpt-5.6-sol` | 98/100 | 83/98 | 17/18 | 4/16 | 0/15 | prior 5×8 broad suite; native 14px pilot: 7/8 exact, 0 inventions, gist/guard pass: [pilot](eval/sol-profile/README.md) |
 | `claude-opus-4-8` | 93/100 | 77/98 | **18/18** | **0/16** | 0/15 | historical profile: [arithmetic](eval/gsm8k/), [gist/state/guards](eval/gist-recall/), [dense hex](eval/needle-haystack/) |
 | `grok-4.5` | **100/100** | **97/98** | 17/18 | **0/16** | 0/15 | native 14px/84 quality suite (live profile); [quality](eval/grok-density/QUALITY_RESULTS.md), [native-sweep](eval/grok-density/native-sweep/RESULTS.md) |
+| `grok-4.6` high | **100/100** | **97/98** | 17/18 | **0/16** | 0/15 | native 14px/84, reasoning high; [quality](eval/grok-profile/QUALITY_RESULTS.md) |
 | `moonshotai/kimi-k3` | 79/100 | 84/98 | 15/18 | 1/16 | 0/15 | generic GPT profile: [quality results](eval/sol-profile/KIMI_K3_QUALITY_RESULTS.md) |
 
 ### Native-profile cost check

diff --git a/eval/gemini-profile/README.md b/eval/gemini-profile/README.md
@@ -0,0 +1,28 @@
+# Gateway visual quality suite
+
+This suite evaluates Gemini image models exposed by the local Cloudflare AI Gateway.
+
+Confirmed model IDs:
+
+- `gemini-3.7-flash` via Google AI Studio
+
+There is no `gemini-4.7` in the gateway catalog. The similarly named
+`workers-ai/@cf/zai-org/glm-4.7-flash` is GLM, not Gemini. Although its catalog
+advertises attachments, its live endpoint rejects image input as non-multimodal.
+
+DeepSeek v4 is available as `workers-ai/@cf/deepseek-ai/deepseek-v4-flash` and
+`workers-ai/@cf/deepseek-ai/deepseek-v4-pro`, but both are declared text-only.
+The client rejects them instead of recording invalid image-reading scores.
+
+Build first, then run the same three quality checks used for Gemini 3.6:
+
+```bash
+pnpm run build
+
+MODEL=gemini-3.7-flash LIVE=1 node eval/gemini-profile/novel-arithmetic.mjs
+MODEL=gemini-3.7-flash LIVE=1 node eval/gemini-profile/gist-recall.mjs
+MODEL=gemini-3.7-flash LIVE=1 node eval/gemini-profile/verbatim-hex.mjs
+
+```
+
+Each run writes a model-qualified result file and does not overwrite the existing Gemini 3.6 receipts.
diff --git a/eval/gemini-profile/gemini-client.mjs b/eval/gemini-profile/gemini-client.mjs
@@ -1,4 +1,15 @@
-// Dedicated Google AI Studio client for Gemini 3.6 Flash evaluations.
+// Google AI Studio eval client through the local Cloudflare AI Gateway.
+
+function gatewayOrigin() {
+  const base = (process.env.OPENAI_BASE_URL || 'http://127.0.0.1:8082/v1').replace(/\/$/, '');
+  return new URL(base).origin;
+}
+
+export function resultFilename(base, model) {
+  const clean = model.replace(/^google\//, '');
+  if (clean === 'gemini-3.6-flash') return `${base}-results.json`;
+  return `${base}-${clean.replace(/[^a-zA-Z0-9._-]+/g, '_')}-results.json`;
+}
 
 export async function callGeminiRequest({ model = 'gemini-3.6-flash', request, maxOutputTokens = 1000, timeoutMs = 120000 }) {
   const key = process.env.OPENAI_API_KEY || process.env.GEMINI_API_KEY;
@@ -7,8 +18,10 @@ export async function callGeminiRequest({ model = 'gemini-3.6-flash', request, m
   const controller = new AbortController();
   const timer = setTimeout(() => controller.abort(), timeoutMs);
   const started = Date.now();
-  const cleanModel = model.replace(/^google\//, '').replace(/^claude-/, '');
-  const url = `http://127.0.0.1:47821/google-ai-studio/v1beta/models/${cleanModel}:generateContent`;
+  const cleanModel = model
+    .replace(/^google\//, '')
+    .replace(/[^a-zA-Z0-9._-]+/g, '_');
+  const url = `${gatewayOrigin()}/google-ai-studio/v1beta/models/${cleanModel}:generateContent`;
 
   try {
     const response = await fetch(url, {

diff --git a/eval/gemini-profile/gist-recall.mjs b/eval/gemini-profile/gist-recall.mjs
@@ -5,7 +5,7 @@ import { fileURLToPath } from 'node:url';
 import { renderTextToPngs } from '../../dist/core/render.js';
 import { resolveGeminiProfile } from '../../dist/core/gemini-model-profiles.js';
 import { factSheetText } from '../../dist/core/factsheet.js';
-import { callGemini } from './gemini-client.mjs';
+import { callGemini, resultFilename } from './gemini-client.mjs';
 
 const HERE = dirname(fileURLToPath(import.meta.url));
 const ROOT = join(HERE, '../gist-recall');
@@ -81,5 +81,5 @@ const out = {
   rows
 };
 
-writeFileSync(join(HERE, 'gist-recall-results.json'), JSON.stringify(out, null, 2));
+writeFileSync(join(HERE, resultFilename('gist-recall', MODEL)), JSON.stringify(out, null, 2));
 console.log(JSON.stringify({ answerable: out.answerable, state: out.state, unanswerable: out.unanswerable }, null, 2));
diff --git a/eval/gemini-profile/novel-arithmetic.mjs b/eval/gemini-profile/novel-arithmetic.mjs
@@ -6,7 +6,7 @@ import { renderTextToPngs } from '../../dist/core/render.js';
 import { resolveGeminiProfile } from '../../dist/core/gemini-model-profiles.js';
 import { factSheetText } from '../../dist/core/factsheet.js';
 import { visionTokensForModel } from '../../dist/core/openai.js';
-import { callGemini } from './gemini-client.mjs';
+import { callGemini, resultFilename } from './gemini-client.mjs';
 
 const HERE = dirname(fileURLToPath(import.meta.url));
 const MODEL = process.env.MODEL || 'gemini-3.6-flash';
@@ -16,7 +16,7 @@ const SEED = Number(process.env.SEED || 20260711);
 const CONCURRENCY = Math.max(1, Number(process.env.CONCURRENCY || 5));
 const TIMEOUT = Number(process.env.TIMEOUT_MS || 180000);
 const profile = resolveGeminiProfile();
-const RESULT = join(HERE, `novel-arithmetic-results.json`);
+const RESULT = join(HERE, resultFilename('novel-arithmetic', MODEL));
 
 function lcg(seed) { let s = seed >>> 0; return () => s = (Math.imul(s, 1664525) + 1013904223) >>> 0; }
 function ri(r, a, b) { return a + (r() % (b - a + 1)); }

diff --git a/eval/gemini-profile/verbatim-hex.mjs b/eval/gemini-profile/verbatim-hex.mjs
@@ -2,7 +2,7 @@
 import { readFileSync, writeFileSync } from 'node:fs';
 import { dirname, join } from 'node:path';
 import { fileURLToPath } from 'node:url';
-import { callGemini } from './gemini-client.mjs';
+import { callGemini, resultFilename } from './gemini-client.mjs';
 import { renderTextToPngs } from '../../dist/core/render.js';
 import { resolveGeminiProfile } from '../../dist/core/gemini-model-profiles.js';
 
@@ -13,7 +13,7 @@ const LIVE = process.env.LIVE === '1';
 const TIMEOUT = Number(process.env.TIMEOUT_MS || 90000);
 const MAX_OUTPUT_TOKENS = Number(process.env.MAX_OUTPUT_TOKENS || 80);
 const trials = JSON.parse(readFileSync(join(ROOT, 'golds.json'), 'utf8'));
-const RESULT = join(HERE, 'verbatim-hex-results.json');
+const RESULT = join(HERE, resultFilename('verbatim-hex', MODEL));
 const profile = resolveGeminiProfile();
 
 function denseLog(trial, totalLines = 80) {

diff --git a/eval/grok-profile/QUALITY_RESULTS.md b/eval/grok-profile/QUALITY_RESULTS.md
@@ -0,0 +1,34 @@
+# Grok 4.6 quality results
+
+Model: `grok-4.6` through the Codex Responses provider, `reasoning.effort=high`.
+Image calls bypassed pxpipe (`OPENAI_BASE_URL` upstream, port 47821 rejected)
+and used the **live production profile** resolved by `resolveGptProfile('grok-4.6')`.
+
+## Native 14px / 84 cols / maxH 512, reasoning high (2026-08-13)
+
+Recipe locked in each receipt: `font=jetbrains-mono-14`, `cols=84`, `maxH=512`,
+factsheet on, `reasoningEffort=high`.
+
+| test | text | production image | notes |
+|---|---:|---:|---|
+| novel arithmetic, N=100 | 100/100 | **100/100** | pure image also 100/100 |
+| gist recall | — | **97/98** | work3 s4 final `BATCH_WINDOW_MS` 4000 vs gold 4800 |
+| state tracking | — | **17/18** | same miss as gist |
+| never-stated guards | — | **0/16** confabulated | lower is better |
+| dense 12-char hex | — | **0/15** | all 15 completed; still byte-unsafe |
+
+Same scores as `grok-4.5` on this corpus. Dense hex remains 0.
+
+Receipts:
+
+- `novel-arithmetic-grok-4.6-results.json`
+- `gist-recall-grok-4.6-results.json`
+- `verbatim-hex-grok-4.6-results.json`
+
+```bash
+export OPENAI_BASE_URL=http://127.0.0.1:8082/v1
+export OPENAI_API_KEY=…
+GROK_QUALITY_LIVE=1 N=100 node eval/grok-profile/novel-arithmetic.mjs
+GROK_QUALITY_LIVE=1 node eval/grok-profile/gist-recall.mjs
+GROK_QUALITY_LIVE=1 node eval/grok-profile/verbatim-hex.mjs
+```