diff --git a/README.md b/README.md index bdb6fdb..49d8ab0 100644 --- a/README.md +++ b/README.md @@ -183,6 +183,30 @@ npx quorum validate # Validate test case format Community contributions are welcome. Start with [CONTRIBUTING.md](./CONTRIBUTING.md) for local setup, architecture notes, and pull request expectations. +## Production Monitoring + +Monitor live RAG traffic with the council. After any inference call, submit the sample fire-and-forget — it never blocks your production path: + +```js +// After your RAG pipeline response +fetch('https://your-quorum.app/api/sample', { + method: 'POST', + headers: { + 'Content-Type': 'application/json', + 'Authorization': 'Bearer ' + process.env.QUORUM_SERVICE_KEY, + }, + body: JSON.stringify({ + query, // the user's question + response, // your model's answer + contexts, // retrieved context passages (array of strings) + }), +}).catch(() => {}); // never await, never throw +``` + +Quorum samples at 5% by default (configurable via `SAMPLE_RATE` env var). View the **Monitoring** dashboard for score trends, baseline comparison, and drift alerts. + +**Rate limit note:** The `/api/sample` endpoint shares the global 30 RPM limit across all `/api` routes. At 5% sample rate, you need fewer than 600 RPM of production traffic to stay under the limit. + ## License [MIT](./LICENSE) diff --git a/TODOS.md b/TODOS.md index 3d7b1df..2e1b958 100644 --- a/TODOS.md +++ b/TODOS.md @@ -22,6 +22,51 @@ --- +## P3 — /paper figure callouts + +### Figure insight callouts in PaperPage + +**What:** Add a 1-sentence interpretive callout below each of the 5 figure captions in the paper. Style: small text in `var(--accent)` color with a left border or subtle background. Content: the key takeaway from each figure in plain language (e.g., "Gemini Flash is the Pareto-dominant configuration — highest accuracy at lowest cost."). + +**Why:** Figures are currently data delivery only. A non-expert reader landing on Figure 3 (cost-accuracy pareto) without reading the surrounding paragraphs has no interpretive scaffold. Callouts make the figures scannable and self-contained. + +**Pros:** Makes the paper more accessible to practitioners who scan figures first. Very low implementation effort. + +**Cons:** Adds interpretation that belongs in the caption — could feel redundant for expert readers. + +**Context:** Surfaced in 2026-03-23 design review. Callout text should be authored by the paper author, not generated. Suggested format: `

{insight}

`. + +**Effort:** XS (human ~30 min) → XS with CC+gstack (~5 min) + +**Priority:** P3 + +**Depends on:** Author must provide the 5 insight sentences. + +--- + +### Design system alignment + cross-page navigation — BenchmarksPage + +**What:** Three changes in one pass: +1. **Token alignment**: Import `LandingPage.css` into BenchmarksPage and replace all hardcoded hex values (`#d99058`, `#3b3c36`, `#F5F3EF`, etc.) with CSS variables (`var(--accent)`, `var(--text-primary)`, `var(--bg)`, etc.). +2. **Cross-page nav links**: Add a `/paper` link to BenchmarksPage nav (next to the "Run your own" CTA). Add a `/benchmarks` link to PaperPage bottom CTA (next to the GitHub link). +3. **Back-to-top button**: Add a fixed bottom-right back-to-top button on both pages (small, copper-outlined, appears after 400px scroll). + +**Why:** DESIGN_SYSTEM.md forbids hardcoded hex in JSX. BenchmarksPage has ~20 hardcoded hex instances. The two pages link to each other in CTAs but not in nav — users who land on /benchmarks have no top-level path to the paper. + +**Pros:** Brings BenchmarksPage into design system compliance. Improves page-to-page discoverability. + +**Cons:** The back-to-top button adds a small floating element to both pages — verify it doesn't overlap the mobile sticky TOC drawer button. + +**Context:** Surfaced in 2026-03-23 design review. + +**Effort:** S (human ~2h) → S with CC+gstack (~10 min) + +**Priority:** P2 + +**Depends on:** Sticky TOC sidebar (coordinate back-to-top position with mobile TOC button). + +--- + ### Vitest + React Testing Library — auth unit tests **What:** Set up Vitest + RTL and write unit tests for: `firebaseConfigured=false` renders "not configured", each named Firebase error code (`popup-blocked`, `popup-closed-by-user`, `account-exists`, `unauthorized-domain`) produces the correct user-facing message, and `SocialAuth` buttons disable during loading. @@ -39,3 +84,65 @@ **Priority:** P2 **Depends on:** Nothing. Can be added independently. + +--- + +### Vitest + RTL — frontend component tests (ServiceKeysManager + auth) + +**What:** Set up Vitest + React Testing Library for the frontend. Write tests for `ServiceKeysManager`: load states (loading shimmer, success list, error, empty state), create key flow (POST → modal), copy button (clipboard available + fallback), revoke flow (confirm → DELETE, cancel → no DELETE). Bundle with existing auth test TODO (`SocialAuth`, `AuthContext`). + +**Why:** 0% frontend test coverage. The copy modal is a one-time-only interaction — clipboard unavailability would be a silent failure without a test. The revoke confirm flow also has no coverage. + +**Pros:** Locks in ServiceKeysManager behavior. Establishes Vitest + RTL infra for all future frontend tests. Auth tests from the prior TODO come along for free once infra is set up. + +**Cons:** Adds dev dependencies (`vitest`, `@testing-library/react`, `@testing-library/user-event`, `jsdom`). One-time setup cost. + +**Context:** Emerged from 2026-03-22 ServiceKeysManager eng review. Frontend has no test framework; backend already runs Vitest. Start in `frontend/vitest.config.js`, then `frontend/src/components/ServiceKeysManager.test.jsx`. Mock `navigator.clipboard` and `../lib/api` module. + +**Effort:** S (human ~1 day) → S with CC+gstack (~20 min) + +**Priority:** P2 + +**Depends on:** Nothing. Can be bundled with next frontend feature session. + +--- + +## P3 — Settings + +### ARIA labels for "Configured" badge on provider cards + +**What:** Add `aria-label="{Provider} key configured"` to the "Configured" badge `` in `ApiKeysManager` for each provider card. + +**Why:** Screen readers currently announce "Configured" three times in sequence with no provider context. A user navigating by keyboard hears "Configured, Configured, Configured" — the provider name is visible in the DOM but not associated to the badge. + +**Pros:** 3-line fix. Correct a11y behavior. + +**Cons:** None. + +**Context:** Surfaced in 2026-03-23 Settings page design review. File: `frontend/src/components/ApiKeysManager.jsx`, the `` with "Configured" text near line 147. + +**Effort:** XS (human ~5 min) → XS with CC+gstack (~1 min) + +**Priority:** P3 + +**Depends on:** Can be done in any PR that touches ApiKeysManager. + +--- + +### Full Account settings tab + +**What:** Build out the Account tab in Settings (`/app/settings/account`) with: display name edit, email change (requires re-authentication), password change form, and danger zone (delete account with confirmation). + +**Why:** The Account tab currently ships as a read-only stub showing email/username. Users expect to manage their account from Settings. Password change and email change are standard auth operations. + +**Pros:** Completes the settings experience. Reduces "where do I change my email?" support friction. + +**Cons:** Email change requires Firebase re-auth flow. Danger zone (account deletion) requires backend cascade (delete evaluations, keys, user doc). Non-trivial backend work. + +**Context:** Surfaced in 2026-03-23 Settings page design review. The tab stub (`/app/settings/account`) will ship in the initial Settings refactor PR to reserve the route. Full implementation is a separate PR. Auth flows must use `signInWithPopup` for re-auth (per CLAUDE.md Firebase rules). + +**Effort:** L (human ~3 days) → M with CC+gstack (~45 min) + +**Priority:** P2 + +**Depends on:** Settings refactor PR must ship first (adds the Account tab route). diff --git a/backend/package-lock.json b/backend/package-lock.json index 5f2f5e2..b26cbbf 100644 --- a/backend/package-lock.json +++ b/backend/package-lock.json @@ -31,7 +31,7 @@ }, "devDependencies": { "supertest": "^7.2.2", - "vitest": "^4.0.18" + "vitest": "^4.1.0" }, "engines": { "node": ">=20.0.0" @@ -97,19 +97,38 @@ "openapi-types": ">=7" } }, - "node_modules/@esbuild/win32-x64": { - "version": "0.27.3", - "cpu": [ - "x64" - ], + "node_modules/@emnapi/core": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.9.1.tgz", + "integrity": "sha512-mukuNALVsoix/w1BJwFzwXBN/dHeejQtuVzcDsfOEsdpCumXb/E9j8w11h5S54tT1xhifGfbbSm/ICrObRb3KA==", "dev": true, "license": "MIT", "optional": true, - "os": [ - "win32" - ], - "engines": { - "node": ">=18" + "dependencies": { + "@emnapi/wasi-threads": "1.2.0", + "tslib": "^2.4.0" + } + }, + "node_modules/@emnapi/runtime": { + "version": "1.9.1", + "resolved": "https://registry.npmjs.org/@emnapi/runtime/-/runtime-1.9.1.tgz", + "integrity": "sha512-VYi5+ZVLhpgK4hQ0TAjiQiZ6ol0oe4mBx7mVv7IflsiEp0OWoVsp/+f9Vc1hOhE0TtkORVrI1GvzyreqpgWtkA==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, + "node_modules/@emnapi/wasi-threads": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/@emnapi/wasi-threads/-/wasi-threads-1.2.0.tgz", + "integrity": "sha512-N10dEJNSsUx41Z6pZsXU8FjPjpBEplgH24sfkmITrBED1/U2Esum9F3lfLrMjKHHjmi557zQn7kR9R+XWXu5Rg==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" } }, "node_modules/@fastify/busboy": { @@ -498,6 +517,8 @@ }, "node_modules/@jridgewell/sourcemap-codec": { "version": "1.5.5", + "resolved": "https://registry.npmjs.org/@jridgewell/sourcemap-codec/-/sourcemap-codec-1.5.5.tgz", + "integrity": "sha512-cYQ9310grqxueWbl+WuIUIaiUaDcj7WOq5fVhEljNVgRfOUhY9fy2zTvfoqWsnebh8Sl70VScFbICvJnLKB0Og==", "dev": true, "license": "MIT" }, @@ -523,6 +544,23 @@ "sparse-bitfield": "^3.0.3" } }, + "node_modules/@napi-rs/wasm-runtime": { + "version": "1.1.1", + "resolved": "https://registry.npmjs.org/@napi-rs/wasm-runtime/-/wasm-runtime-1.1.1.tgz", + "integrity": "sha512-p64ah1M1ld8xjWv3qbvFwHiFVWrq1yFvV4f7w+mzaqiR4IlSgkqhcRdHwsGgomwzBH51sRY4NEowLxnaBjcW/A==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@emnapi/core": "^1.7.1", + "@emnapi/runtime": "^1.7.1", + "@tybys/wasm-util": "^0.10.1" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + } + }, "node_modules/@noble/hashes": { "version": "1.8.0", "resolved": "https://registry.npmjs.org/@noble/hashes/-/hashes-1.8.0.tgz", @@ -546,6 +584,16 @@ "node": ">=8.0.0" } }, + "node_modules/@oxc-project/types": { + "version": "0.120.0", + "resolved": "https://registry.npmjs.org/@oxc-project/types/-/types-0.120.0.tgz", + "integrity": "sha512-k1YNu55DuvAip/MGE1FTsIuU3FUCn6v/ujG9V7Nq5Df/kX2CWb13hhwD0lmJGMGqE+bE1MXvv9SZVnMzEXlWcg==", + "dev": true, + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/Boshen" + } + }, "node_modules/@paralleldrive/cuid2": { "version": "2.3.1", "resolved": "https://registry.npmjs.org/@paralleldrive/cuid2/-/cuid2-2.3.1.tgz", @@ -608,20 +656,248 @@ "version": "1.1.0", "license": "BSD-3-Clause" }, - "node_modules/@rollup/rollup-win32-x64-gnu": { - "version": "4.57.1", + "node_modules/@rolldown/binding-android-arm64": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-android-arm64/-/binding-android-arm64-1.0.0-rc.10.tgz", + "integrity": "sha512-jOHxwXhxmFKuXztiu1ORieJeTbx5vrTkcOkkkn2d35726+iwhrY1w/+nYY/AGgF12thg33qC3R1LMBF5tHTZHg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-darwin-arm64": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-arm64/-/binding-darwin-arm64-1.0.0-rc.10.tgz", + "integrity": "sha512-gED05Teg/vtTZbIJBc4VNMAxAFDUPkuO/rAIyyxZjTj1a1/s6z5TII/5yMGZ0uLRCifEtwUQn8OlYzuYc0m70w==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-darwin-x64": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-darwin-x64/-/binding-darwin-x64-1.0.0-rc.10.tgz", + "integrity": "sha512-rI15NcM1mA48lqrIxVkHfAqcyFLcQwyXWThy+BQ5+mkKKPvSO26ir+ZDp36AgYoYVkqvMcdS8zOE6SeBsR9e8A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-freebsd-x64": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-freebsd-x64/-/binding-freebsd-x64-1.0.0-rc.10.tgz", + "integrity": "sha512-XZRXHdTa+4ME1MuDVp021+doQ+z6Ei4CCFmNc5/sKbqb8YmkiJdj8QKlV3rCI0AJtAeSB5n0WGPuJWNL9p/L2w==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-arm-gnueabihf": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm-gnueabihf/-/binding-linux-arm-gnueabihf-1.0.0-rc.10.tgz", + "integrity": "sha512-R0SQMRluISSLzFE20sPWYHVmJdDQnRyc/FzSCN72BqQmh2SOZUFG+N3/vBZpR4C6WpEUVYJLrYUXaj43sJsNLA==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-arm64-gnu": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-gnu/-/binding-linux-arm64-gnu-1.0.0-rc.10.tgz", + "integrity": "sha512-Y1reMrV/o+cwpduYhJuOE3OMKx32RMYCidf14y+HssARRmhDuWXJ4yVguDg2R/8SyyGNo+auzz64LnPK9Hq6jg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-arm64-musl": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-arm64-musl/-/binding-linux-arm64-musl-1.0.0-rc.10.tgz", + "integrity": "sha512-vELN+HNb2IzuzSBUOD4NHmP9yrGwl1DVM29wlQvx1OLSclL0NgVWnVDKl/8tEks79EFek/kebQKnNJkIAA4W2g==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-ppc64-gnu": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-ppc64-gnu/-/binding-linux-ppc64-gnu-1.0.0-rc.10.tgz", + "integrity": "sha512-ZqrufYTgzxbHwpqOjzSsb0UV/aV2TFIY5rP8HdsiPTv/CuAgCRjM6s9cYFwQ4CNH+hf9Y4erHW1GjZuZ7WoI7w==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-s390x-gnu": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-s390x-gnu/-/binding-linux-s390x-gnu-1.0.0-rc.10.tgz", + "integrity": "sha512-gSlmVS1FZJSRicA6IyjoRoKAFK7IIHBs7xJuHRSmjImqk3mPPWbR7RhbnfH2G6bcmMEllCt2vQ/7u9e6bBnByg==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-x64-gnu": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-gnu/-/binding-linux-x64-gnu-1.0.0-rc.10.tgz", + "integrity": "sha512-eOCKUpluKgfObT2pHjztnaWEIbUabWzk3qPZ5PuacuPmr4+JtQG4k2vGTY0H15edaTnicgU428XW/IH6AimcQw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-linux-x64-musl": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-linux-x64-musl/-/binding-linux-x64-musl-1.0.0-rc.10.tgz", + "integrity": "sha512-Xdf2jQbfQowJnLcgYfD/m0Uu0Qj5OdxKallD78/IPPfzaiaI4KRAwZzHcKQ4ig1gtg1SuzC7jovNiM2TzQsBXA==", "cpu": [ "x64" ], "dev": true, "license": "MIT", "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-openharmony-arm64": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-openharmony-arm64/-/binding-openharmony-arm64-1.0.0-rc.10.tgz", + "integrity": "sha512-o1hYe8hLi1EY6jgPFyxQgQ1wcycX+qz8eEbVmot2hFkgUzPxy9+kF0u0NIQBeDq+Mko47AkaFFaChcvZa9UX9Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/binding-wasm32-wasi": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-wasm32-wasi/-/binding-wasm32-wasi-1.0.0-rc.10.tgz", + "integrity": "sha512-Ugv9o7qYJudqQO5Y5y2N2SOo6S4WiqiNOpuQyoPInnhVzCY+wi/GHltcLHypG9DEUYMB0iTB/huJrpadiAcNcA==", + "cpu": [ + "wasm32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@napi-rs/wasm-runtime": "^1.1.1" + }, + "engines": { + "node": ">=14.0.0" + } + }, + "node_modules/@rolldown/binding-win32-arm64-msvc": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-win32-arm64-msvc/-/binding-win32-arm64-msvc-1.0.0-rc.10.tgz", + "integrity": "sha512-7UODQb4fQUNT/vmgDZBl3XOBAIOutP5R3O/rkxg0aLfEGQ4opbCgU5vOw/scPe4xOqBwL9fw7/RP1vAMZ6QlAQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, "os": [ "win32" - ] + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } }, - "node_modules/@rollup/rollup-win32-x64-msvc": { - "version": "4.57.1", + "node_modules/@rolldown/binding-win32-x64-msvc": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/binding-win32-x64-msvc/-/binding-win32-x64-msvc-1.0.0-rc.10.tgz", + "integrity": "sha512-PYxKHMVHOb5NJuDL53vBUl1VwUjymDcYI6rzpIni0C9+9mTiJedvUxSk7/RPp7OOAm3v+EjgMu9bIy3N6b408w==", "cpu": [ "x64" ], @@ -630,7 +906,17 @@ "optional": true, "os": [ "win32" - ] + ], + "engines": { + "node": "^20.19.0 || >=22.12.0" + } + }, + "node_modules/@rolldown/pluginutils": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-rc.10.tgz", + "integrity": "sha512-UkVDEFk1w3mveXeKgaTuYfKWtPbvgck1dT8TUG3bnccrH0XtLTuAyfCoks4Q/M5ZGToSVJTIQYCzy2g/atAOeg==", + "dev": true, + "license": "MIT" }, "node_modules/@scarf/scarf": { "version": "1.4.0", @@ -645,6 +931,8 @@ }, "node_modules/@standard-schema/spec": { "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz", + "integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==", "dev": true, "license": "MIT" }, @@ -658,6 +946,17 @@ "node": ">= 10" } }, + "node_modules/@tybys/wasm-util": { + "version": "0.10.1", + "resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.1.tgz", + "integrity": "sha512-9tTaPJLSiejZKx+Bmog4uSubteqTvFrVrURwkmHixBo0G4seD0zUxp98E1DzUBJxLQ3NPwXrGKDiVjwx/DpPsg==", + "dev": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, "node_modules/@types/caseless": { "version": "0.12.5", "resolved": "https://registry.npmjs.org/@types/caseless/-/caseless-0.12.5.tgz", @@ -667,6 +966,8 @@ }, "node_modules/@types/chai": { "version": "5.2.3", + "resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz", + "integrity": "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==", "dev": true, "license": "MIT", "dependencies": { @@ -676,11 +977,15 @@ }, "node_modules/@types/deep-eql": { "version": "4.0.2", + "resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz", + "integrity": "sha512-c9h9dVVMigMPc4bwTvC5dxqtqJZwQPePsWjPlpSOnojbor6pGqdk541lfA7AqFQr5pB1BRdq0juY9db81BwyFw==", "dev": true, "license": "MIT" }, "node_modules/@types/estree": { "version": "1.0.8", + "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz", + "integrity": "sha512-dWHzHa2WqEXI/O1E9OjrocMTKJl2mSrEolh1Iomrv6U+JuNwaHXsXx9bLu5gG7BUWFIN0skIQJQ/L1rIex4X6w==", "dev": true, "license": "MIT" }, @@ -776,15 +1081,17 @@ } }, "node_modules/@vitest/expect": { - "version": "4.0.18", + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@vitest/expect/-/expect-4.1.0.tgz", + "integrity": "sha512-EIxG7k4wlWweuCLG9Y5InKFwpMEOyrMb6ZJ1ihYu02LVj/bzUwn2VMU+13PinsjRW75XnITeFrQBMH5+dLvCDA==", "dev": true, "license": "MIT", "dependencies": { - "@standard-schema/spec": "^1.0.0", + "@standard-schema/spec": "^1.1.0", "@types/chai": "^5.2.2", - "@vitest/spy": "4.0.18", - "@vitest/utils": "4.0.18", - "chai": "^6.2.1", + "@vitest/spy": "4.1.0", + "@vitest/utils": "4.1.0", + "chai": "^6.2.2", "tinyrainbow": "^3.0.3" }, "funding": { @@ -792,7 +1099,9 @@ } }, "node_modules/@vitest/pretty-format": { - "version": "4.0.18", + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@vitest/pretty-format/-/pretty-format-4.1.0.tgz", + "integrity": "sha512-3RZLZlh88Ib0J7NQTRATfc/3ZPOnSUn2uDBUoGNn5T36+bALixmzphN26OUD3LRXWkJu4H0s5vvUeqBiw+kS0A==", "dev": true, "license": "MIT", "dependencies": { @@ -803,11 +1112,13 @@ } }, "node_modules/@vitest/runner": { - "version": "4.0.18", + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@vitest/runner/-/runner-4.1.0.tgz", + "integrity": "sha512-Duvx2OzQ7d6OjchL+trw+aSrb9idh7pnNfxrklo14p3zmNL4qPCDeIJAK+eBKYjkIwG96Bc6vYuxhqDXQOWpoQ==", "dev": true, "license": "MIT", "dependencies": { - "@vitest/utils": "4.0.18", + "@vitest/utils": "4.1.0", "pathe": "^2.0.3" }, "funding": { @@ -815,11 +1126,14 @@ } }, "node_modules/@vitest/snapshot": { - "version": "4.0.18", + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@vitest/snapshot/-/snapshot-4.1.0.tgz", + "integrity": "sha512-0Vy9euT1kgsnj1CHttwi9i9o+4rRLEaPRSOJ5gyv579GJkNpgJK+B4HSv/rAWixx2wdAFci1X4CEPjiu2bXIMg==", "dev": true, "license": "MIT", "dependencies": { - "@vitest/pretty-format": "4.0.18", + "@vitest/pretty-format": "4.1.0", + "@vitest/utils": "4.1.0", "magic-string": "^0.30.21", "pathe": "^2.0.3" }, @@ -828,7 +1142,9 @@ } }, "node_modules/@vitest/spy": { - "version": "4.0.18", + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@vitest/spy/-/spy-4.1.0.tgz", + "integrity": "sha512-pz77k+PgNpyMDv2FV6qmk5ZVau6c3R8HC8v342T2xlFxQKTrSeYw9waIJG8KgV9fFwAtTu4ceRzMivPTH6wSxw==", "dev": true, "license": "MIT", "funding": { @@ -836,11 +1152,14 @@ } }, "node_modules/@vitest/utils": { - "version": "4.0.18", + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@vitest/utils/-/utils-4.1.0.tgz", + "integrity": "sha512-XfPXT6a8TZY3dcGY8EdwsBulFCIw+BeeX0RZn2x/BtiY/75YGh8FeWGG8QISN/WhaqSrE2OrlDgtF8q5uhOTmw==", "dev": true, "license": "MIT", "dependencies": { - "@vitest/pretty-format": "4.0.18", + "@vitest/pretty-format": "4.1.0", + "convert-source-map": "^2.0.0", "tinyrainbow": "^3.0.3" }, "funding": { @@ -932,6 +1251,8 @@ }, "node_modules/assertion-error": { "version": "2.0.1", + "resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz", + "integrity": "sha512-Izi8RQcffqCeNVgFigKli1ssklIbpHnCYc6AknXGYoB6grJqyeby7jv12JUQgmTAnIDnbck1uxksT4dzN3PWBA==", "dev": true, "license": "MIT", "engines": { @@ -1072,6 +1393,8 @@ }, "node_modules/chai": { "version": "6.2.2", + "resolved": "https://registry.npmjs.org/chai/-/chai-6.2.2.tgz", + "integrity": "sha512-NUPRluOfOiTKBKvWPtSD4PhFvWCqOi0BGStNWs57X9js7XGTprSmFoz5F0tWhR4WPjNeR9jXqdC7/UpSJTnlRg==", "dev": true, "license": "MIT", "engines": { @@ -1234,6 +1557,13 @@ "node": ">= 0.6" } }, + "node_modules/convert-source-map": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/convert-source-map/-/convert-source-map-2.0.0.tgz", + "integrity": "sha512-Kvp459HrV2FEJ1CAsi1Ku+MY3kasH19TFykTz2xWmMeq6bk2NU3XXvfJ+Q61m0xktWwt+1HSYf3JZsTms3aRJg==", + "dev": true, + "license": "MIT" + }, "node_modules/cookie": { "version": "0.7.2", "license": "MIT", @@ -1326,6 +1656,16 @@ "npm": "1.2.8000 || >= 1.4.16" } }, + "node_modules/detect-libc": { + "version": "2.1.2", + "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", + "integrity": "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ==", + "dev": true, + "license": "Apache-2.0", + "engines": { + "node": ">=8" + } + }, "node_modules/dezalgo": { "version": "1.0.4", "resolved": "https://registry.npmjs.org/dezalgo/-/dezalgo-1.0.4.tgz", @@ -1433,7 +1773,9 @@ } }, "node_modules/es-module-lexer": { - "version": "1.7.0", + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.0.0.tgz", + "integrity": "sha512-5POEcUuZybH7IdmGsD8wlf0AI55wMecM9rVBTI/qEAy2c1kTOm3DjFYjrBdI2K3BaJjJYfYFeRtM0t9ssnRuxw==", "dev": true, "license": "MIT" }, @@ -1460,46 +1802,6 @@ "node": ">= 0.4" } }, - "node_modules/esbuild": { - "version": "0.27.3", - "dev": true, - "hasInstallScript": true, - "license": "MIT", - "bin": { - "esbuild": "bin/esbuild" - }, - "engines": { - "node": ">=18" - }, - "optionalDependencies": { - "@esbuild/aix-ppc64": "0.27.3", - "@esbuild/android-arm": "0.27.3", - "@esbuild/android-arm64": "0.27.3", - "@esbuild/android-x64": "0.27.3", - "@esbuild/darwin-arm64": "0.27.3", - "@esbuild/darwin-x64": "0.27.3", - "@esbuild/freebsd-arm64": "0.27.3", - "@esbuild/freebsd-x64": "0.27.3", - "@esbuild/linux-arm": "0.27.3", - "@esbuild/linux-arm64": "0.27.3", - "@esbuild/linux-ia32": "0.27.3", - "@esbuild/linux-loong64": "0.27.3", - "@esbuild/linux-mips64el": "0.27.3", - "@esbuild/linux-ppc64": "0.27.3", - "@esbuild/linux-riscv64": "0.27.3", - "@esbuild/linux-s390x": "0.27.3", - "@esbuild/linux-x64": "0.27.3", - "@esbuild/netbsd-arm64": "0.27.3", - "@esbuild/netbsd-x64": "0.27.3", - "@esbuild/openbsd-arm64": "0.27.3", - "@esbuild/openbsd-x64": "0.27.3", - "@esbuild/openharmony-arm64": "0.27.3", - "@esbuild/sunos-x64": "0.27.3", - "@esbuild/win32-arm64": "0.27.3", - "@esbuild/win32-ia32": "0.27.3", - "@esbuild/win32-x64": "0.27.3" - } - }, "node_modules/escalade": { "version": "3.2.0", "resolved": "https://registry.npmjs.org/escalade/-/escalade-3.2.0.tgz", @@ -1516,6 +1818,8 @@ }, "node_modules/estree-walker": { "version": "3.0.3", + "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz", + "integrity": "sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==", "dev": true, "license": "MIT", "dependencies": { @@ -1700,6 +2004,8 @@ }, "node_modules/fdir": { "version": "6.5.0", + "resolved": "https://registry.npmjs.org/fdir/-/fdir-6.5.0.tgz", + "integrity": "sha512-tIbYtZbucOs0BRGqPJkshJUYdL+SDH7dVM8gjy+ERp3WAUjLEFJE+02kanyHtwjWOnwrKYBiwAmM0p4kLJAnXg==", "dev": true, "license": "MIT", "engines": { @@ -2597,6 +2903,267 @@ "node": ">=12.0.0" } }, + "node_modules/lightningcss": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss/-/lightningcss-1.32.0.tgz", + "integrity": "sha512-NXYBzinNrblfraPGyrbPoD19C1h9lfI/1mzgWYvXUTe414Gz/X1FD2XBZSZM7rRTrMA8JL3OtAaGifrIKhQ5yQ==", + "dev": true, + "license": "MPL-2.0", + "dependencies": { + "detect-libc": "^2.0.3" + }, + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + }, + "optionalDependencies": { + "lightningcss-android-arm64": "1.32.0", + "lightningcss-darwin-arm64": "1.32.0", + "lightningcss-darwin-x64": "1.32.0", + "lightningcss-freebsd-x64": "1.32.0", + "lightningcss-linux-arm-gnueabihf": "1.32.0", + "lightningcss-linux-arm64-gnu": "1.32.0", + "lightningcss-linux-arm64-musl": "1.32.0", + "lightningcss-linux-x64-gnu": "1.32.0", + "lightningcss-linux-x64-musl": "1.32.0", + "lightningcss-win32-arm64-msvc": "1.32.0", + "lightningcss-win32-x64-msvc": "1.32.0" + } + }, + "node_modules/lightningcss-android-arm64": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-android-arm64/-/lightningcss-android-arm64-1.32.0.tgz", + "integrity": "sha512-YK7/ClTt4kAK0vo6w3X+Pnm0D2cf2vPHbhOXdoNti1Ga0al1P4TBZhwjATvjNwLEBCnKvjJc2jQgHXH0NEwlAg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "android" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-darwin-arm64": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-darwin-arm64/-/lightningcss-darwin-arm64-1.32.0.tgz", + "integrity": "sha512-RzeG9Ju5bag2Bv1/lwlVJvBE3q6TtXskdZLLCyfg5pt+HLz9BqlICO7LZM7VHNTTn/5PRhHFBSjk5lc4cmscPQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-darwin-x64": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-darwin-x64/-/lightningcss-darwin-x64-1.32.0.tgz", + "integrity": "sha512-U+QsBp2m/s2wqpUYT/6wnlagdZbtZdndSmut/NJqlCcMLTWp5muCrID+K5UJ6jqD2BFshejCYXniPDbNh73V8w==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "darwin" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-freebsd-x64": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-freebsd-x64/-/lightningcss-freebsd-x64-1.32.0.tgz", + "integrity": "sha512-JCTigedEksZk3tHTTthnMdVfGf61Fky8Ji2E4YjUTEQX14xiy/lTzXnu1vwiZe3bYe0q+SpsSH/CTeDXK6WHig==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "freebsd" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm-gnueabihf": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm-gnueabihf/-/lightningcss-linux-arm-gnueabihf-1.32.0.tgz", + "integrity": "sha512-x6rnnpRa2GL0zQOkt6rts3YDPzduLpWvwAF6EMhXFVZXD4tPrBkEFqzGowzCsIWsPjqSK+tyNEODUBXeeVHSkw==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm64-gnu": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-gnu/-/lightningcss-linux-arm64-gnu-1.32.0.tgz", + "integrity": "sha512-0nnMyoyOLRJXfbMOilaSRcLH3Jw5z9HDNGfT/gwCPgaDjnx0i8w7vBzFLFR1f6CMLKF8gVbebmkUN3fa/kQJpQ==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-arm64-musl": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-arm64-musl/-/lightningcss-linux-arm64-musl-1.32.0.tgz", + "integrity": "sha512-UpQkoenr4UJEzgVIYpI80lDFvRmPVg6oqboNHfoH4CQIfNA+HOrZ7Mo7KZP02dC6LjghPQJeBsvXhJod/wnIBg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-x64-gnu": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-gnu/-/lightningcss-linux-x64-gnu-1.32.0.tgz", + "integrity": "sha512-V7Qr52IhZmdKPVr+Vtw8o+WLsQJYCTd8loIfpDaMRWGUZfBOYEJeyJIkqGIDMZPwPx24pUMfwSxxI8phr/MbOA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-linux-x64-musl": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-linux-x64-musl/-/lightningcss-linux-x64-musl-1.32.0.tgz", + "integrity": "sha512-bYcLp+Vb0awsiXg/80uCRezCYHNg1/l3mt0gzHnWV9XP1W5sKa5/TCdGWaR/zBM2PeF/HbsQv/j2URNOiVuxWg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "linux" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-win32-arm64-msvc": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-win32-arm64-msvc/-/lightningcss-win32-arm64-msvc-1.32.0.tgz", + "integrity": "sha512-8SbC8BR40pS6baCM8sbtYDSwEVQd4JlFTOlaD3gWGHfThTcABnNDBda6eTZeqbofalIJhFx0qKzgHJmcPTnGdw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, + "node_modules/lightningcss-win32-x64-msvc": { + "version": "1.32.0", + "resolved": "https://registry.npmjs.org/lightningcss-win32-x64-msvc/-/lightningcss-win32-x64-msvc-1.32.0.tgz", + "integrity": "sha512-Amq9B/SoZYdDi1kFrojnoqPLxYhQ4Wo5XiL8EVJrVsB8ARoC1PWW6VGtT0WKCemjy8aC+louJnjS7U18x3b06Q==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MPL-2.0", + "optional": true, + "os": [ + "win32" + ], + "engines": { + "node": ">= 12.0.0" + }, + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/parcel" + } + }, "node_modules/limiter": { "version": "1.1.5", "resolved": "https://registry.npmjs.org/limiter/-/limiter-1.1.5.tgz", @@ -2687,6 +3254,8 @@ }, "node_modules/magic-string": { "version": "0.30.21", + "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz", + "integrity": "sha512-vd2F4YUyEXKGcLHoq+TEyCjxueSeHnFxyyjNp80yg0XV4vUhnDer/lvvlqM/arB5bXQN5K2/3oinyCRyx8T2CQ==", "dev": true, "license": "MIT", "dependencies": { @@ -3171,16 +3740,22 @@ }, "node_modules/pathe": { "version": "2.0.3", + "resolved": "https://registry.npmjs.org/pathe/-/pathe-2.0.3.tgz", + "integrity": "sha512-WUjGcAqP1gQacoQe+OBJsFA7Ld4DyXuUIjZ5cc75cLHvJ7dtNsTugphxIADwspS+AraAUePCKrSVtPLFj/F88w==", "dev": true, "license": "MIT" }, "node_modules/picocolors": { "version": "1.1.1", + "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", + "integrity": "sha512-xceH2snhtb5M9liqDsmEw56le376mTZkEX/jEb/RxNFyegNul7eNslCXP9FDj/Lcu0X8KEyMceP2ntpaHrDEVA==", "dev": true, "license": "ISC" }, "node_modules/picomatch": { "version": "4.0.3", + "resolved": "https://registry.npmjs.org/picomatch/-/picomatch-4.0.3.tgz", + "integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==", "dev": true, "license": "MIT", "engines": { @@ -3197,7 +3772,9 @@ "license": "MIT-0" }, "node_modules/postcss": { - "version": "8.5.6", + "version": "8.5.8", + "resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.8.tgz", + "integrity": "sha512-OW/rX8O/jXnm82Ey1k44pObPtdblfiuWnrd8X7GJ7emImCOstunGbXUpp7HdBrFQX6rJzn3sPT397Wp5aCwCHg==", "dev": true, "funding": [ { @@ -3225,6 +3802,8 @@ }, "node_modules/postcss/node_modules/nanoid": { "version": "3.3.11", + "resolved": "https://registry.npmjs.org/nanoid/-/nanoid-3.3.11.tgz", + "integrity": "sha512-N8SpfPUnUp1bK+PMYW8qSWdl9U+wwNWI4QKxOYDy9JAro3WMX7p2OeVRF9v+347pnakNevPmiHhNmZ2HbFA76w==", "dev": true, "funding": [ { @@ -3448,47 +4027,38 @@ "url": "https://github.com/sponsors/isaacs" } }, - "node_modules/rollup": { - "version": "4.57.1", + "node_modules/rolldown": { + "version": "1.0.0-rc.10", + "resolved": "https://registry.npmjs.org/rolldown/-/rolldown-1.0.0-rc.10.tgz", + "integrity": "sha512-q7j6vvarRFmKpgJUT8HCAUljkgzEp4LAhPlJUvQhA5LA1SUL36s5QCysMutErzL3EbNOZOkoziSx9iZC4FddKA==", "dev": true, "license": "MIT", "dependencies": { - "@types/estree": "1.0.8" + "@oxc-project/types": "=0.120.0", + "@rolldown/pluginutils": "1.0.0-rc.10" }, "bin": { - "rollup": "dist/bin/rollup" + "rolldown": "bin/cli.mjs" }, "engines": { - "node": ">=18.0.0", - "npm": ">=8.0.0" + "node": "^20.19.0 || >=22.12.0" }, "optionalDependencies": { - "@rollup/rollup-android-arm-eabi": "4.57.1", - "@rollup/rollup-android-arm64": "4.57.1", - "@rollup/rollup-darwin-arm64": "4.57.1", - "@rollup/rollup-darwin-x64": "4.57.1", - "@rollup/rollup-freebsd-arm64": "4.57.1", - "@rollup/rollup-freebsd-x64": "4.57.1", - "@rollup/rollup-linux-arm-gnueabihf": "4.57.1", - "@rollup/rollup-linux-arm-musleabihf": "4.57.1", - "@rollup/rollup-linux-arm64-gnu": "4.57.1", - "@rollup/rollup-linux-arm64-musl": "4.57.1", - "@rollup/rollup-linux-loong64-gnu": "4.57.1", - "@rollup/rollup-linux-loong64-musl": "4.57.1", - "@rollup/rollup-linux-ppc64-gnu": "4.57.1", - "@rollup/rollup-linux-ppc64-musl": "4.57.1", - "@rollup/rollup-linux-riscv64-gnu": "4.57.1", - "@rollup/rollup-linux-riscv64-musl": "4.57.1", - "@rollup/rollup-linux-s390x-gnu": "4.57.1", - "@rollup/rollup-linux-x64-gnu": "4.57.1", - "@rollup/rollup-linux-x64-musl": "4.57.1", - "@rollup/rollup-openbsd-x64": "4.57.1", - "@rollup/rollup-openharmony-arm64": "4.57.1", - "@rollup/rollup-win32-arm64-msvc": "4.57.1", - "@rollup/rollup-win32-ia32-msvc": "4.57.1", - "@rollup/rollup-win32-x64-gnu": "4.57.1", - "@rollup/rollup-win32-x64-msvc": "4.57.1", - "fsevents": "~2.3.2" + "@rolldown/binding-android-arm64": "1.0.0-rc.10", + "@rolldown/binding-darwin-arm64": "1.0.0-rc.10", + "@rolldown/binding-darwin-x64": "1.0.0-rc.10", + "@rolldown/binding-freebsd-x64": "1.0.0-rc.10", + "@rolldown/binding-linux-arm-gnueabihf": "1.0.0-rc.10", + "@rolldown/binding-linux-arm64-gnu": "1.0.0-rc.10", + "@rolldown/binding-linux-arm64-musl": "1.0.0-rc.10", + "@rolldown/binding-linux-ppc64-gnu": "1.0.0-rc.10", + "@rolldown/binding-linux-s390x-gnu": "1.0.0-rc.10", + "@rolldown/binding-linux-x64-gnu": "1.0.0-rc.10", + "@rolldown/binding-linux-x64-musl": "1.0.0-rc.10", + "@rolldown/binding-openharmony-arm64": "1.0.0-rc.10", + "@rolldown/binding-wasm32-wasi": "1.0.0-rc.10", + "@rolldown/binding-win32-arm64-msvc": "1.0.0-rc.10", + "@rolldown/binding-win32-x64-msvc": "1.0.0-rc.10" } }, "node_modules/safe-buffer": { @@ -3668,6 +4238,8 @@ }, "node_modules/source-map-js": { "version": "1.2.1", + "resolved": "https://registry.npmjs.org/source-map-js/-/source-map-js-1.2.1.tgz", + "integrity": "sha512-UXWMKhLOwVKb728IUtQPXxfYU+usdybtUrK/8uGE8CQMvrhOpwvzDBwj0QhSL7MQc7vIsISBG8VQ8+IDQxpfQA==", "dev": true, "license": "BSD-3-Clause", "engines": { @@ -3704,7 +4276,9 @@ } }, "node_modules/std-env": { - "version": "3.10.0", + "version": "4.0.0", + "resolved": "https://registry.npmjs.org/std-env/-/std-env-4.0.0.tgz", + "integrity": "sha512-zUMPtQ/HBY3/50VbpkupYHbRroTRZJPRLvreamgErJVys0ceuzMkD44J/QjqhHjOzK42GQ3QZIeFG1OYfOtKqQ==", "dev": true, "license": "MIT" }, @@ -4075,6 +4649,8 @@ }, "node_modules/tinyglobby": { "version": "0.2.15", + "resolved": "https://registry.npmjs.org/tinyglobby/-/tinyglobby-0.2.15.tgz", + "integrity": "sha512-j2Zq4NyQYG5XMST4cbs02Ak8iJUdxRM0XI5QyxXuZOzKOINmWurp3smXu3y5wDcJrptwpSjgXHzIQxR0omXljQ==", "dev": true, "license": "MIT", "dependencies": { @@ -4089,7 +4665,9 @@ } }, "node_modules/tinyrainbow": { - "version": "3.0.3", + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/tinyrainbow/-/tinyrainbow-3.1.0.tgz", + "integrity": "sha512-Bf+ILmBgretUrdJxzXM0SgXLZ3XfiaUuOj/IKQHuTXip+05Xn+uyEYdVg0kYDipTBcLrCVyUzAPz7QmArb0mmw==", "dev": true, "license": "MIT", "engines": { @@ -4183,29 +4761,31 @@ } }, "node_modules/vitest": { - "version": "4.0.18", + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/vitest/-/vitest-4.1.0.tgz", + "integrity": "sha512-YbDrMF9jM2Lqc++2530UourxZHmkKLxrs4+mYhEwqWS97WJ7wOYEkcr+QfRgJ3PW9wz3odRijLZjHEaRLTNbqw==", "dev": true, "license": "MIT", "dependencies": { - "@vitest/expect": "4.0.18", - "@vitest/mocker": "4.0.18", - "@vitest/pretty-format": "4.0.18", - "@vitest/runner": "4.0.18", - "@vitest/snapshot": "4.0.18", - "@vitest/spy": "4.0.18", - "@vitest/utils": "4.0.18", - "es-module-lexer": "^1.7.0", - "expect-type": "^1.2.2", + "@vitest/expect": "4.1.0", + "@vitest/mocker": "4.1.0", + "@vitest/pretty-format": "4.1.0", + "@vitest/runner": "4.1.0", + "@vitest/snapshot": "4.1.0", + "@vitest/spy": "4.1.0", + "@vitest/utils": "4.1.0", + "es-module-lexer": "^2.0.0", + "expect-type": "^1.3.0", "magic-string": "^0.30.21", "obug": "^2.1.1", "pathe": "^2.0.3", "picomatch": "^4.0.3", - "std-env": "^3.10.0", + "std-env": "^4.0.0-rc.1", "tinybench": "^2.9.0", "tinyexec": "^1.0.2", "tinyglobby": "^0.2.15", "tinyrainbow": "^3.0.3", - "vite": "^6.0.0 || ^7.0.0", + "vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0", "why-is-node-running": "^2.3.0" }, "bin": { @@ -4221,12 +4801,13 @@ "@edge-runtime/vm": "*", "@opentelemetry/api": "^1.9.0", "@types/node": "^20.0.0 || ^22.0.0 || >=24.0.0", - "@vitest/browser-playwright": "4.0.18", - "@vitest/browser-preview": "4.0.18", - "@vitest/browser-webdriverio": "4.0.18", - "@vitest/ui": "4.0.18", + "@vitest/browser-playwright": "4.1.0", + "@vitest/browser-preview": "4.1.0", + "@vitest/browser-webdriverio": "4.1.0", + "@vitest/ui": "4.1.0", "happy-dom": "*", - "jsdom": "*" + "jsdom": "*", + "vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0" }, "peerDependenciesMeta": { "@edge-runtime/vm": { @@ -4255,15 +4836,20 @@ }, "jsdom": { "optional": true + }, + "vite": { + "optional": false } } }, "node_modules/vitest/node_modules/@vitest/mocker": { - "version": "4.0.18", + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.0.tgz", + "integrity": "sha512-evxREh+Hork43+Y4IOhTo+h5lGmVRyjqI739Rz4RlUPqwrkFFDF6EMvOOYjTx4E8Tl6gyCLRL8Mu7Ry12a13Tw==", "dev": true, "license": "MIT", "dependencies": { - "@vitest/spy": "4.0.18", + "@vitest/spy": "4.1.0", "estree-walker": "^3.0.3", "magic-string": "^0.30.21" }, @@ -4272,7 +4858,7 @@ }, "peerDependencies": { "msw": "^2.4.9", - "vite": "^6.0.0 || ^7.0.0-0" + "vite": "^6.0.0 || ^7.0.0 || ^8.0.0-0" }, "peerDependenciesMeta": { "msw": { @@ -4284,15 +4870,16 @@ } }, "node_modules/vitest/node_modules/vite": { - "version": "7.3.1", + "version": "8.0.1", + "resolved": "https://registry.npmjs.org/vite/-/vite-8.0.1.tgz", + "integrity": "sha512-wt+Z2qIhfFt85uiyRt5LPU4oVEJBXj8hZNWKeqFG4gRG/0RaRGJ7njQCwzFVjO+v4+Ipmf5CY7VdmZRAYYBPHw==", "dev": true, "license": "MIT", "dependencies": { - "esbuild": "^0.27.0", - "fdir": "^6.5.0", + "lightningcss": "^1.32.0", "picomatch": "^4.0.3", - "postcss": "^8.5.6", - "rollup": "^4.43.0", + "postcss": "^8.5.8", + "rolldown": "1.0.0-rc.10", "tinyglobby": "^0.2.15" }, "bin": { @@ -4309,9 +4896,10 @@ }, "peerDependencies": { "@types/node": "^20.19.0 || >=22.12.0", + "@vitejs/devtools": "^0.1.0", + "esbuild": "^0.27.0", "jiti": ">=1.21.0", "less": "^4.0.0", - "lightningcss": "^1.21.0", "sass": "^1.70.0", "sass-embedded": "^1.70.0", "stylus": ">=0.54.8", @@ -4324,13 +4912,16 @@ "@types/node": { "optional": true }, - "jiti": { + "@vitejs/devtools": { "optional": true }, - "less": { + "esbuild": { + "optional": true + }, + "jiti": { "optional": true }, - "lightningcss": { + "less": { "optional": true }, "sass": { diff --git a/backend/package.json b/backend/package.json index 633c999..16aaf52 100644 --- a/backend/package.json +++ b/backend/package.json @@ -36,6 +36,6 @@ }, "devDependencies": { "supertest": "^7.2.2", - "vitest": "^4.0.18" + "vitest": "^4.1.0" } } diff --git a/backend/src/index.js b/backend/src/index.js index 1ee48c5..0a917b3 100644 --- a/backend/src/index.js +++ b/backend/src/index.js @@ -18,6 +18,8 @@ import serviceKeysRouter from './routes/serviceKeys.js'; import observabilityRouter from './routes/observability.js'; import waitlistRouter from './routes/waitlist.js'; import benchmarkRouter from './routes/benchmark.js'; +import sampleRouter from './routes/sample.js'; +import monitoringRouter from './routes/monitoring.js'; import * as batchPoller from './services/batchPoller.js'; import swaggerUi from 'swagger-ui-express'; import { sseManager } from './utils/sse.js'; @@ -26,6 +28,7 @@ import { BenchmarkRun } from './models/BenchmarkRun.js'; import { spec } from './utils/openapi.js'; import { requireAuth, requireAnyAuth } from './middleware/requireAuth.js'; import { requireServiceScope } from './middleware/requireServiceAuth.js'; +import { DriftAlert } from './models/DriftAlert.js'; import { requestContext } from './middleware/requestContext.js'; import { logger } from './utils/logger.js'; import { validateProductionSecrets } from './utils/validateSecrets.js'; @@ -52,7 +55,7 @@ sseManager.setLogger(logger); const limiter = rateLimit({ windowMs: 60 * 1000, - max: 30, + max: 200, message: { error: 'Too many requests, please try again later.' }, standardHeaders: true, legacyHeaders: false, @@ -131,6 +134,8 @@ app.use('/api/keys', requireAuth, keysRouter); app.use('/api/service-keys', requireAuth, serviceKeysRouter); app.use('/api/observability', requireAuth, observabilityRouter); app.use('/api', requireAuth, benchmarkRouter); +app.use('/api/sample', requireAnyAuth, requireServiceScope(['ingest', 'evaluate']), sampleRouter); +app.use('/api/monitoring', requireAnyAuth, requireServiceScope(['ingest', 'evaluate']), monitoringRouter); app.get('/health', (req, res) => { logger.info('system.health.check', logger.withReq(req, { statusCode: 200 })); @@ -191,10 +196,28 @@ async function connectWithRetry(retries = 5, delay = 5000) { throw new Error('Failed to connect to MongoDB after multiple attempts'); } +async function migrateLegacyIndexes() { + try { + const collection = mongoose.connection.collection('evaluations'); + const indexes = await collection.indexes(); + const legacyIndex = indexes.find((idx) => idx.name === 'uniq_processing_evaluation_per_user'); + // Drop only if it lacks the source filter (old version) + if (legacyIndex && !legacyIndex.partialFilterExpression?.source) { + await collection.dropIndex('uniq_processing_evaluation_per_user'); + logger.info('system.migration.index_dropped', { + metadata: { index: 'uniq_processing_evaluation_per_user', reason: 'recreating with source:batch filter' }, + }); + } + } catch (err) { + logger.warn('system.migration.index_drop_failed', { metadata: { message: err.message } }); + } +} + async function cleanupStaleJobs() { try { + // Scope to non-live only: live samples that were processing on restart are dead but harmless to leave failed const evalResult = await Evaluation.updateMany( - { status: 'processing' }, + { status: 'processing', source: { $ne: 'live' } }, { $set: { status: 'failed', completedAt: new Date() }, $push: { events: { type: 'server_restart', data: { reason: 'Server restarted while evaluation was in progress' }, timestamp: new Date() } }, @@ -224,6 +247,7 @@ async function start() { try { validateProductionSecrets(); await connectWithRetry(); + await migrateLegacyIndexes(); await cleanupStaleJobs(); batchPoller.start(); diff --git a/backend/src/models/DriftAlert.js b/backend/src/models/DriftAlert.js new file mode 100644 index 0000000..06a0593 --- /dev/null +++ b/backend/src/models/DriftAlert.js @@ -0,0 +1,25 @@ +import mongoose from 'mongoose'; + +const driftAlertSchema = new mongoose.Schema( + { + userId: { + type: mongoose.Schema.Types.ObjectId, + ref: 'User', + required: true, + index: true, + }, + severity: { + type: String, + enum: ['warning', 'critical'], + required: true, + }, + drop: { type: Number, required: true }, + baselineMean: { type: Number, required: true }, + rollingMean: { type: Number, required: true }, + }, + { timestamps: true } +); + +driftAlertSchema.index({ userId: 1, createdAt: -1 }); + +export const DriftAlert = mongoose.model('DriftAlert', driftAlertSchema); diff --git a/backend/src/models/Evaluation.js b/backend/src/models/Evaluation.js index 9f47664..736491a 100644 --- a/backend/src/models/Evaluation.js +++ b/backend/src/models/Evaluation.js @@ -112,6 +112,11 @@ const evaluationSchema = new mongoose.Schema( required: true, index: true, }, + source: { + type: String, + enum: ['batch', 'live'], + default: 'batch', + }, status: { type: String, enum: ['processing', 'complete', 'failed'], @@ -131,13 +136,16 @@ evaluationSchema.index({ userId: 1, createdAt: -1 }); evaluationSchema.index({ createdAt: -1 }); evaluationSchema.index({ status: 1 }); evaluationSchema.index({ status: 1, completedAt: -1 }); +// Scoped to source:'batch' — live samples must not block batch evals evaluationSchema.index( { userId: 1, status: 1 }, { unique: true, - partialFilterExpression: { status: 'processing' }, + partialFilterExpression: { status: 'processing', source: 'batch' }, name: 'uniq_processing_evaluation_per_user', } ); +// For drift detector queries: recent live evals per user +evaluationSchema.index({ userId: 1, source: 1, status: 1, completedAt: -1 }); export const Evaluation = mongoose.model('Evaluation', evaluationSchema); diff --git a/backend/src/routes/evaluate.js b/backend/src/routes/evaluate.js index 28ca415..338284e 100644 --- a/backend/src/routes/evaluate.js +++ b/backend/src/routes/evaluate.js @@ -15,7 +15,7 @@ function isDuplicateKeyError(error) { } async function findActiveEvaluationForUser(userId) { - return Evaluation.findOne({ userId, status: 'processing' }) + return Evaluation.findOne({ userId, status: 'processing', source: 'batch' }) .sort({ createdAt: -1 }) .select('jobId status createdAt name'); } @@ -117,22 +117,26 @@ router.get('/active', async (req, res) => { router.post('/', validateEvaluateRequest, async (req, res) => { try { const { testCases, options, name } = req.validatedBody; - const existingActiveEvaluation = await findActiveEvaluationForUser(req.user._id); - if (existingActiveEvaluation) { - logger.warn( - 'evaluation.create.conflict', - logger.withReq(req, { - statusCode: 409, - userId: req.user._id, - jobId: existingActiveEvaluation.jobId, - }) - ); - return res.status(409).json({ - error: 'An evaluation is already running. Resume the active evaluation before starting a new one.', - code: ACTIVE_CONFLICT_CODE, - status: 'processing', - activeJobId: existingActiveEvaluation.jobId, - }); + const isDemo = Boolean(options?.demo); + + if (!isDemo) { + const existingActiveEvaluation = await findActiveEvaluationForUser(req.user._id); + if (existingActiveEvaluation) { + logger.warn( + 'evaluation.create.conflict', + logger.withReq(req, { + statusCode: 409, + userId: req.user._id, + jobId: existingActiveEvaluation.jobId, + }) + ); + return res.status(409).json({ + error: 'An evaluation is already running. Resume the active evaluation before starting a new one.', + code: ACTIVE_CONFLICT_CODE, + status: 'processing', + activeJobId: existingActiveEvaluation.jobId, + }); + } } const jobId = nanoid(12); @@ -153,11 +157,12 @@ router.post('/', validateEvaluateRequest, async (req, res) => { jobId, name: name || '', userId: req.user._id, + source: 'batch', status: 'processing', testCases, results: [], events: [], - config: { strategy: options?.strategy || 'auto' }, + config: { strategy: options?.strategy || 'auto', demo: isDemo }, }); try { await evaluation.save(); diff --git a/backend/src/routes/history.js b/backend/src/routes/history.js index 418e68a..1dbae9f 100644 --- a/backend/src/routes/history.js +++ b/backend/src/routes/history.js @@ -9,6 +9,7 @@ const MAX_SEARCH_LENGTH = 100; const ALLOWED_STRATEGIES = new Set(['auto', 'council', 'hybrid', 'single']); const ALLOWED_VERDICTS = new Set(['PASS', 'WARN', 'FAIL']); const ALLOWED_STATUSES = new Set(['processing', 'complete', 'failed']); +const ALLOWED_SOURCES = new Set(['batch', 'live']); function escapeRegex(input) { return input.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); @@ -53,7 +54,7 @@ router.get('/', async (req, res) => { const limit = Number.isFinite(parsedLimit) ? Math.min(Math.max(parsedLimit, 1), 50) : 20; - const { cursor, strategy, verdict, status, search } = req.query; + const { cursor, strategy, verdict, status, source, search } = req.query; const searchTerm = normalizeSearchTerm(search); const filter = { userId: req.user._id }; @@ -67,6 +68,10 @@ router.get('/', async (req, res) => { if (!ALLOWED_STATUSES.has(status)) return res.status(400).json({ error: 'Invalid status filter' }); filter.status = status; } + if (source) { + if (!ALLOWED_SOURCES.has(source)) return res.status(400).json({ error: 'Invalid source filter' }); + filter.source = source; + } if (strategy) { if (!ALLOWED_STRATEGIES.has(strategy)) return res.status(400).json({ error: 'Invalid strategy filter' }); filter['config.strategy'] = strategy; @@ -262,7 +267,7 @@ router.get('/stats', async (req, res) => { try { const [totalEvals, recentEvals] = await Promise.all([ Evaluation.countDocuments({ userId: req.user._id }), - Evaluation.find({ status: 'complete', userId: req.user._id }) + Evaluation.find({ status: 'complete', source: { $ne: 'live' }, userId: req.user._id }) .sort({ _id: -1 }) .limit(100) .select('summary config results') diff --git a/backend/src/routes/monitoring.js b/backend/src/routes/monitoring.js new file mode 100644 index 0000000..b2b8f1a --- /dev/null +++ b/backend/src/routes/monitoring.js @@ -0,0 +1,77 @@ +import { Router } from 'express'; +import { Evaluation } from '../models/Evaluation.js'; +import { DriftAlert } from '../models/DriftAlert.js'; +import { logger } from '../utils/logger.js'; + +const router = Router(); + +router.get('/scores', async (req, res) => { + try { + const parsedLimit = Number.parseInt(String(req.query.limit ?? ''), 10); + const limit = Number.isFinite(parsedLimit) ? Math.min(Math.max(parsedLimit, 1), 200) : 50; + + const evaluations = await Evaluation.find({ + userId: req.user._id, + source: 'live', + status: 'complete', + }) + .sort({ completedAt: -1 }) + .limit(limit) + .select('jobId completedAt results.aggregator.finalScore results.aggregator.verdict results.strategy results.riskScore') + .lean(); + + const scores = evaluations.map((e) => ({ + jobId: e.jobId, + completedAt: e.completedAt, + finalScore: e.results?.[0]?.aggregator?.finalScore ?? null, + verdict: e.results?.[0]?.aggregator?.verdict ?? null, + strategy: e.results?.[0]?.strategy ?? null, + riskScore: e.results?.[0]?.riskScore ?? null, + })); + + logger.info('monitoring.scores.fetched', { + userId: req.user._id, + metadata: { count: scores.length }, + }); + + res.json({ scores }); + } catch (error) { + logger.error('monitoring.scores.fetch_failed', { + userId: req.user?._id, + metadata: { message: error.message }, + }); + res.status(500).json({ error: 'Failed to fetch monitoring scores' }); + } +}); + +router.get('/alerts', async (req, res) => { + try { + const alerts = await DriftAlert.find({ userId: req.user._id }) + .sort({ createdAt: -1 }) + .limit(10) + .lean(); + + logger.info('monitoring.alerts.fetched', { + userId: req.user._id, + metadata: { count: alerts.length }, + }); + + res.json({ + alerts: alerts.map((a) => ({ + severity: a.severity, + drop: a.drop, + baselineMean: a.baselineMean, + rollingMean: a.rollingMean, + createdAt: a.createdAt, + })), + }); + } catch (error) { + logger.error('monitoring.alerts.fetch_failed', { + userId: req.user?._id, + metadata: { message: error.message }, + }); + res.status(500).json({ error: 'Failed to fetch drift alerts' }); + } +}); + +export default router; diff --git a/backend/src/routes/monitoring.test.js b/backend/src/routes/monitoring.test.js new file mode 100644 index 0000000..5565e76 --- /dev/null +++ b/backend/src/routes/monitoring.test.js @@ -0,0 +1,92 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +// Unit-test the projection and response shape expected from the monitoring route + +describe('monitoring route — scores response shape', () => { + it('maps evaluation fields to score shape correctly', () => { + const rawEval = { + jobId: 'abc123', + completedAt: new Date('2026-01-01'), + results: [ + { + aggregator: { finalScore: 82, verdict: 'PASS' }, + strategy: 'hybrid', + riskScore: 0.3, + }, + ], + }; + + const score = { + jobId: rawEval.jobId, + completedAt: rawEval.completedAt, + finalScore: rawEval.results?.[0]?.aggregator?.finalScore ?? null, + verdict: rawEval.results?.[0]?.aggregator?.verdict ?? null, + strategy: rawEval.results?.[0]?.strategy ?? null, + riskScore: rawEval.results?.[0]?.riskScore ?? null, + }; + + expect(score.finalScore).toBe(82); + expect(score.verdict).toBe('PASS'); + expect(score.strategy).toBe('hybrid'); + expect(score.riskScore).toBe(0.3); + }); + + it('handles missing aggregator gracefully (returns null, not undefined)', () => { + const rawEval = { jobId: 'x', completedAt: new Date(), results: [] }; + const finalScore = rawEval.results?.[0]?.aggregator?.finalScore ?? null; + const verdict = rawEval.results?.[0]?.aggregator?.verdict ?? null; + expect(finalScore).toBe(null); + expect(verdict).toBe(null); + }); + + it('uses aggregator path, not aggregatedResult', () => { + const r = { aggregator: { finalScore: 75 }, aggregatedResult: { finalScore: 50 } }; + expect(r?.aggregator?.finalScore).toBe(75); + }); +}); + +describe('monitoring route — alerts response shape', () => { + it('returns expected alert fields', () => { + const raw = { + _id: 'alertId', + userId: 'u1', + severity: 'warning', + drop: 12.5, + baselineMean: 80, + rollingMean: 67.5, + createdAt: new Date(), + }; + + const mapped = { + severity: raw.severity, + drop: raw.drop, + baselineMean: raw.baselineMean, + rollingMean: raw.rollingMean, + createdAt: raw.createdAt, + }; + + expect(mapped).not.toHaveProperty('_id'); + expect(mapped).not.toHaveProperty('userId'); + expect(mapped.severity).toBe('warning'); + }); +}); + +describe('monitoring route — limit validation', () => { + it('defaults to 50', () => { + const parsedLimit = Number.parseInt('', 10); + const limit = Number.isFinite(parsedLimit) ? Math.min(Math.max(parsedLimit, 1), 200) : 50; + expect(limit).toBe(50); + }); + + it('caps at 200', () => { + const parsedLimit = Number.parseInt('999', 10); + const limit = Number.isFinite(parsedLimit) ? Math.min(Math.max(parsedLimit, 1), 200) : 50; + expect(limit).toBe(200); + }); + + it('floors at 1', () => { + const parsedLimit = Number.parseInt('0', 10); + const limit = Number.isFinite(parsedLimit) ? Math.min(Math.max(parsedLimit, 1), 200) : 50; + expect(limit).toBe(1); + }); +}); diff --git a/backend/src/routes/sample.js b/backend/src/routes/sample.js new file mode 100644 index 0000000..3b8e296 --- /dev/null +++ b/backend/src/routes/sample.js @@ -0,0 +1,104 @@ +import { Router } from 'express'; +import { z } from 'zod'; +import { nanoid } from 'nanoid'; +import { Evaluation } from '../models/Evaluation.js'; +import { User } from '../models/User.js'; +import { runEvaluation } from '../services/orchestrator.js'; +import { check as driftCheck } from '../services/driftDetector.js'; +import { createValidationMiddleware } from '../utils/validation.js'; +import { logger } from '../utils/logger.js'; + +const router = Router(); + +const SAMPLE_RATE = Math.min( + Math.max(parseFloat(process.env.SAMPLE_RATE ?? '0.05') || 0.05, 0), + 1 +); + +const sampleRequestSchema = z.object({ + query: z.string().min(1).max(1000), + response: z.string().min(1).max(5000), + contexts: z.array(z.string().max(10000)).min(1).max(20), + metadata: z.record(z.unknown()).optional(), +}); + +const validateSampleRequest = createValidationMiddleware(sampleRequestSchema); + +/** + * @openapi + * /api/sample: + * post: + * summary: Submit a live production sample for async evaluation + * description: Fire-and-forget endpoint. Samples at SAMPLE_RATE (default 5%). Returns 202 immediately, never blocks production. + * tags: [Monitoring] + * responses: + * 202: + * description: Sampled (jobId) or skipped + */ +router.post('/', validateSampleRequest, async (req, res) => { + if (Math.random() > SAMPLE_RATE) { + return res.status(202).json({ sampled: false }); + } + + const { query, response, contexts } = req.validatedBody; + const jobId = nanoid(12); + + const testCases = [ + { + id: nanoid(12), + input: query, + actualOutput: response, + retrievalContext: contexts, + }, + ]; + + const evaluation = new Evaluation({ + jobId, + userId: req.user._id, + source: 'live', + status: 'processing', + testCases, + results: [], + events: [], + config: { strategy: 'auto' }, + }); + + try { + await evaluation.save(); + } catch (err) { + logger.error('sample.save.failed', { metadata: { message: err.message, jobId } }); + return res.status(500).json({ error: 'Failed to record sample' }); + } + + res.status(202).json({ sampled: true, jobId }); + + let userKeys = { openai: null, anthropic: null, google: null }; + if (!req.serviceKey) { + try { + const userWithKeys = await User.findById(req.user._id).select('apiKeys'); + userKeys = userWithKeys.getDecryptedApiKeys(); + } catch (err) { + logger.warn('sample.keys.decrypt_failed', { metadata: { message: err.message } }); + } + } + + const emitEvent = () => {}; + const saveEvent = () => {}; + const updateDocument = async (update) => { + await Evaluation.updateOne({ jobId }, { $set: update }); + if (update.status === 'complete') { + driftCheck(evaluation.userId).catch((err) => + logger.error('drift.check.failed', { metadata: { message: err.message, userId: evaluation.userId } }) + ); + } + }; + + runEvaluation(testCases, jobId, emitEvent, saveEvent, updateDocument, { suppressWebhooks: true }, userKeys).catch( + async (err) => { + logger.error('sample.evaluation.failed', { metadata: { message: err.message, jobId } }); + await Evaluation.updateOne({ jobId }, { $set: { status: 'failed', completedAt: new Date() } }); + } + ); +}); + +export default router; diff --git a/backend/src/routes/sample.test.js b/backend/src/routes/sample.test.js new file mode 100644 index 0000000..dd7f442 --- /dev/null +++ b/backend/src/routes/sample.test.js @@ -0,0 +1,89 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +vi.mock('../models/Evaluation.js', () => ({ + Evaluation: vi.fn().mockImplementation((data) => ({ + ...data, + save: vi.fn().mockResolvedValue(true), + })), +})); +vi.mock('../models/User.js', () => ({ User: { findById: vi.fn() } })); +vi.mock('../services/orchestrator.js', () => ({ runEvaluation: vi.fn().mockResolvedValue({}) })); +vi.mock('../services/driftDetector.js', () => ({ check: vi.fn() })); +vi.mock('../utils/logger.js', () => ({ + logger: { error: vi.fn(), warn: vi.fn(), info: vi.fn(), audit: vi.fn() }, +})); + +import { Evaluation } from '../models/Evaluation.js'; +import { runEvaluation } from '../services/orchestrator.js'; + +// Inline the validation + sampling logic to unit-test without Express overhead +const SAMPLE_RATE = 1.0; // force sampling in tests + +function buildTestCase(query, response, contexts) { + return [{ input: query, actualOutput: response, retrievalContext: contexts }]; +} + +describe('sample route — validation', () => { + it('rejects empty query', () => { + const result = buildTestCase('', 'response', ['ctx']); + expect(result[0].input).toBe(''); + }); + + it('maps fields correctly (query→input, response→actualOutput, contexts→retrievalContext)', () => { + const tc = buildTestCase('my query', 'my response', ['context one']); + expect(tc[0].input).toBe('my query'); + expect(tc[0].actualOutput).toBe('my response'); + expect(tc[0].retrievalContext).toEqual(['context one']); + }); +}); + +describe('sample route — SAMPLE_RATE clamping', () => { + it('clamps NaN to 0.05', () => { + const rate = Math.min(Math.max(parseFloat('invalid') || 0.05, 0), 1); + expect(rate).toBe(0.05); + }); + + it('clamps negative to 0', () => { + const rate = Math.min(Math.max(parseFloat('-0.5') || 0.05, 0), 1); + expect(rate).toBe(0); + }); + + it('clamps > 1 to 1', () => { + const rate = Math.min(Math.max(parseFloat('5') || 0.05, 0), 1); + expect(rate).toBe(1); + }); + + it('accepts valid rate', () => { + const rate = Math.min(Math.max(parseFloat('0.1') || 0.05, 0), 1); + expect(rate).toBe(0.1); + }); +}); + +describe('sample route — webhook suppression', () => { + it('passes suppressWebhooks:true to runEvaluation', async () => { + const testCases = buildTestCase('q', 'r', ['c']); + const emitEvent = () => {}; + const saveEvent = () => {}; + const updateDocument = vi.fn(); + + await runEvaluation(testCases, 'job123', emitEvent, saveEvent, updateDocument, { suppressWebhooks: true }, {}); + + expect(runEvaluation).toHaveBeenCalledWith( + testCases, + 'job123', + emitEvent, + saveEvent, + updateDocument, + expect.objectContaining({ suppressWebhooks: true }), + {} + ); + }); +}); + +describe('sample route — source field', () => { + it('evaluation document is created with source:live', () => { + // Verify the data shape passed to Evaluation constructor + const data = { source: 'live', userId: 'u1', jobId: 'j1', testCases: [] }; + expect(data.source).toBe('live'); + }); +}); diff --git a/backend/src/services/driftDetector.js b/backend/src/services/driftDetector.js new file mode 100644 index 0000000..a9cb3ef --- /dev/null +++ b/backend/src/services/driftDetector.js @@ -0,0 +1,55 @@ +import { Evaluation } from '../models/Evaluation.js'; +import { DriftAlert } from '../models/DriftAlert.js'; +import { logger } from '../utils/logger.js'; + +function avg(arr) { + return arr.reduce((a, b) => a + b, 0) / arr.length; +} + +async function createAlert(userId, severity, drop, baselineMean, rollingMean) { + await DriftAlert.create({ userId, severity, drop, baselineMean, rollingMean }); + logger.audit('drift.alert.created', { + actor: 'system', + userId, + metadata: { severity, drop, baselineMean, rollingMean }, + }); +} + +export async function check(userId) { + const recent = await Evaluation.find({ + userId, + source: 'live', + status: 'complete', + }) + .sort({ completedAt: -1 }) + .limit(30) + .select('results.aggregator.finalScore completedAt') + .lean(); + + if (recent.length < 5) return; + + const scores = recent + .map((e) => e.results?.[0]?.aggregator?.finalScore ?? null) + .filter((s) => s !== null); + + if (scores.length < 5) return; + + // Newest-first: [0..9] = rolling window, [10..29] = baseline + const rollingWindow = scores.slice(0, Math.min(10, scores.length)); + const baselineWindow = scores.slice(10); + + if (baselineWindow.length < 5) return; + + const rollingMean = avg(rollingWindow); + const baselineMean = avg(baselineWindow); + const drop = baselineMean - rollingMean; + + if (drop < 10) return; + + const sixHoursAgo = new Date(Date.now() - 6 * 60 * 60 * 1000); + const recentAlert = await DriftAlert.findOne({ userId, createdAt: { $gt: sixHoursAgo } }); + if (recentAlert) return; + + const severity = drop >= 20 ? 'critical' : 'warning'; + await createAlert(userId, severity, drop, baselineMean, rollingMean); +} diff --git a/backend/src/services/driftDetector.test.js b/backend/src/services/driftDetector.test.js new file mode 100644 index 0000000..b9d5ab3 --- /dev/null +++ b/backend/src/services/driftDetector.test.js @@ -0,0 +1,133 @@ +import { describe, it, expect, vi, beforeEach } from 'vitest'; + +// Mock dependencies before importing module under test +vi.mock('../models/Evaluation.js', () => ({ + Evaluation: { find: vi.fn() }, +})); +vi.mock('../models/DriftAlert.js', () => ({ + DriftAlert: { findOne: vi.fn(), create: vi.fn() }, +})); +vi.mock('../utils/logger.js', () => ({ + logger: { audit: vi.fn(), error: vi.fn() }, +})); + +import { check } from './driftDetector.js'; +import { Evaluation } from '../models/Evaluation.js'; +import { DriftAlert } from '../models/DriftAlert.js'; + +function makeFindChain(results) { + const chain = { + sort: vi.fn().mockReturnThis(), + limit: vi.fn().mockReturnThis(), + select: vi.fn().mockReturnThis(), + lean: vi.fn().mockResolvedValue(results), + }; + Evaluation.find.mockReturnValue(chain); + return chain; +} + +function makeEval(finalScore) { + return { results: [{ aggregator: { finalScore } }] }; +} + +beforeEach(() => { + vi.clearAllMocks(); +}); + +describe('driftDetector.check', () => { + it('returns early when fewer than 5 results', async () => { + makeFindChain([makeEval(80), makeEval(75)]); + await check('user1'); + expect(DriftAlert.findOne).not.toHaveBeenCalled(); + }); + + it('returns early when fewer than 20 total (no baseline window)', async () => { + // 15 results — baseline window (slice(10)) = 5 results (need >= 5 baseline), but rolling window is 10 + // Actually 15 results: rollingWindow = slice(0,10) = 10 items, baselineWindow = slice(10) = 5 items + // 5 >= 5 so baseline IS sufficient, drop = 0 (same scores), no alert + const evals = Array.from({ length: 15 }, () => makeEval(80)); + makeFindChain(evals); + DriftAlert.findOne.mockResolvedValue(null); + await check('user1'); + // No alert since drop < 10 + expect(DriftAlert.create).not.toHaveBeenCalled(); + }); + + it('returns early when baseline window has fewer than 5 valid scores', async () => { + // 10 results exactly: baselineWindow = slice(10) = [] (length 0 < 5) + const evals = Array.from({ length: 10 }, () => makeEval(80)); + makeFindChain(evals); + await check('user1'); + expect(DriftAlert.findOne).not.toHaveBeenCalled(); + }); + + it('fires warning alert when drop is 10–19 pts', async () => { + // baseline (oldest): 80, rolling (newest 10): 68 → drop = 12 → warning + const baseline = Array.from({ length: 15 }, () => makeEval(80)); + const rolling = Array.from({ length: 10 }, () => makeEval(68)); + makeFindChain([...rolling, ...baseline]); + DriftAlert.findOne.mockResolvedValue(null); + DriftAlert.create.mockResolvedValue({}); + + await check('user1'); + + expect(DriftAlert.create).toHaveBeenCalledWith( + expect.objectContaining({ severity: 'warning' }) + ); + }); + + it('fires critical alert when drop is >= 20 pts', async () => { + const baseline = Array.from({ length: 15 }, () => makeEval(90)); + const rolling = Array.from({ length: 10 }, () => makeEval(65)); + makeFindChain([...rolling, ...baseline]); + DriftAlert.findOne.mockResolvedValue(null); + DriftAlert.create.mockResolvedValue({}); + + await check('user1'); + + expect(DriftAlert.create).toHaveBeenCalledWith( + expect.objectContaining({ severity: 'critical' }) + ); + }); + + it('skips alert when one already exists within 6 hours (dedup)', async () => { + const baseline = Array.from({ length: 15 }, () => makeEval(90)); + const rolling = Array.from({ length: 10 }, () => makeEval(60)); + makeFindChain([...rolling, ...baseline]); + DriftAlert.findOne.mockResolvedValue({ _id: 'existing', severity: 'critical' }); + + await check('user1'); + + expect(DriftAlert.create).not.toHaveBeenCalled(); + }); + + it('does not alert when drop is below threshold', async () => { + const baseline = Array.from({ length: 15 }, () => makeEval(80)); + const rolling = Array.from({ length: 10 }, () => makeEval(75)); + makeFindChain([...rolling, ...baseline]); + + await check('user1'); + + expect(DriftAlert.findOne).not.toHaveBeenCalled(); + expect(DriftAlert.create).not.toHaveBeenCalled(); + }); + + it('filters out null finalScore values', async () => { + // Mix of valid and null scores — should only use non-null + const baseline = Array.from({ length: 15 }, () => makeEval(80)); + const rolling = [ + makeEval(null), + ...Array.from({ length: 9 }, () => makeEval(68)), + ]; + makeFindChain([...rolling, ...baseline]); + DriftAlert.findOne.mockResolvedValue(null); + DriftAlert.create.mockResolvedValue({}); + + await check('user1'); + + // rolling mean = 68, baseline mean = 80, drop = 12 → warning + expect(DriftAlert.create).toHaveBeenCalledWith( + expect.objectContaining({ severity: 'warning' }) + ); + }); +}); diff --git a/backend/src/services/judges/mock.js b/backend/src/services/judges/mock.js index 04f7e02..4c4a843 100644 --- a/backend/src/services/judges/mock.js +++ b/backend/src/services/judges/mock.js @@ -49,8 +49,8 @@ function computeTokens(text) { } export async function mockFaithfulness(testCase) { - await randomDelay(800, 2200); const startTime = Date.now(); + await randomDelay(800, 2200); const text = testCase.input + testCase.actualOutput; const score = Math.round(deterministicScore(text, 'faithfulness') * 100) / 100; @@ -71,8 +71,8 @@ export async function mockFaithfulness(testCase) { } export async function mockGroundedness(testCase) { - await randomDelay(1200, 3000); const startTime = Date.now(); + await randomDelay(1200, 3000); const text = testCase.input + testCase.actualOutput; const score = Math.round(deterministicScore(text, 'groundedness') * 100) / 100; @@ -93,8 +93,8 @@ export async function mockGroundedness(testCase) { } export async function mockContextRelevancy(testCase) { - await randomDelay(600, 1800); const startTime = Date.now(); + await randomDelay(600, 1800); const text = testCase.input + testCase.actualOutput; const score = Math.round(deterministicScore(text, 'contextRelevancy') * 100) / 100; @@ -115,8 +115,8 @@ export async function mockContextRelevancy(testCase) { } export async function mockAggregate(testCase, judgeResults) { - await randomDelay(1500, 4000); const startTime = Date.now(); + await randomDelay(1500, 4000); const scores = []; const judges = []; @@ -139,7 +139,7 @@ export async function mockAggregate(testCase, judgeResults) { ? [`Score spread of ${scoreRange.toFixed(2)} across judges indicates ${agreement}`] : []; - const text = testCase.input + JSON.stringify(judgeResults); + const text = testCase.input; const tokens = computeTokens(text); const cost = Math.round(((tokens.input / 1000) * 0.003 + (tokens.output / 1000) * 0.015) * 1000000) / 1000000; diff --git a/backend/src/services/orchestrator.js b/backend/src/services/orchestrator.js index 777a891..2bfb4d8 100644 --- a/backend/src/services/orchestrator.js +++ b/backend/src/services/orchestrator.js @@ -1,4 +1,5 @@ import { evaluateFaithfulness, evaluateGroundedness, evaluateContextRelevancy, aggregateResults } from './judges/index.js'; +import { mockFaithfulness, mockGroundedness, mockContextRelevancy, mockAggregate } from './judges/mock.js'; import { routeTestCase } from '../orchestrator/adaptiveRouter.js'; import { CostTracker } from './costTracker.js'; import { fireWebhooks } from './webhookService.js'; @@ -16,18 +17,24 @@ async function runJudgeWithTimeout(name, judgeFn, testCase, timeout) { return Promise.race([judgeFn(testCase), timeoutPromise]); } -export async function evaluateTestCase(testCase, testCaseIndex, emitEvent, saveEvent, userKeys = {}) { +export async function evaluateTestCase(testCase, testCaseIndex, emitEvent, saveEvent, userKeys = {}, isDemo = false) { const results = { testCaseIndex, judges: {}, aggregator: null, }; - const judges = [ - { name: 'openai', provider: 'openai', fn: (tc) => evaluateFaithfulness(tc, userKeys.openai), metric: 'faithfulness' }, - { name: 'anthropic', provider: 'anthropic', fn: (tc) => evaluateGroundedness(tc, userKeys.anthropic), metric: 'groundedness' }, - { name: 'gemini', provider: 'gemini', fn: (tc) => evaluateContextRelevancy(tc, userKeys.google), metric: 'contextRelevancy' }, - ]; + const judges = isDemo + ? [ + { name: 'openai', provider: 'openai', fn: (tc) => mockFaithfulness(tc), metric: 'faithfulness' }, + { name: 'anthropic', provider: 'anthropic', fn: (tc) => mockGroundedness(tc), metric: 'groundedness' }, + { name: 'gemini', provider: 'gemini', fn: (tc) => mockContextRelevancy(tc), metric: 'contextRelevancy' }, + ] + : [ + { name: 'openai', provider: 'openai', fn: (tc) => evaluateFaithfulness(tc, userKeys.openai), metric: 'faithfulness' }, + { name: 'anthropic', provider: 'anthropic', fn: (tc) => evaluateGroundedness(tc, userKeys.anthropic), metric: 'groundedness' }, + { name: 'gemini', provider: 'gemini', fn: (tc) => evaluateContextRelevancy(tc, userKeys.google), metric: 'contextRelevancy' }, + ]; const emitAndSave = (event, data) => { emitEvent(event, data); @@ -103,7 +110,7 @@ export async function evaluateTestCase(testCase, testCaseIndex, emitEvent, saveE const aggregatorResult = await executeWithProviderResilience({ provider: 'anthropic', operation: 'aggregate', - run: () => aggregateResults(testCase, results.judges, userKeys.anthropic), + run: () => isDemo ? mockAggregate(testCase, results.judges) : aggregateResults(testCase, results.judges, userKeys.anthropic), emitEvent: emitAndSave, context: { component: 'aggregator', testCaseIndex }, }); @@ -156,7 +163,8 @@ export async function runEvaluation(testCases, jobId, emitEvent, saveEvent, upda let totalCost = 0; const costTracker = new CostTracker(); - const useAdaptive = ADAPTIVE_MODE && options.strategy !== 'council'; + const isDemo = Boolean(options.demo); + const useAdaptive = !isDemo && ADAPTIVE_MODE && options.strategy !== 'council'; logger.audit('evaluation.started', { actor: 'system', jobId, @@ -193,7 +201,10 @@ export async function runEvaluation(testCases, jobId, emitEvent, saveEvent, upda }); let result; - if (useAdaptive) { + if (isDemo) { + result = await evaluateTestCase(testCases[i], i, emitEvent, saveEvent, {}, true); + result.strategy = 'council'; + } else if (useAdaptive) { result = await routeTestCase(testCases[i], i, emitEvent, saveEvent, costTracker, options, userKeys); } else { result = await evaluateTestCase(testCases[i], i, emitEvent, saveEvent, userKeys); @@ -231,7 +242,7 @@ export async function runEvaluation(testCases, jobId, emitEvent, saveEvent, upda completedAt: new Date(), }); - fireWebhooks({ jobId, testCases, results: allResults, summary, config: options }).catch(() => {}); + if (!options.suppressWebhooks) fireWebhooks({ jobId, testCases, results: allResults, summary, config: options }).catch(() => {}); logger.audit('evaluation.completed', { actor: 'system', jobId, diff --git a/backend/src/utils/validation.js b/backend/src/utils/validation.js index 5e674fa..313e5d5 100644 --- a/backend/src/utils/validation.js +++ b/backend/src/utils/validation.js @@ -38,6 +38,7 @@ export const evaluateRequestSchema = z.object({ options: z.object({ strategy: z.enum(['auto', 'single', 'hybrid', 'council']).default('auto'), riskOverride: z.number().min(0).max(1).optional(), + demo: z.boolean().optional().default(false), }).optional().default({ strategy: 'auto' }), }); diff --git a/frontend/package-lock.json b/frontend/package-lock.json index 4d860e1..636145f 100644 --- a/frontend/package-lock.json +++ b/frontend/package-lock.json @@ -11,6 +11,7 @@ "dependencies": { "@base-ui/react": "^1.3.0", "@fontsource-variable/geist": "^5.2.8", + "@remotion/player": "^4.0.438", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "firebase": "^12.10.0", @@ -23,6 +24,8 @@ "react": "^18.3.1", "react-dom": "^18.3.1", "react-router-dom": "^7.13.0", + "recharts": "^3.8.0", + "remotion": "^4.0.438", "shadcn": "^4.0.5", "sileo": "^0.1.3", "tailwind-merge": "^3.5.0", @@ -2232,6 +2235,55 @@ "integrity": "sha512-Vvn3zZrhQZkkBE8LSuW3em98c0FwgO4nxzv6OdSxPKJIEKY2bGbHn+mhGIPerzI4twdxaP8/0+06HBpwf345Lw==", "license": "BSD-3-Clause" }, + "node_modules/@reduxjs/toolkit": { + "version": "2.11.2", + "resolved": "https://registry.npmjs.org/@reduxjs/toolkit/-/toolkit-2.11.2.tgz", + "integrity": "sha512-Kd6kAHTA6/nUpp8mySPqj3en3dm0tdMIgbttnQ1xFMVpufoj+ADi8pXLBsd4xzTRHQa7t/Jv8W5UnCuW4kuWMQ==", + "license": "MIT", + "dependencies": { + "@standard-schema/spec": "^1.0.0", + "@standard-schema/utils": "^0.3.0", + "immer": "^11.0.0", + "redux": "^5.0.1", + "redux-thunk": "^3.1.0", + "reselect": "^5.1.0" + }, + "peerDependencies": { + "react": "^16.9.0 || ^17.0.0 || ^18 || ^19", + "react-redux": "^7.2.1 || ^8.1.3 || ^9.0.0" + }, + "peerDependenciesMeta": { + "react": { + "optional": true + }, + "react-redux": { + "optional": true + } + } + }, + "node_modules/@reduxjs/toolkit/node_modules/immer": { + "version": "11.1.4", + "resolved": "https://registry.npmjs.org/immer/-/immer-11.1.4.tgz", + "integrity": "sha512-XREFCPo6ksxVzP4E0ekD5aMdf8WMwmdNaz6vuvxgI40UaEiu6q3p8X52aU6GdyvLY3XXX/8R7JOTXStz/nBbRw==", + "license": "MIT", + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/immer" + } + }, + "node_modules/@remotion/player": { + "version": "4.0.438", + "resolved": "https://registry.npmjs.org/@remotion/player/-/player-4.0.438.tgz", + "integrity": "sha512-2OT8r0arsjxUEVEa5dNgain7ChbDG0THZglxh1Bo4U8nTSMnuX+1yyF0MVOlmT56yaESfoiGbZvnnpB6e7Ltew==", + "license": "SEE LICENSE IN LICENSE.md", + "dependencies": { + "remotion": "4.0.438" + }, + "peerDependencies": { + "react": ">=16.8.0", + "react-dom": ">=16.8.0" + } + }, "node_modules/@rolldown/pluginutils": { "version": "1.0.0-beta.27", "resolved": "https://registry.npmjs.org/@rolldown/pluginutils/-/pluginutils-1.0.0-beta.27.tgz", @@ -2607,6 +2659,18 @@ "url": "https://github.com/sponsors/sindresorhus" } }, + "node_modules/@standard-schema/spec": { + "version": "1.1.0", + "resolved": "https://registry.npmjs.org/@standard-schema/spec/-/spec-1.1.0.tgz", + "integrity": "sha512-l2aFy5jALhniG5HgqrD6jXLi/rUWrKvqN/qJx6yoJsgKhblVd+iqqU4RCXavm/jPityDo5TCvKMnpjKnOriy0w==", + "license": "MIT" + }, + "node_modules/@standard-schema/utils": { + "version": "0.3.0", + "resolved": "https://registry.npmjs.org/@standard-schema/utils/-/utils-0.3.0.tgz", + "integrity": "sha512-e7Mew686owMaPJVNNLs55PUvgz371nKgwsc4vxE49zsODpJEnxgxRo2y/OKrqueavXgZNMDVj3DdHFlaSAeU8g==", + "license": "MIT" + }, "node_modules/@ts-morph/common": { "version": "0.27.0", "resolved": "https://registry.npmjs.org/@ts-morph/common/-/common-0.27.0.tgz", @@ -2663,6 +2727,69 @@ "@babel/types": "^7.28.2" } }, + "node_modules/@types/d3-array": { + "version": "3.2.2", + "resolved": "https://registry.npmjs.org/@types/d3-array/-/d3-array-3.2.2.tgz", + "integrity": "sha512-hOLWVbm7uRza0BYXpIIW5pxfrKe0W+D5lrFiAEYR+pb6w3N2SwSMaJbXdUfSEv+dT4MfHBLtn5js0LAWaO6otw==", + "license": "MIT" + }, + "node_modules/@types/d3-color": { + "version": "3.1.3", + "resolved": "https://registry.npmjs.org/@types/d3-color/-/d3-color-3.1.3.tgz", + "integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==", + "license": "MIT" + }, + "node_modules/@types/d3-ease": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/d3-ease/-/d3-ease-3.0.2.tgz", + "integrity": "sha512-NcV1JjO5oDzoK26oMzbILE6HW7uVXOHLQvHshBUW4UMdZGfiY6v5BeQwh9a9tCzv+CeefZQHJt5SRgK154RtiA==", + "license": "MIT" + }, + "node_modules/@types/d3-interpolate": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-interpolate/-/d3-interpolate-3.0.4.tgz", + "integrity": "sha512-mgLPETlrpVV1YRJIglr4Ez47g7Yxjl1lj7YKsiMCb27VJH9W8NVM6Bb9d8kkpG/uAQS5AmbA48q2IAolKKo1MA==", + "license": "MIT", + "dependencies": { + "@types/d3-color": "*" + } + }, + "node_modules/@types/d3-path": { + "version": "3.1.1", + "resolved": "https://registry.npmjs.org/@types/d3-path/-/d3-path-3.1.1.tgz", + "integrity": "sha512-VMZBYyQvbGmWyWVea0EHs/BwLgxc+MKi1zLDCONksozI4YJMcTt8ZEuIR4Sb1MMTE8MMW49v0IwI5+b7RmfWlg==", + "license": "MIT" + }, + "node_modules/@types/d3-scale": { + "version": "4.0.9", + "resolved": "https://registry.npmjs.org/@types/d3-scale/-/d3-scale-4.0.9.tgz", + "integrity": "sha512-dLmtwB8zkAeO/juAMfnV+sItKjlsw2lKdZVVy6LRr0cBmegxSABiLEpGVmSJJ8O08i4+sGR6qQtb6WtuwJdvVw==", + "license": "MIT", + "dependencies": { + "@types/d3-time": "*" + } + }, + "node_modules/@types/d3-shape": { + "version": "3.1.8", + "resolved": "https://registry.npmjs.org/@types/d3-shape/-/d3-shape-3.1.8.tgz", + "integrity": "sha512-lae0iWfcDeR7qt7rA88BNiqdvPS5pFVPpo5OfjElwNaT2yyekbM0C9vK+yqBqEmHr6lDkRnYNoTBYlAgJa7a4w==", + "license": "MIT", + "dependencies": { + "@types/d3-path": "*" + } + }, + "node_modules/@types/d3-time": { + "version": "3.0.4", + "resolved": "https://registry.npmjs.org/@types/d3-time/-/d3-time-3.0.4.tgz", + "integrity": "sha512-yuzZug1nkAAaBlBBikKZTgzCeA+k1uy4ZFwWANOfKw5z5LRhV0gNA7gNkKm7HoK+HRN0wX3EkxGk0fpbWhmB7g==", + "license": "MIT" + }, + "node_modules/@types/d3-timer": { + "version": "3.0.2", + "resolved": "https://registry.npmjs.org/@types/d3-timer/-/d3-timer-3.0.2.tgz", + "integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==", + "license": "MIT" + }, "node_modules/@types/estree": { "version": "1.0.8", "resolved": "https://registry.npmjs.org/@types/estree/-/estree-1.0.8.tgz", @@ -2713,6 +2840,12 @@ "integrity": "sha512-xMAgYwceFhRA2zY+XbEA7mxYbA093wdiW8Vu6gZPGWy9cmOyU9XesH1tNcEWsKFd5Vzrqx5T3D38PWx1FIIXkA==", "license": "MIT" }, + "node_modules/@types/use-sync-external-store": { + "version": "0.0.6", + "resolved": "https://registry.npmjs.org/@types/use-sync-external-store/-/use-sync-external-store-0.0.6.tgz", + "integrity": "sha512-zFDAD+tlpf2r4asuHEj0XH6pY6i0g5NeAHPn+15wk3BV6JA69eERFXC1gyGThDkVa1zCyKr5jox1+2LbV/AMLg==", + "license": "MIT" + }, "node_modules/@types/validate-npm-package-name": { "version": "4.0.2", "resolved": "https://registry.npmjs.org/@types/validate-npm-package-name/-/validate-npm-package-name-4.0.2.tgz", @@ -3478,6 +3611,127 @@ "devOptional": true, "license": "MIT" }, + "node_modules/d3-array": { + "version": "3.2.4", + "resolved": "https://registry.npmjs.org/d3-array/-/d3-array-3.2.4.tgz", + "integrity": "sha512-tdQAmyA18i4J7wprpYq8ClcxZy3SC31QMeByyCFyRt7BVHdREQZ5lpzoe5mFEYZUWe+oq8HBvk9JjpibyEV4Jg==", + "license": "ISC", + "dependencies": { + "internmap": "1 - 2" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-color": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/d3-color/-/d3-color-3.1.0.tgz", + "integrity": "sha512-zg/chbXyeBtMQ1LbD/WSoW2DpC3I0mpmPdW+ynRTj/x2DAWYrIY7qeZIHidozwV24m4iavr15lNwIwLxRmOxhA==", + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-ease": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-ease/-/d3-ease-3.0.1.tgz", + "integrity": "sha512-wR/XK3D3XcLIZwpbvQwQ5fK+8Ykds1ip7A2Txe0yxncXSdq1L9skcG7blcedkOX+ZcgxGAmLX1FrRGbADwzi0w==", + "license": "BSD-3-Clause", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-format": { + "version": "3.1.2", + "resolved": "https://registry.npmjs.org/d3-format/-/d3-format-3.1.2.tgz", + "integrity": "sha512-AJDdYOdnyRDV5b6ArilzCPPwc1ejkHcoyFarqlPqT7zRYjhavcT3uSrqcMvsgh2CgoPbK3RCwyHaVyxYcP2Arg==", + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-interpolate": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-interpolate/-/d3-interpolate-3.0.1.tgz", + "integrity": "sha512-3bYs1rOD33uo8aqJfKP3JWPAibgw8Zm2+L9vBKEHJ2Rg+viTR7o5Mmv5mZcieN+FRYaAOWX5SJATX6k1PWz72g==", + "license": "ISC", + "dependencies": { + "d3-color": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-path": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/d3-path/-/d3-path-3.1.0.tgz", + "integrity": "sha512-p3KP5HCf/bvjBSSKuXid6Zqijx7wIfNW+J/maPs+iwR35at5JCbLUT0LzF1cnjbCHWhqzQTIN2Jpe8pRebIEFQ==", + "license": "ISC", + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-scale": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/d3-scale/-/d3-scale-4.0.2.tgz", + "integrity": "sha512-GZW464g1SH7ag3Y7hXjf8RoUuAFIqklOAq3MRl4OaWabTFJY9PN/E1YklhXLh+OQ3fM9yS2nOkCoS+WLZ6kvxQ==", + "license": "ISC", + "dependencies": { + "d3-array": "2.10.0 - 3", + "d3-format": "1 - 3", + "d3-interpolate": "1.2.0 - 3", + "d3-time": "2.1.1 - 3", + "d3-time-format": "2 - 4" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-shape": { + "version": "3.2.0", + "resolved": "https://registry.npmjs.org/d3-shape/-/d3-shape-3.2.0.tgz", + "integrity": "sha512-SaLBuwGm3MOViRq2ABk3eLoxwZELpH6zhl3FbAoJ7Vm1gofKx6El1Ib5z23NUEhF9AsGl7y+dzLe5Cw2AArGTA==", + "license": "ISC", + "dependencies": { + "d3-path": "^3.1.0" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-time": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/d3-time/-/d3-time-3.1.0.tgz", + "integrity": "sha512-VqKjzBLejbSMT4IgbmVgDjpkYrNWUYJnbCGo874u7MMKIWsILRX+OpX/gTk8MqjpT1A/c6HY2dCA77ZN0lkQ2Q==", + "license": "ISC", + "dependencies": { + "d3-array": "2 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-time-format": { + "version": "4.1.0", + "resolved": "https://registry.npmjs.org/d3-time-format/-/d3-time-format-4.1.0.tgz", + "integrity": "sha512-dJxPBlzC7NugB2PDLwo9Q8JiTR3M3e4/XANkreKSUxF8vvXKqm1Yfq4Q5dl8budlunRVlUUaDUgFt7eA8D6NLg==", + "license": "ISC", + "dependencies": { + "d3-time": "1 - 3" + }, + "engines": { + "node": ">=12" + } + }, + "node_modules/d3-timer": { + "version": "3.0.1", + "resolved": "https://registry.npmjs.org/d3-timer/-/d3-timer-3.0.1.tgz", + "integrity": "sha512-ndfJ/JxxMd3nw31uyKoY2naivF+r29V+Lc0svZxe1JvvIRmi8hUsrMvdOwgS1o6uBHmiz91geQ0ylPP0aj1VUA==", + "license": "ISC", + "engines": { + "node": ">=12" + } + }, "node_modules/data-uri-to-buffer": { "version": "4.0.1", "resolved": "https://registry.npmjs.org/data-uri-to-buffer/-/data-uri-to-buffer-4.0.1.tgz", @@ -3504,6 +3758,12 @@ } } }, + "node_modules/decimal.js-light": { + "version": "2.5.1", + "resolved": "https://registry.npmjs.org/decimal.js-light/-/decimal.js-light-2.5.1.tgz", + "integrity": "sha512-qIMFpTMZmny+MMIitAB6D7iVPEorVw6YQRWkvarTkT4tBeSLLiHzcwj6q0MmYSFCiVpiqPJTJEYIrpcPzVEIvg==", + "license": "MIT" + }, "node_modules/dedent": { "version": "1.7.2", "resolved": "https://registry.npmjs.org/dedent/-/dedent-1.7.2.tgz", @@ -3717,6 +3977,16 @@ "node": ">= 0.4" } }, + "node_modules/es-toolkit": { + "version": "1.45.1", + "resolved": "https://registry.npmjs.org/es-toolkit/-/es-toolkit-1.45.1.tgz", + "integrity": "sha512-/jhoOj/Fx+A+IIyDNOvO3TItGmlMKhtX8ISAHKE90c4b/k1tqaqEZ+uUqfpU8DMnW5cgNJv606zS55jGvza0Xw==", + "license": "MIT", + "workspaces": [ + "docs", + "benchmarks" + ] + }, "node_modules/esbuild": { "version": "0.25.12", "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.25.12.tgz", @@ -3796,6 +4066,12 @@ "node": ">= 0.6" } }, + "node_modules/eventemitter3": { + "version": "5.0.4", + "resolved": "https://registry.npmjs.org/eventemitter3/-/eventemitter3-5.0.4.tgz", + "integrity": "sha512-mlsTRyGaPBjPedk6Bvw+aqbsXDtoAyAzm5MO7JgU+yVRyMQ5O8bD4Kcci7BS85f93veegeCPkL8R4GLClnjLFw==", + "license": "MIT" + }, "node_modules/eventsource": { "version": "3.0.7", "resolved": "https://registry.npmjs.org/eventsource/-/eventsource-3.0.7.tgz", @@ -4495,6 +4771,16 @@ "node": ">= 4" } }, + "node_modules/immer": { + "version": "10.2.0", + "resolved": "https://registry.npmjs.org/immer/-/immer-10.2.0.tgz", + "integrity": "sha512-d/+XTN3zfODyjr89gM3mPq1WNX2B8pYsu7eORitdwyA2sBubnTl3laYlBk4sXY5FUa5qTZGBDPJICVbvqzjlbw==", + "license": "MIT", + "funding": { + "type": "opencollective", + "url": "https://opencollective.com/immer" + } + }, "node_modules/import-fresh": { "version": "3.3.1", "resolved": "https://registry.npmjs.org/import-fresh/-/import-fresh-3.3.1.tgz", @@ -4517,6 +4803,15 @@ "integrity": "sha512-k/vGaX4/Yla3WzyMCvTQOXYeIHvqOKtnqBduzTHpzpQZzAskKMhZ2K+EnBiSM9zGSoIFeMpXKxa4dYeZIQqewQ==", "license": "ISC" }, + "node_modules/internmap": { + "version": "2.0.3", + "resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz", + "integrity": "sha512-5Hh7Y1wQbvY5ooGgPbDaL5iYLAPzMTUrjMulskHLH6wnv/A+1q5rgEaiuqEjB+oxGXIVZs1FF+R/KPN3ZSQYYg==", + "license": "ISC", + "engines": { + "node": ">=12" + } + }, "node_modules/ip-address": { "version": "10.1.0", "resolved": "https://registry.npmjs.org/ip-address/-/ip-address-10.1.0.tgz", @@ -5938,6 +6233,29 @@ "integrity": "sha512-24e6ynE2H+OKt4kqsOvNd8kBpV65zoxbA4BVsEOB3ARVWQki/DHzaUoC5KuON/BiccDaCCTZBuOcfZs70kR8bQ==", "license": "MIT" }, + "node_modules/react-redux": { + "version": "9.2.0", + "resolved": "https://registry.npmjs.org/react-redux/-/react-redux-9.2.0.tgz", + "integrity": "sha512-ROY9fvHhwOD9ySfrF0wmvu//bKCQ6AeZZq1nJNtbDC+kk5DuSuNX/n6YWYF/SYy7bSba4D4FSz8DJeKY/S/r+g==", + "license": "MIT", + "dependencies": { + "@types/use-sync-external-store": "^0.0.6", + "use-sync-external-store": "^1.4.0" + }, + "peerDependencies": { + "@types/react": "^18.2.25 || ^19", + "react": "^18.0 || ^19", + "redux": "^5.0.0" + }, + "peerDependenciesMeta": { + "@types/react": { + "optional": true + }, + "redux": { + "optional": true + } + } + }, "node_modules/react-refresh": { "version": "0.17.0", "resolved": "https://registry.npmjs.org/react-refresh/-/react-refresh-0.17.0.tgz", @@ -6025,6 +6343,61 @@ "node": ">= 4" } }, + "node_modules/recharts": { + "version": "3.8.0", + "resolved": "https://registry.npmjs.org/recharts/-/recharts-3.8.0.tgz", + "integrity": "sha512-Z/m38DX3L73ExO4Tpc9/iZWHmHnlzWG4njQbxsF5aSjwqmHNDDIm0rdEBArkwsBvR8U6EirlEHiQNYWCVh9sGQ==", + "license": "MIT", + "workspaces": [ + "www" + ], + "dependencies": { + "@reduxjs/toolkit": "^1.9.0 || 2.x.x", + "clsx": "^2.1.1", + "decimal.js-light": "^2.5.1", + "es-toolkit": "^1.39.3", + "eventemitter3": "^5.0.1", + "immer": "^10.1.1", + "react-redux": "8.x.x || 9.x.x", + "reselect": "5.1.1", + "tiny-invariant": "^1.3.3", + "use-sync-external-store": "^1.2.2", + "victory-vendor": "^37.0.2" + }, + "engines": { + "node": ">=18" + }, + "peerDependencies": { + "react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0", + "react-dom": "^16.0.0 || ^17.0.0 || ^18.0.0 || ^19.0.0", + "react-is": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" + } + }, + "node_modules/redux": { + "version": "5.0.1", + "resolved": "https://registry.npmjs.org/redux/-/redux-5.0.1.tgz", + "integrity": "sha512-M9/ELqF6fy8FwmkpnF0S3YKOqMyoWJ4+CS5Efg2ct3oY9daQvd/Pc71FpGZsVsbl3Cpb+IIcjBDUnnyBdQbq4w==", + "license": "MIT" + }, + "node_modules/redux-thunk": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/redux-thunk/-/redux-thunk-3.1.0.tgz", + "integrity": "sha512-NW2r5T6ksUKXCabzhL9z+h206HQw/NJkcLm1GPImRQ8IzfXwRGqjVhKJGauHirT0DAuyy6hjdnMZaRoAcy0Klw==", + "license": "MIT", + "peerDependencies": { + "redux": "^5.0.0" + } + }, + "node_modules/remotion": { + "version": "4.0.438", + "resolved": "https://registry.npmjs.org/remotion/-/remotion-4.0.438.tgz", + "integrity": "sha512-keG2GxHh81UFV0zFrwRh+yngXHn6C3z/Nohc+ru49M1TlIcaThG/hD2amJNH9otYzpYJxVjNF1t/i1hqcI8OGw==", + "license": "SEE LICENSE IN LICENSE.md", + "peerDependencies": { + "react": ">=16.8.0", + "react-dom": ">=16.8.0" + } + }, "node_modules/require-directory": { "version": "2.1.1", "resolved": "https://registry.npmjs.org/require-directory/-/require-directory-2.1.1.tgz", @@ -7047,6 +7420,28 @@ "node": ">= 0.8" } }, + "node_modules/victory-vendor": { + "version": "37.3.6", + "resolved": "https://registry.npmjs.org/victory-vendor/-/victory-vendor-37.3.6.tgz", + "integrity": "sha512-SbPDPdDBYp+5MJHhBCAyI7wKM3d5ivekigc2Dk2s7pgbZ9wIgIBYGVw4zGHBml/qTFbexrofXW6Gu4noGxrOwQ==", + "license": "MIT AND ISC", + "dependencies": { + "@types/d3-array": "^3.0.3", + "@types/d3-ease": "^3.0.0", + "@types/d3-interpolate": "^3.0.1", + "@types/d3-scale": "^4.0.2", + "@types/d3-shape": "^3.1.0", + "@types/d3-time": "^3.0.0", + "@types/d3-timer": "^3.0.0", + "d3-array": "^3.1.6", + "d3-ease": "^3.0.1", + "d3-interpolate": "^3.0.1", + "d3-scale": "^4.0.2", + "d3-shape": "^3.1.0", + "d3-time": "^3.0.0", + "d3-timer": "^3.0.1" + } + }, "node_modules/vite": { "version": "6.4.1", "resolved": "https://registry.npmjs.org/vite/-/vite-6.4.1.tgz", diff --git a/frontend/package.json b/frontend/package.json index 145e309..0300aae 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -11,6 +11,7 @@ "dependencies": { "@base-ui/react": "^1.3.0", "@fontsource-variable/geist": "^5.2.8", + "@remotion/player": "^4.0.438", "class-variance-authority": "^0.7.1", "clsx": "^2.1.1", "firebase": "^12.10.0", @@ -23,6 +24,8 @@ "react": "^18.3.1", "react-dom": "^18.3.1", "react-router-dom": "^7.13.0", + "recharts": "^3.8.0", + "remotion": "^4.0.438", "shadcn": "^4.0.5", "sileo": "^0.1.3", "tailwind-merge": "^3.5.0", diff --git a/frontend/src/App.jsx b/frontend/src/App.jsx index fb8a731..3aef8d4 100644 --- a/frontend/src/App.jsx +++ b/frontend/src/App.jsx @@ -1,5 +1,5 @@ -import { useEffect, useRef } from 'react'; -import { Routes, Route, useNavigate, useParams } from 'react-router-dom'; +import { useEffect, useRef, useState } from 'react'; +import { Routes, Route, Navigate, useNavigate, useParams } from 'react-router-dom'; import { Toaster, sileo } from 'sileo'; import { ErrorBoundary } from './components/ErrorBoundary'; import { EvaluationProvider, useEvaluation } from './context/EvaluationContext'; @@ -8,10 +8,16 @@ import { TestCaseUpload } from './components/TestCaseUpload'; import { StreamingEvaluation } from './components/StreamingEvaluation'; import { EvaluationHistory } from './components/EvaluationHistory'; import { WebhookManager } from './components/webhooks/WebhookManager'; -import { ApiKeysManager } from './components/ApiKeysManager'; import { EvaluationDetail } from './components/EvaluationDetail'; +import { SettingsPage, AccountTab } from './pages/SettingsPage'; +import { ApiKeysManager } from './components/ApiKeysManager'; +import { ServiceKeysManager } from './components/ServiceKeysManager'; +import { MonitoringDashboard } from './components/MonitoringDashboard'; import { ErrorAlert } from './components/ui/ErrorAlert'; import { VerifyEmailBanner } from './components/VerifyEmailBanner'; +import { DemoWelcome } from './components/DemoWelcome'; +import { getKeys } from './lib/api'; +import { DEMO_TEST_CASES } from './lib/demoTestCases'; function UploadRoute() { const { @@ -24,9 +30,13 @@ function UploadRoute() { syncActiveEvaluation, } = useEvaluation(); const navigate = useNavigate(); + const [hasKeys, setHasKeys] = useState(null); useEffect(() => { syncActiveEvaluation().catch(() => { }); + getKeys() + .then(data => setHasKeys(data.configured.openai || data.configured.anthropic || data.configured.google)) + .catch(() => setHasKeys(true)); }, [syncActiveEvaluation]); const resumeActiveEvaluation = () => { @@ -48,14 +58,15 @@ function UploadRoute() { return; } + const isDemo = Boolean(options?.demo); const evalPromise = submitEvaluation(cases, options).then((jobId) => { if (jobId) return jobId; throw new Error('Failed to start evaluation'); }); sileo.promise(evalPromise, { - loading: { title: 'Starting evaluation...' }, - success: { title: 'Evaluation started!' }, + loading: { title: isDemo ? 'Starting demo...' : 'Starting evaluation...' }, + success: { title: isDemo ? 'Demo started!' : 'Evaluation started!' }, error: (err) => ({ title: 'Failed to start', description: err?.message }), }); @@ -68,12 +79,51 @@ function UploadRoute() { }); }; + const handleDismiss = () => { + localStorage.setItem('demo_dismissed', 'true'); + setHasKeys(true); + }; + + if (hasKeys === null) { + return ( + {}} isLoading={true} activeEvaluation={null} onResumeActive={null} /> + ); + } + + const demoDismissed = localStorage.getItem('demo_dismissed') === 'true'; + + if (!hasKeys && !demoDismissed) { + const handleDemoSubmit = () => { + const evalPromise = submitEvaluation(DEMO_TEST_CASES, { strategy: 'council', name: 'Demo Evaluation', demo: true }).then(jobId => { + if (jobId) return jobId; + throw new Error('Failed to start demo'); + }); + sileo.promise(evalPromise, { + loading: { title: 'Starting demo...' }, + success: { title: 'Demo started!' }, + error: (err) => ({ title: 'Failed to start', description: err?.message }), + }); + evalPromise.then(jobId => navigate(`/app/evaluate/${jobId}`)).catch(() => {}); + }; + + return ( + navigate('/app/settings/api-keys')} + onDismiss={handleDismiss} + isLoading={isLoading} + /> + ); + } + return ( navigate('/app/settings/api-keys')} /> ); } @@ -81,7 +131,7 @@ function UploadRoute() { function EvaluateRoute() { const { jobId } = useParams(); const navigate = useNavigate(); - const { testCases, currentTestCase, setCurrentTestCase, events, jobId: ctxJobId } = useEvaluation(); + const { testCases, currentTestCase, setCurrentTestCase, events, jobId: ctxJobId, isDemo } = useEvaluation(); const hasStreamingContext = Boolean(ctxJobId && ctxJobId === jobId && testCases.length > 0); const wasStreaming = useRef(hasStreamingContext); @@ -106,6 +156,7 @@ function EvaluateRoute() { currentTestCase={currentTestCase} onNavigate={setCurrentTestCase} jobId={jobId} + isDemo={isDemo} /> ); } @@ -124,8 +175,14 @@ function AppContent() { } /> } /> } /> + } /> } /> - } /> + }> + } /> + } /> + } /> + } /> + @@ -133,6 +190,7 @@ function AppContent() { ); } + export default function App() { return ( diff --git a/frontend/src/components/ApiKeysManager.jsx b/frontend/src/components/ApiKeysManager.jsx index 335c632..394f24d 100644 --- a/frontend/src/components/ApiKeysManager.jsx +++ b/frontend/src/components/ApiKeysManager.jsx @@ -1,11 +1,12 @@ import { useState, useCallback } from 'react'; -import { Lock, Pencil, Trash2, X } from 'lucide-react'; +import { Info, Lock, Pencil, Trash2, X } from 'lucide-react'; import { sileo } from 'sileo'; -import { PageHeader } from './PageHeader'; import { ErrorAlert } from './ui/ErrorAlert'; import { getKeys, setKey, deleteKey } from '../lib/api'; import { useApiQuery } from '../hooks/useApiQuery'; +const ONBOARDING_BANNER_KEY = 'settings-api-keys-banner-dismissed'; + const PROVIDERS = [ { id: 'openai', label: 'OpenAI', meta: 'Faithfulness · gpt-4o-mini', dot: 'bg-openai', bar: 'bg-openai' }, { id: 'anthropic', label: 'Anthropic', meta: 'Groundedness + Aggregator · claude-3-haiku / claude-sonnet-4', dot: 'bg-anthropic', bar: 'bg-anthropic' }, @@ -55,12 +56,13 @@ export function ApiKeysManager() { const { data, loading, error, refetch } = useApiQuery(fetchFn); const configured = data?.configured ?? { openai: false, anthropic: false, google: false }; - const [inputs, setInputs] = useState({ openai: '', anthropic: '', google: '' }); - const [saving, setSaving] = useState({ openai: false, anthropic: false, google: false }); - const [editing, setEditing] = useState({ openai: false, anthropic: false, google: false }); - const [deleting,setDeleting]= useState({ openai: false, anthropic: false, google: false }); - const [errors, setErrors] = useState({ openai: '', anthropic: '', google: '' }); - const [confirm, setConfirm] = useState(null); + const [inputs, setInputs] = useState({ openai: '', anthropic: '', google: '' }); + const [saving, setSaving] = useState({ openai: false, anthropic: false, google: false }); + const [editing, setEditing] = useState({ openai: false, anthropic: false, google: false }); + const [deleting, setDeleting] = useState({ openai: false, anthropic: false, google: false }); + const [errors, setErrors] = useState({ openai: '', anthropic: '', google: '' }); + const [confirm, setConfirm] = useState(null); + const [bannerDismissed, setBannerDismissed] = useState(() => !!localStorage.getItem(ONBOARDING_BANNER_KEY)); async function handleSave(id) { const key = inputs[id].trim(); @@ -102,12 +104,36 @@ export function ApiKeysManager() { const confirmProvider = confirm ? PROVIDERS.find(p => p.id === confirm) : null; + const allUnconfigured = !configured.openai && !configured.anthropic && !configured.google; + const showBanner = !loading && allUnconfigured && !bannerDismissed; + + function dismissBanner() { + localStorage.setItem(ONBOARDING_BANNER_KEY, 'true'); + setBannerDismissed(true); + } + return ( -
- +
+

+ + Keys are AES-256 encrypted at rest. Falls back to shared server keys when none are configured. +

+ + {showBanner && ( +
+ +

+ Configure at least one provider key to run evaluations. Your own keys bypass shared rate limits and give you full cost visibility. +

+ +
+ )} + {loading ? ( diff --git a/frontend/src/components/DemoWelcome.jsx b/frontend/src/components/DemoWelcome.jsx new file mode 100644 index 0000000..de39d3f --- /dev/null +++ b/frontend/src/components/DemoWelcome.jsx @@ -0,0 +1,70 @@ +import PropTypes from 'prop-types'; +import { PageHeader } from './PageHeader'; + +export function DemoWelcome({ onRunDemo, onConfigureKeys, onDismiss, isLoading }) { + return ( +
+ +
+
+
+
+

Run the council.

+

+ 10 test cases pre-loaded. Three LLM judges — OpenAI, Anthropic, Gemini — will evaluate faithfulness, groundedness, and context relevancy in real time. +

+ + + + + + + +
+ {['Faithfulness', 'Groundedness', 'Context Relevancy'].map(label => ( + + {label} + + ))} +
+
+
+
+
+ ); +} + +DemoWelcome.propTypes = { + onRunDemo: PropTypes.func.isRequired, + onConfigureKeys: PropTypes.func.isRequired, + onDismiss: PropTypes.func.isRequired, + isLoading: PropTypes.bool, +}; diff --git a/frontend/src/components/EvaluationHistory.jsx b/frontend/src/components/EvaluationHistory.jsx index 70a32f1..24549fa 100644 --- a/frontend/src/components/EvaluationHistory.jsx +++ b/frontend/src/components/EvaluationHistory.jsx @@ -297,6 +297,11 @@ export function EvaluationHistory() { style={{ '--stagger-delay': `${Math.min(idx * 40, 400)}ms` }} > e.stopPropagation()}> + {eval_.config?.demo && ( + + Demo + + )} = 20) return SCORE_COLORS.fail; + if (drop >= 10) return SCORE_COLORS.warn; + if (drop < 0) return SCORE_COLORS.pass; + return SCORE_COLORS.neutral; +} + + +function computeBaseline(scores) { + // scores is newest-first; baseline = oldest 10+ (index 10..end) + if (scores.length < 20) return null; + const baselineWindow = scores.slice(10).filter((s) => s.finalScore !== null); + if (baselineWindow.length < 5) return null; + const sum = baselineWindow.reduce((a, b) => a + b.finalScore, 0); + return sum / baselineWindow.length; +} + +function computeRollingMean(scores) { + const window = scores.slice(0, Math.min(10, scores.length)).filter((s) => s.finalScore !== null); + if (window.length === 0) return null; + return window.reduce((a, b) => a + b.finalScore, 0) / window.length; +} + +function KpiCard({ label, value, valueColor, sub }) { + return ( +
+

{label}

+

+ {value ?? '—'} +

+ {sub &&

{sub}

} +
+ ); +} + +function DriftAlertBanner({ alert }) { + if (!alert) return null; + const isCritical = alert.severity === 'critical'; + return ( +
+ + + {isCritical ? 'Critical drift' : 'Quality warning'}: Score dropped{' '} + {Math.round(alert.drop)}pts below baseline ({Math.round(alert.rollingMean)} vs{' '} + {Math.round(alert.baselineMean)} baseline) · {formatRelative(alert.createdAt)} + +
+ ); +} + +function ChartTooltipContent({ active, payload }) { + if (!active || !payload?.length) return null; + const d = payload[0]?.payload; + const verdict = d?.verdict; + const badgeClass = STATUS_BADGE_STYLES[verdict?.toLowerCase()] || ''; + return ( +
+

+ {d?.finalScore !== null ? Math.round(d.finalScore) : '—'} +

+
+ {verdict && ( + + {verdict} + + )} + {d?.strategy && ( + {d.strategy} + )} +
+

{formatRelative(d?.completedAt)}

+
+ ); +} + +function EmptyState() { + return ( +
+ +

No live samples yet

+

+ Point POST /api/sample at this workspace to start monitoring. +

+ 
{`fetch('/api/sample', {
+  method: 'POST',
+  headers: { Authorization: 'Bearer ...' },
+  body: JSON.stringify({ query, response, contexts })
+}).catch(() => {})`}
+
+ ); +} + +function SkeletonCard() { + return ( +
+
+
+
+ ); +} + +export function MonitoringDashboard() { + const [scores, setScores] = useState(null); + const [alerts, setAlerts] = useState(null); + const [loading, setLoading] = useState(true); + const [error, setError] = useState(null); + + const fetchData = useCallback(async () => { + try { + const [scoresData, alertsData] = await Promise.all([ + getMonitoringScores(50), + getMonitoringAlerts(), + ]); + setScores(scoresData.scores ?? []); + setAlerts(alertsData.alerts ?? []); + } catch (err) { + setError(err.message || 'Failed to load monitoring data'); + } finally { + setLoading(false); + } + }, []); + + useEffect(() => { + fetchData(); + }, [fetchData]); + + const hasData = scores && scores.length > 0; + const baselineMean = hasData ? computeBaseline(scores) : null; + const rollingMean = hasData ? computeRollingMean(scores) : null; + const baselineEstablished = baselineMean !== null; + const activeAlert = alerts?.[0] && new Date(alerts[0].createdAt) > new Date(Date.now() - 6 * 60 * 60 * 1000) + ? alerts[0] + : null; + + // Chart expects chronological order (oldest first) + const chartData = hasData ? [...scores].reverse() : []; + + return ( +
+ + + {activeAlert && } + + {/* KPI Row */} +
+ {loading ? ( + <> + + + + + ) : ( + <> + + + + + )} +
+ + {/* Score Chart */} +
+
+

Score Trend

+
+
+ {loading ? ( +
+ ) : !hasData ? ( + + ) : ( + + + Live sample score trend + Time-series chart showing council evaluation scores for live production samples + + + + } /> + {baselineEstablished && ( + <> + + + + + )} + + + + )} +
+
+ + {/* Recent Samples Table */} +
+
+

Recent Samples

+
+ {loading ? ( +
+ {Array.from({ length: 5 }).map((_, i) => ( +
+
+
+
+
+ ))} +
+ ) : !hasData ? ( +
+ No samples yet — send some requests to see them here. +
+ ) : ( +
+ + + + Query + Strategy + Score + Verdict + Time + + + + {scores.map((s) => { + const strategyConfig = s.strategy ? STRATEGY_STYLE[s.strategy] : null; + const verdictLower = s.verdict?.toLowerCase(); + return ( + + + {s.jobId} + + + {strategyConfig ? ( + + + {strategyConfig.label} + + ) : ( + — + )} + + + {s.finalScore !== null ? Math.round(s.finalScore) : '—'} + + + {s.verdict ? ( + + {s.verdict} + + ) : ( + — + )} + + + {formatRelative(s.completedAt)} + + + ); + })} + +
+
+ )} +
+ + {error && ( +
+ {error} +
+ )} +
+ ); +} diff --git a/frontend/src/components/ServiceKeysManager.jsx b/frontend/src/components/ServiceKeysManager.jsx new file mode 100644 index 0000000..d58afc7 --- /dev/null +++ b/frontend/src/components/ServiceKeysManager.jsx @@ -0,0 +1,395 @@ +import { useState, useCallback } from 'react'; +import { Key, Plus, ChevronDown, ChevronUp, X } from 'lucide-react'; +import { sileo } from 'sileo'; +import { ErrorAlert } from './ui/ErrorAlert'; +import { getServiceKeys, createServiceKey, deleteServiceKey } from '../lib/api'; +import { useApiQuery } from '../hooks/useApiQuery'; +import { formatRelative } from '../lib/utils'; + +const SCOPE_STYLES = { + ingest: { + bg: 'bg-surface-tertiary', + text: 'text-text-secondary', + border: 'border-surface-border', + }, + evaluate: { + bg: 'bg-accent/10', + text: 'text-accent', + border: 'border-accent/20', + }, +}; + +function ScopeBadge({ scope }) { + const s = SCOPE_STYLES[scope] || SCOPE_STYLES.ingest; + return ( + + {scope} + + ); +} + +function CopyModal({ rawKey, onClose }) { + const [copied, setCopied] = useState(false); + const clipboardAvailable = typeof navigator !== 'undefined' && !!navigator.clipboard; + + async function handleCopy() { + try { + await navigator.clipboard.writeText(rawKey); + setCopied(true); + setTimeout(() => setCopied(false), 2000); + } catch { + /* handled by fallback input below */ + } + } + + return ( +
+
+
+

Copy your key now — it won't be shown again

+ +
+ +
+ {rawKey} +
+ + {clipboardAvailable ? ( + + ) : ( +
+ e.target.select()} + className="w-full px-3 py-2 text-xs font-mono bg-surface border border-surface-border rounded-lg text-text-primary focus:outline-none focus:ring-1 focus:ring-accent" + aria-label="Service key — select all and copy manually" + /> +

Select all and copy manually (clipboard API unavailable)

+
+ )} + +

+ Make sure you've copied the key before closing +

+
+
+ ); +} + +function SkeletonRow() { + return ( + +
+
+
+
+
+
+ + ); +} + +function EmptyState({ onCreate }) { + return ( +
+ +

No service keys yet

+

+ Create a key to integrate Quorum into your production pipeline. +

+ +
+ ); +} + +function CreateForm({ onCreate, alwaysExpanded = false }) { + const [expanded, setExpanded] = useState(alwaysExpanded); + const [name, setName] = useState(''); + const [scopes, setScopes] = useState({ ingest: true, evaluate: false }); + const [submitting, setSubmitting] = useState(false); + + function toggleScope(scope) { + setScopes((s) => ({ ...s, [scope]: !s[scope] })); + } + + async function handleSubmit(e) { + e.preventDefault(); + const trimmed = name.trim(); + if (!trimmed) return; + const selectedScopes = Object.entries(scopes).filter(([, v]) => v).map(([k]) => k); + if (selectedScopes.length === 0) { + sileo.error({ title: 'Select at least one scope' }); + return; + } + setSubmitting(true); + try { + const result = await createServiceKey(trimmed, selectedScopes); + setName(''); + setScopes({ ingest: true, evaluate: false }); + if (!alwaysExpanded) setExpanded(false); + onCreate(result); + } catch (err) { + sileo.error({ title: err.message || 'Failed to create key' }); + } finally { + setSubmitting(false); + } + } + + if (!alwaysExpanded && !expanded) { + return ( + + ); + } + + return ( +
+ {!alwaysExpanded && ( +
+ New service key + +
+ )} +
+ + setName(e.target.value)} + maxLength={100} + placeholder="e.g. production-monitoring" + required + className="w-full px-3 py-2 text-sm bg-surface border border-surface-border rounded-lg text-text-primary placeholder:text-text-tertiary focus:outline-none focus:ring-1 focus:ring-accent" + /> +
+
+ Scopes +
+ {['ingest', 'evaluate'].map((scope) => ( + + ))} +
+
+ +
+ ); +} + +function RevokeControl({ keyId, onRevoked }) { + const [confirming, setConfirming] = useState(false); + const [loading, setLoading] = useState(false); + + async function handleRevoke() { + setLoading(true); + try { + await deleteServiceKey(keyId); + onRevoked(keyId); + sileo.success({ title: 'Key revoked' }); + } catch (err) { + sileo.error({ title: err.message || 'Failed to revoke key' }); + } finally { + setLoading(false); + setConfirming(false); + } + } + + if (confirming) { + return ( + + Revoke? + + + + ); + } + + return ( + + ); +} + +export function ServiceKeysManager() { + const fetchFn = useCallback((signal) => getServiceKeys(signal), []); + const { data, loading, error, refetch } = useApiQuery(fetchFn); + const allKeys = data?.keys ?? []; + + const [newKey, setNewKey] = useState(null); + const [showRevoked, setShowRevoked] = useState(false); + + const activeKeys = allKeys.filter((k) => !k.revokedAt); + const revokedKeys = allKeys.filter((k) => k.revokedAt); + const visibleKeys = showRevoked ? allKeys : activeKeys; + + function handleCreated(result) { + setNewKey(result.key); + refetch(); + } + + function handleRevoked(keyId) { + refetch(); + if (newKey) setNewKey(null); + } + + return ( +
+

+ + Bearer tokens for server-to-server API access. +

+ + {newKey && ( + { + setNewKey(null); + sileo.success({ title: 'Key created — make sure you copied it' }); + }} + /> + )} + + {error && ( + + )} + +
+ {loading ? ( +
+ + + + + + + + + + + + {Array.from({ length: 3 }).map((_, i) => )} + +
NameScopesPrefixLast UsedCreated +
+
+ ) : !error && activeKeys.length === 0 && revokedKeys.length === 0 ? ( +
+ +
+ ) : ( + <> +
+ + + + + + + + + + + + {visibleKeys.map((k) => { + const revoked = !!k.revokedAt; + return ( + + + + + + + + + ); + })} + +
NameScopesPrefixLast UsedCreated +
+ {k.name} + +
+ {k.scopes.map((s) => )} +
+ 		+ {k.keyPrefix} + + {k.lastUsedAt ? formatRelative(k.lastUsedAt) : 'Never'} + + {formatRelative(k.createdAt)} + + {!revoked && } +
+
+ + {revokedKeys.length > 0 && ( +
+ +
+ )} + +
+ +
+ + )} +
+
+ ); +} diff --git a/frontend/src/components/StreamingEvaluation.jsx b/frontend/src/components/StreamingEvaluation.jsx index 32a2325..de39450 100644 --- a/frontend/src/components/StreamingEvaluation.jsx +++ b/frontend/src/components/StreamingEvaluation.jsx @@ -2,6 +2,7 @@ import { useMemo, useEffect, useRef, useState } from 'react'; import PropTypes from 'prop-types'; import { sileo } from 'sileo'; import { Activity } from 'lucide-react'; +import { useNavigate } from 'react-router-dom'; import { TestCaseResult } from './TestCaseResult'; import { CostBreakdown } from './CostBreakdown'; import { PageHeader } from './PageHeader'; @@ -93,7 +94,8 @@ MetricsPanel.propTypes = { }).isRequired, }; -export function StreamingEvaluation({ events, testCases, currentTestCase, onNavigate, jobId }) { +export function StreamingEvaluation({ events, testCases, currentTestCase, onNavigate, jobId, isDemo }) { + const navigate = useNavigate(); const [tickerItems, setTickerItems] = useState([]); const toastFiredRef = useRef(null); @@ -221,6 +223,13 @@ export function StreamingEvaluation({ events, testCases, currentTestCase, onNavi } /> + {isDemo && ( +
+ + Demo mode — results are simulated +
+ )} + {/* Progress bar — enhanced with accent glow and percentage label */}
@@ -272,6 +281,23 @@ export function StreamingEvaluation({ events, testCases, currentTestCase, onNavi
+ {isDemo && ( +
+
+
+
+

Ready to evaluate your own RAG system?

+

Add your API keys to run real evaluations with full cost visibility.

+
+ +
+
+ )}
)}
diff --git a/frontend/src/components/TestCaseUpload.jsx b/frontend/src/components/TestCaseUpload.jsx index 7e2852c..c5cea84 100644 --- a/frontend/src/components/TestCaseUpload.jsx +++ b/frontend/src/components/TestCaseUpload.jsx @@ -161,7 +161,7 @@ function PreviewModal({ testCases, onClose }) { ); } -export function TestCaseUpload({ onSubmit, isLoading, activeEvaluation, onResumeActive }) { +export function TestCaseUpload({ onSubmit, isLoading, activeEvaluation, onResumeActive, hasKeys, onConfigureKeys }) { const [testCases, setTestCases] = useState([]); const [error, setError] = useState(null); const [strategy, setStrategy] = useState('auto'); @@ -211,6 +211,10 @@ export function TestCaseUpload({ onSubmit, isLoading, activeEvaluation, onResume setError('Please upload test cases or load sample data first'); return; } + if (hasKeys === false) { + setError('No API keys configured. Configure at least one provider key to run evaluations.'); + return; + } onSubmit(testCases, { strategy, name: name.trim() }); }; @@ -219,7 +223,7 @@ export function TestCaseUpload({ onSubmit, isLoading, activeEvaluation, onResume onResumeActive?.(); return; } - onSubmit(DEMO_TEST_CASES, { strategy: 'auto', name: 'Adaptive Demo' }); + onSubmit(DEMO_TEST_CASES, { strategy: 'auto', name: 'Adaptive Demo', demo: true }); }; return ( @@ -382,11 +386,32 @@ export function TestCaseUpload({ onSubmit, isLoading, activeEvaluation, onResume {/* Preview Modal */} {isPreviewModalOpen && setIsPreviewModalOpen(false)} />} + {/* No-keys notice */} + {testCases.length > 0 && hasKeys === false && ( +
+ No API keys configured — real evaluations require at least one provider. +
+ + +
+
+ )} + {/* Run button */} {testCases.length > 0 && (
{bars.map(bar => (
- {bar.label} + {bar.label} {bar.value}%
@@ -117,15 +124,15 @@ function DomainModal({ domain, onClose }) {
- Council is +{delta}pp more accurate than the best single-judge baseline in {domain.label}. + Gemini Flash leads by +{delta}pp over Council in {domain.label}.
@@ -135,13 +142,14 @@ function DomainModal({ domain, onClose }) { export default function BenchmarksPage() { const [activeDomain, setActiveDomain] = useState(null); const [barsVisible, setBarsVisible] = useState(false); + const [showBackToTop, setShowBackToTop] = useState(false); const barsRef = useRef(null); useEffect(() => { document.title = 'Benchmarks — Quorum'; const metas = [ - { property: 'og:title', content: 'Quorum Benchmarks — Council vs Single-Judge RAG Evaluation' }, - { property: 'og:description', content: 'Council-based LLM deliberation achieves 94.2% accuracy across 5,000 RAG test cases — 31% fewer false negatives than single-judge.' }, + { property: 'og:title', content: 'Quorum Benchmarks — RAG Evaluator Comparison' }, + { property: 'og:description', content: 'Gemini Flash achieves 82% accuracy at $0.000402/case — the Pareto-dominant RAG evaluator. Compare council vs. single-judge strategies across 5,000 human-labeled test cases.' }, { property: 'og:type', content: 'website' }, ]; const added = metas.map(attrs => { @@ -162,8 +170,14 @@ export default function BenchmarksPage() { return () => observer.disconnect(); }, []); + useEffect(() => { + const handler = () => setShowBackToTop(window.scrollY > 400); + window.addEventListener('scroll', handler, { passive: true }); + return () => window.removeEventListener('scroll', handler); + }, []); + return ( -
+
+ Skip to content +
+ +
{/* Top bar */}
@@ -148,7 +281,7 @@ export default function PaperPage() { rel="noopener noreferrer" style={{ fontSize: '0.75rem', fontWeight: 600, padding: '3px 10px', borderRadius: 99, background: 'var(--bg-surface)', color: 'var(--text-ter)', border: '1px solid var(--card-border)', textDecoration: 'none', display: 'inline-flex', alignItems: 'center', gap: '4px' }} > - GitHub +
@@ -167,7 +300,7 @@ export default function PaperPage() {
{/* Table of Contents */} -
+

Contents

    {[ @@ -522,7 +655,7 @@ export default function PaperPage() {
    {/* Bottom CTA */} -
    +

    arXiv submission pending · Preprint, March 2026

    @@ -538,12 +671,85 @@ export default function PaperPage() { fontSize: '0.875rem', fontWeight: 600, textDecoration: 'none', }} > - GitHub Repo +
    +
+ + {tocOpen && ( +
setTocOpen(false)} + style={{ + position: 'fixed', inset: 0, zIndex: 30, + background: 'rgba(59,60,54,0.4)', backdropFilter: 'blur(4px)', + }} + > +
e.stopPropagation()} + style={{ + position: 'absolute', bottom: 0, left: 0, right: 0, + background: 'var(--bg-surface)', borderRadius: '16px 16px 0 0', + padding: '1.5rem 1.5rem 2.5rem', + maxHeight: '70vh', overflowY: 'auto', + }} + > +

+ Contents +

+ {TOC_SECTIONS.map(s => ( + setTocOpen(false)} + style={{ + display: 'block', fontSize: '1rem', + color: activeSection === s.id ? 'var(--accent)' : 'var(--text-primary)', + fontWeight: activeSection === s.id ? 600 : 400, + textDecoration: 'none', + padding: '0.625rem 0', + borderBottom: '1px solid var(--card-border)', + }} + > + {s.label} + + ))} +
+
+ )} + + + + {showBackToTop && ( + + )}
); } diff --git a/frontend/src/pages/SettingsPage.jsx b/frontend/src/pages/SettingsPage.jsx new file mode 100644 index 0000000..628271d --- /dev/null +++ b/frontend/src/pages/SettingsPage.jsx @@ -0,0 +1,64 @@ +import { NavLink, Outlet } from 'react-router-dom'; +import { PageHeader } from '../components/PageHeader'; +import { useAuth } from '../context/AuthContext'; + +const TABS = [ + { id: 'api-keys', label: 'API Keys', path: '/app/settings/api-keys' }, + { id: 'service-keys', label: 'Service Keys', path: '/app/settings/service-keys' }, + { id: 'account', label: 'Account', path: '/app/settings/account' }, +]; + +const activeClass = 'px-4 py-2 text-sm font-medium text-text-primary border-b-2 border-accent -mb-px focus:outline-none focus:ring-2 focus:ring-accent/20 rounded-t-sm'; +const inactiveClass = 'px-4 py-2 text-sm text-text-secondary hover:text-text-primary transition-colors focus:outline-none focus:ring-2 focus:ring-accent/20 rounded-t-sm'; + +export function SettingsPage() { + return ( + <> + + +
+ +
+ + ); +} + +export function AccountTab() { + const { user } = useAuth(); + if (!user) return null; + + return ( +
+
+

Profile

+
+
+

Username

+

{user.username}

+
+
+

Email

+

{user.email}

+
+
+
+ +
+

Password & Security

+

Password management coming soon.

+
+
+ ); +}