[Testing 10] test: offline eval harness — citation accuracy, injection resistance, privilege leakage

[amal66]

Summary

The offline eval harness from the reference index (#205, row amal66#45): deterministic prompt-safety and behavior gates that run against committed fixtures — no network, no LLM calls, no secrets. Fully standalone (no dependency on any other PR in the testing series; doesn't even need the vitest harness).

Changes

• evals/ — runner (run.mjs), engine, scorers, fixture runner, golden dataset (8 cases + JSON schema), fixture model outputs, and a README covering how to add cases and how to plug in a live-model runner later.
• Checks cover: citation accuracy against source text, prompt-injection resistance, and privilege/PII leakage.

Why

An LLM legal product can regress in ways unit tests can't see — a prompt tweak that starts leaking privileged content or following injected instructions. This gives a cheap, deterministic scorecard for exactly those behaviors, gated at --threshold 1.0, cheap enough to run on every PR (wired up by the CI workflow PR in this series, but runnable on its own today).

Testing

• node evals/run.mjs --threshold 1.0 — 8/8 cases passing locally on this branch.
• The failure gate works: the fork row proved a failing fixture fails the run (non-zero exit).

Based directly on main; no shared commits with the other testing PRs.

🤖 Generated with Claude Code

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.

---

QA Runner seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[willchen96]

Thanks for this. Going to reject this for now due to limited usefulness of the evals and format. A fuller real evals suite can be implemented in the future which aligns with the format of actual assistant responses and involves real documents.