Conversation
…old numbers (#2346) #2206 renumbered six colliding spec directories to 087-092 and #2345 rewrote their headings, but each still names its OLD test file in the body of spec.md/plan.md -- a file that no longer exists under that number: 087-benchmark-tie-order-share test_spec_042_tie_order_share -> 087 088-benchmark-repo-task-mean test_spec_046_repo_task_mean -> 088 089-benchmark-partition-task-share test_spec_047_partition_task_share -> 089 090-benchmark-repo-task-mean test_spec_048_repo_task_mean -> 090 091-benchmark-improvement test_spec_068_improvement -> 091 092-agent-context test_spec_011_agent_context -> 092 That is the same renumbering drift #2345 closed for headings, just in the in-body "Tests" cross-reference: a reader following it lands on a dead path (or, for 011, a different spec's test). Repoint each reference at the correctly-numbered test file that actually exists (tests/test_spec_08X_*.py / 09X), matching the directory. Docs only; no code or test change. Refs #2143
… not an old number (#2347) test_spec_numbering already enforces unique directory numbers and that each doc's HEADING matches its directory (#2143 / #2345). It does not enforce the same for the in-body `tests/test_spec_NNN_*.py` cross-reference, so a renumbered spec can keep pointing a reader at a number another spec now owns -- exactly the drift #2346 had to repair by hand after the 087-092 renumbering. Add test_spec_documents_reference_their_own_test_file: scan every spec.md/plan.md for test_spec_NNN references and assert NNN equals the directory number. Passes on the current tree (all references were repointed in #2346) and fails closed the next time a renumber leaves a reference behind. Tests only; no product change. Refs #2143
The first product-runtime deliverable: a private control plane that turns vanguarstew from an operator-run dev tool into a self-hostable maintainer-assist service. In this phase it NEVER mutates GitHub -- it accepts work, stores private results locally, and exposes only loopback health checks. Additive and self-contained: depends only on modules already on test (benchmark.sealed_*, agent.review.review_pr, agent.llm), touches no scored agent/ or benchmark/ code, and adds no third-party runtime dependency (stdlib only). - vanguarstew_runtime/: cli, config (env-only secrets), state (durable SQLite queue), github (read-only client), service (signed webhook intake + loopback /healthz /readyz). - openvang/: agent factory, scheduler, subnet, isolated execution, memory. - deploy/ + Dockerfile + docker-compose.yml: Compose/systemd templates. - ignore files exclude private runtime state (.env, data/, *.sqlite3, private-review-results/) from git and build contexts. - pyproject: 'vanguarstew' console entry point + the two new packages. 45 runtime/openvang tests, ruff clean. Scope note: first of several PRs splitting a large local branch. The .dockerignore change here only APPENDS the runtime ignore entries and preserves the attested eval image's existing allowlist; the full rewrite is held for the attestation review where that TCB tradeoff belongs.
openvang/memory.py requires the optional 'cryptography' package. The CI test job installs only pytest/pytest-cov, so the openvang memory tests errored on a clean runner (ModuleNotFoundError) even though they pass where the dep is present. Install cryptography in the test job, and declare it as a pyproject extra (dev + private-memory) so real installs get it. Runtime code stays stdlib-only; this is a test-time dependency of the openvang memory feature.
feat(runtime): private local product runtime + openvang agent factory
…rand Second slice of the local branch (runtime landed in the prior PR). Three coherent-but-interwoven changes that touch the same core files and so land together: - Persistent memory: benchmark/memory.py + source_memory.py + memory_coverage.py + ablation.py, threaded through agent/context.py, benchmark/runner.py, and scripts/run_eval.py, with a receipt-safe memory_commitment bound into benchmark/attestation.py (never binds raw recalled content). Memory-coverage and ablation protocols + their CLIs and tests. - Review-policy migration: REVIEW.md + CONTRIBUTING.md content moves into specs/009-agent-review; CODEOWNERS, the PR template, and the review scripts (agent/review.py, review_pr.py, benchmark_pr_policy.py, pr_reopen_policy.py) updated to match. - Identity: pyproject authors + URLs move gittensor-vanguard -> openvang, completing the org transfer. Blog posts are kept (only spec-driven-development.md is content-updated). Rebased onto the runtime PR's test; inherits its cryptography CI install.
feat: persistent agent memory + review-policy migration; openvang rebrand
The root .dockerignore was a deny-all-then-allowlist scoped to the attested eval image. With the openvang/vanguarstew_runtime product runtime now in-tree, the product Dockerfile's `COPY . /app` was silently dropping both packages (they were never on the eval allowlist), shipping a product image that can't import its own entrypoint. Give the eval image its own build-context ignore at docker/eval.Dockerfile.dockerignore (BuildKit uses a <dockerfile>.dockerignore in place of the context-root one when building `-f docker/eval.Dockerfile`), holding the exact same deny-all-then-allowlist rules as before -- so the attested TCB is unchanged, just relocated. The root .dockerignore becomes a normal product deny-list: secrets, receipts, VCS metadata, caches and dev/test artifacts stay out, all source ships. - publish-eval-image.yml now triggers on the eval-specific ignore (the file that actually governs the eval TCB) rather than the root one. - test_runtime_packaging.py gains two guards: the product context must ship the runtime packages and must not be a deny-all-allowlist; the eval context must stay a deny-all-allowlist with no private runtime state on it.
docker: split eval TCB allowlist from product .dockerignore
github-actions
Bot
added
agent
|
Agent benchmark smoke check (offline stub — crash/shape only, not a reasoning-quality score)
This does not determine a perf:* label. That requires a maintainer-bot-run live benchmark comparison against both the public and private repo targets — see the benchmark policy. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Routine promotion of
test→main. Payload — 10 commits since #2344:Features
Fixes
.dockerignoreso the product image ships the runtime packages (docker: split eval TCB allowlist from product .dockerignore #2375)Infra / docs
cryptographyfor the openvang private-memory testsNo version bump or leaderboard re-anchor — v0.8.0 stays the released tag; those remain separate deliberate steps.