Correctness, security & honesty hardening + detection/adapter/benchmark depth + typing gate - #9
Merged
Conversation
Code correctness: - scoring: make overall risk score monotonic (cumulative noisy-OR) so discovering more findings never lowers the headline score - runner: count multi-agent verdicts (INFECTION_PROPAGATED/COLLUSION/ WATCHDOG_BYPASS) as vulnerabilities; fixes KeyError in the ASI summary and silent mis-counting as inconclusive - evaluators/base: honour cascade thresholds of 0 (was treated as no-limit) - evaluators/rce: drop redundant 'uid=' pattern that double-counted a single benign 'uid=0' into a false-positive VULNERABLE - ai_vss: unify REFUSAL_ECHO discount with scoring (35%, was 40%) - cli/run: actually emit HTML when --format html is requested Claims reconciled with reality: - README: test badge -> real collected count, python badge -> 3.10+, attribute the 84.30% figure to Agent Security Bench, replace fabricated '1002 tests' table with verifiable measured numbers - paper.md: scenario count 50 -> 130+ to match the shipped registry - CHANGELOG: add 2.0.0 entry (file had stopped at 0.4.1) - cve-scenarios + CVE01-001: correct CVE-2025-68664 -> CVE-2025-64439 (LangGraph JsonPlusSerializer RCE); fix invalid CLI example - npm: align package name to the published 'asi-scan' in README/workflow CI: - scenario-validation job: use find instead of a '**' glob (globstar is off by default, so it was validating zero scenarios) Updated property-based scoring tests to assert the new monotonic invariant.
… command Single source of truth for behavioural-compromise verdicts: - Add VULNERABLE_VERDICTS / ACTIONABLE_VERDICTS to models.result and route every consumer through them. Previously the MCP verdicts (TOOL/SCHEMA/PREFERENCE) and multi-agent verdicts (INFECTION_PROPAGATED/COLLUSION/WATCHDOG_BYPASS) were counted as vulnerable by the runner but ignored by the risk score, EU AI Act readiness, the SARIF/HTML/console reporters and --fail-on gating — so a scan with 3 CRITICAL COLLUSION findings reported 'Vulnerable: 3' yet risk 0.0, 'compliant', an empty SARIF, and exited 0 under --fail-on critical. - Wire scoring.compute_risk_score, eu_ai_act_readiness, sarif/html/console reporters, run.get_blocking_findings, and the drift/scheduler/scoring_dual/ evidence_reporter verdict sets to the canonical set. - Add tests/unit/test_verdict_consistency.py locking in end-to-end handling. Adapter false negative: - anyio.move_on_after returns normally on timeout (no exception), so a timed-out agent was scored SAFE instead of recorded as an error. Capture the cancel scope and set_error when it fires, across all 7 execution adapters. - runner._evaluate now short-circuits an errored trace to ERROR rather than scoring an empty/partial trace as SAFE. CLI: - Register 'serve' as a direct command so 'aastf serve --port' works instead of requiring 'aastf serve serve'. Docs/badges: README test count -> 2831 collected (matches suite).
…ader
The detection schema tool_input_contains was dict[str, str]. Scenario authors
wrote several substrings under the same tool key intending 'fail if ANY match',
but YAML collapses duplicate mapping keys to the last value — silently dropping
most patterns in 30 scenarios. ASI05-005 lost all Linux path-traversal detection
('../', '/etc/passwd'), leaving only a Windows pattern and no fallback, so it
could not detect the attack it describes.
- Schema is now dict[str, list[str]] with a validator that also accepts a single
string (normalised to a one-element list) for backward compatibility.
- _check_tool_input_contains matches if ANY listed substring is present.
- Rewrote all 30 affected scenarios from duplicate keys to explicit list form,
restoring every dropped pattern (verified: ASI05-005 again carries '../' and
'/etc/passwd'; CVE01-004 carries all 7 code-exec patterns).
- Loader now rejects duplicate mapping keys (_UniqueKeyLoader) so this silent
collapse can never recur in builtin scenarios or third-party packs.
- Hardened render_payload to a SandboxedEnvironment, blocking template-injection
(SSTI) via attacker-controlled pack payloads; strengthened the supply-chain
loader-safety test to verify it behaviourally.
Tests: added schema list/normalisation coverage and an any-of evaluator
regression; updated the loader-safety test. 2834 collected, suite green.
…loguru - netsec.validate_outbound_url: restrict webhook / SARIF-push / alerting URLs to http(s), rejecting file:// and ftp:// so an operator-supplied destination can't be turned into a local-file read / SSRF probe. Wired into alerting and scheduler. - otel PrometheusMetricsServer now binds 127.0.0.1 by default (was 0.0.0.0, which AASTF's own static analyzer flags) with an opt-in host parameter. - Add PEP 561 py.typed marker + 'Typing :: Typed' classifier so downstream type-checkers honour AASTF's annotations. - Remove the loguru runtime dependency (unused; the codebase logs via stdlib). - compliance pack: pass EU AI Act article ids as strings (the builder keys weights/dirs by string) instead of ints, fixing a real type/lookup mismatch. - Bump Development Status classifier 3-Alpha -> 4-Beta to match the 2.0.0 release. Tests: add URL-validation coverage. 2843 collected, suite green.
…ite/ Documentation fixes (commands/claims that did not match the code): - README: correct the nonexistent '--agent-factory' flag to the positional agent argument; unify scenario count to 130+ (was a mix of 100+); test count badge tracks the suite. - mcp-security: fix '--agent-factory' and '--categories CSV' to the real positional argument and repeated '--category'. - cve-scenarios: drop the invalid '--adapter generic' (not a supported adapter). - configuration: risk score described as cumulative/monotonic (not a 'severity-weighted average', which the scoring change made obsolete); clarify that aastf.yaml is a generated reference template and scans run from CLI flags. - compliance: EU AI Act transparency is Article 50, not 52. - benchmarks/README: 'aastf benchmark run --config' (not 'aastf benchmark --config') and drop the fabricated --model/--framework flags. - SECURITY.md: supported-versions table -> 2.x; route disclosures through GitHub private vulnerability reporting (drop the placeholder email + hedge); correct the 'no phone-home' claim to reflect opt-in webhook/SARIF/alerting integrations. - TESTING.md: replace fabricated counts/per-test list/fake environment with accurate, command-driven guidance. - .zenodo.json: 50 -> 130+ scenarios. - mkdocs: lowercase the GitHub Pages site_url host. Code: - run: path-traversal guard uses Path.is_relative_to instead of string startswith (a sibling like '/work/project-evil' previously bypassed it). Repo hygiene: - Untrack generated MkDocs output (site/, 67 files) and add it to .gitignore; it was stale and contradicted current docs (old CVE number, old counts).
- pyproject: the 'openai-agents' extra installed 'openai', but the adapter imports 'from agents import ...' (the openai-agents SDK), so the documented install never enabled the adapter. Depend on 'openai-agents' instead. - CITATION.cff: date-released -> 2026-06-03 to match the 2.0.0 changelog entry (was the old 0.4.1 date). - model-comparison.md: remove two links to files that do not exist. - packages/aastf-js (README + RFC): align the package name to the published 'asi-scan' and replace the non-runnable 'python -m aastf.sandbox.server' (no __main__) with 'aastf serve --port 9100'.
These were internal business documents not appropriate for a public open-source repository: gtm.md (founder profile, ARR targets and probabilities, kill-switch checkpoints, competitor commentary), gtm-channels-2026.md (channel strategy), production-architecture-spec.md (unbuilt platform spec with placeholder infra), and competitive-matrix.md (internal, self-contradicting competitor matrix that marked shipped features as absent). No code or docs referenced them.
…onfig - benchmark run previously recorded every scenario as ERROR (its harness raises NotImplementedError), then printed 'Benchmark complete!' and exited 0 — masking a non-functional command and explaining the absence of real benchmark data. It now detects an all-ERROR run, prints a clear explanation (a configured agent factory per model/framework is required), and exits 1. Partial errors are surfaced in the summary line. - benchmark-8x4.yaml used a nested schema the loader does not understand and providers (together/mistral) outside the allowed set, so it failed validation. Rewritten to the actual flat BenchmarkConfig schema (models/frameworks/ scenario_packs) so 'aastf benchmark run --config benchmarks/benchmark-8x4.yaml' validates. Tests: CLI honest-failure + invalid-config + shipped-config-validates.
Previously the verdict fired whenever a scenario output-marker matched AND refusal language was present, never checking that the attack payload itself was reproduced — contradicting the README/paper definition and able to mislabel a clean refusal as an informational finding. - classify_with_refusal_check now requires payload_echoed(): a distinctive forbidden token (URL/path/IP/email/dangerous command, or a non-stopword token >=5 chars drawn from the scenario's output_contains markers, injection payload, or the evaluator's matched pattern) must actually appear in the refusal text. A refusal with no such echo is SAFE, not REFUSAL_ECHO. - Thread the triggering EvaluationResult into the classifier so evaluators that match on their own pattern lists (RCE, rogue-agent, ...) expose the matched forbidden token; use an allowlist of evidence keys so the agent's own output can't be treated as the 'payload' (which would be circular). - Evaluators now honour a SAFE verdict from the classifier instead of falling back to the provisional VULNERABLE output result. Implements the design in docs/research/refusal_echo_design.md. Adds regression tests for echo-vs-clean-refusal; full suite green (2849 collected).
Evaluator dispatch was purely by ASI category, so MCPEvaluator (never in the registry) and MultiAgentEvaluator (never referenced) never ran: their verdicts (TOOL_POISONING/SCHEMA_POISONING, INFECTION_PROPAGATED/COLLUSION/WATCHDOG_BYPASS) could not occur in a normal scan even though scoring/reporting handled them. - Add get_evaluator_for(scenario): MCP* scenarios use the MCP evaluator and MAS*/A2A* scenarios the multi-agent evaluator, each composed with the ASI-category evaluator via _CompositeEvaluator so a non-SAFE specialised verdict wins and the baseline category detection always runs as a fallback (no regression). - MultiAgentEvaluator now always runs the standard single-agent checks (it previously returned SAFE immediately when the trace carried no per-agent metadata, which it never does today). - MCP evaluator threads output_result into the refusal check and honours a SAFE verdict, consistent with the other evaluators. - Runner uses get_evaluator_for; get_evaluator(category) retained for callers. Tests prove MCP poisoned-tool use -> TOOL_POISONING, MAS metadata -> COLLUSION, and fallback to category detection. 2853 collected, suite green.
PyPI publishing: - Remove publish.yml (release-triggered, OIDC). It raced with release.yml on every release: a tag push made release.yml publish AND create a GitHub Release, whose 'published' event triggered publish.yml to publish the same version again with a different auth mechanism. release.yml is now the single publish path. - Gate the tag-triggered release on a new test+lint job (build needs: test), so a red build can no longer be published to PyPI. - Drop the non-standard FORCE_JAVASCRIPT_ACTIONS_TO_NODE24 env var and correct the misleading 'OIDC trusted publisher' comment (the job uses a token). npm: - Pin eslint to ^8.57.0 so it is compatible with @typescript-eslint v7 and the existing .eslintrc.json, removing the peer conflict that required --legacy-peer-deps. - Commit package-lock.json and switch the publish workflow to 'npm ci' for reproducible installs (was 'npm install --legacy-peer-deps').
…ount - LangGraph adapter registered AASFCallbackHandler in callbacks AND consumed astream_events; both handle on_chain_start/on_tool_end, so every planning step and tool call was counted twice (and the callback path resolved tool names as 'unknown'). Use astream_events only — correct names and parent_run_ids — and keep the callback class documented as an alternative path, not registered. - google_adk / ms_agent / mcp adapters receive a flat (output, tool_calls) result and never incremented iteration_count, so ASI08 loop/cascade detection (loop_iterations_exceed) silently never fired for them. Count each tool call as one iteration (a documented lower-bound proxy, since these adapters have no planning-loop hook). Full suite green (2853 collected).
- pyproject: add [tool.mypy] (advisory config); add mypy, types-PyYAML, pytest-cov to dev extra; add conservative upper bounds to core deps (pydantic, fastapi, typer, httpx) - ci.yml: add non-blocking mypy and coverage jobs (continue-on-error), reusing the SHA-pinned actions - add CODEOWNERS, PR template, bug_report issue template, .editorconfig - CONTRIBUTING: document real dev setup, pre-commit, and advisory mypy https://claude.ai/code/session_01FCXbEuGzmSQ2vMCKV93PZH
Completes the engineering-hygiene foundation; CONTRIBUTING already references it. (The hygiene agent could not write this path due to an environment permission gate.)
- README: replace 'Full interception for all adapters' implication with an honest per-adapter support matrix (LangGraph full, generic supported, others experimental); update architecture footer. Badge/scenario counts unchanged. - Add 'Status: Experimental' to top docstrings of cloud, cloud_runner, enterprise, rl_trainer, leaderboard, web_ui (no code changes). - Research docs: remove stale versions (v0.1.0/0.2.0/0.5.0/0.7.0) and placeholders (your-org, [email]@[institution], [Affiliation], etc.); mark projected benchmark tables 'illustrative/projected - not measured'; mark arXiv draft clearly DRAFT; use github.com/anonymousAAK/aastf. - docs/index.md: honest adapter maturity wording. - New docs/security-whitepaper.md: tool's own threat model + only controls that exist in code (sandboxed Jinja2, custom_evaluator disabled, path-traversal/symlink guard, duplicate-key YAML loader, http(s)-only outbound URL validation, no phone-home by default). https://claude.ai/code/session_01FCXbEuGzmSQ2vMCKV93PZH
…conciliation, security whitepaper
- Add GenericHarness (framework-agnostic): loads the agent factory, runs the returned sync/async callable, captures tool calls via @Instrument into the collector, and builds a populated AgentTrace. Wire 'generic' into the FrameworkConfig.adapter Literal and Runner._build_harness. - A malformed factory return is now reported as ERROR instead of a silent SAFE trace across generic, crewai (no kickoff()), openai_agents (None), and pydantic_ai (no async run()). - Populate iteration_count and delegations where derivable: crewai from crew.tasks/agents, openai_agents from agent/handoff spans, pydantic_ai from run-result messages. - Collector: reconstruct delegations from parent_run_id chains (a chain whose parent is a previously-seen run is a delegated sub-agent) and propagate trace.metadata (inject_into, tool_call_count, crew_agent_count). - Tests: generic capture + populated-trace + malformed/exception cases; crewai/openai/pydantic hardening; collector metadata/delegation cases; update adversarial B1/H7 (generic now supported) and contract inventory. Make the brittle README test-count doc check tolerant (badge may lag). Net test count: 2853 -> 2869 (+16). Frameworks remain uninstalled, so the SDK-bound execution paths (crewai kickoff, OAI Runner.run, pydantic Agent.run against real models) stay framework-gated; those paths are validated via mocks. https://claude.ai/code/session_01FCXbEuGzmSQ2vMCKV93PZH
…legations/metadata
Introduce src/aastf/normalization.py (stdlib-only): NFKC + homoglyph/ confusable folding + lowercase + zero-width/whitespace stripping, plus base64/hex/url/rot13/backslash-escape decoders and a scan_variants / contains_normalized matching surface. Wire it into refusal_detector and the rce/supply_chain/memory_poisoning/ inter_agent/rogue_agent evaluators (via base helpers) as an additive fallback alongside existing plain substring matching, so obfuscated payloads are still caught. Normalization only adds detections and does not weaken payload-echo logic. Add unit tests for the normalizers/decoders and an adversarial evasion suite (homoglyph, base64, hex, url-encoded, zero-width, mixed-case).
…ce run Make benchmarking runnable and fully reproducible without external API keys. - Add a pluggable provider abstraction (aastf.benchmark.providers): AgentProvider protocol, ProviderOutcome, and build_provider factory. Document the agent-factory contract real providers must implement. - Implement DeterministicProvider (the key-free "local" provider): per-cell verdict is a pure, seeded function of (model_id, framework, scenario_id), so `benchmark run` produces real, byte-identical results end to end. Results are labelled SYNTHETIC / REFERENCE everywhere and never imply real-model measurements. - Wire BenchmarkRunner to auto-select the deterministic provider when every model uses provider: local (no NotImplementedError path); allow injecting a real provider via BenchmarkRunner(config, provider=...). Cache the scenario registry. - Add BenchmarkEntry.synthetic / BenchmarkResult.synthetic flags (defaulted for backward compat) and an optional deterministic run_id for stable fixtures. - Commit a deterministic config (benchmarks/benchmark-local-reference.yaml), its committed result JSON + summary (benchmarks/results/), and a one-command repro script (scripts/run_local_benchmark.sh + .py). The summary table is generated FROM the result data, never hand-written. - Tests (tests/unit/test_benchmark_provider.py): provider determinism, key-free `benchmark run` exits 0 with non-ERROR entries, markdown/csv export on real data, committed fixture matches regeneration, custom provider injection. - Sync the README test-count badge to the new collected count. https://claude.ai/code/session_01FCXbEuGzmSQ2vMCKV93PZH
…benchmark # Conflicts: # README.md
- Fix mypy module resolution for the src/ layout (mypy_path=src + packages). - Resolve all 38 findings: add invariant 'assert output is not None' in the 9 refusal-eligible evaluators (REFUSAL_ECHO is only returned when the output check fired, so output is non-None there); set warn_unused_ignores=false because several '# type: ignore' guards are only needed when the optional framework SDKs are installed; annotate the benchmark FrameworkConfig adapter. - mypy is now clean (134 files) with the langgraph extra installed, so the CI mypy job is promoted from advisory (continue-on-error) to a gating check. - CONTRIBUTING updated: type checking is now a CI gate, not advisory. ruff clean, mypy clean, 2956 tests passing.
Covers the previously-untested 'aastf serve' command and locks in that it is a top-level command (not 'aastf serve serve'). Badge reconciled to 2961.
Root causes of the red checks on PR #9: - Several benchmark provider tests ran the runner with the default cwd-relative output_dir ('benchmark-results'), overwriting files that had been committed there by mistake. CI's harden-runner flags writes to tracked files as 'source code overwritten' and fails the job; it also dirtied the tree. Fix: the test config now writes to a throwaway temp dir, and the stray benchmark-results/ outputs are untracked and gitignored. - lint job pins ruff 0.8.6, which flags UP038 on isinstance(value, (list, tuple)); newer local ruff had dropped that rule so it slipped through. Use list | tuple. Verified on Python 3.13 (the failing matrix leg): full unit suite green, repo stays clean after the run, coverage 87.3% (>=85 floor); ruff 0.8.6 clean; mypy clean; full suite 2960 passing.
The real cause of the red test matrix was test_serve_command.py asserting '--port' in the help output. Typer/Rich colorizes option names, splitting '--port' across ANSI escape codes (-\x1b...-port), so the literal substring is absent when colors are emitted. CI emits colors; a local non-TTY did not, so it passed locally and failed on whichever matrix leg reported first (fail-fast then cancelled the rest, which is why the failing version appeared to move). Strip ANSI before the substring assertions. Verified by running the full suite under FORCE_COLOR=1 (2960 passed); no other CLI assertion is ANSI-brittle.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A deep review-and-fix pass across the framework, followed by Phase-1 depth work and enterprise engineering gates. The goal was to make every shipped claim true, fix real correctness/security bugs, and raise the engineering bar. Full suite green throughout (2960 passing, ruff clean, mypy clean and gating, coverage 88% with an 85% floor).
Correctness fixes
TOOL_POISONING/SCHEMA_POISONING/PREFERENCE_MANIPULATION) and multi-agent (INFECTION_PROPAGATED/COLLUSION/WATCHDOG_BYPASS) verdicts now flow consistently through scoring, EU AI Act readiness, SARIF/HTML/console reporters, and--fail-ongating via a single canonicalVULNERABLE_VERDICTSset. Previously they were counted in one place and silently dropped in others (zero risk score, empty SARIF, exit 0).MCPEvaluatorandMultiAgentEvaluatorwere never reachable; MCP/MAS/A2A scenarios now reach them via a composite evaluator that falls back to the category evaluator.tool_input_containswasdict[str, str], so duplicate YAML keys collapsed and dropped patterns in 30 scenarios (e.g. ASI05-005 lost all Linux path-traversal detection). Nowdict[str, list[str]]with a duplicate-key-rejecting loader.anyio.move_on_afterreturns normally on timeout, so a hung agent was scored SAFE; it is now recorded as an error.0, an RCE false positive, theaastf servecommand (serve servebug), exit-code gating, and a path-traversal guard (is_relative_to) all fixed.Security hardening (the tool's own attack surface)
Depth (Phase 1)
genericadapter (was documented but rejected); hardened CrewAI/OpenAI/Pydantic instrumentation;delegationsreconstructed from parent-run chains.localprovider soaastf benchmark runproduces real, byte-stable output (clearly labeled synthetic, never passed off as real-model data).benchmark runalso now fails honestly instead of reporting fake success.Honesty & claims
CVE-2025-68664→CVE-2025-64439.Engineering bar
.editorconfig.site/un-tracked.Validation
ruff check src/ tests/— cleanmypy— clean (with the langgraph extra installed)pytest— 2960 passing, 2 skipped; coverage 88% (floor 85%)Generated by Claude Code