Skip to content

feat: add adversarial prompt-injection document corpus - #27

Merged
arthurpanhku merged 1 commit into
arthurpanhku:mainfrom
yuehua1213:avora/issue-20-adversarial-corpus
Aug 12, 2026
Merged

feat: add adversarial prompt-injection document corpus#27
arthurpanhku merged 1 commit into
arthurpanhku:mainfrom
yuehua1213:avora/issue-20-adversarial-corpus

Conversation

@yuehua1213

Copy link
Copy Markdown
Contributor

What changed

  • add an MIT-licensed manifest of six synthetic indirect prompt-injection samples
  • cover TXT, Markdown comments, metadata, hidden DOCX table text, hidden XLSX rows, and policy-looking PDF text
  • add a deterministic offline generator for all declared document formats
  • add regression tests for manifest quality, expected behavior, generated artifacts, and hidden carrier preservation

Why

This implements the initial corpus requested in #20. DocSentinel already tests several injection strings directly, but it did not have a reusable, documented, multi-format corpus that guardrail changes could evaluate offline.

Each sample declares a trusted goal, attack carrier, synthetic payload, risk classification, and the same expected safety boundary: document content remains untrusted data and cannot change the deterministic rule-engine decision.

Validation

  • Before implementation: test collection failed because the corpus generator did not exist
  • After implementation: 4 passed
  • Ruff: all changed Python files pass
  • No external LLM or network call is used
python -m pytest --noconftest tests/test_prompt_injection_corpus.py -q
ruff check scripts/generate_prompt_injection_corpus.py tests/test_prompt_injection_corpus.py

The contribution was prepared with AI assistance and manually reviewed and tested, following CONTRIBUTING_WITH_AI.md.

Closes #20.

@yuehua1213
yuehua1213 marked this pull request as ready for review August 8, 2026 11:04
@arthurpanhku
arthurpanhku merged commit 6ad19ae into arthurpanhku:main Aug 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create an adversarial document corpus for prompt-injection testing

2 participants