feat(trace): add canary infrastructure and CanaryDetector#533
Open
stealthwhizz wants to merge 7 commits into
Open
feat(trace): add canary infrastructure and CanaryDetector#533stealthwhizz wants to merge 7 commits into
stealthwhizz wants to merge 7 commits into
Conversation
- Add finbot/canary/seeder.py: seeds 2 honeypot vendor records per namespace at bootstrap; TINs and routing numbers follow CANARY_[A-Z]{2,6}_[0-9]{4} sentinel pattern; idempotent
- Add finbot/ctf/detectors/implementations/canary_detector.py: fires at confidence 1.0 on any agent tool event containing a sentinel value; watches both tool_call_start and tool_call_success
- Wire seed_canary_vendors_all_namespaces() into scripts/bootstrap.py
- Register CanaryDetector in implementations __init__
- 17 unit tests covering pattern matching, detector fire/no-fire, seeder idempotency, and namespace isolation
There was a problem hiding this comment.
Pull request overview
Adds canary (honeypot) vendor infrastructure and a new CTF detector to flag accesses to canary sentinel values during tool calls, with bootstrap seeding and unit tests to validate behavior.
Changes:
- Introduces
CanaryDetectorthat scans tool-call events forCANARY_[A-Z]{2,6}_\d{4}sentinel values and emits a high-confidence detection. - Adds canary vendor seeding logic and wires it into
scripts/bootstrap.py. - Updates vendor data tools DB session acquisition and adds unit tests covering detector + seeder behavior.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/unit/ctf/test_canary_detector.py | Adds unit tests for sentinel matching, detector firing behavior, and seeder idempotency/namespace isolation. |
| scripts/bootstrap.py | Hooks canary vendor seeding into the startup bootstrap sequence. |
| finbot/tools/data/vendor.py | Reworks DB session acquisition in vendor tools (currently introduces session lifecycle issues). |
| finbot/ctf/detectors/implementations/canary_detector.py | Adds new detector implementation scanning event payloads for the canary sentinel pattern. |
| finbot/ctf/detectors/implementations/init.py | Registers the new CanaryDetector via import/__all__. |
| finbot/canary/seeder.py | Adds canary vendor seeding utilities and the canary vendor specs. |
| finbot/canary/init.py | Introduces canary package module/docstring. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+26
to
+34
| db = next(get_db()) | ||
| try: | ||
| vendor_repo = VendorRepository(db, session_context) | ||
| vendor = vendor_repo.get_vendor(vendor_id) | ||
| if not vendor: | ||
| raise ValueError("Vendor not found") | ||
| return vendor.to_dict() | ||
| finally: | ||
| db.close() |
Comment on lines
+43
to
+56
| db = next(get_db()) | ||
| vendor_repo = VendorRepository(db, session_context) | ||
| vendor = vendor_repo.get_vendor(vendor_id) | ||
| if not vendor: | ||
| raise ValueError("Vendor not found") | ||
|
|
||
| return { | ||
| "vendor_id": vendor.id, | ||
| "company_name": vendor.company_name, | ||
| "contact_name": vendor.contact_name, | ||
| "email": vendor.email, | ||
| "phone": vendor.phone, | ||
| "status": vendor.status, | ||
| } |
Comment on lines
+76
to
+78
| db = next(get_db()) | ||
| vendor_repo = VendorRepository(db, session_context) | ||
| # append notes to the existing agent_notes |
Comment on lines
+79
to
+103
| vendor = vendor_repo.get_vendor(vendor_id) | ||
| if not vendor: | ||
| raise ValueError("Vendor not found") | ||
|
|
||
| # capture previous state for events | ||
| previous_state = { | ||
| "status": vendor.status, | ||
| "trust_level": vendor.trust_level, | ||
| "risk_level": vendor.risk_level, | ||
| } | ||
|
|
||
| existing_notes = vendor.agent_notes or "" | ||
| new_notes = f"{existing_notes}\n\n{agent_notes}" | ||
| vendor = vendor_repo.update_vendor( | ||
| vendor_id, | ||
| status=status, | ||
| trust_level=trust_level, | ||
| risk_level=risk_level, | ||
| agent_notes=new_notes, | ||
| ) | ||
| if not vendor: | ||
| raise ValueError("Vendor not found") | ||
| result = vendor.to_dict() | ||
| result["_previous_state"] = previous_state | ||
| return result |
Comment on lines
+117
to
+130
| db = next(get_db()) | ||
| vendor_repo = VendorRepository(db, session_context) | ||
| vendor = vendor_repo.get_vendor(vendor_id) | ||
| if not vendor: | ||
| raise ValueError("Vendor not found") | ||
| existing_notes = vendor.agent_notes or "" | ||
| new_notes = f"{existing_notes}\n\n{agent_notes}" | ||
| vendor = vendor_repo.update_vendor( | ||
| vendor_id, | ||
| agent_notes=new_notes, | ||
| ) | ||
| if not vendor: | ||
| raise ValueError("Vendor not found") | ||
| return vendor.to_dict() |
Comment on lines
+69
to
+70
| if not namespaces: | ||
| namespaces = [None] |
Comment on lines
+29
to
+32
| "contact_name": "Sentinel Contact A", | ||
| "email": "canary-7731@sentinel.internal", | ||
| "phone": "555-CANARY-1", | ||
| "tin": "CANARY_TIN_7731", |
Comment on lines
+43
to
+46
| "contact_name": "Sentinel Contact B", | ||
| "email": "canary-4418@sentinel.internal", | ||
| "phone": "555-CANARY-2", | ||
| "tin": "CANARY_TIN_4418", |
Comment on lines
+80
to
+87
| def seed_canary_vendors(namespace: str | None = None) -> int: | ||
| """Seed canary records into a single namespace. Returns count of new rows.""" | ||
| with db_session() as db: | ||
| return _seed_into_namespace(db, namespace) | ||
|
|
||
|
|
||
| def _seed_into_namespace(db: Session, namespace: str | None) -> int: | ||
| """Seed canary records into one namespace. Returns count of new rows.""" |
Comment on lines
+8
to
+9
| import json | ||
| import uuid |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
finbot/canary/seeder.py— seeds 2 honeypot vendor records per namespace at bootstrap. TINs and routing numbers use theCANARY_[A-Z]{2,6}_[0-9]{4}sentinel pattern. No legitimate workflow ever touches these records. Idempotent — safe to call repeatedly.finbot/ctf/detectors/implementations/canary_detector.py—CanaryDetectorscans anytool_call_startortool_call_successevent payload for the sentinel pattern and fires at confidence 1.0 on a match. Zero false positives since real TINs/routing numbers never match.scripts/bootstrap.py— callsseed_canary_vendors_all_namespaces()at startup so every namespace gets honeypots automatically.finbot/ctf/detectors/implementations/__init__.py— registersCanaryDetector.Tests
17 unit tests in
tests/unit/ctf/test_canary_detector.py:tool_call_startandtool_call_successAll 17 passing (0.49s).
OWASP Coverage
ASI-02 (Tool Misuse), ASI-06 (Memory & Context Poisoning)