Detector Shields for testing LLM Application Firewalls

[Eric-Hacker]

This implements a Shields detector as describe in #1055

Verification

Use this detector with an LLM Guardrail such as Granite-Guardian

• Rest generator config for running Granite-Guardian 2B locally with Ollama

{
   "rest": {
      "RestGenerator": {
         "name": "Ollama GraniteGuardian",
         "uri": "http://localhost:11434/api/chat",
         "method": "post",
         "headers": {
            "Content-Type": "application/json"
         },
         "req_template_json_object": {
            "model": "granite3-guardian:2b",
            "messages": [ {"role": "user", "content": "$INPUT"} ],
            "temperature": 0,
           "stream": false
         },
         "response_json": true,
         "response_json_field": "$.message.content",
         "request_timeout": 120
      }
   }
}

• Run the tests and ensure they pass python -m pytest tests/
• python -m garak --model_type rest \ --generator_option_file garak_runs/ollama_graniteguardian.json \ --generations 1 \ --probes test.Test \ --detectors shields.Up \ --report_prefix promptinject.ollama_graniteguardian.up
• Verify shields.Up: FAIL ok on 3/ 8 (failure rate: 62.50%)
• python -m garak --model_type rest \ --generator_option_file garak_runs/ollama_graniteguardian.json \ --generations 1 \ --probes test.Test \ --detectors shields.Down \ --report_prefix promptinject.ollama_graniteguardian.down
• Verify shields.Down: FAIL ok on 5/ 8 (failure rate: 37.50%)
• Documented

[github-actions]

DCO Assistant Lite bot All contributors have signed the DCO ✍️ ✅

[jmartin-tech]

Awesome, I like this idea. There has been some discussion in discord about boolean detection for rule triggers. I wonder if there is a reasonable way to incorporate that here.

The solution for users may be as simple as setting upstrings: [ "true" ] or downstrings: [ "false" ] to detect rule match response.

[Eric-Hacker]

I have read the DCO Document and I hereby sign the DCO

[Eric-Hacker]

It looks like I messed up my username when I switched machines and now there is a commit from what Github thinks is someone else. I'll have to figure out how to fix that. Pulled it back to draft.

[Eric-Hacker]

Regarding adding other boolean detection strings, I'd be happy to do something, but looking at the thread it looks like the firewall returned a key of "flagged" that is set to true rather than something like "flagged: true" in the normal response_json_field. It seems that would need non-string detector which should probably be its own class.

Also, one of the default Up strings that is matched is "flag" at the beginning of the response. So if a firewall was returning "flagged true" and "flagged false" as strings, this would treat them both as a pass. That is one of the reasons the upstrings and downstrings are params. I didn't call that out in the docs, but I could do so since I know this can get really confusing.

[Eric-Hacker]

TL;DR Fixed the DCO identity issue. Putting back up for review.

Details. My first commit was using a canary email address on my domain that wasn't registered with Github. Then I found out that github provides email addresses for public exposure and I switched to that. I had to add the canary email address to my github profile and the DCO issue cleared up.

So now I have an email address that is only exposed in one public commit. It will be interesting to see how much spam it gets.

[jmartin-tech]

@Eric-Hacker if you would like to limit the exposure you can rebase or squash the branch before we land it to remove the email and allowing removal from your github account, while not a full scrub since git commits are rarely truly deleted it could mitigate future spam.

Either way we will consider this ready for review and provide feedback soon.

[Eric-Hacker]

@jmartin-tech No need to try and remove that email, thank you. I already have filters to direct compromised canary email addresses to spam. Now I'm curious to see what it receives.

[jmartin-tech]

Testing looks good, #1075 has added the startswith capability here and incorporated the tests in test_detectors_base.py making test_detectors_string.py here redundant.

I will push a commit removing test_detectors_string.py and land this shortly.