[DRAFT] FEAT support adversarial_chat and scoring in scanner to enable automated multi-turn-orchestrators #706
Conversation
| # scoring: | ||
| # scorer: ... | ||
| # objective_scorer: ... |
There was a problem hiding this comment.
For PSO we can technically accept "scorers" (plural). Options I considered so far:
- just take 1 scorer (the current version) and perhaps allow
additional_scorerswhich can take a list - take a list of scorers, and only the first one is passed to orchestrators that need a single scorer
Thoughts?
There was a problem hiding this comment.
I like the first option a bit better. Seems like most of the time (maybe I'm wrong!) people are just using one scorer so could be good to optimize for that experience. And then having the additional_scorers would be a nice clear place to add others for anyone experimenting with more than one.
There was a problem hiding this comment.
I agree, mostly because it's the happy path and it makes it clear what the important one is.
| if "type" not in scorer_args: | ||
| raise KeyError("Scorer definition must contain a 'type' key.") | ||
|
|
||
| scorer_type = scorer_args.pop("type") |
There was a problem hiding this comment.
NIT: this pop is clean as opposed to the del in validate_target above, can we change validate_target to match this?
| adversarial_chat: | ||
| type: "OpenAIChatTarget" | ||
| is_azure_target: true | ||
| scoring: |
There was a problem hiding this comment.
Sorry if this is too broad or has been answered already, but if in the future, we want to run RTO and Crescendo each with a different scorer, how could we do that?
There was a problem hiding this comment.
In different files...
…scanner_scorer_adv_chat_rto
…scanner_scorer_adv_chat_rto
Description
So far, only prompt sending orchestrator works with the scanner. This PR bridges the gap to multi-turn orchestrators with automated red teaming by adding
adversarial_chatand scoring support. For scoring, this meansobjective_scorerorscoring_targetdepending on what the attack requires. Since #673 already implemented orchestrators in a very generic way this just generalized to red teaming orchestrator, crescendo orchestrator, and TAP orchestrator without major changes!Tests and Documentation
Lots to be added! Looking for validation for the approach first, though.