Is there a best practice for human-in-the-loop with multiple agents ?

[jb-ross-bingham]

Hi, we are looking into breaking our AI Tools and behaviors into multiple agents, using the Semantic Kernel Agent Framework. However, in order to do this, the agents would need to be able to take requests from the user, interact with each other and with the user. I believe the behavior we're interested in is described in the documentation as 'Human-in-the-Loop collaboration'. However, I haven't seen this concept mentioned anywhere else. Is there a sample, a document, or anything else that might provide some direction for implementing this type of pattern in Semantic Kernel ?

Currently, we have an extremely long prompt that describes supported behaviors and advertises dozens of tools. This works well for interacting with the user, but it's not sustainable.

It might be worth noting that this an aspnetcore web application, where a transient instance of the Kernel is created via Http Request each time the user sends a message in the chat window. The history and state are stored, so it can be recreated with each request. When the invocation completes, a response is sent back to the user. I'm not sure if this architecture adds additional limitations to implementing a 'Human-in-the-Loop Collaboration' pattern.

I'm happy to provide some context of the problem I'm looking to solve. I started to type it, but I wasn't sure that it added anything helpful to the question.

Thank you for any information you can provide.

Note, I did do a search for similar discussion, and found one which is closed:
#6027

I wasn't sure if I should post a comment to a closed discussion, or open a new one...

[moonbox3]

@jb-ross-bingham thanks for your question. Have you taken a look at our Process framework? I am not sure if you're targeting SK C# or Python, but we have a getting started sample that shows a simulated "human input" step:

C# Example:

semantic-kernel/dotnet/samples/GettingStartedWithProcesses/Step01/Step01_Processes.cs

Line 89 in f3633e2

private sealed class ChatUserInputStep : ScriptedUserInputStep

Python Example:

semantic-kernel/python/samples/getting_started_with_processes/step01/step01_processes.py

Line 82 in f3633e2

class ChatUserInputStep(ScriptedUserInputStep):

In C# (coming soon to Python), we then have a Processes sample around configuring steps to run the agent framework: https://github.com/microsoft/semantic-kernel/tree/main/dotnet/samples/GettingStartedWithProcesses/Step04

Would this help your use-case?

[jb-ross-bingham]

@moonbox3 , Thank you for your response, I really appreciate it. I will be looking at the Process Framework as a solution. I was aware of it, but I was hesitant to look at it in detail because of the dependency on Dapr. At least, I think I read there's a dependency on Dapr; not for local development, but for production environments.

Just to answer your question, from the limited reading I've done on the Process Framework, I believe you've nailed it, and this would solve our human interaction needs. I'm looking forward to trying it out.

Also, FWIW, we did make the switch from a no-agent ChatHistory implementation to the AgentGroupChat implementation with N number of agents. We're still building out agents, and migrating the thousands of lines of instructions into the much more compartmentalized, much more maintainable agents.

For simple scenarios, we've have early success with the human interaction. However, as the number of agents grow, and as we add increasingly complex workflow scenarios, I can see the Process Framework being even more valuable.

Thanks again for your response.