chat memory and conversation id

[hstaudacher]

I have a bit of trouble understanding the ideas behind the chat memory API.

The MessageWindowChatMemory has a static DEFAULT_CHAT_MEMORY_REPOSITORY which is the default if not overridden. When advising a request the doGetConversationId method of the AbstractChatMemoryAdvisor is called. The chat memory repository's intention is to store conversations by id. So far so good.

However, the conversation id always falls back to the DEFAULT_CONVERSATION_ID because there is no id set on the advise context. I also can't find an API to set the conversation id.

The result of this is very nasty because conversation history is shared between ChatClient instances. I currently work around this by creating new InMemoryChatMemoryRepository per client instance.

I'm not sure if this intended or simply a bug. Any help would be very. much appreciated 😄.

[markpollack]

Sorry, we need to document this better. Here is some example code that shows how to do it.

https://github.com/ThomasVitale/llm-apps-java-spring-ai/blob/main/patterns/memory/memory-jdbc/src/main/java/com/thomasvitale/ai/spring/MemoryController.java

@RestController
class MemoryController {

    private final ChatClient chatClient;

    MemoryController(ChatClient.Builder chatClientBuilder, ChatMemory chatMemory) {
        this.chatClient = chatClientBuilder.clone()
                .defaultAdvisors(new MessageChatMemoryAdvisor(chatMemory))
                .build();
    }

    @PostMapping("/memory/jdbc/{conversationId}")
    String chat(@PathVariable String conversationId, @RequestBody String question) {
        return chatClient.prompt()
                .user(question)
                .advisors(a -> a.param(CHAT_MEMORY_CONVERSATION_ID_KEY, conversationId))
                .call()
                .content();
    }

}