Deeper usage analysis

[dhvcc]

I was wondering, since this already is a load balancer with option to track usage - wouldn't it be nice to track or be able to analyze tool usage?

I personally do it via hooks, but that's pretty limited

The idea is to be able to see where your models make mistakes more often to optimize your own prompts and tooling

It was very useful to me to change and wrap my existing mcps and prompts to reduce token usage and therefore increase the accuracy of the models

[dhvcc]

Personally I'll most likely try to implement that in a personal fork, but I try to avoid non upstream changes/ideas since it makes it hard to always maintain and sync with upstream