Serving extra models by intercepting calls via proxy

[jpmunix]

Hey everyone!

I have a new idea I've been experimenting with: using the proxy to intercept model calls and serve extra models directly in the selector. Take a look at the attached screenshot—specifically the last model in the selector—to see what I mean.

I don't have the time to continue working on this, but I wanted to leave the idea here for anyone who wants to take it over. The logic behind it is actually pretty simple:

1. Intercept the incoming model calls.
2. Decompress the payload (since it comes in gzip).
3. Reformulate the JSON to inject the extra models.
4. Re-compress it back to gzip and forward it.

Hopefully, someone finds this useful and wants to bring it to life!

[jpmunix]

Hi everyone,

A few days ago I shared an idea about intercepting model calls and injecting additional models into the selector through the proxy. I've now completed a working implementation.

At this point, I can override the model list and inject as many custom models as I want directly into the selector, making them behave like native options from the user's perspective.

I also implemented an additional vision compatibility layer.

When a user attaches one or more images and selects a model that does not support vision, the proxy automatically intercepts the images and routes them to a vision-capable model. A dedicated prompt analyzes the user's intent and generates a description of the image specifically tailored to what the user is trying to accomplish, rather than producing a generic caption.

That contextual description is then forwarded to the original non-vision model, allowing it to continue the conversation as if it had native image understanding.

As a bonus, the generated image analysis is also injected into the reasoning/thinking stream, so it can be inspected directly from the AG response when reasoning visibility is enabled.

Everything is working surprisingly well so far, and the experience feels seamless from the user's side.

I'll be preparing a pull request with the changes soon.

Stay tuned. 🚀

[12errh]

@jpmunix I Am Wating