Grafana 10.2.2 GenAI with PrivateGPT

[tedraymond]

Grafana's latest release allows for interaction with OpenAI for their LLM plugin. Creating potential of self managed generative AI.

Now the documentation in privateGPT says it should work the same as OpenAI API but testing today was not successful.

I can query privateGPT via my own API call.

I'm not very familiar with OpenAPI call basically because I like free stuff.

Has anyone else explored this new opportunity to tie in privateGPT with Grafana?

Maybe kindly explain any differences between OpenAi API and PrivateGPT API?

Tried using the v1/chat/completions endpoint, it works fine on my own API call but Grafana errors out with a 500 internal server error.

Assuming Grafana is passing some sort of key that is not recognized by the privateGPT api.

I'll likely try and capture the api from Grafana and figure out the differences tomorrow.

Best Regards,

[tedraymond]

I figured it out.

Initial testing was on CPU LLM endooint, grafana seems to time out before the CPU llm can respond.

Building GPU based local LLM helps tremendously.

Now i have all the GENAI features.
👍👍

[tedraymond]

Also improved performance moving to Grafana 11 as well.