feat: added option to select default inferencing runtime

[gastoner]

What does this PR do?

Adds settings to chose default inference runtime to preferences
Implements mockup in the issue

Screenshot / video of UI

Screencast_20250428_150354.webm

What issues does this PR fix or reference?

Closes #2611

How to test this PR?

Select different inference runtime -> models should be filtered out

[Copilot]

Copilot wasn't able to review any files in this pull request.

Files not reviewed (1)

• packages/backend/package.json: Language not supported

[axel7083]

We cannot have a default for inference server not dealing with the same kind of files, like you cannot default to llamacpp instead of whispercpp as model handled by whispercpp cannot be handled by llamacpp.

This need to be specific to GGUF models, or safetensors. Choice should be llamacpp, openvino, vllm.

Meaning this PR should be delayed after the following PRs

• feat(inference): vllm support #2900
• feat: add OpenVINO inference provider #2848

In the meantime we cannot have a default, as the inference provided depends on the models, not the other way around

[jeffmaury]

I did not configure anything but when I go to the settings page the settings tell me that llama-cpp is the prefered (I would expect to be none) but if I go to the recipes page I see only one recipe (object detection)
So I think we are missing the no value case

[jeffmaury]

LGTM.

However, I though you said you add a select in ModelSelect so that we can select models even if they are not for the preferred runtime ?

[gastoner]

LGTM.

However, I though you said you add a select in ModelSelect so that we can select models even if they are not for the preferred runtime ?

In the ModelSelect there is a mechanism of filtering the models based on selected Recipe ~ runime for this recipe

[jeffmaury]

LGTM.
However, I though you said you add a select in ModelSelect so that we can select models even if they are not for the preferred runtime ?

In the ModelSelect there is a mechanism of filtering the models based on selected Recipe ~ runime for this recipe

I think ModelSelect is used in other contexts ?

[gastoner]

LGTM.
However, I though you said you add a select in ModelSelect so that we can select models even if they are not for the preferred runtime ?

In the ModelSelect there is a mechanism of filtering the models based on selected Recipe ~ runime for this recipe

I think ModelSelect is used in other contexts ?

Yes it is used in 3 or 4 places I think. Should I remove the filter from ModelSelect then?

[gastoner]

Should I now also add the openvino filter? since the openvino PR is now closed?

[jeffmaury]

LGTM

[gastoner]

@axel7083 can you PTAL?

[slemeur]

I think the behavior is fine. And we have a couple of PRs that are waiting to be merged after this one, which will make the experience more logical.

What we are proposing the user is to use different Inferencing Runtimes with AI Lab. With this ticket, it will default the entire scope of AI Lab to a specific runtime.
For example, a user who is defaulting to Whisper, will get the list of recipes filtered to those who are compatible with Whisper, but the user will still be able to remove the filter and see the entire list.

For other situation (and probably the most common case), the user are most likely going to change the inferencing runtime at the time they are going to start one. For example, when the user is starting a recipe, when the user is starting to serve a model or starting a playground. We have the corresponding tickets for that: #2612, #2613

So the current behavior is good from my point of view.

Dismissing my request change as @slemeur agree to go ahead with this change.

---

My personal opinion is that this feature will not be used because it is a configuration deep in the settings, and having it increase the complexity of AI Lab, for QE and maintenance