Simplify LLM config #120
Simplify LLM config #120
Conversation
| llm_model: "anthropic.claude-3-5-sonnet-20241022-v2:0" | ||
| # llm_provider: openai | ||
| # llm_model: gpt-4o-2024-08-06 | ||
| llm_provider: ${llm.provider} |
There was a problem hiding this comment.
how and where is llm.provider defined?
There was a problem hiding this comment.
it is defined in the config file under the llm section
and similarly for llm.model
There was a problem hiding this comment.
So we define them only once, and use the same model and provider defined for task inference.
There was a problem hiding this comment.
Why do we not supporting different models again?
There was a problem hiding this comment.
Could you please elaborate "not supporting different models again"? We will be able to support all the models that were also supported before. It is just easier that the user has to pass their model and their provider once, instead of doing it twice.
There was a problem hiding this comment.
Why are we not supporting different models for task inference and feature generation?
There was a problem hiding this comment.
Is that a use-case? If yes, we don't have that option in the UI. I can revert this change, and keep the redundancy if that is a use-case we foresee.
There was a problem hiding this comment.
And btw if someone wants to do that it is possible via the overrides. It's just the defaults that will be the same, which were already same.
There was a problem hiding this comment.
Example: caafe should be more expensive, so the customer want to use a cheaper LLM. but I think I'm fine for now.
There was a problem hiding this comment.
yes so that is possible you just have to override the parameter and provide it explicitly.
Description of changes:
This PR makes Bedrock & OpenAI config setup similar, and removes the requirement of passing in
api_key_locationparameter. Also I've removed the redundancy of specifying llm model and provider again if CAAFE is used, by loading in the same llm parameters defined for task inference.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.