Have you searched for this feature request?
Problem Statement
The streaming configuration is currently hard-coded to True: https://github.com/cnoe-io/cnoe-agent-utils/blob/main/cnoe_agent_utils/llm_factory.py#L255, which might create issues for users that don't have streaming supported in their infrastructure.
Possible Solution
Similar to other the Azure and OpenAI providers, we shall allow users to control this streaming option via a flag: https://github.com/cnoe-io/cnoe-agent-utils/blob/main/cnoe_agent_utils/llm_factory.py#L342-L343
Alternatives Considered
No response
Have you searched for this feature request?
Problem Statement
The streaming configuration is currently hard-coded to
True: https://github.com/cnoe-io/cnoe-agent-utils/blob/main/cnoe_agent_utils/llm_factory.py#L255, which might create issues for users that don't have streaming supported in their infrastructure.Possible Solution
Similar to other the Azure and OpenAI providers, we shall allow users to control this streaming option via a flag: https://github.com/cnoe-io/cnoe-agent-utils/blob/main/cnoe_agent_utils/llm_factory.py#L342-L343
Alternatives Considered
No response