Bump: ChatQnA Example with OpenAI-Compatible Endpoint

[edlee123]

Description

Allows ChatQnA to be used with hundreds of OpenAI-like endpoints e.g. OpenRouter.ai, Hugging Face, Denvr, and improve the developer experience to use OPEA quickly even on low resource environments.

Key Changes Made:

• Created ChatQnA/docker_compose/intel/cpu/xeon/README_endpoint_openai.md: instructions to spin up example.
• Created ChatQnA/docker_compose/intel/cpu/xeon/compose_endpoint_openai.yaml: replaces vLLM with an opeai-like endpoint

Also:

• Fixed align_generator function to properly detect and skip chunks where content is null in open-ai like endpoints. Previously it'd show the null json in the UI.
• Added better error handling and debug logging for easier troubleshooting of endpoint issues.

Issues

N/A

Type of change

List the type of change like below. Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds new functionality)
• Breaking change (fix or feature that would break existing design and interface)
• Others (enhancement, documentation, validation, etc.)

Dependencies

N/A

Tests

OpenRouter.ai: anthropic/claude-3.7-sonnet
Denvr: meta-llama/Llama-3.1-70B-Instruct
Hugging Face Inference Endpoint: microsoft/phi-4

[github-actions]

Dependency Review

✅ No vulnerabilities or license issues found.

Scanned Files

None

[Copilot]

Pull Request Overview

This pull request introduces an OpenAI-compatible endpoint for ChatQnA, updates the deployment documentation, and includes improvements in error handling and logging.

• Added new Docker Compose file (compose_endpoint_openai.yaml) to support OpenAI-like endpoints.
• Updated README files for clearer deployment instructions and configuration details.
• Fixed the align_generator function in chatqna.py to better handle and filter null content chunks.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
CodeGen/docker_compose/intel/cpu/xeon/README.md	Updated docker compose command and environment variable documentation; note a markdown table formatting issue.
ChatQnA/docker_compose/intel/cpu/xeon/compose_endpoint_openai.yaml	Added new compose file for OpenAI-compatible endpoint integration.
ChatQnA/docker_compose/intel/cpu/xeon/README_endpoint_openai.md	New documentation with detailed instructions for deploying ChatQnA using the new endpoint.
ChatQnA/chatqna.py	Improved logging and error handling in input/output alignment and generator functions.

Comments suppressed due to low confidence (1)

CodeGen/docker_compose/intel/cpu/xeon/README.md:111

• The table row for LLM_ENDPOINT appears to be broken into two columns due to an unintended pipe character. Please merge the content into a single cell to ensure the URL displays correctly.

| `LLM_ENDPOINT`                          | Internal URL for the LLM serving endpoint (used by `codegen-llm-server`). Configured in `compose.yaml`.             | `http://codegen-vllm                           | tgi-server:9000/v1/chat/completions` |

[edlee123]

Hi @yao531441 @letonghan if either of you can, I'm looking for one more reviewer please :)