Skip to content

fix(model): wire openai_compatible configuration across trainer, eval, and config pipeline - #262

Open
RohithPariki wants to merge 3 commits into
microsoft:mainfrom
RohithPariki:fix/openai-compatible-config-pipeline
Open

fix(model): wire openai_compatible configuration across trainer, eval, and config pipeline#262
RohithPariki wants to merge 3 commits into
microsoft:mainfrom
RohithPariki:fix/openai-compatible-config-pipeline

Conversation

@RohithPariki

Copy link
Copy Markdown
Contributor

Description

This PR connects the generic openai_compatible backend across the configuration, trainer, and evaluation pipelines so that models served via OpenAI-compatible endpoints (such as DeepSeek, Groq, Together AI, vLLM, Ollama, LiteLLM, or local servers) can be fully configured via YAML configs, --cfg-options, or CLI flags and properly initialized during training and evaluation runs.

Key Changes

  1. Config pipeline (skillopt/config.py): Added all 18 model.openai_compatible_* mappings to _FLATTEN_MAP so YAML configs retain general and per-role (optimizer_ / target_) settings during flattening.
  2. Backend runtime configuration (skillopt/model/openai_compatible_backend.py & skillopt/model/__init__.py): Extended configure_openai_compatible to support per-role overrides for temperature, timeout_seconds, and max_tokens across OPTIMIZER_CONFIG and TARGET_CONFIG.
  3. Trainer initialization (skillopt/engine/trainer.py): Added configure_openai_compatible(...) call during ReflACTTrainer.train() initialization so parameters configured via YAML or CLI take effect during training and episode rollouts.
  4. Standalone evaluation harness (scripts/eval_only.py): Added "openai_compatible", "qwen", and "qwen_chat" to --backend choices, added CLI flags, mapped them to structured keys in load_config, and invoked configure_openai_compatible in main().
  5. Training CLI (scripts/train.py): Added CLI arguments for openai_compatible_*, mapped them in _LEGACY_TO_STRUCTURED, and added environment variable guidance (OPENAI_COMPATIBLE_API_KEY) for secure credential passing.
  6. Unit Tests (tests/test_openai_compatible_config.py & tests/test_openai_compatible_backend.py): Added comprehensive test coverage for YAML config flattening, CLI overrides, role separation, trainer initialization, credential warnings, and eval_only script parsing.

Verification

  • Ran test suite across config and backend modules (pytest tests/test_openai_compatible_config.py tests/test_openai_compatible_backend.py tests/test_azure_openai_compat.py tests/test_minimax_backend.py tests/test_minimax_region.py tests/test_qwen_backend.py tests/test_role_backend_resolution.py tests/test_codex_config_aliases.py tests/test_retired_cli_options.py tests/test_env_section_survives_dedup.py). All 175 tests passed.

@RohithPariki
RohithPariki marked this pull request as ready for review August 29, 2026 00:21
@Yif-Yang

Copy link
Copy Markdown
Contributor

Thanks — the new OpenAI-compatible fields flatten and reach the runtime configurator, but current head still has several entry-point gaps:

  1. scripts/train.py does not include openai_compatible in the --backend choices, so --backend openai_compatible exits with argparse status 2 even though the related flags were added.
  2. scripts/eval_only.py has no qwen_chat routing branch. Because both qwen and qwen_chat normalize to qwen_chat, --backend qwen currently falls through to the generic branch and selects openai_chat for both roles; the target therefore never uses Qwen. Eval-only also lacks the Qwen model-default normalization already present in train.py, so an inherited gpt-5.5 can remain on a Qwen role.
  3. Eval-only parses and maps all optimizer_qwen_chat_* options, but its configure_qwen_chat(...) call forwards only shared and target values, so every optimizer-specific Qwen setting is silently ignored.
  4. OpenAI-compatible role selection is correct, but model fallback is not handled in either train or eval-only. With the shipped base config, selecting openai_compatible leaves both role models at the inherited Azure gpt-5.5; with no openai_compatible_model override, that becomes the actual compatible wire model instead of the backend's declared gpt-4o-mini fallback.

Please wire these paths consistently while preserving explicit per-role model overrides, and add regressions for train CLI backend selection, eval-only qwen/qwen_chat role and model resolution, optimizer Qwen kwargs, and OpenAI-compatible fallback/explicit-model precedence. The newly added compatible tests currently all pass but do not exercise these cases.

@RohithPariki

Copy link
Copy Markdown
Contributor Author

Thanks Yifan Yang (@Yif-Yang) for the detailed review! I have addressed all 4 points in commit 451f2e8:

  1. train.py --backend choices: Added "openai_compatible" to the argument choices in scripts/train.py.
  2. eval_only.py Qwen routing & model normalization: Added the qwen_chat role routing branch (optimizer_backend = "openai_chat", target_backend = "qwen_chat") and model-default normalization for qwen_chat roles to resolve the target model to Qwen/Qwen3.5-4B.
  3. eval_only.py optimizer Qwen kwargs: Updated the configure_qwen_chat(...) invocation to forward all optimizer_qwen_chat_* options.
  4. OpenAI-compatible model fallback & precedence: Added normalization in both train.py and eval_only.py so that unoverridden base gpt-5.5 sentinels fall back to --openai_compatible_model / per-role compatible models or the declared default gpt-4o-mini, while strictly preserving explicit per-role model overrides (--optimizer_model / --target_model).
  5. Regressions: Added unit tests in tests/test_openai_compatible_config.py covering train CLI backend selection, eval-only Qwen role and model resolution, optimizer Qwen kwargs forwarding, and OpenAI-compatible fallback / explicit-model precedence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants