Skip to content

feat: Add EcoTune-based inference tuning module#1156

Open
ust-xu wants to merge 2 commits intoflagos-ai:mainfrom
ust-xu:feat/ecotune-inference-tuning
Open

feat: Add EcoTune-based inference tuning module#1156
ust-xu wants to merge 2 commits intoflagos-ai:mainfrom
ust-xu:feat/ecotune-inference-tuning

Conversation

@ust-xu
Copy link
Copy Markdown

@ust-xu ust-xu commented Mar 19, 2026

  • Adds a lightweight EcoTune core for inference-time hyperparameter optimization
  • Keeps the contribution minimal: no benchmarks, demos, datasets, or docs in this PR
  • Designed to be reused by future inference-tuning entrypoints and config-driven workflows
  • Supports bootstrap from default decoding parameters

@ust-xu ust-xu requested a review from zhaoyinglia as a code owner March 19, 2026 15:40
Copilot AI review requested due to automatic review settings March 19, 2026 15:40
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 19, 2026

CLA assistant check
All committers have signed the CLA.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces an EcoTune-based inference-time tuning core under flagscale.inference.tuning, implementing a small Bayesian-optimization style loop with a multi-fidelity GP surrogate and token-cost-aware acquisition to propose decoding/config suggestions under a budget.

Changes:

  • Added multi-fidelity GP surrogate model for score prediction across config + fidelity.
  • Added token-aware Expected Improvement acquisition and an optimizer implementing ask/tell with promotion to max fidelity.
  • Added search space utilities and public package exports for flagscale.inference.tuning.ecotune.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
flagscale/inference/tuning/ecotune/surrogate.py Implements a multi-fidelity GP surrogate with fit/predict and incumbent querying.
flagscale/inference/tuning/ecotune/search_space.py Defines parameter dimensions, sampling, and config↔vector transforms.
flagscale/inference/tuning/ecotune/optimizer.py Implements EcoTune optimizer loop (budgeting, suggestion, promotion, history).
flagscale/inference/tuning/ecotune/acquisition.py Adds token-cost-aware Expected Improvement acquisition.
flagscale/inference/tuning/ecotune/init.py Exposes EcoTune public API symbols.
flagscale/inference/tuning/init.py Exposes tuning API from flagscale.inference.tuning.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants