Skip to content

[DON'T MERGE] Testing workflow optimality#52

Closed
jamesbond19925-spec wants to merge 3 commits into
RouteWorks:auto-metricsfrom
jamesbond19925-spec:testing-workflow-optimality
Closed

[DON'T MERGE] Testing workflow optimality#52
jamesbond19925-spec wants to merge 3 commits into
RouteWorks:auto-metricsfrom
jamesbond19925-spec:testing-workflow-optimality

Conversation

@jamesbond19925-spec

Copy link
Copy Markdown
Contributor

No description provided.

@jamesbond19925-spec jamesbond19925-spec force-pushed the testing-workflow-optimality branch from b52b4fb to ada12e8 Compare December 2, 2025 08:16
@github-actions

github-actions Bot commented Dec 2, 2025

Copy link
Copy Markdown

📊 Router Evaluation Results

Router: test-glm4air-optimality
Dataset Split: full

Error reading metrics: Expected ',' or '}' after property value in JSON at position 400


Evaluation completed by RouterArena automated workflow

@github-actions

github-actions Bot commented Dec 2, 2025

Copy link
Copy Markdown

📊 Router Evaluation Results

Router: test-glm4air-optimality
Dataset Split: full

RouterArena Metrics

Metric Value
RouterArena Score 0.5617
Accuracy 54.65%
Total Cost $0.396728
Avg Cost per Query $0.000047
Avg Cost per 1K Queries $0.0472
Number of Queries 8400

Optimality Metrics (Sub_10 Split)

Metric Value
Opt.Sel (Optimal Selection) 0.7701
Opt.Cost (Cost Efficiency) 0.9337
Opt.Acc (Accuracy vs Optimal) 1.0000

Evaluation completed by RouterArena automated workflow

@yl231 yl231 closed this Dec 2, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants