From 2578f2d49aaa5bd7cdc3f7755d897a1dd977b19f Mon Sep 17 00:00:00 2001 From: Louie Lu Date: Wed, 27 May 2026 23:26:13 -0500 Subject: [PATCH] Add AgentForge Router to leaderboard (#111) Insert AgentForge Router at rank 12 (Arena 60.12, Acc 59.16%, $0.09/1K, Robustness 100.00) from the PR #111 automated evaluation, and add the Sqwish Router project link. Co-Authored-By: Claude Opus 4.7 (1M context) --- README.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 39b3dd3b..24b96486 100644 --- a/README.md +++ b/README.md @@ -37,7 +37,7 @@ For more details, please see our [website](https://routeworks.github.io/leaderbo | Rank | Router | Affiliation | Acc-Cost Arena | Accuracy | Cost/1K Queries | Optimal Selection | Optimal Cost | Optimal Accuracy | Latency | Robustness | |------|--------------------|-----------------------------|--------|----------|---------|-----------------|--------------|----------------|---------|------------| -| ๐Ÿฅ‡ | [Sqwish Router]() | | 75.27 | 76.40 | $0.18 | 7.41 | 25.10 | 90.47 | โ€” | 100.00 | +| ๐Ÿฅ‡ | [Sqwish Router](https://www.sqwish.ai/) | | 75.27 | 76.40 | $0.18 | 7.41 | 25.10 | 90.47 | โ€” | 100.00 | | ๐Ÿฅˆ | [OrcaRouter-Adaptive](https://www.orcarouter.ai/) | | 72.08 | 75.54 | $1.00 | โ€” | โ€” | โ€” | โ€” | 22.62 | | ๐Ÿฅ‰ | [Azure-Model-Router](https://ai.azure.com/catalog/models/model-router) [[Web]](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/concepts/model-router) | ๐Ÿ’ผ Microsoft | 71.87 | 72.82 | $0.22 | โ€” | โ€” | โ€” | โ€” | 71.43 | | 4 | [R2-Router](https://arxiv.org/abs/2602.02823/) | ๐ŸŽ“ UCF | 71.60 | 71.23 | $0.06 | 24.51 | 48.70 | 99.85 | โ€” | 45.71 | @@ -48,12 +48,13 @@ For more details, please see our [website](https://routeworks.github.io/leaderbo | 9 | [GPTโ€‘5](https://openai.com/index/introducing-gpt-5/) | ๐Ÿ’ผ OpenAI | 64.32 | 73.96 | $10.02 | โ€” | โ€” | โ€” | โ€” | โ€” | | 10 | [CARROT](https://arxiv.org/abs/2502.03261) [[Code]](https://github.com/somerstep/CARROT) [[HF]](https://huggingface.co/CARROT-LLM-Routing) | ๐ŸŽ“ UMich | 63.87 | 67.21 | $2.06 | 2.68 | 6.77 | 78.63 | 1.50 | 89.05 | | 11 | [Chayan](https://huggingface.co/adaptive-classifier/chayan) [[HF]](https://huggingface.co/adaptive-classifier/chayan) | ๐ŸŽ“ Adaptive Classifier | 63.83 | 64.89 | $0.56 | 43.03 | 43.75 | 88.74 | โ€” | โ€” | -| 12 | [RouterBenchโ€‘MLP](https://arxiv.org/pdf/2403.12031) [[Code]](https://github.com/withmartian/routerbench) [[HF]](https://huggingface.co/datasets/withmartian/routerbench) | ๐ŸŽ“ Martian | 57.56 | 61.62 | $4.83 | 13.39 | 24.45 | 83.32 | 90.91 | 80.00 | -| 13 | [NotDiamond](https://www.notdiamond.ai/) | ๐Ÿ’ผ NotDiamond | 57.29 | 60.83 | $4.10 | 1.55 | 2.14 | 76.81 | โ€” | 55.91 | -| 14 | [GraphRouter](https://arxiv.org/abs/2410.03834) [[Code]](https://github.com/ulab-uiuc/GraphRouter) | ๐ŸŽ“ UIUC | 57.22 | 57.00 | $0.34 | 4.73 | 38.33 | 74.25 | 2.70 | 94.29 | -| 15 | [RouterBenchโ€‘KNN](https://arxiv.org/pdf/2403.12031) [[Code]](https://github.com/withmartian/routerbench) [[HF]](https://huggingface.co/datasets/withmartian/routerbench) | ๐ŸŽ“ Martian | 55.48 | 58.69 | $4.27 | 13.09 | 25.49 | 78.77 | 1.33 | 83.33 | -| 16 | [RouteLLM](https://arxiv.org/abs/2406.18665) [[Code]](https://github.com/lm-sys/RouteLLM) [[HF]](https://huggingface.co/routellm) | ๐ŸŽ“ Berkeley | 48.07 | 47.04 | $0.27 | 99.72 | 99.63 | 68.76 | 0.40 | 100.00 | -| 17 | [RouterDC](https://arxiv.org/abs/2409.19886) [[Code]](https://github.com/shuhao02/RouterDC) | ๐ŸŽ“ SUSTech | 33.75 | 32.01 | $0.07 | 39.84 | 73.00 | 49.05 | 10.75 | 85.24 | +| 12 | [AgentForge Router]() | | 60.12 | 59.16 | $0.09 | โ€” | โ€” | โ€” | โ€” | 100.00 | +| 13 | [RouterBenchโ€‘MLP](https://arxiv.org/pdf/2403.12031) [[Code]](https://github.com/withmartian/routerbench) [[HF]](https://huggingface.co/datasets/withmartian/routerbench) | ๐ŸŽ“ Martian | 57.56 | 61.62 | $4.83 | 13.39 | 24.45 | 83.32 | 90.91 | 80.00 | +| 14 | [NotDiamond](https://www.notdiamond.ai/) | ๐Ÿ’ผ NotDiamond | 57.29 | 60.83 | $4.10 | 1.55 | 2.14 | 76.81 | โ€” | 55.91 | +| 15 | [GraphRouter](https://arxiv.org/abs/2410.03834) [[Code]](https://github.com/ulab-uiuc/GraphRouter) | ๐ŸŽ“ UIUC | 57.22 | 57.00 | $0.34 | 4.73 | 38.33 | 74.25 | 2.70 | 94.29 | +| 16 | [RouterBenchโ€‘KNN](https://arxiv.org/pdf/2403.12031) [[Code]](https://github.com/withmartian/routerbench) [[HF]](https://huggingface.co/datasets/withmartian/routerbench) | ๐ŸŽ“ Martian | 55.48 | 58.69 | $4.27 | 13.09 | 25.49 | 78.77 | 1.33 | 83.33 | +| 17 | [RouteLLM](https://arxiv.org/abs/2406.18665) [[Code]](https://github.com/lm-sys/RouteLLM) [[HF]](https://huggingface.co/routellm) | ๐ŸŽ“ Berkeley | 48.07 | 47.04 | $0.27 | 99.72 | 99.63 | 68.76 | 0.40 | 100.00 | +| 18 | [RouterDC](https://arxiv.org/abs/2409.19886) [[Code]](https://github.com/shuhao02/RouterDC) | ๐ŸŽ“ SUSTech | 33.75 | 32.01 | $0.07 | 39.84 | 73.00 | 49.05 | 10.75 | 85.24 | ๐ŸŽ“ Open-sourceโ€ƒโ€ƒ๐Ÿ’ผ Closed-sourceโ€ƒ