MCP Orchestrator is an enterprise-grade, high-performance gateway middleware positioned between AI coding agents (such as Claude Code, Claude Desktop, Cursor, Windsurf, OpenCode, Qwen-Code, or custom LLM agents) and downstream Model Context Protocol (MCP) servers.
By replacing upfront static schema preloading with LangGraph-driven two-stage vector retrieval and context-aware 3-term ranking, MCP Orchestrator achieves a 90.3% reduction in context window token usage while preserving an 85.0% task execution success rate.
- Overview & Problem Statement
- Key Architectural Innovations
- System Architecture
- LangGraph Query Pipeline
- 3-Term Context-Aware Weighted Ranking
- Cloning & Local Setup Guide
- Deployment Methods (Python & Docker)
- MCP Client Connection Guide
- Analytics & Evaluation Dashboard
- Experimental Evaluation & Benchmark Results
- Academic Viva & Demonstration Cheatsheet
- Engineering Economics & Capital Budgeting (Course 2CSHS404)
- Non-Blocking Event-Loop Architecture & Process Guidelines
Standard AI coding agents load all tool definitions from every connected downstream MCP server directly into the LLM context window upfront. As an enterprise workspace connects multiple MCP servers (e.g., GitHub, Filesystem, Memory, Jira, Slack, AWS), loading dozens of verbose JSON Schema tool definitions causes:
- Context Window Exhaustion: Thousands of tokens wasted on unused schemas per prompt turn.
- Tool Confusion: LLMs misfire or select sub-optimal tools when overloaded with choices.
- Escalating Token Costs: High per-turn API token fees ($3.00/1M tokens).
MCP Orchestrator operates as a dual MCP gateway (acting as an MCP Client downstream and an MCP Server upstream):
- Downstream (MCP Client): Automatically indexes schemas from connected stdio/SSE servers defined in
mcp_orc.json. - Upstream (MCP Server): Exposes only two lightweight meta-tools to the AI agent:
discover_tools(query: string): Dynamically retrieves only top-$K$ relevant candidate tool descriptions tailored to the prompt and workspace context.invoke_tool(server_id: string, tool_id: string, arguments: object): Safely proxies execution to the target downstream MCP server and logs historical success metrics.
A frequent question is: “If my AI agent (such as OpenCode) already uses plugins, extensive skills, and custom rules that take 20k–23k tokens, how does MCP Orchestrator claim an 85% to 90%+ reduction?”
To understand the mathematics, distinguish between the Fixed Client Base and the Dynamic MCP Tool Schema Layer:
When an agent connects 10–20 MCP servers directly (e.g., Playwright, GitHub, Context7, SQLite, Filesystem):
- The raw JSON schemas of 50–70 tools inject ~21,800 tokens into the prompt.
- Total context immediately balloons from ~23.2k to ~45.0k tokens on turn zero.
MCP Orchestrator collapses all connected servers down to just two meta-tools (discover_tools and invoke_tool), consuming only ~394 tokens (or ~725 tokens when
(On the standardized 49-tool benchmark suite, this reduction is 90.3%, from 7,450 tokens down to 725 tokens).
LLMs are stateless APIs. On every generation step or tool-calling sub-turn, the client must re-transmit the entire conversation history and all active tool definitions.
Consider a real-world task requiring 2 tool invocations (e.g. reading a file and creating a GitHub issue across 3 API turns):
-
Without Orchestrator: The 21,800 tokens of schemas are re-sent on every turn:
$$21,800 \text{ tokens} \times 3 \text{ steps} = \mathbf{65,400 \text{ tokens of pure schema overhead}}$$ -
With Orchestrator:
$$(\text{Meta-tools } 394 \times 4) + (\text{Discovered Top-5 tools } 725 \times 3) = \mathbf{3,751 \text{ tokens}}$$ $$\mathbf{\text{Real-world reduction during active tool execution: } 94.3%}$$
When tools actually execute and return large files or test outputs, standard MCP forces that output to squeeze alongside 21,800 tokens of unused schemas. MCP Orchestrator frees up 21,000+ tokens of headroom, allowing full code files, error traces, and complex diffs to fit comfortably within the context window without premature compaction.
-
Two-Stage Hybrid Vector Retrieval:
-
Stage 1 (Server Retrieval): Identifies top candidate MCP servers using semantic embeddings (
all-MiniLM-L6-v2in ChromaDB) or lexical search. -
Stage 2 (Tool Retrieval): Filters tools belonging to candidate servers, reducing search space from
$N$ total tools to top candidate sets.
-
Stage 1 (Server Retrieval): Identifies top candidate MCP servers using semantic embeddings (
-
3-Term Context-Aware Reranking: Combines Query Similarity, Workspace Project Context Alignment (
$\text{ProjMatch}$ ), and Historical Execution Success ($\text{PastSuccess}$ ). -
LangGraph Orchestration: Declarative 5-node acyclic state graph (
ContextParserNode$\rightarrow$ Stage1RetrievalNode$\rightarrow$ Stage2RetrievalNode$\rightarrow$ RankingNode$\rightarrow$ LoggingNode). -
Resilient Infrastructure: Idempotent registration with SQLite/PostgreSQL metadata persistence, minimum relevance thresholding (
$\text{MIN_RELEVANCE_THRESHOLD} = 0.15$ ), and graceful fallback degradation.
+-----------------------------------------------------------------------------------+
| AI CLIENT LAYER |
| (Claude Code / OpenCode / Qwen-Code / Claude Desktop / Cursor / Windsurf) |
+-----------------------------------------------------------------------------------+
|
(stdio or SSE Transport)
v
+-----------------------------------------------------------------------------------+
| MCP ORCHESTRATOR GATEWAY |
| |
| Upstream FastMCP Server (Meta-Tools: discover_tools & invoke_tool) |
| ------------------------------------------------------------------------------ |
| LangGraph Online Execution Pipeline: |
| 1. ContextParserNode --> Reads workspace (.py, .json, dependencies) |
| 2. Stage1RetrievalNode --> Top-N MCP Server lookup |
| 3. Stage2RetrievalNode --> Top-M Candidate Tool retrieval |
| 4. RankingNode --> 3-Term Weighted Scoring (QuerySim, ProjMatch, Past) |
| 5. LoggingNode --> Asynchronous Postgres/SQLite audit logging |
+-----------------------------------------------------------------------------------+
| | |
(Local SQLite / Postgres) (ChromaDB Vector Store) (Downstream stdio/SSE)
| | |
v v v
+------------------+ +--------------------+ +----------------------+
| mcp_orchestrator | | Vector Collections:| | Downstream Servers: |
| - mcp_servers | | - "servers" | | - filesystem-server |
| - mcp_tools | | - "tools" | | - memory-server |
| - query_logs | +--------------------+ | - github-server |
| - tool_logs | +----------------------+
+------------------+
graph TD
A[User Query] --> B[ContextParserNode]
B --> C[Stage1RetrievalNode]
C -->|Top-N Candidate Servers| D[Stage2RetrievalNode]
D -->|Top-M Candidate Tools| E[RankingNode]
E -->|Sorted Top-K Ranked Tools| F[LoggingNode]
F --> G[Return Tools to AI Client]
Candidates are scored using the 3-term normalized formula:
Where:
-
$\mathbf{\text{QuerySim}}$ : Cosine similarity between user query vector and tool description embedding$[0, 1]$ . -
$\mathbf{\text{ProjMatch}}$ : Cosine similarity between project workspace context embedding (dependencies, file extensions) and tool embedding$[0, 1]$ . -
$\mathbf{\text{PastSuccess}}$ : Historical execution success rate tracked in database (default cold-start$= 0.5$ ). -
Base Weights: Default equal weights
$w_1 = 0.33, w_2 = 0.33, w_3 = 0.34$ . If project context is missing,$w_2=0$ and remaining weights are re-normalized.
git clone https://github.com/YourUsername/MCP-Orchestrator.git
cd "MCP Orchestrator"# Create and activate Python 3.11+ virtual environment
python -m venv venv
# Windows PowerShell:
.\venv\Scripts\Activate.ps1
# Linux / macOS:
source venv/bin/activate
# Install package in editable mode with development dependencies
pip install -e .[dev]
### Step 3: Downstream Server Configuration (`mcp_orc.json`)
MCP Orchestrator features **Automatic Machine-Wide AppData Resolution & Dynamic Hot-Reloading**:
1. **Global AppData Location**: MCP Orchestrator automatically creates and checks `~/.mcp-orchestrator/mcp_orc.json` (`C:\Users\patil\.mcp-orchestrator\mcp_orc.json` on Windows).
2. **Automatic Hot-Reloading (`mtime`)**: When a user adds or edits downstream servers in `mcp_orc.json`, Orchestrator detects the file modification timestamp (`mtime`) and **automatically indexes the new tools in the background—zero manual commands required!**
3. **Multi-Root Filesystem Support**: Pass multiple project paths in the `args` array so `filesystem-server` can access all your active workspaces simultaneously.
4. **Downstream Transport Support**: Natively supports both local `stdio` subprocess servers (`npx`, `uvx`, `python`) and remote `sse` HTTP endpoints (`https://gitmcp.io/docs/sse`) with optional HTTP `"headers"`.
Example `mcp_orc.json` (mixing Option A & Option B with multi-root filesystem access):
```json
{
"_info": "MCP Orchestrator Global Downstream Server Registry",
"_guide_option_A": "Option A (Simplicity): Paste raw secret strings directly in 'env' (e.g. 'BRAVE_API_KEY': 'BSA_xyz123')",
"_guide_option_B": "Option B (Security): Use '${VAR_NAME}' template syntax (e.g. 'GITHUB_TOKEN': '${GITHUB_TOKEN}') and keep secrets in .env",
"mcpServers": {
"filesystem": {
"transport": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-filesystem", "C:/Users/patil/OneDrive/Desktop/Projects", "C:/Users/patil/Desktop"],
"env": {}
},
"github": {
"transport": "stdio",
"command": "npx",
"args": ["-y", "@modelcontextprotocol/server-github"],
"env": {
"GITHUB_PERSONAL_ACCESS_TOKEN": "${GITHUB_PERSONAL_ACCESS_TOKEN}"
}
},
"gitmcp": {
"type": "remote",
"url": "https://gitmcp.io/docs/sse",
"headers": {},
"enabled": true
}
}
}📖 Looking for a beginner step-by-step setup guide?
Check out the complete [GUIDE.md](file:///c:/Users/patil/OneDrive/Desktop/Projects/MCP Orchestrator/GUIDE.md) for detailed PyPI, Docker, Python, and client configuration instructions!
You can run MCP Orchestrator using three deployment modes:
Install and launch directly via PyPI CLI executable:
# 1. Install package from PyPI
pip install mcp-orchestrator
# 2. Launch FastMCP SSE Gateway Server (with built-in auto-sync)
mcp-orchestrator --transport sse --port 8000
# Optional: Force sync downstream servers and exit cleanly
mcp-orchestrator --syncUsed for local CLI tools (Claude Code, OpenCode, Qwen-Code) and IDE extensions (Cursor, Claude Desktop):
python -m orchestrator.server --transport stdioUsed for web clients, remote agents, or network connections listening on port 8000:
python -m orchestrator.server --transport sse --host 0.0.0.0 --port 8000Run the complete multi-container stack (PostgreSQL + ChromaDB + MCP Orchestrator Gateway):
docker-compose up -d --buildThis spins up:
- PostgreSQL Database: Port
5432(metadata & invocation audit logs) - ChromaDB Vector Store: Port
8001(server and tool vector collections) - MCP Orchestrator Gateway: Port
8000(SSE transport endpoint athttp://localhost:8000/sse)
To stop the container stack:
docker-compose downTip
Docker Troubleshooting on Windows: If docker-compose up fails with open //./pipe/dockerDesktopLinuxEngine: The system cannot find the file specified, start Docker Desktop from your Windows Start Menu and wait until the engine displays "Running" before running docker-compose.
MCP Inspector is the official developer visual tool created by Anthropic/Model Context Protocol to inspect and test MCP servers interactively in your browser.
In a separate terminal, launch the Inspector:
npx @modelcontextprotocol/inspectorIt will output: MCP Inspector is running at http://localhost:5173.
When the Inspector modal opens in your browser, enter the following exact values:
-
For SSE Mode (server running locally or via Docker Compose on port
8000):- Server ID:
mcp-orchestrator - Transport Type:
sse(Server-Sent Events) - URL:
http://localhost:8000/sse - Click Connect.
- Server ID:
-
For stdio Mode (direct command launch):
- Server ID:
mcp-orchestrator - Transport Type:
stdio - Command:
python - Arguments:
-m orchestrator.server - Environment Variables:
PYTHONPATH=. - Click Connect.
- Server ID:
Once connected, MCP Inspector will display 2 Meta-Tools: discover_tools and invoke_tool.
Select discover_tools in the left menu and enter arguments:
-
Example 1: Filesystem Query
{ "query": "read configuration file in current workspace", "workspace_dir": ".", "bypass": false }Click Execute Tool $\rightarrow$ Returns top-ranked tools (
filesystem-server:read_file,filesystem-server:list_directory) with score breakdowns. -
Example 2: Memory & Knowledge Graph Query
{ "query": "store entity observation notes into memory graph", "workspace_dir": ".", "bypass": false }Click Execute Tool $\rightarrow$ Returns memory candidates (
memory-server:create_entities,memory-server:add_observations).
Select invoke_tool in the left menu and enter field parameters:
Important
Server ID Format: Server IDs use hyphens (filesystem-server), NOT underscores (filesystem_server).
Arguments Format: The arguments field requires a valid JSON object ({}), NOT an empty string.
-
Example 1: Read a Workspace File (
read_file)server_id:filesystem-servertool_id:read_filearguments:{ "path": "mcp_orc.json" }
-
Example 2: List Directory Contents (
list_directory)server_id:filesystem-servertool_id:list_directoryarguments:{ "path": "." }
-
Example 3: Create Knowledge Graph Entities (
create_entities)server_id:memory-servertool_id:create_entitiesarguments:{ "entities": [ { "name": "InspectorTest", "entityType": "BenchmarkTest", "observations": ["Tested invoke_tool via MCP Inspector UI"] } ] }
MCP Orchestrator can be integrated into any MCP-compliant AI client using either stdio (command launch) or SSE (HTTP connection).
Add MCP Orchestrator directly via the Claude Code CLI:
# Option A: Direct stdio launch (Recommended)
claude mcp add mcp-orchestrator -- python -m orchestrator.server
# Option B: SSE network launch (if gateway is running on port 8000)
claude mcp add mcp-orchestrator --url http://localhost:8000/sseFor OpenCode and Qwen-Code, add MCP Orchestrator to your configuration file (e.g., opencode.json or qwen_mcp.json).
{
"mcpServers": {
"mcp-orchestrator": {
"command": "python",
"args": ["-m", "orchestrator.server"],
"env": {
"PYTHONPATH": "C:/path/to/MCP Orchestrator"
}
}
}
}{
"mcp-orchestrator": {
"type": "local",
"command": ["python", "-m", "orchestrator.server"],
"enabled": true
}
}Edit your claude_desktop_config.json:
- Windows:
%APPDATA%\Claude\claude_desktop_config.json - macOS:
~/Library/Application Support/Claude/claude_desktop_config.json
{
"mcpServers": {
"mcp-orchestrator": {
"command": "python",
"args": [
"-m",
"orchestrator.server"
],
"env": {
"PYTHONPATH": "C:/path/to/MCP Orchestrator"
}
}
}
}- Open Cursor Settings
$\rightarrow$ Features$\rightarrow$ MCP. - Click + Add New MCP Server.
-
Name:
mcp-orchestrator. -
Type:
command(stdio) orsse. -
Command / URL:
- Command:
python -m orchestrator.server - SSE URL:
http://localhost:8000/sse
- Command:
MCP Orchestrator includes a single-page web analytics dashboard for inspecting live metrics, token savings, and ablation F1 charts.
python -m uvicorn dashboard.app:app --port 8002Navigate to http://localhost:8002 in your browser to view:
- KPI Metrics: Token Reduction (
90.3%), Prompt Tokens Saved per turn (6,724.9), Task Success Rate (85.0%), and Latency (67.16 ms). - Interactive Chart.js Visualization: F1@5 scores across all ablation variants.
| Metric | Bypass Baseline (Naive Load-All) | Full System ( |
Full System ( |
Tuned System ( |
|---|---|---|---|---|
| Exposed Tokens (Avg) | 7450.0 |
1460.5 |
725.1 |
725.1 |
| Context Window Token Red. | 0.0% |
80.4% |
90.3% |
90.3% |
| Precision @ K | 0.0286 |
0.1250 |
0.2100 |
0.2300 |
| Recall @ K | 1.0000 |
0.9250 |
0.8000 |
0.8500 |
| F1 Score @ K | 0.0554 |
0.2174 |
0.3262 |
0.3547 |
| Task Success Rate | 90.0% |
85.0% |
85.0% |
85.0% |
| Recovery Rate | 90.0% |
85.0% |
85.0% |
85.0% |
| Ranking Latency | 2.05 ms |
64.51 ms |
67.16 ms |
67.16 ms |
-
Top-K Truncation Necessity: Bypassing Top-K retrieval (
No-Top-K) causes statistically significant F1 degradation ($p = 0.0455 < 0.05$ , Cohen's$d = 0.4873$ ). -
Out-of-Sample Weight Calibration: Leave-One-Out Cross-Validation grid search selected optimal weights
$w_1=0.25, w_2=0.70, w_3=0.05$ , yielding$F1@5 = 0.3547$ (+8.7% relative gain over default weights). -
Vector Space Equivalence: Omitting project context (
No-ProjMatch) and corrupting vector space alignment (No-Shared-Space) both yield$F1=0.2976$ , confirming that destroying vector space alignment mathematically collapses context signal.
When demonstrating this project to advisors or evaluators:
-
Demonstrate Token Reduction: Show that preloading 49 tools consumes 7,450 tokens, whereas MCP Orchestrator exposes only 725 tokens (
$K=5$ ), achieving a 90.3% token reduction per turn. -
Explain 3-Term Ranking: Explain how
$w_1 \cdot \text{QuerySim} + w_2 \cdot \text{ProjMatch} + w_3 \cdot \text{PastSuccess}$ incorporates real-time workspace context (.py, dependencies) and historical execution feedback. -
Show Extensibility: Open
mcp_orc.json. Emphasize that adding new downstream MCP tools requires zero gateway code changes—simply update JSON definitions. -
Show Dashboard: Launch
python -m uvicorn dashboard.app:app --port 8002to visually present the ablation chart and benchmark comparison. - Explain Total Context vs. Tool Schema Layer: If asked how this reduces 85%–90%+ when an agent already has 20k tokens of base skills or plugins, explain that Orchestrator optimizes the MCP Tool Schema layer. Without Orchestrator, adding 20 MCP servers adds +21.8k tokens (jumping context to ~45k). With Orchestrator, the tool layer adds only ~0.7k tokens (a 96.7% tool schema reduction), saving hundreds of thousands of tokens across multi-turn sessions.
Developed and benchmarked solely by Sharvil Patil with ₹0.00 out-of-pocket financial outlay, MCP Orchestrator is designed for students, individual developers, and software teams interacting daily with agentic AI systems (OpenCode, Claude Code, Cursor, Windsurf).
As part of the academic case study evaluation for Course 2CSHS404 (Department of Computer Science and Engineering, Annasaheb Dange College of Engineering and Technology, Ashta), a comprehensive financial evaluation report was compiled: ACTIVITY_2_FINANCIAL_EVALUATION_REPORT.md.
| Financial Evaluation Metric | Target Threshold | Student Bootstrapped Reality | 100-User Cohort Model (INR ₹) | Status |
|---|---|---|---|---|
| Initial Capital Outlay ( |
Minimum | ₹0.00 Out-of-Pocket | ₹35,000.00 (Imputed Effort) | Optimal |
| Net Present Value (NPV) | Positive (Day One) | +₹3,92,300.66 | Accepted | |
| Internal Rate of Return (IRR) | 277.07% | Accepted | ||
| Simple Payback Period | 0 Days (Day One) | 4.49 Months | Accepted | |
| Discounted Payback Period | 0 Days (Day One) | 4.94 Months | Accepted | |
| Profitability Index (PI) | 12.21 | Accepted | ||
| Monthly API Bill Savings | Maximized | ~₹4,030.00 / month / dev | ~₹2,23,900 / month (Cohort) | Proven |
To prevent client connection timeouts (such as OpenCode's strict 30-second client timeout), CPU-intensive neural model pre-warming (all-MiniLM-L6-v2) and ChromaDB vector store collection checks are offloaded from the main asyncio thread to background worker threads via asyncio.to_thread:
initializeRPC Call: 2.15s (down from blocking > 30s)tools/listRPC Call: 2.1 ms (0.0021s)prompts/listRPC Call: 1.3 ms (0.0013s)resources/listRPC Call: 0.7 ms (0.0007s)resources/templates/listRPC Call: 22.0 ms (0.0220s)
Because mcp-orchestrator operates as a Python subprocess under stdio, avoid running blanket process termination commands (such as Get-Process -Name python | Stop-Process -Force or taskkill /F /IM python.exe) in agent terminal sessions when restarting project dev servers. Instead, terminate processes specifically by port or command line:
# Recommended: Kill process by listening port (e.g. port 8000)
Get-NetTCPConnection -LocalPort 8000 -ErrorAction SilentlyContinue | ForEach-Object { Stop-Process -Id $_.OwningProcess -Force }Distributed under the MIT License. See LICENSE for details.