feat(mcp): simplify MCP tool architecture and unify platform tools #1507

quanru · 2025-11-25T12:25:17Z

Summary

This PR refactors the MCP packages to use a simplified, consistent tool architecture across all platforms (Web, Android, iOS).

Key Changes

🏗️ Shared Infrastructure (`@midscene/shared/mcp`)

Created BaseMCPServer and BaseMidsceneTools base classes for code reuse
Implemented dynamic tool generation from agent.getActionSpace()
Simplified common tools to only take_screenshot and wait_for (removed assert)
Added unified tool generation pattern in tool-generator.ts

📦 Platform-Specific Packages

@midscene/mcp (Web): Simplified to single web_connect tool
@midscene/android-mcp (New): Android-specific MCP server with android_connect tool
@midscene/ios-mcp (New): iOS-specific MCP server with ios_connect tool

🎯 Simplified Tool Architecture

Each platform now provides exactly 3 types of tools:

ActionSpace tools (dynamic): Generated from agent.getActionSpace()
- Platform-specific interaction capabilities (tap, input, scroll, etc.)
Common tools (2 tools):
- take_screenshot: Capture current page/screen
- wait_for: Wait until condition becomes true
Platform connection (1 tool):
- web_connect: Connect to web page
- android_connect: Connect to Android device (optional deviceId and uri)
- ios_connect: Connect to iOS device

🔧 Improvements

Naming Consistency: Unified {platform}_connect pattern across all platforms
Reduced Tool Bloat: Removed platform-specific helpers (list_devices, check_environment, etc.)
Visual Feedback: All connection tools return screenshots
Better Architecture: Shared base classes reduce code duplication

✅ Testing

Removed obsolete tool snapshot tests
Fixed tests to work with new dynamic tool architecture
All MCP package tests passing
Build and lint checks passing

Migration Notes

Tools have been consolidated:

~~open_new~~ → web_connect
~~android_list_devices~~ → Removed (use android_connect without deviceId)
~~android_launch~~ → Merged into android_connect (use uri parameter)
~~ios_check_environment~~ → Removed (just use ios_connect)
~~assert~~ → Removed from common tools

Related Issues

Part of MCP architecture refactoring to support multiple platforms with consistent patterns.

🤖 Generated with Claude Code

netlify · 2025-11-25T12:25:23Z

✅ Deploy Preview for midscene ready!

Name	Link
🔨 Latest commit	`e18c94b`
🔍 Latest deploy log	https://app.netlify.com/projects/midscene/deploys/6929819d61742f0008456457
😎 Deploy Preview	https://deploy-preview-1507--midscene.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

Copilot

Pull request overview

This PR refactors the MCP (Model Context Protocol) server architecture to use a unified, dynamic tool generation pattern across all platforms (Web, Android, iOS), significantly reducing code duplication and improving maintainability.

Key Changes

Introduced shared base classes (BaseMCPServer and BaseMidsceneTools) in @midscene/shared/mcp to provide common infrastructure for all platform MCP servers
Implemented dynamic tool generation from agent.getActionSpace() instead of hardcoded tool definitions, making the architecture more flexible
Created platform-specific MCP packages: @midscene/android-mcp for Android automation and @midscene/ios-mcp for iOS automation, plus @midscene/web-mcp as an alias for the existing @midscene/mcp

Reviewed changes

Copilot reviewed 38 out of 39 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
packages/shared/src/mcp/base-server.ts	New base MCP server class providing common launch() API and lifecycle management
packages/shared/src/mcp/base-tools.ts	New base tools manager with dynamic tool initialization from action space
packages/shared/src/mcp/tool-generator.ts	Core tool generation logic converting action space to MCP tool definitions
packages/shared/src/mcp/types.ts	Type definitions for tool architecture
packages/shared/src/mcp/index.ts	Export aggregation for shared MCP infrastructure
packages/shared/package.json	Added MCP module export path and SDK dependency
packages/mcp/src/server.ts	Refactored Web MCP server to extend BaseMCPServer
packages/mcp/src/web-tools.ts	New Web-specific tools manager extending BaseMidsceneTools
packages/mcp/src/index.ts	Simplified CLI entry to use new server architecture
packages/mcp/src/tools.ts	Removed (replaced by dynamic tool generation)
packages/mcp/src/midscene.ts	Removed (logic moved to base classes and web-tools)
packages/mcp/package.json	Added exports field for server module
packages/mcp/tests/index.test.ts	Removed obsolete tool snapshot tests
packages/android-mcp/*	New Android MCP server package with android_connect tool
packages/ios-mcp/*	New iOS MCP server package with ios_connect tool
packages/web-mcp/*	New alias package re-exporting @midscene/mcp for naming consistency
pnpm-lock.yaml	Updated dependencies for new packages
package.json	Added new packages to test script

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (4)

packages/android-mcp/rslib.config.ts:1

Unused import path.

import path from 'node:path';

packages/ios-mcp/rslib.config.ts:1

Unused import path.

import path from 'node:path';

packages/ios-mcp/src/ios-tools.ts:5

Unused import z.

import { z } from 'zod';

packages/shared/src/mcp/tool-generator.ts:29

Unused variable result.

      const result = await agent.action(`Use the action "${action.name}"`, {

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/ios-mcp/src/ios-tools.ts

packages/android-mcp/README.md

packages/shared/src/mcp/tool-generator.ts

packages/web-mcp/src/index.ts

packages/android-mcp/rslib.config.ts

packages/shared/src/mcp/tool-generator.ts

packages/ios-mcp/README.md

packages/android-mcp/README.md

packages/shared/src/mcp/base-tools.ts

Copilot

Pull request overview

Copilot reviewed 39 out of 40 changed files in this pull request and generated 7 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

packages/shared/src/mcp/base-tools.ts

packages/shared/src/mcp/tool-generator.ts

packages/android-mcp/src/android-tools.ts

packages/mcp/src/web-tools.ts

packages/android-mcp/src/android-tools.ts

packages/ios-mcp/rslib.config.ts

Refactored MCP packages to use a simplified, consistent tool architecture across all platforms (Web, Android, iOS). - Created base classes for MCP servers and tools - Implemented dynamic tool generation from agent actionSpace - Simplified common tools to only take_screenshot and wait_for - Added unified tool generation pattern - Web (@midscene/mcp): Simplified to single web_connect tool - Android: Consolidated into android_connect with optional params - iOS: Simplified to single ios_connect tool Each platform now provides exactly 3 types of tools: 1. ActionSpace tools (dynamic): From agent.getActionSpace() 2. Common tools (2): take_screenshot, wait_for 3. Platform connection (1): {platform}_connect - Unified naming pattern: {platform}_connect across all platforms - Removed platform helpers (list_devices, check_environment, etc) - All connection tools return screenshots for visual feedback - Removed obsolete tool snapshot tests - Fixed tests to work with new dynamic tool architecture - All MCP package tests passing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <[email protected]>

…oss packages

Co-authored-by: Copilot <[email protected]>

refactor(ios): remove unused path import in rslib.config refactor(shared): change agent.action to agent.aiAction in tool-generator

…tion messages

…lib configuration

… enhance tool initialization handling

…Web tools; enhance action space retrieval

… and Web tools; streamline temporary device creation

Copilot

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

…ers; enhance command line argument parsing

…gic for Android, iOS, and Web MCP servers

… MCP server and tools

… for unified HTTP support and argument parsing

Copilot

Pull request overview

Copilot reviewed 41 out of 45 changed files in this pull request and generated 6 comments.

Files not reviewed (1)

pnpm-lock.yaml: Language not supported

Comments suppressed due to low confidence (1)

packages/mcp/tests/index.test.ts:7

The test file imports deepMerge, getChromePathFromEnv, and getSystemChromePath from ../src/utils, but this file no longer exists in the packages/mcp package. These utilities have been moved to packages/web-mcp/src/utils.ts as part of the refactoring.

Since @midscene/mcp is now just a wrapper for @midscene/web-mcp, these tests should either be:

Moved to the packages/web-mcp package where the actual implementation now lives, or
Updated to import from @midscene/web-mcp if testing the wrapper functionality

The current import path will cause test failures.

import {
  deepMerge,
  getChromePathFromEnv,
  getSystemChromePath,
} from '../src/utils';

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-11-28T11:09:43Z

packages/web-mcp/src/web-tools.ts

+    if (!this.puppeteerMode) {
+      if (!openNewTabWithUrl) {
+        throw new Error(
+          'Bridge mode requires a URL. Use web_connect tool to connect to a page first.',
+        );
+      }


In bridge mode (when puppeteerMode is false), calling ensureAgent() without a URL parameter will throw an error: "Bridge mode requires a URL. Use web_connect tool to connect to a page first."

However, in packages/shared/src/mcp/base-tools.ts line 58, the initTools() method calls await this.ensureAgent() without any parameters during initialization. This means that when the server starts in bridge mode, initialization will fail.

The comment on line 64 mentions "agent creation will be deferred until first tool use", but the error is thrown immediately. Consider either:

Returning null or a placeholder instead of throwing, to allow deferred initialization

Updating the initTools() fallback logic to handle this case gracefully

Always using the temporary device fallback for bridge mode during initialization

Copilot · 2025-11-28T11:09:43Z

packages/ios-mcp/src/server.ts

+import { version } from '../package.json';
+import { IOSMidsceneTools } from './ios-tools.js';


Inconsistent version handling: This file imports version from package.json while web-mcp/src/server.ts uses __VERSION__ (defined via rslib config).

For consistency with web-mcp and to avoid potential TypeScript issues with JSON imports, consider using the __VERSION__ constant which is already defined in the rslib config (rslib.config.ts line 7).

Change:

import { version } from '../package.json';

to:

declare const __VERSION__: string;

And use __VERSION__ instead of version on line 13.

Copilot · 2025-11-28T11:09:43Z

packages/ios-mcp/rslib.config.ts

+        cb();
+      },
+      '@silvia-odwyer/photon',
+      '@silvia-odwyer/photon-node',


Missing externals configuration compared to web-mcp. The ios-mcp rslib config doesn't externalize:

@modelcontextprotocol/sdk

/^@midscene\/.*/ (workspace dependencies)

These are externalized in web-mcp and should be consistent across all MCP packages. Not externalizing workspace dependencies can lead to:

Larger bundle sizes

Duplicate code in the output

Potential version conflicts

Add these to the externals array:

externals: [ // ... existing externals ... /^@midscene\/.*/, '@modelcontextprotocol/sdk', ],

Suggested change

'@silvia-odwyer/photon-node',

'@silvia-odwyer/photon-node',

/^@midscene\/.*/,

'@modelcontextprotocol/sdk',

Copilot · 2025-11-28T11:09:44Z

packages/web-mcp/package.json

+    "@midscene/android": "workspace:*",
+    "@midscene/core": "workspace:*",


The @midscene/android package is listed as a devDependency but is not imported or used anywhere in the web-mcp source code. This appears to be leftover from when the packages were combined.

Since web-mcp is specifically for web automation and doesn't need Android functionality, this dependency should be removed to keep the package lean.

Copilot · 2025-11-28T11:09:44Z

packages/android-mcp/src/server.ts

+import { version } from '../package.json';
+import { AndroidMidsceneTools } from './android-tools.js';


Inconsistent version handling across MCP packages:

web-mcp/src/server.ts uses __VERSION__ (global defined via rslib)

android-mcp/src/server.ts and ios-mcp/src/server.ts import version from package.json

All three rslib configs define __VERSION__, so they should all use it consistently. Using the __VERSION__ constant (like web-mcp does) is preferable because:

It's consistently defined in all rslib configs

It avoids TypeScript issues with JSON imports

It follows the same pattern as web-mcp

Consider updating android-mcp and ios-mcp to use __VERSION__ like web-mcp does.

Copilot · 2025-11-28T11:09:44Z

packages/android-mcp/rslib.config.ts

+        cb();
+      },
+      '@silvia-odwyer/photon',
+      '@silvia-odwyer/photon-node',


Missing externals configuration compared to web-mcp. The android-mcp rslib config doesn't externalize:

@modelcontextprotocol/sdk

/^@midscene\/.*/ (workspace dependencies)

These are externalized in web-mcp (lines 28-29) and should be consistent across all MCP packages. Not externalizing workspace dependencies can lead to:

Larger bundle sizes

Duplicate code in the output

Potential version conflicts

Add these to the externals array:

externals: [ // ... existing externals ... /^@midscene\/.*/, '@modelcontextprotocol/sdk', ],

Suggested change

'@silvia-odwyer/photon-node',

'@silvia-odwyer/photon-node',

/^@midscene\/.*/,

'@modelcontextprotocol/sdk',

github-actions bot added the change: feat label Nov 25, 2025

quanru requested a review from Copilot November 25, 2025 12:36

Copilot started reviewing on behalf of quanru November 25, 2025 12:36 View session

Copilot finished reviewing on behalf of quanru November 25, 2025 12:40

Copilot AI reviewed Nov 25, 2025

View reviewed changes

quanru force-pushed the feat/mcp-simplify-tools branch from 9535f2f to 4613d05 Compare November 25, 2025 12:53

quanru requested a review from Copilot November 27, 2025 02:35

Copilot started reviewing on behalf of quanru November 27, 2025 02:35 View session

Copilot finished reviewing on behalf of quanru November 27, 2025 02:38

Copilot AI reviewed Nov 27, 2025

View reviewed changes

quanru and others added 5 commits November 27, 2025 10:53

refactor(shared): remove zod dependency and clean up README files acr…

77d2bd1

…oss packages

Update packages/shared/src/mcp/base-tools.ts

f3a6a8b

Co-authored-by: Copilot <[email protected]>

Update packages/mcp/src/web-tools.ts

e0ae1ca

Co-authored-by: Copilot <[email protected]>

fix(android): update launchUri to launch in AndroidMidsceneTools

9cfd87e

refactor(ios): remove unused path import in rslib.config refactor(shared): change agent.action to agent.aiAction in tool-generator

quanru force-pushed the feat/mcp-simplify-tools branch from ff57aac to 9cfd87e Compare November 27, 2025 02:53

quanru added 4 commits November 27, 2025 14:30

feat(android, ios): implement AI-driven app loading and update connec…

3299734

…tion messages

fix(mcp): update type definitions path in package.json and enhance rs…

b843d59

…lib configuration

feat(mcp): add default action spaces for Android, iOS, and Web tools;…

bcf7a1a

… enhance tool initialization handling

feat(mcp): implement temporary device creation for Android, iOS, and …

c94a329

…Web tools; enhance action space retrieval

quanru requested a review from Copilot November 28, 2025 07:37

Copilot started reviewing on behalf of quanru November 28, 2025 07:37 View session

Copilot finished reviewing on behalf of quanru November 28, 2025 07:37

refactor(mcp): remove default action space methods from Android, iOS,…

8f47ac4

… and Web tools; streamline temporary device creation

quanru force-pushed the feat/mcp-simplify-tools branch from bec2957 to 8f47ac4 Compare November 28, 2025 08:14

Copilot AI reviewed Nov 28, 2025

View reviewed changes

quanru added 3 commits November 28, 2025 17:16

feat(mcp): add HTTP launch support for Android, iOS, and Web MCP serv…

1c579af

…ers; enhance command line argument parsing

refactor(mcp): streamline command line argument parsing and launch lo…

f900bc6

…gic for Android, iOS, and Web MCP servers

refactor(mcp): enhance type definitions and improve error handling in…

a7fbb7d

… MCP server and tools

quanru added 2 commits November 28, 2025 19:03

refactor(mcp): delegate web MCP server launch to the main MCP package…

5563421

… for unified HTTP support and argument parsing

feat(mcp): refactor MCP server structure and tools

e18c94b

quanru requested a review from Copilot November 28, 2025 11:04

Copilot started reviewing on behalf of quanru November 28, 2025 11:04 View session

Copilot finished reviewing on behalf of quanru November 28, 2025 11:08

Copilot AI reviewed Nov 28, 2025

View reviewed changes

		import { version } from '../package.json';
		import { IOSMidsceneTools } from './ios-tools.js';

		"@midscene/android": "workspace:*",
		"@midscene/core": "workspace:*",

		import { version } from '../package.json';
		import { AndroidMidsceneTools } from './android-tools.js';

feat(mcp): simplify MCP tool architecture and unify platform tools #1507

Are you sure you want to change the base?

feat(mcp): simplify MCP tool architecture and unify platform tools #1507

Uh oh!

Conversation

quanru commented Nov 25, 2025

Summary

Key Changes

🏗️ Shared Infrastructure (@midscene/shared/mcp)

📦 Platform-Specific Packages

🎯 Simplified Tool Architecture

🔧 Improvements

✅ Testing

Migration Notes

Related Issues

Uh oh!

netlify bot commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for midscene ready!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 28, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

🏗️ Shared Infrastructure (`@midscene/shared/mcp`)

netlify bot commented Nov 25, 2025 •

edited

Loading