-
Notifications
You must be signed in to change notification settings - Fork 99
feat(doc): add test section #865
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| --- | ||
| title: "Build Your App" | ||
| description: "The four decisions that make an MCP App" | ||
| icon: "drafting-compass" | ||
| --- | ||
|
|
||
| Building an MCP App means answering four questions: what the model can do, what the user sees, what the model knows, and who is asking. Each page below covers one, in build order, on the same running example: a personal shopper with a product carousel and an authenticated checkout. | ||
|
|
||
| | Define | Page | Core APIs | | ||
| |------------|------|-----------| | ||
| | What the model can do | [Register Tools](/build/tools) | `registerTool` | | ||
| | What the user sees | [Create Views](/build/view) | `useToolInfo`, `useCallTool` | | ||
| | What the model knows | [Manage State](/build/state) | `useViewState`, `data-llm` | | ||
| | Who is asking | [Authenticate Users](/build/auth) | `requireBearerAuth`, `securitySchemes` | | ||
|
|
||
| <Columns cols={2}> | ||
| <Card title="Register Tools" img="/images/tools.png" href="/build/tools"> | ||
| Define what humans and agents can do | ||
| </Card> | ||
| <Card title="Create Views" img="/images/view.png" href="/build/view"> | ||
| Craft interactive UIs rendered in conversation | ||
| </Card> | ||
| <Card title="Manage State" img="/images/state.png" href="/build/state"> | ||
| Decide what the model sees | ||
| </Card> | ||
| <Card title="Authenticate Users" img="/images/auth.png" href="/build/auth"> | ||
| Know who's behind every tool call | ||
| </Card> | ||
| </Columns> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| --- | ||
| title: "Audit" | ||
| description: "Catch spec and platform issues before submission" | ||
| icon: "clipboard-check" | ||
| --- | ||
| <img src="/images/audit.png" alt="Audit Illustration" style={{ width: "100%", maxHeight: "40vh", marginTop:"-0.75rem", objectFit: "cover", borderRadius: "15px" }} /> | ||
|
|
||
| Before a platform accepts your app, the server must conform to the MCP specs, and each platform layers its own requirements on top, which differ between OpenAI and Anthropic. The Audit checks your app's conformance and performs end-to-end tests in real ChatGPT and Claude conversations, triggering tools and rendering views. | ||
|
|
||
| ## Run the Audit | ||
|
|
||
| The Audit needs your server publicly reachable: start the [tunnel](/test/tunnel), then click **Audit** in the [DevTools](/test/devtools) header. | ||
|
|
||
| <CodeGroup> | ||
| ```bash npm | ||
| npm run dev -- --tunnel | ||
| ``` | ||
| ```bash pnpm | ||
| pnpm dev --tunnel | ||
| ``` | ||
| ```bash yarn | ||
| yarn dev --tunnel | ||
| ``` | ||
| ```bash bun | ||
| bun dev --tunnel | ||
| ``` | ||
| ```bash deno | ||
| deno task dev --tunnel | ||
| ``` | ||
| </CodeGroup> | ||
|
|
||
| <Info> | ||
| The Audit is powered by [Alpic Beacon](https://docs.alpic.ai/testing/beacon). It can also be launched from [beacon.alpic.ai](https://beacon.alpic.ai) with your server's public URL, or via the [Alpic CLI](https://docs.alpic.ai/cli/tunnel). | ||
| </Info> | ||
|
|
||
| ## What It Checks | ||
|
|
||
| - **MCP protocol**: tool and resource conformance with the official specification. | ||
| - **MCP Apps**: view resource requirements (metadata, Content Security Policy, annotations) for both ChatGPT and Claude. | ||
| - **Platform requirements**: each platform's own submission guidelines. | ||
| - **App behavior**: the app actually works, verified by triggering tools and rendering views in real ChatGPT and Claude conversations. | ||
|
|
||
| <Frame caption="The Audit interface"> | ||
| <img src="/images/audit-screenshot.png" alt="Audit Illustration" /> | ||
| </Frame> | ||
|
|
||
| The report returns an overall score, a readiness status per platform, and the issues found, each with its severity (error, warning, info), the affected component, and the solution. **Improve your score** turns the report into a prompt for your coding agent to fix them all. | ||
|
|
||
| ## Limitation | ||
|
|
||
| OAuth-protected servers aren't supported yet. | ||
|
|
||
| ## Go Further | ||
| <Columns cols={3}> | ||
| <Card title="DevTools" img="/images/devtools.png" href="/test/devtools"> | ||
| Call tools and render views locally, without a host | ||
| </Card> | ||
| <Card title="Tunnel" img="/images/tunnel.png" href="/test/tunnel"> | ||
| Expose your local server to real hosts | ||
| </Card> | ||
| <Card title="Playground" img="/images/playground.png" href="/test/playground"> | ||
| Chat with a real model running your app | ||
| </Card> | ||
| </Columns> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,85 @@ | ||
| --- | ||
| title: "DevTools" | ||
| description: "Call tools and render views locally, without a host" | ||
| icon: "square-dashed-mouse-pointer" | ||
| --- | ||
| {/* @todo: devtools illustration asset */} | ||
| <img src="/images/devtools.png" alt="DevTools Illustration" style={{ width: "100%", maxHeight: "40vh", marginTop:"-0.75rem", objectFit: "cover", borderRadius: "15px" }} /> | ||
|
|
||
| Testing through a real host means exposing a public URL, registering a connector, starting a conversation, and prompting the model until it calls the right tool. DevTools cuts that loop down to a file save: it runs locally, emulates the host runtime, and lets you call tools directly, no model or host involved. | ||
|
|
||
| ## Start the Emulator | ||
|
|
||
| DevTools ships with the dev server. In a scaffolded project (`npx skybridge create`), start it from the project root: | ||
|
|
||
| <CodeGroup> | ||
| ```bash npm | ||
| npm run dev | ||
| ``` | ||
| ```bash pnpm | ||
| pnpm dev | ||
| ``` | ||
| ```bash yarn | ||
| yarn dev | ||
| ``` | ||
| ```bash bun | ||
| bun dev | ||
| ``` | ||
| ```bash deno | ||
| deno task dev | ||
| ``` | ||
| </CodeGroup> | ||
|
|
||
| Two routes come up: | ||
| - `http://localhost:3000/mcp`: your MCP server | ||
| - `http://localhost:3000/`: the DevTools | ||
|
|
||
| Every tool registered on the server appears in the sidebar automatically, and edits to the server or the views reload the preview on save. | ||
|
|
||
| ## Explore the Features | ||
|
|
||
| Select a tool in the sidebar and DevTools gives you the full exchange: | ||
|
|
||
| <Frame caption="The DevTools interface"> | ||
| <img src="/images/devtools-screenshot.png" alt="DevTools Illustration" /> | ||
| </Frame> | ||
|
|
||
| - **Tool controls**: generated from the tool's input schema. Inputs can be saved and re-run from the tool header. | ||
| - **Tool output**: the raw server response (`content`, `structuredContent`, `_meta`), with status, latency, and payload size. | ||
| - **State inspector**: the view state as a JSON tree, updated as the view mutates it. | ||
| - **View preview**: the rendered view, with live controls to switch display mode, theme, locale, and device type; the preview updates immediately. | ||
| - **Call logs**: every runtime API call the view makes (`setViewState`, `callTool`, `requestDisplayMode`) with its arguments and responses. | ||
|
|
||
| ## Limitations | ||
|
|
||
| DevTools emulates, and three gaps separate the emulation from production: | ||
|
|
||
| - **No model**: you pick the tool and type the arguments yourself. Tool selection, the prompt surface (names, descriptions, schemas), and follow-up messages are never exercised. | ||
| - **One runtime**: as of today the preview mocks the Apps SDK runtime. MCP Apps-specific features are not yet supported. | ||
| - **Loose CSP**: external resources blocked by hosts in production may load locally. | ||
|
|
||
| ## Custom Integration | ||
|
|
||
| Projects created with `npx skybridge create` serve DevTools automatically. If you added Skybridge to an existing Express app instead, register the two DevTools middlewares yourself: | ||
|
|
||
| ```ts server.ts | ||
| import { devtoolsStaticServer, viewsDevServer } from "skybridge/server"; | ||
|
|
||
| if (process.env.NODE_ENV !== "production") { | ||
| app.use(await devtoolsStaticServer()); | ||
| app.use(await viewsDevServer()); | ||
| } | ||
| ``` | ||
|
|
||
| ## Go Further | ||
| <Columns cols={3}> | ||
| <Card title="Tunnel" img="/images/tunnel.png" href="/test/tunnel"> | ||
| Expose your local server to real hosts | ||
| </Card> | ||
| <Card title="Playground" img="/images/playground.png" href="/test/playground"> | ||
| Chat with a real model running your app | ||
| </Card> | ||
| <Card title="Audit" img="/images/audit.png" href="/test/audit"> | ||
| Catch spec and platform issues before submission | ||
| </Card> | ||
| </Columns> | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,29 @@ | ||
| --- | ||
| title: "Test Your App" | ||
| description: "Pick the right tool for what you're testing" | ||
| icon: "test-tube-diagonal" | ||
| --- | ||
|
|
||
| Testing an MCP App means assessing how well it integrates with the host, interacts with the model, and responds to the user. The tools below put these actors in the loop, each trading setup for fidelity. | ||
|
|
||
| | To | Use | Real model | Real host | Interactive | | ||
| |-------------|-----|------------|-----------|-------------| | ||
| | Iterate on tools and views | [DevTools](/test/devtools) | <Icon icon="x" color="#EF4444" /> | <Icon icon="x" color="#EF4444" /> | <Icon icon="check" color="#22C55E" /> | | ||
| | Verify behavior in ChatGPT and Claude | [Tunnel](/test/tunnel) | <Icon icon="check" color="#22C55E" /> | <Icon icon="check" color="#22C55E" /> | <Icon icon="check" color="#22C55E" /> | | ||
| | Check what the model does with your app | [Playground](/test/playground) | <Icon icon="check" color="#22C55E" /> | <Icon icon="x" color="#EF4444" /> | <Icon icon="check" color="#22C55E" /> | | ||
| | Validate before submission | [Audit](/test/audit) | <Icon icon="check" color="#22C55E" /> | <Icon icon="check" color="#22C55E" /> | <Icon icon="x" color="#EF4444" /> | | ||
|
paulleseute marked this conversation as resolved.
|
||
|
|
||
| <Columns cols={2}> | ||
| <Card title="DevTools" img="/images/devtools.png" href="/test/devtools"> | ||
| Call tools and render views locally, without a host | ||
| </Card> | ||
| <Card title="Tunnel" img="/images/tunnel.png" href="/test/tunnel"> | ||
| Expose your local server to real hosts | ||
| </Card> | ||
| <Card title="Playground" img="/images/playground.png" href="/test/playground"> | ||
| Chat with a real model running your app | ||
| </Card> | ||
| <Card title="Audit" img="/images/audit.png" href="/test/audit"> | ||
| Catch spec and platform issues before submission | ||
| </Card> | ||
| </Columns> | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.