ai-qa is an agent-orchestrated QA CLI and Agent Skill for Web, iOS Simulator, and Android Emulator. Your host Agent operates the browser, Simulator, or Emulator through the configured controller; the CLI records and validates readiness, actions, evidence, assertions, cases, verdicts, RunGroups, and reports.
Real iOS and Android devices are not supported.
- Node.js 22 or 24.
- An Agent host that can use Agent Skills and the controller for each target platform.
- Web: Chrome DevTools MCP.
- iOS Simulator: Pepper.
- Android Emulator: Appium with UiAutomator2.
Install the public package globally, then install its bundled Agent Skill:
npm install --global @narra-im/ai-qa
ai-qa --helpInstall the Skill into your Agent host's skill root. The root is explicit — there is no default:
# Codex
ai-qa skill install --agents-home ~/.agents
# Claude Code
ai-qa skill install --agents-home ~/.claude
ai-qa skill check --agents-home ~/.claudeAn explicit skill install or skill sync automatically replaces the global
Skill's CLI-managed content and bundled references/ and migrations/ during
a forward version upgrade. It preserves the user region and files outside
those managed roots. If a forward upgrade is interrupted, rerun skill sync;
the retry remains automatic and needs no confirmation. Same-version managed
repair or downgrade requires --confirm-managed-replacement.
Global Skill installation is separate from project migration: it only updates the selected Agent skill root and never scans or edits projects. On the next entry to a trusted project, the Agent checks the project format before doctor. Ordinary releases with no project-format change do not touch Project Skill files. Breaking updates load their installed, versioned Migration Skills and run the chain automatically without confirmation on the normal path. Conflicts fail closed, interrupted migrations remain recoverable, and historical QA records remain immutable.
Run AI QA from the exact project you want to test. Humans normally describe the work to an Agent; the Agent uses the installed Skill, platform controller, and CLI.
First ask the Agent to configure the project:
Configure AI QA for this project. The deployed platforms are Web and iOS Simulator. Keep reports local. Show me the complete proposed files before writing anything.
After configuration and readiness checks pass, ask for QA:
On Web, explore sign-in. Start from the sign-in page with a valid test account. A successful sign-in must open the dashboard without an error. Keep the report local and show me the verdict with its evidence.
The Agent handles readiness, controller actions, evidence, verdicts, and report generation.
On first use, QA remains blocked until the Agent completes this setup with you:
- Check the project: the Agent resolves the exact project root, derives its own host label and skill root, confirms them on first use, runs
project upgrade check, completes any returned migration chain, and only then runs doctor with that host context. - Select deployed platforms: choose at least one deployed Web, iOS Simulator, or Android Emulator target. Real devices are not supported.
- Choose result handling: explicitly select
local-onlyorproject-skill; neither mode is selected by default. - Collect safe configuration: the Agent gathers the target and controller settings for every selected platform and keeps literal secrets out of config.
- Review both proposals: the Agent validates
.ai-qa/config.yamland the canonical.ai-qa/project-skill/SKILL.mdtogether, then shows complete new-file content or complete existing-file diffs. One confirmation covers both files; cancellation writes neither. - Write and check: after that confirmation, the Agent writes the config and canonical Project Skill once, then repeats upgrade preflight before doctor for every configured platform.
- Provision the current host when required: if doctor returns
provision-host-skill, the Agent renders the canonical Project Skill in that host's normal project-skill location, shows the complete proposal or diff, obtains a separate confirmation, writes it, and registers the rendering. Each host has its own registered rendering; rerun doctor until every requested platform is ready before starting QA.
During project migration recovery, every history-writing command is
unavailable. The Agent inspects the declared targets and validation invariants,
retries completion when every invariant already holds, or aborts when one does
not. Abort restores declared targets and CLI-owned project state while
preserving undeclared config edits. The Agent then reruns upgrade check and the
migration, and runs doctor only after check reports up_to_date.
The Agent handles schema validation, path and symlink checks, directory creation, and controller-specific readiness details.
A useful request states:
- Platform: which configured Web, iOS Simulator, or Android Emulator targets to run now.
- Goal: the user behavior or product result to verify.
- Preconditions: starting screen, login state, feature flags, or required data.
- Acceptance criteria: observable results that determine success or failure.
- Test data: account or data requirements, using secret references instead of literal credentials.
- Result handling: keep verified reports local or use an already approved project recording procedure.
You do not need to provide work-order JSON, action IDs, evidence IDs, verdict payloads, or case revisions. Describe the outcome; the Agent manages the protocol.
Configure AI QA for this project. Web is deployed at
https://example.test, and reports should stay local. Inspect the project, show the complete configuration and project Skill proposal, and wait for my confirmation before writing.
On iOS Simulator, explore password reset. Start from the sign-in screen with a test account that can receive a reset link. The user must be able to request a reset and reach the confirmation state without an error. Capture evidence and return the verified report.
On Web, reproduce BUG-123 before the fix. Start on the sign-in page with a valid test account. Submitting valid credentials should open the dashboard, but the reported behavior stays on the sign-in page. Preserve an evidence-backed fail baseline and show me the report.
BUG-123 is fixed and deployed. On Web, start a new run with the same preconditions and acceptance criteria. Verify that valid sign-in opens the dashboard without an error. Keep this result separate from the pre-fix run and show me the new report.
I reviewed the passing BUG-123 result. Prepare it as regression case
bug-123-sign-in, show me the proposed case, and activate it only after I confirm.
On Web, replay the active
bug-123-sign-inregression case and return the verified report.
On Web and iOS Simulator, replay all active sign-in regression cases. Report every case/platform result and any coverage gaps.
Bug verification uses separate before-fix and after-fix runs. The failed run remains the reproduction record; only an evidence-valid passing run can be activated as a regression case.
Agents implementing these requests should read AI QA Agent Workflow. It maps human requests to project setup, controller work, CLI lifecycles, evidence, cases, RunGroups, reports, recording, repair, and cleanup. The installed AI QA Agent Skill remains the source of truth.
Each target project owns its .ai-qa/config.yaml, cases, runs, RunGroups, evidence, reports, and recording receipts. The canonical .ai-qa/project-skill/SKILL.md defines an optional existing result-management procedure; each host uses its own registered rendering of that canonical file. Neither file grants the CLI controller or external-system access.
The host Agent owns project access, permissions, authentication, controller sessions, and file writes. The CLI validates and records host-supplied events. It never invokes Chrome DevTools MCP, Pepper, Appium, or UiAutomator2.
Remove project configuration without deleting cases, runs, evidence, or reports:
ai-qa clear
ai-qa --project /exact/project/path clearThis immediately removes .ai-qa/config.yaml, .ai-qa/project-state.yaml, the canonical .ai-qa/project-skill/SKILL.md, .ai-qa/project-skill/renderings.yaml, and each exact registered rendering file. Rendering parent directories and unrelated sibling files remain untouched. The command is idempotent and does not prompt for confirmation. A corrupt or unsafe rendering registry stops clear before any deletion. Doctor reports unregistered project-skill leftovers, but clear never deletes them automatically.
To also delete every project-local AI QA record, including cases, runs, RunGroups, evidence, reports, and recording receipts:
ai-qa clear --records--records immediately removes each exact registered rendering file and the complete .ai-qa/ directory. Rendering parent directories, unrelated sibling files, and other project skills remain untouched.
If clear reports storage.recovery_required, inspect and manually resolve the project-relative recoveryPath before retrying. Clear never automatically deletes, restores, or resumes a retained recovery entry.
If a project-format migration is in progress, both clear modes report
project_upgrade.recovery_required and remove nothing. Complete or abort the
recorded migration before retrying clear.
Requirements for source development: Node.js 22 or 24 and pnpm 11.9.0.
corepack enable
pnpm install
pnpm check
pnpm buildThe bundled Skill is version 3.1.1 and accepts work protocol ^3.0.0. Sync installs four managed references plus the versioned Migration Skills and their metadata. User content outside managed markers and files outside references/ and migrations/ are preserved.