diff --git a/docs/architecture/adrs/003-agent-workflow-tracking.md b/docs/architecture/adrs/003-agent-workflow-tracking.md new file mode 100644 index 0000000..9ab2f93 --- /dev/null +++ b/docs/architecture/adrs/003-agent-workflow-tracking.md @@ -0,0 +1,24 @@ +# ADR 003: Agent Workflow Tracking & 100% CLI Interception + +## Status +Accepted + +## Context +FlyCLI is increasingly being operated by autonomous AI agents (like Google Gemini / Antigravity). These agents operate contextually, meaning if a session reboots or another agent takes over, the context of what was done previously is lost. The users required a mechanism to explicitly set plans and log actions (`flycli agent log`, `flycli agent plan`). +However, we discovered that agents often "forget" to explicitly log their actions. The business requirement demands 100% auditability: every command executed through FlyCLI must be tracked, even if the agent fails to explicitly declare it. + +## Decision +1. **JSONL Storage with Rotation**: + We will store the audit logs as local `JSONL` files (`.flycli/agent_logs/agent_workflow_X.jsonl`) instead of using SQLite. This bypasses binary build issues encountered when using `@yao-pkg/pkg` for binary distribution. A manifest file will track the list of all generated logs. Logs rotate automatically when exceeding 1MB. + +2. **Interface Decoration for 100% Interception**: + Instead of using `Commander.js` lifecycle hooks (e.g., `program.hook('preAction')`) which fail to catch built-in arguments like `--version` or `--help` (because they exit the process before hooks run), we will implement a top-level **Interface Decoration** pattern. + - At the absolute top-level of `index.js`, we read `process.argv.slice(2)`. + - We log the raw execution arguments directly to `AgentWorkflowService` as `[SYSTEM] CLI Execution`. + - This guarantees 100% interception regardless of how Commander parses the command or if it early-exits. + +## Consequences +- **Positive**: Absolute auditability. Agents cannot secretly execute commands without leaving a local footprint. +- **Positive**: JSONL is easily parsable and append-only, reducing corruption risks. +- **Negative**: Adds a slight overhead to every CLI execution (a few milliseconds to append to a file). +- **Negative**: Need to be careful to filter out internal commands (e.g., `agent log`) from the global interceptor to prevent infinite loops or redundancy. diff --git a/docs/business/OKR.md b/docs/business/OKR.md index 6363c10..a9220d6 100644 --- a/docs/business/OKR.md +++ b/docs/business/OKR.md @@ -18,3 +18,11 @@ Transform FlyCLI from a mere configuration tool into a hardware development assi - **KR2.1:** Implement the `flycli cad` command that seamlessly orchestrates the FreeCAD GUI from Node.js by `v1.3.0`. - **KR2.2:** Establish a reliable IPC/CadQuery bridge allowing the AI (Gemini) to generate and instantly visualize solid models without user scripting. - **KR2.3:** Ensure a zero-friction distribution strategy where users don't need to manually configure FreeCAD paths or Python environments. + +## Objective 3: Agent Auditability & Context Recovery +Ensure that autonomous AI agents operating FlyCLI have a robust, machine-readable audit trail that tracks 100% of actions and maintains contextual continuity across sessions. + +**Key Results:** +- **KR3.1:** Implement a 100% interception layer for all CLI command executions, logging them automatically without requiring explicit agent calls. +- **KR3.2:** Provide a local, rotation-based JSONL storage mechanism (max 1MB per file) for agent logs to bypass binary packaging constraints. +- **KR3.3:** Expose a `flycli agent context` command that immediately yields a structured summary of recent actions and current active plans. diff --git a/docs/business/REQUIREMENTS.md b/docs/business/REQUIREMENTS.md index 64454d6..93af540 100644 --- a/docs/business/REQUIREMENTS.md +++ b/docs/business/REQUIREMENTS.md @@ -16,3 +16,8 @@ Currently, AI Agents configuring Flight Controllers (FC) lack the ability to ver ## 4. Constraints - Must not break the existing textual CLI execution architecture (`src/interfaces/cli/execute.js`). - Must operate over the same USB VCP connection without requiring external tools. + +## 5. Agent Workflow Tracking (New Feature) +- **As an Autonomous Agent**, I want to explicitly log my active plan and completed actions (`flycli agent plan set`, `flycli agent log`) so that if I am restarted or a new agent takes over, the context is preserved locally. +- **As a System Auditor**, I want 100% of CLI command executions to be automatically intercepted and logged without relying on the agent's explicit logging commands, ensuring absolute accountability. +- **As an AI Agent**, I want to retrieve a clean, machine-readable summary of my recent context (`flycli agent context`) without ANSI colors or verbose formatting. diff --git a/index.js b/index.js index a62c63c..0f44f36 100755 --- a/index.js +++ b/index.js @@ -7,13 +7,16 @@ import healthCommand from './src/interfaces/cli/health.js'; import contextCommand from './src/interfaces/cli/context.js'; import wizardCommand from './src/interfaces/cli/wizard.js'; import cadCommand from './src/interfaces/cli/cad.js'; +import agentCommand from './src/interfaces/cli/agent.js'; +import AgentWorkflowService from './src/application/AgentWorkflowService.js'; +import AgentStorage from './src/infrastructure/storage/AgentStorage.js'; const program = new Command(); program .name('flycli') .description('CLI tool for Betaflight flight controller interaction') - .version('1.4.0'); + .version('1.5.0'); program .command('scan') @@ -60,4 +63,18 @@ program .description('Start an interactive AI-powered CAD session with FreeCAD (MCP Server)') .action(cadCommand); +// 100% Command Execution Logging (Interface Decoration) +try { + const rawArgs = process.argv.slice(2).join(' ') || 'empty command'; + + // Prevent infinite loops/redundancy for agent tracking commands + if (!rawArgs.startsWith('agent log') && !rawArgs.startsWith('agent context') && !rawArgs.startsWith('agent plan set')) { + const storage = new AgentStorage(); + const service = new AgentWorkflowService(storage); + await service.logAction('CLI Execution', rawArgs, 'SYSTEM'); + } +} catch (err) { + // Ignore logging errors silently +} +program.addCommand(agentCommand); program.parse(); diff --git a/package-lock.json b/package-lock.json index 103680c..e026e91 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "flycli", - "version": "1.3.1", + "version": "1.5.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "flycli", - "version": "1.3.1", + "version": "1.5.0", "license": "MIT", "dependencies": { "@modelcontextprotocol/sdk": "^1.29.0", diff --git a/package.json b/package.json index 4d512ee..cdd3ba9 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "flycli", - "version": "1.4.0", + "version": "1.5.0", "description": "A reliable CLI tool for Betaflight flight controller interaction and automation.", "license": "MIT", "keywords": [ diff --git a/src/application/AgentWorkflowService.js b/src/application/AgentWorkflowService.js new file mode 100644 index 0000000..827e73e --- /dev/null +++ b/src/application/AgentWorkflowService.js @@ -0,0 +1,39 @@ +export default class AgentWorkflowService { + constructor(storage) { + if (!storage) throw new Error('storage dependency required'); + this.storage = storage; + } + + async logAction(actionName, description, role = null) { + const entry = { action: actionName }; + if (description) entry.description = description; + if (role) entry.role = role; + await this.storage.logAction(entry); + } + + async setPlan(planText) { + await this.storage.setPlan(planText); + } + + async getContextSummary() { + const plan = await this.storage.getPlan(); + const recent = await this.storage.getRecentActions(10); + + let summary = '=== AGENT CONTEXT ===\n\n'; + summary += '## Current Plan:\n'; + summary += plan || 'No plan set. Please define a plan using `flycli agent plan set `.\n'; + + summary += '\n## Recent Actions:\n'; + if (recent.length === 0) { + summary += 'No recent actions logged.\n'; + } else { + recent.forEach((a, i) => { + const roleStr = a.role ? `[${a.role}] ` : ''; + const descStr = a.description ? `\n ${a.description}` : ''; + summary += `${i + 1}. ${roleStr}${a.action} (${a.timestamp})${descStr}\n`; + }); + } + + return summary; + } +} diff --git a/src/infrastructure/ai/McpCadTools.js b/src/infrastructure/ai/McpCadTools.js index 8d32dee..26321d0 100644 --- a/src/infrastructure/ai/McpCadTools.js +++ b/src/infrastructure/ai/McpCadTools.js @@ -66,9 +66,11 @@ function assertValidScript(code) { export class McpCadTools { /** * @param {import('../cad/CadEngineProcess.js').CadEngineProcess} engine + * @param {Function} [lazyStart] - Optional async function to lazily start the engine */ - constructor(engine) { + constructor(engine, lazyStart) { this.engine = engine; + this.lazyStart = lazyStart || (async () => {}); } /** @@ -81,6 +83,7 @@ export class McpCadTools { */ async renderCadQuery(code) { assertValidScript(code); + await this.lazyStart(); return this.engine.executeScript(code); } @@ -91,6 +94,7 @@ export class McpCadTools { * @returns {Promise} Serialized EngineState (object tree) */ async getEngineState() { + await this.lazyStart(); return this.engine.getDocumentState(); } diff --git a/src/infrastructure/storage/AgentStorage.js b/src/infrastructure/storage/AgentStorage.js new file mode 100644 index 0000000..8fd0383 --- /dev/null +++ b/src/infrastructure/storage/AgentStorage.js @@ -0,0 +1,88 @@ +import fs from 'fs-extra'; +import path from 'path'; + +const MAX_FILE_SIZE = 1024 * 1024; + +export default class AgentStorage { + constructor(projectRoot = process.cwd()) { + this.storageDir = path.join(projectRoot, '.flycli', 'agent_logs'); + this.planFile = path.join(this.storageDir, 'current_plan.md'); + this.manifestFile = path.join(this.storageDir, 'manifest.json'); + } + + async init() { + await fs.ensureDir(this.storageDir); + if (!(await fs.pathExists(this.manifestFile))) { + await fs.writeJson(this.manifestFile, { currentFileIndex: 1, files: ['agent_workflow_1.jsonl'] }); + } + } + + async getManifest() { + return fs.readJson(this.manifestFile); + } + + async saveManifest(manifest) { + await fs.writeJson(this.manifestFile, manifest); + } + + async logAction(actionEntry) { + await this.init(); + const manifest = await this.getManifest(); + let currentFileName = manifest.files[manifest.files.length - 1]; + let currentFilePath = path.join(this.storageDir, currentFileName); + + const stats = await fs.stat(currentFilePath).catch(() => ({ size: 0 })); + + if (stats.size >= MAX_FILE_SIZE) { + manifest.currentFileIndex += 1; + currentFileName = `agent_workflow_${manifest.currentFileIndex}.jsonl`; + currentFilePath = path.join(this.storageDir, currentFileName); + manifest.files.push(currentFileName); + await this.saveManifest(manifest); + } + + const logLine = `${JSON.stringify({ ...actionEntry, timestamp: new Date().toISOString() })}\n`; + await fs.appendFile(currentFilePath, logLine); + } + + async setPlan(planText) { + await this.init(); + await fs.writeFile(this.planFile, planText, 'utf-8'); + } + + async getPlan() { + await this.init(); + try { + return await fs.readFile(this.planFile, 'utf-8'); + } catch (e) { + return null; + } + } + + async getRecentActions(count = 10) { + await this.init(); + const manifest = await this.getManifest(); + + let actions = []; + const readPromises = manifest.files.slice().reverse().map((fileName) => { + const fPath = path.join(this.storageDir, fileName); + return fs.readFile(fPath, 'utf-8').catch(() => ''); + }); + + const fileContents = await Promise.all(readPromises); + + for (let i = 0; i < fileContents.length; i += 1) { + const content = fileContents[i]; + if (content) { + const lines = content.trim().split('\n').filter((l) => l.length > 0); + const fileActions = lines.map((l) => JSON.parse(l)).reverse(); + actions = actions.concat(fileActions); + if (actions.length >= count) { + break; + } + } + } + + return actions.slice(0, count).reverse(); + } +} diff --git a/src/interfaces/cli/agent.js b/src/interfaces/cli/agent.js new file mode 100644 index 0000000..24082d6 --- /dev/null +++ b/src/interfaces/cli/agent.js @@ -0,0 +1,59 @@ +import { Command } from 'commander'; +import AgentWorkflowService from '../../application/AgentWorkflowService.js'; +import AgentStorage from '../../infrastructure/storage/AgentStorage.js'; +import ConsoleLogger from '../../infrastructure/Logger.js'; + +const agentCommand = new Command('agent') + .description('Commands for autonomous AI agent continuity and context tracking'); + +const storage = new AgentStorage(); +const service = new AgentWorkflowService(storage); +const logger = new ConsoleLogger(); + +agentCommand + .command('log') + .description('Log an agent action to the local tracking history') + .argument('', 'Short name or description of the action (e.g. "Create File")') + .argument('[description...]', 'Detailed description of what was done') + .option('-r, --role ', 'The role or mode the agent is acting as (e.g. DEVELOPER, ARCHITECT)') + .action(async (action, descriptionArray, options) => { + try { + const description = descriptionArray.join(' '); + await service.logAction(action, description, options.role); + logger.info('Agent action logged successfully.'); + } catch (err) { + logger.error('Failed to log agent action:', err.message); + process.exit(1); + } + }); + +agentCommand + .command('plan') + .description('Manage the overarching execution plan') + .command('set ') + .description('Set the current execution plan for context recovery') + .action(async (planText) => { + try { + await service.setPlan(planText); + logger.info('Agent plan updated successfully.'); + } catch (err) { + logger.error('Failed to set agent plan:', err.message); + process.exit(1); + } + }); + +agentCommand + .command('context') + .description('Get the current execution plan and recent history context') + .action(async () => { + try { + const summary = await service.getContextSummary(); + // Output directly to stdout so it can be easily read by LLMs without logger prefixes + process.stdout.write(`${summary}\n`); + } catch (err) { + logger.error('Failed to get context:', err.message); + process.exit(1); + } + }); + +export default agentCommand; diff --git a/src/interfaces/cli/cad.js b/src/interfaces/cli/cad.js index 04a692b..30a3e81 100644 --- a/src/interfaces/cli/cad.js +++ b/src/interfaces/cli/cad.js @@ -24,29 +24,31 @@ import ConsoleLogger from '../../infrastructure/Logger.js'; function buildDependencies(logger) { const env = new EnvironmentManager(logger); const engine = new CadEngineProcess(logger); - const tools = new McpCadTools(engine); + + let startPromise = null; + const lazyStart = async () => { + if (engine.process) return; + if (!startPromise) { + startPromise = (async () => { + const executablePath = await env.ensureEnvironmentReady(); + await engine.start(executablePath); + })(); + } + await startPromise; + }; + + const tools = new McpCadTools(engine, lazyStart); const mcpServer = new McpServerAdapter(tools, logger); return { env, engine, mcpServer }; } -/** - * Resolves the FreeCAD executable and starts the CadEngine process. - * @param {EnvironmentManager} env - * @param {CadEngineProcess} engine - * @returns {Promise} - */ -async function startEngine(env, engine) { - const executablePath = await env.ensureEnvironmentReady(); - await engine.start(executablePath); -} - /** * Prints the CadAgent welcome banner to stdout. */ function printBanner() { process.stderr.write('\n🚀 FlyCLI CAD Agent — Model Context Protocol (MCP) Server\n'); process.stderr.write(`${'-'.repeat(50)}\n`); - process.stderr.write('💡 FreeCAD is starting… Please wait.\n'); + process.stderr.write('💡 FreeCAD will be started lazily when a tool is invoked.\n'); } /** @@ -82,7 +84,7 @@ async function stopServices(mcpServer, engine, resolveExit) { */ export default async function cadCommand() { const logger = new ConsoleLogger(); - const { env, engine, mcpServer } = buildDependencies(logger); + const { engine, mcpServer } = buildDependencies(logger); printBanner(); @@ -100,11 +102,11 @@ export default async function cadCommand() { process.on('SIGTERM', cleanup); try { - await startEngine(env, engine); /* * Note: Start the MCP Server on stdio. * The Promise from mcpServer.start() will resolve when the server is ready, * but the node process will stay alive because of the stdio event listeners. + * FreeCAD will be started lazily when a tool is first invoked. */ await mcpServer.start(); diff --git a/test/integration/GlobalIntercept.test.js b/test/integration/GlobalIntercept.test.js new file mode 100644 index 0000000..65a738c --- /dev/null +++ b/test/integration/GlobalIntercept.test.js @@ -0,0 +1,50 @@ +import { execSync } from 'child_process'; +import path from 'path'; +import fs from 'fs-extra'; + +describe('Global CLI Execution Intercept (Interface Decoration)', () => { + const cliPath = path.join(process.cwd(), 'index.js'); + const storageDir = path.join(process.cwd(), '.flycli', 'agent_logs'); + const manifestFile = path.join(storageDir, 'manifest.json'); + + beforeAll(async () => { + // Clear logs to ensure clean slate + await fs.remove(storageDir); + }); + + afterAll(async () => { + await fs.remove(storageDir); + }); + + it('should intercept and log "node index.js --version" execution', async () => { + // Execute command + execSync(`node ${cliPath} --version`); + + // Verify manifest exists + expect(fs.existsSync(manifestFile)).toBe(true); + + const manifest = await fs.readJson(manifestFile); + expect(manifest.files.length).toBeGreaterThan(0); + + const lastLogFile = path.join(storageDir, manifest.files[manifest.files.length - 1]); + const logContent = await fs.readFile(lastLogFile, 'utf-8'); + + // Verify it contains the version command + expect(logContent).toContain('CLI Execution'); + expect(logContent).toContain('--version'); + }); + + it('should not redundantly log "node index.js agent context" to avoid infinite loops', async () => { + // Count current lines + const lastLogFile = path.join(storageDir, 'agent_workflow_1.jsonl'); + const initialLines = (await fs.readFile(lastLogFile, 'utf-8')).trim().split('\n').length; + + // Execute agent context + execSync(`node ${cliPath} agent context`); + + const finalLines = (await fs.readFile(lastLogFile, 'utf-8')).trim().split('\n').length; + + // Should exactly be the same, meaning no new logs were appended by the interceptor + expect(finalLines).toBe(initialLines); + }); +}); diff --git a/test/unit/AgentWorkflowService.test.js b/test/unit/AgentWorkflowService.test.js new file mode 100644 index 0000000..c18461a --- /dev/null +++ b/test/unit/AgentWorkflowService.test.js @@ -0,0 +1,58 @@ +import fs from 'fs-extra'; +import path from 'path'; +import { jest } from '@jest/globals'; +import AgentStorage from '../../src/infrastructure/storage/AgentStorage.js'; +import AgentWorkflowService from '../../src/application/AgentWorkflowService.js'; + +describe('AgentWorkflowService', () => { + const testDir = path.join(process.cwd(), '.test_agent_storage'); + let service; + + beforeEach(async () => { + await fs.remove(testDir); + const storage = new AgentStorage(testDir); + /* + * override the hardcoded storageDir just for testing if needed + * Actually AgentStorage uses projectRoot + '.flycli' + */ + storage.storageDir = path.join(testDir, '.flycli', 'agent_logs'); + storage.planFile = path.join(storage.storageDir, 'current_plan.md'); + storage.manifestFile = path.join(storage.storageDir, 'manifest.json'); + service = new AgentWorkflowService(storage); + }); + + afterEach(async () => { + await fs.remove(testDir); + }); + + test('should log an action and retrieve it', async () => { + await service.logAction('TEST_ACTION', 'This is a test action', 'DEVELOPER'); + + const summary = await service.getContextSummary(); + expect(summary).toContain('TEST_ACTION'); + expect(summary).toContain('This is a test action'); + expect(summary).toContain('[DEVELOPER]'); + }); + + test('should set and get the plan', async () => { + const planText = '# Test Plan\n- Step 1\n- Step 2'; + await service.setPlan(planText); + + const summary = await service.getContextSummary(); + expect(summary).toContain('# Test Plan'); + expect(summary).toContain('- Step 1'); + }); + + test('should rotate logs if file size exceeds limit', async () => { + const originalStat = fs.stat; + fs.stat = jest.fn().mockImplementation(async () => ({ size: 1024 * 1024 * 2 })); + await service.logAction('ACTION_1'); + await service.logAction('ACTION_2'); + + const manifest = await fs.readJson(service.storage.manifestFile); + // Because stat returns 2MB, it rotates EVERY time it logs + expect(manifest.files.length).toBeGreaterThan(1); + + fs.stat = originalStat; + }); +}); diff --git a/test/unit/CadCommand.test.js b/test/unit/CadCommand.test.js index 2d96826..591e7dc 100644 --- a/test/unit/CadCommand.test.js +++ b/test/unit/CadCommand.test.js @@ -66,8 +66,6 @@ describe('cadCommand — Happy Path', () => { await cadCommand(); - expect(mockEnsureEnvironmentReady).toHaveBeenCalledTimes(1); - expect(mockEngineStart).toHaveBeenCalledWith('/usr/bin/freecad'); expect(mockMcpStart).toHaveBeenCalledTimes(1); expect(mockMcpStop).toHaveBeenCalledTimes(1); expect(mockEngineStop).toHaveBeenCalledTimes(1); @@ -75,17 +73,6 @@ describe('cadCommand — Happy Path', () => { }); describe('cadCommand — Error Handling', () => { - it('should call engine.stop() even when EnvironmentManager fails', async () => { - setup(); - mockEnsureEnvironmentReady.mockRejectedValue(new Error('FreeCAD not found')); - - // Should not throw — graceful exit - await expect(cadCommand()).resolves.not.toThrow(); - - expect(mockMcpStop).toHaveBeenCalledTimes(1); - expect(mockEngineStop).toHaveBeenCalledTimes(1); - }); - it('should call mcpServer.stop() and engine.stop() when McpServer fails', async () => { setup(); mockMcpStart.mockRejectedValue(new Error('Port in use'));