Repository navigation
Expand file tree
/
Copy path.cursorrules
More file actions
284 lines (210 loc) · 12.6 KB
/
Copy path.cursorrules
File metadata and controls
284 lines (210 loc) · 12.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
# Juno AI Computer Use Agent - Cursor Rules
**CRITICAL**: See [LLMs.txt](LLMs.txt) for comprehensive project instructions optimized for AI agents.
This is a TAURI v2 App.
Documentation:
@<https://v2.tauri>
We have this A.I. Anthropic Computer Use Bot.
We want it to have a complete implementation. Make sure that you research anthropic computer use and look to see if we're missing anything completely.
@Web
@<https://docs.anthropic.com/en/docs/agents-and-tools/computer-use>
@main.rs
## THIS IS NOT
- An AI-powered platform automating business loan processing for lenders and brokers.
We are building a completely unrelated AI agent that can use the computer.
## Development Rules
Remember this is a new build so we don't need to leave any legacy functionality or anything like that. Just change the functionality. You can make breaking changes if they work.
Since you're working on a new app when you create a feature, focus on the feature but don't be afraid to refactor things that are very unmaintainable or convoluted and clean up the repo while you work.
### Mandatory Compilation Check
After substantial changes in the Rust code, please run this check to verify everything still builds. Please note this command takes a long time so use it wisely. But it is a requirement to all acceptance criteria.
```bash
cargo check --manifest-path src-tauri/Cargo.toml --message-format=short 2>&1 | tee cargo-check-results.log
```
### Hierarchical Agent Architecture (Enhanced)
- **Orchestrator**: `src-tauri/src/anthropic.rs` - personality + memory + delegation + advanced workflow management
- **Specialists**: Domain-specific agents (browser, desktop, file) with isolated memory + enhanced coordination + security controls
- **Tools**: Shared providers with lazy initialization + advanced configuration management + enhanced security validation
- **Memory**: Enhanced memory managers with token-aware pruning and conversation summarization
- **MCP Integration**: External tool servers via `src-tauri/src/agent/tools/mcp_integration.rs` + orchestrator integration
- **Self-Awareness**: Active in debug mode via `src-tauri/src/agent/tools/self_awareness_tools.rs`
- **Command System**: Comprehensive command registry with 50+ categorized commands
### Key Patterns (Enhanced)
- Use `AgentError` enum for errors, never `std::process::exit()`
- Dynamic escape key registration ONLY during agent execution
- All persistent state in `AppState`, access via getters
- Clone memory managers safely (Arc-based) with enhanced memory tracking
- Follow async/await patterns consistently with parallel tool execution
- Proper import organization (no duplicate imports)
- **NEW**: Sophisticated workflow orchestration with task dependencies
- **NEW**: Advanced memory management with automatic pruning
- **NEW**: Comprehensive tool configuration with real implementations
- **NEW**: All tool operations must use security validation and audit logging
- **NEW**: Hardware monitoring data collection should be async and parallel
- **CLEAN**: All deprecated methods eliminated - modern APIs only
- **STORE-BASED**: All configuration uses Tauri store, no file-based legacy patterns
- **MEMORY SAFETY**: NEVER use `.unwrap()` in production code - always handle errors properly
### Implementation Status (Enhanced)
✅ All 17 Computer Use actions
✅ Timer system with context resumption
✅ Voice integration (Agent/Dictation modes)
✅ Multi-agent orchestration with advanced workflow management
✅ Browser automation
✅ MCP integration for extensibility + orchestrator coordination
✅ Cloud control system and authentication
✅ Streaming AI responses
✅ **Enhanced Security Framework** - Production-grade file and command security
✅ **Real-Time Hardware Monitoring** - CPU, memory, disk, connection metrics
✅ **Complete Frontend Functionality** - All user interface features implemented
✅ **Self-Awareness System** - Agent knows its source code location, creator, and can build itself
✅ **macOS Accessibility Permission Fixes** - Built apps properly detect permissions
✅ **Advanced Tool Configuration** - Real implementations with comprehensive management
✅ **Orchestrator Command System** - Sophisticated task and workflow management
✅ **Memory Management Enhancement** - Token-aware memory with intelligent optimization
✅ **Deprecated Code Elimination Complete** - Modern Rust patterns, removed OLD-* files, clean codebase
✅ **Dynamic System Tray Integration** - State-aware tray icons with automatic updates and comprehensive menu system
✅ **ModelSelector Synchronization Fix** - Resolved critical frontend state synchronization issues with provider/model selection
✅ **MCP Request Batching System** - Intelligent tool batching for 33% performance improvement with pattern detection, JSON-RPC 2.0 support, and AI prompt guidance for optimal batching behavior
✅ **Safari Tools Implementation** - Native Safari DOM automation with AppleScript/JavaScript injection for 3-5x faster Safari operations
✅ **Enhanced Accessibility Tools** - Comprehensive multi-tier accessibility system with both accessibility_interface and native macOS accessibility APIs
### Advanced Features 🚀 NEW
#### **Tool Configuration System**
- **Location**: `src-tauri/src/commands/tools.rs`
- **Features**: Real tool management, category-based configuration, MCP integration
- **Commands**: 8 new tool configuration commands with persistent storage
#### **Orchestrator Enhancement**
- **Location**: `src-tauri/src/commands/orchestrator.rs`
- **Features**: Task prioritization, workflow templates, MCP tool execution
- **Commands**: 5 new orchestrator commands with advanced task management
- **Workflow Templates**: Web research, file processing, desktop automation patterns
#### **Memory Management**
- **Location**: `src-tauri/src/commands/memory.rs`
- **Features**: Token-aware management, conversation summarization, optimization
- **Commands**: 5 new memory commands with real-time monitoring
#### **System Tray Integration**
- **Location**: `src-tauri/src/menu/tray_menu.rs`
- **Features**: Dynamic state-aware tray icons, automatic event-driven updates, state priority system
- **Commands**: 9 new tray icon control commands with comprehensive state management
- **Icon States**: Default, Agent Active, Dictation Active, Always Listening, Processing, Error
- **Integration**: Automatic state detection through application event monitoring
#### **Safari Tools Integration**
- **Location**: `src-tauri/src/agent/tools/safari_tools.rs`
- **Features**: Native Safari DOM automation via AppleScript/JavaScript injection
- **Commands**: 6 Safari-specific automation commands with element caching
- **Performance**: 3-5x faster Safari operations vs traditional browser automation
- **Capabilities**: Direct DOM access, element interaction, Safari-optimized workflows
#### **Enhanced Accessibility Tools**
- **Location**: `src-tauri/src/agent/tools/accessibility_tools.rs`
- **Features**: Multi-tier accessibility system with comprehensive UI element interaction
- **Commands**: 5 accessibility commands with native macOS accessibility API integration
- **Capabilities**: Element scanning, semantic clicking, accessibility_interface integration
- **Reliability**: 15-25% improvement in click reliability through native accessibility APIs
### Security Features 🔒 NEW
**Development Mode vs Production Mode**:
- **Development Mode** (`RUST_LOG=debug`): Relaxed security for development workflow
- **Production Mode**: Strict security controls with comprehensive validation
**File Access Security**:
- Path traversal prevention (`../`, `~/` blocking)
- Directory access control (workspace-only)
- File extension validation (safe text files only)
- File size limits (10MB production, 50MB development)
**Command Execution Security**:
- Command whitelist enforcement (safe development tools only)
- Dangerous pattern detection and blocking
- Execution timeouts and resource monitoring
- Comprehensive audit logging
**Location**: `src-tauri/src/agent/tools/basic_tools.rs`
**Configuration**: `SecurityConfig::default()` and `SecurityConfig::development_mode()`
### Hardware Monitoring Features 📊 NEW
**Real-Time Metrics**:
- CPU usage percentage via macOS `top` command
- Memory usage calculation via `vm_stat`
- Disk usage monitoring via `df` command
- Screen resolution detection via `system_profiler`
**Performance Analytics**:
- Command execution timing and success rates
- Connection statistics with latency measurement
- System health reporting in heartbeats
- Hardware data collection optimization
**Location**: `src-tauri/src/cloud/connector.rs`
**Implementation**: `HardwareMonitor::get_comprehensive_hardware_info()`
### Frontend Features 🎨 NEW
**Complete Modal System**:
- **Help Modal**: Comprehensive documentation and user guides
- **Feedback System**: GitHub issue integration with priority levels
- **Import/Export**: JSON-based conversation backup and restore
- **Window Management**: Full window control capabilities
- **Update System**: Application update checking and installation
**User Experience Enhancements**:
- Accessible modal design with keyboard navigation
- Comprehensive error handling and user feedback
- Loading states and progress indicators
- Data validation and format checking
**Location**: `src/App.tsx`
**Integration**: Backend file operations via secure Tauri commands
### Self-Awareness Features 🤖
**Development Mode Only** (activated with `RUST_LOG=debug bun run tauri dev`):
- **Source Code Awareness**: Knows location at `~/repo/juno`
- **Creator Recognition**: Acknowledges Lacy as "magnanimous benefactor"
- **Self-Building**: Can compile itself using Cargo tools
- **System Understanding**: Knows its prompt system and architecture
- **Purpose Awareness**: Understands mission to unite AI and humanity
**Location**: `src-tauri/src/agent/tools/self_awareness_tools.rs`
**Integration**: Automatic activation in debug builds via `cfg!(debug_assertions)`
### macOS Permission Handling
**CRITICAL**: Always test built apps, not just development builds for permission issues.
**Required Files for Built Apps**:
- `src-tauri/juno.entitlements` - macOS security permissions
- `src-tauri/Info.plist` - Usage descriptions for permission dialogs
- `src-tauri/tauri.conf.json` - Bundle configuration including entitlements and Info.plist
**Permission Check Architecture**:
- Primary: `computer_use_ai_sdk` permission checks
- Fallback: `try_accessibility_test()` with actual Desktop operations
- Multiple detection mechanisms for robust permission validation
See `.cursor/rules/accessibility-permission-fixes.mdc` for complete implementation details.
### Memory Safety Rules 🛡️ CRITICAL
**ZERO TOLERANCE for `.unwrap()` in production code**
```rust
// ❌ BANNED in production code
value.unwrap()
result.unwrap()
lock.unwrap()
// ✅ REQUIRED patterns
value.ok_or("Error message")?
value.unwrap_or_default()
value.unwrap_or_else(|| default)
// ✅ SystemTime MUST use this pattern
SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap_or_else(|_| Duration::from_secs(0))
// ✅ Mutex locking MUST use match
match mutex.lock() {
Ok(guard) => /* use guard */,
Err(e) => return Err(format!("Lock failed: {}", e))
}
// ✅ Regex compilation MUST handle errors
match Regex::new(pattern) {
Ok(regex) => /* use regex */,
Err(e) => /* fallback logic */
}
```
**Race Condition Prevention**:
- Use `Arc<TokioMutex<T>>` for shared state
- Use `AtomicBool`/`AtomicUsize` for simple flags
- Implement RAII patterns for resource cleanup
- Use semaphores for limiting concurrent operations
### Quick Reference (Enhanced)
- **Entry Point**: `src-tauri/src/anthropic.rs::submit_query()`
- **Agent Tools**: `src-tauri/src/agent/tools/`
- **Tool Configuration**: `src-tauri/src/commands/tools.rs`
- **Orchestrator**: `src-tauri/src/commands/orchestrator.rs`
- **Memory Management**: `src-tauri/src/commands/memory.rs`
- **Command Registry**: `src-tauri/src/commands/registry.rs` (50+ categorized commands)
- **Security Framework**: `src-tauri/src/agent/tools/basic_tools.rs`
- **Hardware Monitoring**: `src-tauri/src/cloud/connector.rs`
- **Frontend Features**: `src/App.tsx`
- **Self-Awareness**: `src-tauri/src/agent/tools/self_awareness_tools.rs`
- **Safari Tools**: `src-tauri/src/agent/tools/safari_tools.rs` + `src-tauri/src/commands/safari_tools.rs`
- **Accessibility Tools**: `src-tauri/src/agent/tools/accessibility_tools.rs` + `src-tauri/src/commands/accessibility.rs`
- **macOS Integration**: `src-tauri/mcp-server-os-level/src/platforms/macos/`
- **Voice System**: `tauri-plugin-voice-transcription/`
- **Permission System**: `src-tauri/src/commands/permissions.rs`
- **System Tray**: `src-tauri/src/menu/tray_menu.rs` + `src-tauri/src/commands/tray_commands.rs`