Storage programs

.gitignore

@@ -149,3 +149,5 @@ docs/src
 src/features/bridges/EVMSmartContract/docs
 src/features/bridges/LiquidityTank_UserGuide.md
 local_tests
+docs/storage_features
+STORAGE_PROGRAMS_SPEC.md

.serena/memories/session_2025-10-11_storage_programs_review_fixes.md

@@ -0,0 +1,126 @@
+# Storage Programs Code Review Fixes Session
+
+## Session Summary
+**Date**: 2025-10-11  
+**Branch**: storage  
+**Focus**: Addressing critical code review feedback for Storage Programs implementation
+
+## Commits Created
+
+### Commit 1: `224a8fe9` - Initial Code Review Fixes
+**Files Modified**: 5 files (+70, -56 lines)
+
+1. **CRITICAL: Access Control Bypass Fixed** (manageNodeCall.ts, server_rpc.ts)
+   - Added `sender?: string` parameter to manageNodeCall()
+   - Pass authenticated caller from RPC server headers
+   - Enforce access control in getStorageProgram endpoint
+   - Return 403 Forbidden without data leakage
+   - Private/restricted programs now properly protected
+
+2. **MAJOR: Logger Fix** (handleGCR.ts:527)
+   - Fixed `log.error("[StorageProgram] Error applying edit:", error)`
+   - Embedded error message and stack trace in single string
+   - Prevents TypeScript type error (second param expects boolean)
+
+3. **MAJOR: JSONB Index Removed** (GCR_Main.ts)
+   - Removed `@Index("idx_gcr_main_data_gin")` decorator
+   - Storage Programs only use primary key lookups
+   - Avoids PostgreSQL B-tree rejection on JSONB
+   - No GIN index needed for current query patterns
+
+### Commit 2: `2e46a704` - Operation Validation Fixes
+**Files Modified**: 2 files (+37, -12 lines)
+
+1. **CRITICAL: Validation Guard Added** (handleGCR.ts:324-329)
+   - Added `context.data.variables` validation
+   - Prevents runtime error before unsafe access
+   - Clear error message returned
+
+2. **CRITICAL: Operation-Specific Logic Fixed** (handleGCR.ts:339-406)
+   - **CREATE**: Requires account does NOT exist
+   - **WRITE/UPDATE/DELETE**: Requires account DOES exist
+   - Fixed broken non-CREATE operations
+   - Moved CREATE into operation-specific branch
+   - Fixed unreachable code issue
+
+## Key Technical Decisions
+
+### Access Control Architecture
+- Caller identity from RPC headers (`"identity"` header)
+- Validation using `validateStorageProgramAccess()` function
+- Access modes: private, public, restricted, deployer-only
+- 401 for missing auth, 403 for denied access
+
+### Storage Program Query Pattern
+```typescript
+// All queries use primary key lookup
+const storageProgram = await gcrRepo.findOne({
+    where: { pubkey: storageAddress }  // Uses primary key index
+})
+// Then access JSONB in JavaScript
+if (storageProgram.data.metadata.deployer === sender) { ... }
+```
+
+### Operation Flow
+1. **CREATE**: Validate non-existence → Create account → Save → Return success
+2. **WRITE**: Validate existence → Check access → Merge data → Validate size → Save
+3. **UPDATE_ACCESS_CONTROL**: Validate existence → Check deployer → Update metadata
+4. **DELETE**: Validate existence → Check deployer → Delete program
+
+## Patterns Discovered
+
+### Error Handling Pattern
+```typescript
+log.error(`[Context] Error: ${error instanceof Error ? `${error.message}\nStack: ${error.stack || 'N/A'}` : String(error)}`)
+```
+
+### Validation Guard Pattern
+```typescript
+if (!context.operation) return { success: false, message: "..." }
+if (!context.data) return { success: false, message: "..." }
+if (!context.data.variables) return { success: false, message: "..." }
+// Safe to access context.data.variables
+```
+
+### Operation-Specific Existence Pattern
+```typescript
+if (operation === "CREATE") {
+    if (account) return { success: false, message: "Already exists" }
+    // Create logic
+    return { success: true, message: "Created" }
+}
+// For all other operations
+if (!account) return { success: false, message: "Does not exist" }
+// Update/delete logic
+```
+
+## Files Modified Summary
+
+1. **src/libs/network/manageNodeCall.ts**
+   - Added sender parameter
+   - Added access control enforcement in getStorageProgram
+
+2. **src/libs/network/server_rpc.ts**
+   - Pass sender to manageNodeCall
+
+3. **src/libs/blockchain/gcr/handleGCR.ts**
+   - Fixed logger error call
+   - Added validation guards
+   - Fixed operation-specific existence logic
+
+4. **src/libs/blockchain/validators/validateStorageProgramAccess.ts**
+   - Fixed duplicate if statement
+   - Updated validateCreateAccess comment
+
+5. **src/model/entities/GCRv2/GCR_Main.ts**
+   - Removed unnecessary JSONB index
+
+## Quality Verification
+- All changes verified with `bun run lint:fix`
+- No new ESLint errors introduced
+- All errors in unrelated test files only
+
+## Next Steps
+- Monitor for any access control edge cases in production
+- Consider adding metrics for storage program operations
+- Potential future optimization: GIN index if JSONB queries added

.serena/memories/storage_programs_access_control_patterns.md

@@ -0,0 +1,121 @@
+# Storage Programs Access Control Patterns
+
+## Access Control Implementation
+
+### RPC Endpoint Security
+**File**: `src/libs/network/manageNodeCall.ts`
+
+```typescript
+// Caller authentication required
+if (!sender) {
+    response.result = 401
+    response.response = { error: "Caller address required for storage access" }
+    break
+}
+
+// Access control validation
+const accessCheck = validateStorageProgramAccess(
+    "READ_STORAGE",
+    sender,
+    storageProgram.data,
+)
+
+if (!accessCheck.success) {
+    response.result = 403
+    response.response = { error: accessCheck.error || "Access denied" }
+    break
+}
+```
+
+### Access Control Modes
+
+**private / deployer-only**:
+- Only deployer can read and write
+- Most restrictive mode
+
+**public**:
+- Anyone can read
+- Only deployer can write
+- Good for public datasets
+
+**restricted**:
+- Only deployer or allowlisted addresses
+- Configured via allowedAddresses array
+- Good for shared team storage
+
+### Validator Logic
+**File**: `src/libs/blockchain/validators/validateStorageProgramAccess.ts`
+
+```typescript
+const isDeployer = requestingAddress === deployer
+
+// Admin operations always require deployer
+if (operation === "UPDATE_ACCESS_CONTROL" || operation === "DELETE_STORAGE_PROGRAM") {
+    return isDeployer ? { success: true } : { success: false, error: "..." }
+}
+
+// Mode-specific rules
+switch (accessControl) {
+    case "private":
+    case "deployer-only":
+        return isDeployer ? { success: true } : { success: false }
+
+    case "public":
+        if (operation === "READ_STORAGE") return { success: true }
+        if (operation === "WRITE_STORAGE") {
+            return isDeployer ? { success: true } : { success: false }
+        }
+
+    case "restricted":
+        if (isDeployer || allowedAddresses.includes(requestingAddress)) {
+            return { success: true }
+        }
+        return { success: false }
+}
+```
+
+### Authentication Flow
+
+1. **RPC Request** → Headers contain `"identity"` field
+2. **Server Validation** → `validateHeaders()` verifies signature
+3. **Extract Sender** → `sender = headers.get("identity")`
+4. **Pass to Handler** → `manageNodeCall(payload, sender)`
+5. **Enforce Access** → `validateStorageProgramAccess(operation, sender, data)`
+6. **Return 403/401** → Appropriate error without data leakage
+
+### Security Considerations
+
+**Never leak data on denial**:
+```typescript
+// ✅ Good - no data in error response
+response.response = { error: "Access denied" }
+
+// ❌ Bad - leaks metadata
+response.response = { error: "Access denied", metadata: program.metadata }
+```
+
+**Always validate sender**:
+```typescript
+// ✅ Good - check sender exists
+if (!sender) return 401
+
+// ❌ Bad - assume sender exists
+const accessCheck = validateStorageProgramAccess(operation, sender, data)
+```
+
+## Integration Points
+
+### Transaction Handler
+**File**: `src/libs/network/routines/transactions/handleStorageProgramTransaction.ts`
+
+- Queues GCR edits with sender context
+- Access validation happens in HandleGCR.applyStorageProgramEdit()
+- Sender included in context for deferred validation
+
+### GCR Handler
+**File**: `src/libs/blockchain/gcr/handleGCR.ts`
+
+- Receives sender from transaction context
+- Validates access before applying edits
+- Returns error if access denied
+- No state changes on validation failure