Storage programs

.gitignore

@@ -149,3 +149,6 @@ docs/src
 src/features/bridges/EVMSmartContract/docs
 src/features/bridges/LiquidityTank_UserGuide.md
 local_tests
+docs/storage_features
+STORAGE_PROGRAMS_SPEC.md
+temp

.serena/memories/session_2025-10-11_storage_programs_review_fixes.md

@@ -0,0 +1,126 @@
+# Storage Programs Code Review Fixes Session
+
+## Session Summary
+**Date**: 2025-10-11  
+**Branch**: storage  
+**Focus**: Addressing critical code review feedback for Storage Programs implementation
+
+## Commits Created
+
+### Commit 1: `224a8fe9` - Initial Code Review Fixes
+**Files Modified**: 5 files (+70, -56 lines)
+
+1. **CRITICAL: Access Control Bypass Fixed** (manageNodeCall.ts, server_rpc.ts)
+   - Added `sender?: string` parameter to manageNodeCall()
+   - Pass authenticated caller from RPC server headers
+   - Enforce access control in getStorageProgram endpoint
+   - Return 403 Forbidden without data leakage
+   - Private/restricted programs now properly protected
+
+2. **MAJOR: Logger Fix** (handleGCR.ts:527)
+   - Fixed `log.error("[StorageProgram] Error applying edit:", error)`
+   - Embedded error message and stack trace in single string
+   - Prevents TypeScript type error (second param expects boolean)
+
+3. **MAJOR: JSONB Index Removed** (GCR_Main.ts)
+   - Removed `@Index("idx_gcr_main_data_gin")` decorator
+   - Storage Programs only use primary key lookups
+   - Avoids PostgreSQL B-tree rejection on JSONB
+   - No GIN index needed for current query patterns
+
+### Commit 2: `2e46a704` - Operation Validation Fixes
+**Files Modified**: 2 files (+37, -12 lines)
+
+1. **CRITICAL: Validation Guard Added** (handleGCR.ts:324-329)
+   - Added `context.data.variables` validation
+   - Prevents runtime error before unsafe access
+   - Clear error message returned
+
+2. **CRITICAL: Operation-Specific Logic Fixed** (handleGCR.ts:339-406)
+   - **CREATE**: Requires account does NOT exist
+   - **WRITE/UPDATE/DELETE**: Requires account DOES exist
+   - Fixed broken non-CREATE operations
+   - Moved CREATE into operation-specific branch
+   - Fixed unreachable code issue
+
+## Key Technical Decisions
+
+### Access Control Architecture
+- Caller identity from RPC headers (`"identity"` header)
+- Validation using `validateStorageProgramAccess()` function
+- Access modes: private, public, restricted, deployer-only
+- 401 for missing auth, 403 for denied access
+
+### Storage Program Query Pattern
+```typescript
+// All queries use primary key lookup
+const storageProgram = await gcrRepo.findOne({
+    where: { pubkey: storageAddress }  // Uses primary key index
+})
+// Then access JSONB in JavaScript
+if (storageProgram.data.metadata.deployer === sender) { ... }
+```
+
+### Operation Flow
+1. **CREATE**: Validate non-existence → Create account → Save → Return success
+2. **WRITE**: Validate existence → Check access → Merge data → Validate size → Save
+3. **UPDATE_ACCESS_CONTROL**: Validate existence → Check deployer → Update metadata
+4. **DELETE**: Validate existence → Check deployer → Delete program
+
+## Patterns Discovered
+
+### Error Handling Pattern
+```typescript
+log.error(`[Context] Error: ${error instanceof Error ? `${error.message}\nStack: ${error.stack || 'N/A'}` : String(error)}`)
+```
+
+### Validation Guard Pattern
+```typescript
+if (!context.operation) return { success: false, message: "..." }
+if (!context.data) return { success: false, message: "..." }
+if (!context.data.variables) return { success: false, message: "..." }
+// Safe to access context.data.variables
+```
+
+### Operation-Specific Existence Pattern
+```typescript
+if (operation === "CREATE") {
+    if (account) return { success: false, message: "Already exists" }
+    // Create logic
+    return { success: true, message: "Created" }
+}
+// For all other operations
+if (!account) return { success: false, message: "Does not exist" }
+// Update/delete logic
+```
+
+## Files Modified Summary
+
+1. **src/libs/network/manageNodeCall.ts**
+   - Added sender parameter
+   - Added access control enforcement in getStorageProgram
+
+2. **src/libs/network/server_rpc.ts**
+   - Pass sender to manageNodeCall
+
+3. **src/libs/blockchain/gcr/handleGCR.ts**
+   - Fixed logger error call
+   - Added validation guards
+   - Fixed operation-specific existence logic
+
+4. **src/libs/blockchain/validators/validateStorageProgramAccess.ts**
+   - Fixed duplicate if statement
+   - Updated validateCreateAccess comment
+
+5. **src/model/entities/GCRv2/GCR_Main.ts**
+   - Removed unnecessary JSONB index
+
+## Quality Verification
+- All changes verified with `bun run lint:fix`
+- No new ESLint errors introduced
+- All errors in unrelated test files only
+
+## Next Steps
+- Monitor for any access control edge cases in production
+- Consider adding metrics for storage program operations
+- Potential future optimization: GIN index if JSONB queries added

.serena/memories/session_2025_10_11_storage_programs_fixes.md

@@ -0,0 +1,100 @@
+# Storage Programs Critical Fixes - Session Summary
+
+## Date: 2025-10-11
+
+## Context
+Resolved three critical blocking issues identified by code reviewer for Storage Programs feature. All issues prevented TypeScript compilation and runtime execution.
+
+## Issues Resolved
+
+### Issue #1: DELETE Operation Data Field Validation ✅
+**Problem**: `handleGCR.ts:320` rejected DELETE operations because context.data was required
+**Location**: `src/libs/blockchain/gcr/handleGCR.ts:318-323`
+**Solution**: Made data field optional for DELETE operations
+```typescript
+if (context.operation !== "DELETE" && !context.data) {
+    return { success: false, message: "Storage program edit missing data context" }
+}
+```
+**Impact**: DELETE_STORAGE_PROGRAM transactions now process correctly
+
+### Issue #2: Missing SDK Storage Export ✅
+**Problem**: Import `@kynesyslabs/demosdk/storage` failed - module not exported
+**Location**: `../sdks/package.json:36`
+**Solution**: Added storage export to SDK package.json
+```json
+"./storage": "./build/storage/index.js"
+```
+**Impact**: All storage type imports now resolve correctly
+
+### Issue #3: Missing GCREdit Type Variant ✅
+**Problem**: GCREdit union type missing "storageProgram" variant
+**Location**: `../sdks/src/types/blockchain/GCREdit.ts:134-147`
+**Solution**: Added complete GCREditStorageProgram interface
+```typescript
+export interface GCREditStorageProgram {
+    type: "storageProgram"
+    target: string
+    isRollback: boolean
+    txhash: string
+    context: {
+        operation: string
+        sender: string
+        data?: { variables: any; metadata: any }
+    }
+}
+```
+**Impact**: TypeScript compilation successful, all storage operations type-safe
+
+## Additional Fixes Applied
+
+### Type Narrowing in handleGCR.ts
+Added type guard for storage program edits to enable property access:
+```typescript
+if (editOperation.type !== "storageProgram") {
+    return { success: false, message: "Invalid edit type for storage program handler" }
+}
+```
+
+### GCREdit Creation Updates
+Updated all 4 storage program GCREdit creation points to include required fields:
+- CREATE: Added sender to context
+- WRITE: Already correct
+- UPDATE_ACCESS_CONTROL: Added variables: {} to data
+- DELETE: Already correct (no data field)
+
+### SDK GCRGeneration.ts Fix
+Updated address normalization to handle target field for storage programs:
+```typescript
+if (edit.type === "storageProgram") {
+    if (!edit.target.startsWith("0x")) {
+        edit.target = "0x" + edit.target
+    }
+} else if ("account" in edit && !edit.account.startsWith("0x")) {
+    edit.account = "0x" + edit.account
+}
+```
+
+## Files Modified
+
+### Node Repository
+1. `src/libs/blockchain/gcr/handleGCR.ts` - DELETE validation, type narrowing
+2. `src/libs/network/routines/transactions/handleStorageProgramTransaction.ts` - GCREdit creation fixes
+
+### SDK Repository
+1. `package.json` - Added storage export, version bump to 2.4.22
+2. `src/types/blockchain/GCREdit.ts` - Added GCREditStorageProgram interface
+3. `src/websdk/GCRGeneration.ts` - Handle target field for storage programs
+
+## Verification Results
+- ✅ TypeScript compilation: 0 storage-related errors
+- ✅ All imports resolve correctly
+- ✅ All 4 storage operations (CREATE, WRITE, UPDATE_ACCESS_CONTROL, DELETE) type-safe
+- ✅ SDK published successfully as v2.4.22
+
+## Key Learnings
+1. GCREdit interfaces require isRollback and txhash fields consistently
+2. Storage programs use 'target' field while other edits use 'account'
+3. DELETE operations intentionally exclude data field (only sender required)
+4. UPDATE_ACCESS_CONTROL needs variables: {} even when not modifying variables
+5. Type narrowing essential for accessing union type-specific properties

.serena/memories/storage_programs_access_control_patterns.md

@@ -0,0 +1,121 @@
+# Storage Programs Access Control Patterns
+
+## Access Control Implementation
+
+### RPC Endpoint Security
+**File**: `src/libs/network/manageNodeCall.ts`
+
+```typescript
+// Caller authentication required
+if (!sender) {
+    response.result = 401
+    response.response = { error: "Caller address required for storage access" }
+    break
+}
+
+// Access control validation
+const accessCheck = validateStorageProgramAccess(
+    "READ_STORAGE",
+    sender,
+    storageProgram.data,
+)
+
+if (!accessCheck.success) {
+    response.result = 403
+    response.response = { error: accessCheck.error || "Access denied" }
+    break
+}
+```
+
+### Access Control Modes
+
+**private / deployer-only**:
+- Only deployer can read and write
+- Most restrictive mode
+
+**public**:
+- Anyone can read
+- Only deployer can write
+- Good for public datasets
+
+**restricted**:
+- Only deployer or allowlisted addresses
+- Configured via allowedAddresses array
+- Good for shared team storage
+
+### Validator Logic
+**File**: `src/libs/blockchain/validators/validateStorageProgramAccess.ts`
+
+```typescript
+const isDeployer = requestingAddress === deployer
+
+// Admin operations always require deployer
+if (operation === "UPDATE_ACCESS_CONTROL" || operation === "DELETE_STORAGE_PROGRAM") {
+    return isDeployer ? { success: true } : { success: false, error: "..." }
+}
+
+// Mode-specific rules
+switch (accessControl) {
+    case "private":
+    case "deployer-only":
+        return isDeployer ? { success: true } : { success: false }
+
+    case "public":
+        if (operation === "READ_STORAGE") return { success: true }
+        if (operation === "WRITE_STORAGE") {
+            return isDeployer ? { success: true } : { success: false }
+        }
+
+    case "restricted":
+        if (isDeployer || allowedAddresses.includes(requestingAddress)) {
+            return { success: true }
+        }
+        return { success: false }
+}
+```
+
+### Authentication Flow
+
+1. **RPC Request** → Headers contain `"identity"` field
+2. **Server Validation** → `validateHeaders()` verifies signature
+3. **Extract Sender** → `sender = headers.get("identity")`
+4. **Pass to Handler** → `manageNodeCall(payload, sender)`
+5. **Enforce Access** → `validateStorageProgramAccess(operation, sender, data)`
+6. **Return 403/401** → Appropriate error without data leakage
+
+### Security Considerations
+
+**Never leak data on denial**:
+```typescript
+// ✅ Good - no data in error response
+response.response = { error: "Access denied" }
+
+// ❌ Bad - leaks metadata
+response.response = { error: "Access denied", metadata: program.metadata }
+```
+
+**Always validate sender**:
+```typescript
+// ✅ Good - check sender exists
+if (!sender) return 401
+
+// ❌ Bad - assume sender exists
+const accessCheck = validateStorageProgramAccess(operation, sender, data)
+```
+
+## Integration Points
+
+### Transaction Handler
+**File**: `src/libs/network/routines/transactions/handleStorageProgramTransaction.ts`
+
+- Queues GCR edits with sender context
+- Access validation happens in HandleGCR.applyStorageProgramEdit()
+- Sender included in context for deferred validation
+
+### GCR Handler
+**File**: `src/libs/blockchain/gcr/handleGCR.ts`
+
+- Receives sender from transaction context
+- Validates access before applying edits
+- Returns error if access denied
+- No state changes on validation failure