Persistent storage in Ocean nodes for users/jobs

[bogdanfazakas]

based on the research we'll drop here

[alexcos20]

Epic: User‑Managed Buckets & File Integration Into Compute Jobs

Overview

We want to enable users to manage their own storage buckets and seamlessly use stored files as inputs to compute jobs. This includes bucket creation, file lifecycle operations, and the ability to reference a in compute workflows.
This epic introduces a unified storage interface that ties together user‑controlled buckets, access control via + contract address, and compute job integration.

Goals

• Allow users to create buckets with accessLists.
• Provide file management API (upload, list, delete).
• Enable compute jobs to consume fileObjects created in previous calls (make them available in containers)

User Stories

1. Bucket Creation
   As a user,
   I want to create a bucket by specifying chainId and accessList contract address,
   so that bucket access is governed by on‑chain permissions.

2. Upload File to Bucket
   As a user,
   I want to upload files to a bucket,
   so that I can use it later on compute tasks.

3. List Files in a Bucket
   As a user,
   I want to list all files in a bucket,

4. Delete File
   As a user,
   I want to delete a file from a bucket

5. Retrieve FileObject
   As a user,
   I want to retrieve a fileObject for a specific file,
   so that I can reference it in compute jobs

6. Use FileObject in Compute Jobs
   As a user,
   I want to pass a previously obtained fileObject into a compute job,
   so that the compute engine can use the file as an input.

Technical Requirements

Storage Layer
• Binary storage backend: internal storage (filesystem) + S3
Compute Integration
• Folder mounting (bucket) mounting

Out of Scope

• File versioning.
• Cross‑bucket file sharing.
• Public buckets or public file URLs.
• Automatic cleanup or lifecycle policies.