Skip to content

feat(s3): add S3-compatible object storage connections - #6582

Open
perundio wants to merge 4 commits into
t8y2:mainfrom
perundio:feat/s3-object-storage
Open

feat(s3): add S3-compatible object storage connections#6582
perundio wants to merge 4 commits into
t8y2:mainfrom
perundio:feat/s3-object-storage

Conversation

@perundio

@perundio perundio commented Aug 18, 2026

Copy link
Copy Markdown

Summary

  • Add native S3-compatible object storage connections for AWS S3, MinIO, and other SigV4-compatible endpoints across desktop and web.
  • Finish the S3 browser with bucket management, folder/prefix creation, paginated object listing, upload/download/preview/delete, and copy/move/rename workflows.
  • Harden the implementation for merge by moving temporary S3 session tokens into the existing secret/session lifecycle, fixing SigV4 path canonicalization for special-character keys, rebasing onto the latest main, and expanding targeted test coverage.

What’s Included

  • Connection dialog support for endpoint, region, addressing style, optional session token, and read-only mode.
  • Native S3 runtime integration in dbx-core, dbx-web, and Tauri commands.
  • S3 browser UI with:
    • list/create/delete bucket
    • create folder/prefix
    • paginated ListObjectsV2 browsing with NextContinuationToken
    • upload, download, preview, delete object
    • copy, move, and rename object
  • Updated S3 driver branding/icon.
  • Secret-handling fixes so temporary S3 session tokens do not persist in plaintext when save_password = false.
  • SigV4 fixes for already-escaped paths plus regression coverage for path-style and virtual-hosted URLs with spaces, Unicode, %, and ? in keys.

Test Plan

  • cargo test -p dbx-core s3:: --lib
  • cargo test -p dbx-core session_credentials::tests --lib
  • pnpm typecheck
  • pnpm vitest run apps/desktop/src/components/s3/__tests__/S3Browser.spec.ts
  • Manual smoke test against local MinIO / AWS-compatible endpoint
    • connect using endpoint + credentials
    • create/delete bucket
    • create prefix, upload/download/preview/delete object
    • copy/move/rename object
    • browse paginated folders with 200+ entries
    • verify read-only mode blocks write operations

Notes For Review

  • The PR head has been rebased onto the current main and the previous merge conflict state is resolved.
  • The review-requested security and signing fixes are included in the latest head commit.

@github-actions github-actions Bot added area/core Shared DBX core runtime area/desktop Desktop application or Tauri shell area/web Web backend or web API dependencies/backend Adds a backend dependency enhancement New feature or request ui-change Changes user-visible interface, text, or visual assets labels Aug 18, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🌐 I18n autofill completed

✅ Added translations for 33 new zh-CN i18n keys.

Locale Added entries
es 33
it 33
ja 33
pt-BR 33
zh-TW 33

@perundio
perundio force-pushed the feat/s3-object-storage branch from c8e82cb to 8305aee Compare August 18, 2026 10:18

@t8y2 t8y2 left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm requesting changes for now. Three issues need fixing before this is safe to merge:

  1. ConnectionDialog.vue stores the temporary S3 session token in external_config. The existing persistence, no-save-password, and cloud-sync scrubbers do not treat this field as a secret, so it can remain in plaintext configuration and shared runtime state. Please move it through the same secret-store/session-isolation lifecycle as the secret access key.

  2. sigv4.rs passes the already escaped Url::path() through percent_encode_path again. Keys containing spaces, Unicode, %, ?, or similar characters are therefore signed with %25 escapes that differ from the transmitted path, causing SignatureDoesNotMatch. Canonicalize the raw key exactly once and add path-style and virtual-hosted regression tests with special-character keys.

    The url crate documents that Url::set_path percent-encodes its input, while AWS SigV4 requires the canonical URI to match the transmitted absolute path: https://docs.rs/url/2.5.8/url/struct.Url.html#method.set_path and https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_sigv-create-signed-request.html

  3. S3Browser.vue always requests 200 entries with a null continuation token and ignores isTruncated/nextContinuationToken, so larger folders silently hide every later object and common prefix. Add load-more or bounded page accumulation with a 201+ object/prefix test. AWS documents that truncated ListObjectsV2 results must continue with NextContinuationToken: https://docs.aws.amazon.com/AmazonS3/latest/API/API_ListObjectsV2.html#AmazonS3-ListObjectsV2-response-NextContinuationToken

GitHub also currently reports this head as conflicting, while the main CI workflow stopped at action_required without running any jobs.

anutre and others added 3 commits August 19, 2026 14:57
Introduce a native SigV4 S3 client with bucket browser UI for MinIO and AWS,
unifying desktop and web on PoolKind::S3 and fixing query-string signing for
list-objects delimiters plus Range header support for previews.

Co-authored-by: Cursor <cursoragent@cursor.com>
Move temporary S3 session tokens through the existing secret lifecycle, fix SigV4 path handling, and finish the S3 browser with pagination plus bucket/object management flows so the connector is safe and review-ready.

Co-authored-by: Cursor <cursoragent@cursor.com>
@perundio
perundio force-pushed the feat/s3-object-storage branch from 8305aee to abf0e6c Compare August 19, 2026 12:07
@perundio

Copy link
Copy Markdown
Author

Thanks for the detailed review and for calling out the risky parts so precisely.

I’ve addressed the requested changes in the latest head:

  • moved the temporary S3 session token into the existing secret-store / session-isolation flow so it no longer stays in plaintext config when save_password = false
  • fixed SigV4 canonical URI handling so already-escaped paths are signed exactly as transmitted, and added regression tests for path-style and virtual-hosted URLs with special-character keys
  • updated S3Browser.vue to continue truncated ListObjectsV2 results via NextContinuationToken, and added a 201+ object/prefix UI test
  • rebased the branch onto the latest main and resolved the previous conflict state

Also took the opportunity to finish the missing object-browser workflows in the same pass: bucket delete, folder/prefix creation, download, and copy/move/rename operations.

Thanks again for the careful review.

Fill the new S3 browser strings in secondary locales so the autofill check no longer depends on a partial machine-translation response.

Co-authored-by: Cursor <cursoragent@cursor.com>
@perundio
perundio requested a review from t8y2 August 19, 2026 12:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Shared DBX core runtime area/desktop Desktop application or Tauri shell area/web Web backend or web API dependencies/backend Adds a backend dependency enhancement New feature or request ui-change Changes user-visible interface, text, or visual assets

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants