Skip to content

feat(storage): add S3 Vectors search strategy for S3Storage in Python - #5552

Draft
lizradway wants to merge 1 commit into
strands-agents:mainfrom
lizradway:lizradway/py-s3-vectors
Draft

lizradway wants to merge 1 commit into
strands-agents:mainfrom
lizradway:lizradway/py-s3-vectors

Conversation

@lizradway

Copy link
Copy Markdown
Member

Human Overview

Description

Python port of #3967: vector search for S3Storage, backed by Amazon S3 Vectors, plus a provider-agnostic embedder. Same concepts and names as the TypeScript SDK, in the Python SDK's idioms.

  • Embedder (strands.storage.embeddings, also re-exported from strands.storage.search): an ABC with one abstract method, async embed(text) -> list[float].
  • BedrockEmbedder (strands.storage.embeddings.bedrock): wraps Bedrock Runtime invoke_model for Amazon Titan Text Embeddings (V2 by default). Takes model_id, region_name, boto_session and boto_client_config, like BedrockModel and S3Storage; region_name and boto_session together raise StorageError. The model ID is passed through unchecked; a response without an embedding list raises StorageError.
  • S3VectorSearchStrategy(S3VectorSearchStrategyConfig(...)) (strands.storage.search): a config dataclass, like Bm25SearchStrategy. vector_bucket_name and index_name are required; embedder, max_results (10), boto_session and boto_client_config are optional.
    • index() embeds the value and calls PutVectors, keyed by storage.prefix + key, so storages and namespace() views sharing one index do not collide. Empty and non-UTF-8 values are not embedded; any earlier vector for the key is removed instead.
    • remove() calls DeleteVectors; search() embeds the query, calls QueryVectors, keeps keys under the storage's prefix (stripped) and scores them 1 / (1 + distance).
    • The default BedrockEmbedder and the S3 Vectors client are created once on first use, in the storage's region unless a boto_session is given.
    • Typed SearchStrategy[S3Storage], so it only attaches to S3Storage.
  • S3Storage: new read-only region_name (the configured region, else the boto_session's) and prefix properties. delete() removes the vector when the strategy is an S3VectorSearchStrategy, and a strategy failure in search() or vector removal is wrapped in StorageError (write() already wrapped indexing failures).

Differences from TypeScript: the strategy takes a boto_session instead of a pre-built s3VectorsClient, matching S3Storage; and BedrockEmbedder is imported only when the default embedder is first needed, but boto3 is a core dependency so nothing optional is avoided.

The vector bucket and index must already exist (dimension 1024 for Titan V2). At runtime the strategy needs s3vectors:PutVectors, DeleteVectors and QueryVectors on the index, plus bedrock:InvokeModel on the embedding model with the default embedder.

from strands.storage import S3Storage
from strands.storage.search import S3VectorSearchStrategy, S3VectorSearchStrategyConfig

storage = S3Storage(
    "my-bucket",
    search_strategy=S3VectorSearchStrategy(
        S3VectorSearchStrategyConfig(vector_bucket_name="my-vectors", index_name="notes")
    ),
)
await storage.write("notes/topic.md", b"content")
results = await storage.search("find relevant notes")

Related Issues

Python port of #3967. Related: #5551 (Python S3Storage trailing-slash prefix fix), #5547 (TS integration test and the CI role permissions this integ test also needs).

Documentation PR

Site docs for S3 Vectors search will follow in a separate PR, covering both SDKs.

Type of Change

New feature

Testing

  • Unit tests for the strategy (indexing, non-text values, removal, search ranking and filtering, prefixes and namespace(), default embedder and client creation, error propagation), BedrockEmbedder (request shape, model pass-through, non-Titan responses, region/session handling, user agent) and S3Storage (vector removal on delete, error wrapping, region_name and prefix). Moto has no query_vectors, so the strategy tests mock the boto client, as the TS tests do.

  • hatch test on storage, memory stores and context offloader: 582 passed. hatch fmt --formatter and hatch fmt --linter (ruff + mypy) clean.

  • New integration test tests_integ/test_s3_vector_search_strategy.py (the three TS cases plus a namespace() check) passed 3/3 against real S3, S3 Vectors and Bedrock in us-east-1, with a personal account's credentials, and teardown left no resources. It creates test-strands-s3v-* buckets, the same prefix as the TS test, so in CI it needs the integ role permissions from test(storage): add S3VectorSearchStrategy integration test #5547.

  • I ran hatch run prepare

Checklist

  • I have read the CONTRIBUTING document
  • I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
  • My change is focused and reasonably small; I have split unrelated work into separate PRs
  • I have added any necessary tests that prove my fix is effective or my feature works
  • I have updated the documentation accordingly
  • I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
  • My changes generate no new warnings
  • Any dependent changes have been merged and published

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Python port of strands-agents#3967. Adds an Embedder base class, a Titan-backed
BedrockEmbedder, and S3VectorSearchStrategy, which embeds content on
write, removes the vector on delete and queries S3 Vectors on search.
S3Storage gains region_name and prefix properties, removes the vector
on delete when using S3VectorSearchStrategy, and wraps search strategy
failures in StorageError. Includes unit tests and an integration test
against real S3, S3 Vectors and Bedrock.
@github-actions github-actions Bot added python Pull requests that update python code enhancement New feature or request area-persistence Session management or checkpointing complexity/low Touched functions have low cognitive complexity (<=10) size/m labels Oct 26, 2026
@codecov

codecov Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.85039% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
strands-py/src/strands/storage/search/s3_vector.py 95.65% 2 Missing and 1 partial ⚠️
...rands-py/src/strands/storage/embeddings/bedrock.py 96.42% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-persistence Session management or checkpointing complexity/low Touched functions have low cognitive complexity (<=10) enhancement New feature or request python Pull requests that update python code size/m

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant