diff --git a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/TOC.yml b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/TOC.yml index b4c46306..de9c5f57 100644 --- a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/TOC.yml +++ b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/TOC.yml @@ -6,6 +6,8 @@ href: azure-cosmosdb-mongodb-connector.md - name: Azure CosmosDB NoSQL connector href: azure-cosmosdb-nosql-connector.md +- name: Chroma connector + href: chroma-connector.md - name: Couchbase connector href: couchbase-connector.md - name: Elasticsearch connector diff --git a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/azure-ai-search-connector.md b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/azure-ai-search-connector.md index 305ce851..a85ee0d4 100644 --- a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/azure-ai-search-connector.md +++ b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/azure-ai-search-connector.md @@ -17,27 +17,59 @@ ms.service: semantic-kernel The Azure AI Search Vector Store connector can be used to access and manage data in Azure AI Search. The connector has the following characteristics. -| Feature Area | Support | -|-----------------------------------|----------------------------------------------------------------------------------------------------------------------------------| -| Collection maps to | Azure AI Search Index | -| Supported key property types | string | -| Supported data property types | | -| Supported vector property types | ReadOnlyMemory\ | -| Supported index types | | -| Supported distance functions | | -| Supported filter clauses | | -| Supports multiple vectors in a record | Yes | -| IsFilterable supported? | Yes | -| IsFullTextSearchable supported? | Yes | -| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) | +::: zone pivot="programming-language-csharp" + +| Feature Area | Support | +| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | Azure AI Search Index | +| Supported key property types | string | +| Supported data property types | | +| Supported vector property types | ReadOnlyMemory\ | +| Supported index types | | +| Supported distance functions | | +| Supported filter clauses | | +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | Yes | +| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) | +::: zone-end +::: zone pivot="programming-language-python" +| Feature Area | Support | +| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | Azure AI Search Index | +| Supported key property types | string | +| Supported data property types | | +| Supported vector property types | list[float], list[int], ndarray | +| Supported index types | | +| Supported distance functions | | +| Supported filter clauses | | +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | Yes | +::: zone-end +::: zone pivot="programming-language-java" +| Feature Area | Support | +| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | Azure AI Search Index | +| Supported key property types | string | +| Supported data property types | | +| Supported vector property types | ReadOnlyMemory\ | +| Supported index types |
  • Hnsw
  • Flat
| +| Supported distance functions |
  • CosineSimilarity
  • DotProductSimilarity
  • EuclideanDistance
| +| Supported filter clauses |
  • AnyTagEqualTo
  • EqualTo
| +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | Yes | +| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) | +::: zone-end ## Limitations Notable Azure AI Search connector functionality limitations. -| Feature Area | Workaround | -|--------------------------------------------------------------------------------------| -----------------------------------------------------------------------------------------------| -| Configuring full text search analyzers during collection creation is not supported. | Use the Azure AI Search Client SDK directly for collection creation | +| Feature Area | Workaround | +| ----------------------------------------------------------------------------------- | ------------------------------------------------------------------- | +| Configuring full text search analyzers during collection creation is not supported. | Use the Azure AI Search Client SDK directly for collection creation | ::: zone pivot="programming-language-csharp" diff --git a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/azure-cosmosdb-mongodb-connector.md b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/azure-cosmosdb-mongodb-connector.md index 46fbbf96..c50ed06f 100644 --- a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/azure-cosmosdb-mongodb-connector.md +++ b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/azure-cosmosdb-mongodb-connector.md @@ -13,30 +13,50 @@ ms.service: semantic-kernel > [!WARNING] > The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release. -::: zone pivot="programming-language-csharp" - ## Overview The Azure CosmosDB MongoDB Vector Store connector can be used to access and manage data in Azure CosmosDB MongoDB (vCore). The connector has the following characteristics. -| Feature Area | Support | -|-----------------------------------|----------------------------------------------------------------------------------------------------------------------------------| -| Collection maps to | Azure Cosmos DB MongoDB (vCore) Collection + Index | -| Supported key property types | string | -| Supported data property types |
  • string
  • int
  • long
  • double
  • float
  • decimal
  • bool
  • DateTime
  • *and enumerables of each of these types*
| -| Supported vector property types |
  • ReadOnlyMemory\
  • ReadOnlyMemory\
| -| Supported index types |
  • Hnsw
  • IvfFlat
| -| Supported distance functions |
  • CosineDistance
  • DotProductSimilarity
  • EuclideanDistance
| -| Supported filter clauses |
  • EqualTo
| -| Supports multiple vectors in a record | Yes | -| IsFilterable supported? | Yes | -| IsFullTextSearchable supported? | No | -| StoragePropertyName supported? | No, use BsonElementAttribute instead. [See here for more info.](#data-mapping) | +::: zone pivot="programming-language-csharp" + +| Feature Area | Support | +| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | Azure Cosmos DB MongoDB (vCore) Collection + Index | +| Supported key property types | string | +| Supported data property types |
  • string
  • int
  • long
  • double
  • float
  • decimal
  • bool
  • DateTime
  • *and enumerables of each of these types*
| +| Supported vector property types |
  • ReadOnlyMemory\
  • ReadOnlyMemory\
| +| Supported index types |
  • Hnsw
  • IvfFlat
| +| Supported distance functions |
  • CosineDistance
  • DotProductSimilarity
  • EuclideanDistance
| +| Supported filter clauses |
  • EqualTo
| +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | No | +| StoragePropertyName supported? | No, use BsonElementAttribute instead. [See here for more info.](#data-mapping) | +::: zone-end +::: zone pivot="programming-language-python" +| Feature Area | Support | +| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | Azure Cosmos DB MongoDB (vCore) Collection + Index | +| Supported key property types | string | +| Supported data property types |
  • string
  • int
  • long
  • double
  • float
  • decimal
  • bool
  • DateTime
  • *and iterables of each of these types*
| +| Supported vector property types |
  • list[float]
  • list[int]
  • ndarray
    • | +| Supported index types |
    • Hnsw
    • IvfFlat
    | +| Supported distance functions |
    • CosineDistance
    • DotProductSimilarity
    • EuclideanDistance
    | +| Supported filter clauses |
    • EqualTo
    • AnyTagsEqualTo
    | +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | No | +::: zone-end +::: zone pivot="programming-language-java" +More info coming soon. +::: zone-end ## Limitations This connector is compatible with Azure Cosmos DB MongoDB (vCore) and is *not* designed to be compatible with Azure Cosmos DB MongoDB (RU). +::: zone pivot="programming-language-csharp" + ## Getting started Add the Azure CosmosDB MongoDB Vector Store connector NuGet package to your project. @@ -165,9 +185,52 @@ public class Hotel ::: zone-end ::: zone pivot="programming-language-python" -## Coming soon +## Getting started -More info coming soon. +Add the Azure CosmosDB MongoDB Vector Store dependencies to your environment. Because the Azure CosmosDB MongoDB connector is built on the MongoDB Atlas connector and uses the same client as that one, you need to install with these extras: + +```bash +pip install semantic-kernel[azure, mongo] +``` + +You can then create the vector store. + +```python +from semantic_kernel.connectors.memory.azure_cosmos_db import AzureCosmosDBforMongoDBStore + +# If the right environment settings are set, namely AZURE_COSMOS_DB_MONGODB_CONNECTION_STRING and optionally AZURE_COSMOS_DB_MONGODB_DATABASE_NAME, this is enough to create the Store: +store = AzureCosmosDBforMongoDBStore() +``` + +Alternatively, you can also pass in your own mongodb client if you want to have more control over the client construction: + +```python +from pymongo import AsyncMongoClient +from semantic_kernel.connectors.memory.azure_cosmos_db import AzureCosmosDBforMongoDBStore + +client = AsyncMongoClient(...) +store = AzureCosmosDBforMongoDBStore(mongo_client=client) +``` + +When a client is passed in, Semantic Kernel will not close the connection for you, so you need to ensure to close it, for instance with a `async with` statement. + +You can also create a collection directly, without the store. + +```python +from semantic_kernel.connectors.memory.azure_cosmos_db import AzureCosmosDBforMongoDBCollection + +# `hotel` is a class created with the @vectorstoremodel decorator +collection = AzureCosmosDBforMongoDBCollection( + collection_name="my_collection", + data_model_type=hotel +) +``` + +## Serialization + +Since the Azure CosmosDB for MongoDB connector needs a simple dict with the fields corresponding to the index as the input, the serialization is quite easy, it only uses a predetermined key `_id`, so we replace the key of the data model with that if it is not already `_id`. + +For more details on this concept see the [serialization documentation](./../serialization.md). ::: zone-end ::: zone pivot="programming-language-java" diff --git a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/chroma-connector.md b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/chroma-connector.md new file mode 100644 index 00000000..17dc15cc --- /dev/null +++ b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/chroma-connector.md @@ -0,0 +1,104 @@ +--- +title: Using the Semantic Kernel Chroma Vector Store connector (Preview) +description: Contains information on how to use a Semantic Kernel Vector store connector to access and manipulate data in ChromaDB. +zone_pivot_groups: programming-languages +author: eavanvalkenburg +ms.topic: conceptual +ms.author: edvan +ms.date: 02/26/2025 +ms.service: semantic-kernel +--- + +# Using the Chroma connector (Preview) + +> [!WARNING] +> The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release. + +::: zone pivot="programming-language-csharp" + +## Not supported + +Not supported. + +::: zone-end +::: zone pivot="programming-language-python" + +## Overview + +The Chroma Vector Store connector can be used to access and manage data in Chroma. The connector has the +following characteristics. + +| Feature Area | Support | +| ------------------------------------- | ------------------------------------------------------------------------------------------------- | +| Collection maps to | Chroma collection | +| Supported key property types | string | +| Supported data property types | All types that are supported by System.Text.Json (either built-in or by using a custom converter) | +| Supported vector property types |
    • list[float]
    • list[int]
    • ndarray
    | +| Supported index types |
    • HNSW
    | +| Supported distance functions |
    • CosineSimilarity
    • DotProductSimilarity
    • EuclideanSquaredDistance
    | +| Supported filter clauses |
    • AnyTagEqualTo
    • EqualTo
    | +| Supports multiple vectors in a record | No | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | Yes | + +## Limitations + +Notable Chroma connector functionality limitations. + +| Feature Area | Workaround | +| ------------------ | ------------------------------------------------------------------------------------------------------------------------- | +| Client-server mode | Use the client.HttpClient and pass the result to the `client` parameter, we do not support a AsyncHttpClient at this time | +| Chroma Cloud | Unclear at this time, as Chroma Cloud is still in private preview | + +## Getting Started + +Add the Chroma Vector Store connector dependencies to your project. + +```bash +pip install semantic-kernel[chroma] +``` + +You can then create the vector store. + +```python +from semantic_kernel.connectors.memory.chroma import ChromaStore + +store = ChromaStore() +``` + +Alternatively, you can also pass in your own mongodb client if you want to have more control over the client construction: + +```python +from chromadb import Client +from semantic_kernel.connectors.memory.chroma import ChromaStore + +client = Client(...) +store = ChromaStore(client=client) +``` + +You can also create a collection directly, without the store. + +```python +from semantic_kernel.connectors.memory.chroma import ChromaCollection + +# `hotel` is a class created with the @vectorstoremodel decorator +collection = ChromaCollection( + collection_name="my_collection", + data_model_type=hotel +) +``` + +## Serialization + +The Chroma client returns both `get` and `search` results in tabular form, this means that there are between 3 and 5 lists being returned in a dict, the lists are 'keys', 'documents', 'embeddings', and optionally 'metadatas' and 'distances'. The Semantic Kernel Chroma connector will automatically convert this into a list of `dict` objects, which are then parsed back to your data model. + +It could be very interesting performance wise to do straight serialization from this format into a dataframe-like structure as that saves a lot of rebuilding of the data structure. This is not done for you, even when using container mode, you would have to specify this yourself, for more details on this concept see the [serialization documentation](./../serialization.md). + +::: zone-end +::: zone pivot="programming-language-java" + +## Not supported + +Not supported. + +::: zone-end diff --git a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md index 15b30d1d..ef0c44cc 100644 --- a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md +++ b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/index.md @@ -22,60 +22,59 @@ Semantic Kernel provides a number of out-of-the-box Vector Store integrations ma ::: zone pivot="programming-language-csharp" -| Vector Store Connectors | C# | Uses officially supported SDK | Maintainer / Vendor | -|------------------------------------------------------------|:---------------:|:---------------------------------:|:----------------------------------:| -| [Azure AI Search](./azure-ai-search-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Cosmos DB MongoDB (vCore)](./azure-cosmosdb-mongodb-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Cosmos DB No SQL](./azure-cosmosdb-nosql-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Couchbase](./couchbase-connector.md) | ✅ | ✅ | Couchbase | -| [Elasticsearch](./elasticsearch-connector.md) | ✅ | ✅ | Elastic | -| Chroma | Planned | | | -| [In-Memory](./inmemory-connector.md) | ✅ | N/A | Microsoft Semantic Kernel Project | -| Milvus | Planned | | | -| [MongoDB](./mongodb-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Pinecone](./pinecone-connector.md) | ✅ | ❌ | Microsoft Semantic Kernel Project | -| [Postgres](./postgres-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Qdrant](./qdrant-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Redis](./redis-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| Sql Server | Planned | | | -| [SQLite](./sqlite-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Volatile (In-Memory)](./volatile-connector.md) | Deprecated (use In-Memory) | N/A | Microsoft Semantic Kernel Project | -| [Weaviate](./weaviate-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| Vector Store Connectors | C# | Uses officially supported SDK | Maintainer / Vendor | +| ------------------------------------------------------------------ | :------------------------: | :---------------------------: | :-------------------------------: | +| [Azure AI Search](./azure-ai-search-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Cosmos DB MongoDB (vCore)](./azure-cosmosdb-mongodb-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Cosmos DB No SQL](./azure-cosmosdb-nosql-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Couchbase](./couchbase-connector.md) | ✅ | ✅ | Couchbase | +| [Elasticsearch](./elasticsearch-connector.md) | ✅ | ✅ | Elastic | +| Chroma | Planned | | | +| [In-Memory](./inmemory-connector.md) | ✅ | N/A | Microsoft Semantic Kernel Project | +| Milvus | Planned | | | +| [MongoDB](./mongodb-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Pinecone](./pinecone-connector.md) | ✅ | ❌ | Microsoft Semantic Kernel Project | +| [Postgres](./postgres-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Qdrant](./qdrant-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Redis](./redis-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| Sql Server | Planned | | | +| [SQLite](./sqlite-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Volatile (In-Memory)](./volatile-connector.md) | Deprecated (use In-Memory) | N/A | Microsoft Semantic Kernel Project | +| [Weaviate](./weaviate-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | ::: zone-end ::: zone pivot="programming-language-python" -| Vector Store Connectors | Python | Uses officially supported SDK | Maintainer / Vendor | -|------------------------------------------------------------|:---------------:|:----------------------------------:|:----------------------------------:| -| [Azure AI Search](./azure-ai-search-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Cosmos DB MongoDB (vCore)](./azure-cosmosdb-mongodb-connector.md) | In Development | ✅ | Microsoft Semantic Kernel Project | -| [Cosmos DB No SQL](./azure-cosmosdb-nosql-connector.md) | In Development | ✅ | Microsoft Semantic Kernel Project | -| [Elasticsearch](./elasticsearch-connector.md) | Planned | | | -| Chroma | Planned | | | -| [In-Memory](./inmemory-connector.md) | ✅ | N/A | Microsoft Semantic Kernel Project | -| Milvus | Planned | | | -| [MongoDB](./mongodb-connector.md) | In Development | ✅ | Microsoft Semantic Kernel Project | -| [Pinecone](./pinecone-connector.md) | In Development | ✅ | Microsoft Semantic Kernel Project | -| [Postgres](./postgres-connector.md) | ✅ | | Microsoft Semantic Kernel Project | -| [Qdrant](./qdrant-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| [Redis](./redis-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| Sql Server | Planned | | | -| [SQLite](./sqlite-connector.md) | In Development | ✅ | Microsoft Semantic Kernel Project | -| [Volatile (In-Memory)](./volatile-connector.md) | Deprecated (use In-Memory) | N/A | Microsoft Semantic Kernel Project | -| [Weaviate](./weaviate-connector.md) | ✅ | N/A | Microsoft Semantic Kernel Project | +| Vector Store Connectors | Python | Uses officially supported SDK | Maintainer / Vendor | +| ------------------------------------------------------------------ | :-----: | :---------------------------: | :-------------------------------: | +| [Azure AI Search](./azure-ai-search-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Cosmos DB MongoDB (vCore)](./azure-cosmosdb-mongodb-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Cosmos DB No SQL](./azure-cosmosdb-nosql-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Chroma](./chroma-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Elasticsearch](./elasticsearch-connector.md) | Planned | | | +| Faiss | Planned | | | +| [In-Memory](./inmemory-connector.md) | ✅ | N/A | Microsoft Semantic Kernel Project | +| [MongoDB](./mongodb-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Pinecone](./pinecone-connector.md) | Planned | ✅ | Microsoft Semantic Kernel Project | +| [Postgres](./postgres-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Qdrant](./qdrant-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| [Redis](./redis-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| SQL Server | Planned | ✅ | Microsoft Semantic Kernel Project | +| SQLite | Planned | ✅ | Microsoft Semantic Kernel Project | +| [Weaviate](./weaviate-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | ::: zone-end ::: zone pivot="programming-language-java" -| Vector Store Connectors | Java | Uses officially supported SDK | Maintainer / Vendor | -|------------------------------------------------------------|:--------------:|:----------------------------------:|:----------------------------------:| -| [Azure AI Search](./azure-ai-search-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| HSQLDB | Use [JDBC](./jdbc-connector.md) | ✅ | Microsoft Semantic Kernel Project | -| [JDBC](./jdbc-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| MySQL | Use [JDBC](./jdbc-connector.md) | ✅ | Microsoft Semantic Kernel Project | -| Postgres | Use [JDBC](./jdbc-connector.md) | | Microsoft Semantic Kernel Project | -| [Redis](./redis-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | -| SQLite | Use [JDBC](./jdbc-connector.md) | ✅ | Microsoft Semantic Kernel Project | -| [Volatile (In-Memory)](./volatile-connector.md) | ✅ | N/A | Microsoft Semantic Kernel Project | +| Vector Store Connectors | Java | Uses officially supported SDK | Maintainer / Vendor | +| ------------------------------------------------- | :-----------------------------: | :---------------------------: | :-------------------------------: | +| [Azure AI Search](./azure-ai-search-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| HSQLDB | Use [JDBC](./jdbc-connector.md) | ✅ | Microsoft Semantic Kernel Project | +| [JDBC](./jdbc-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| MySQL | Use [JDBC](./jdbc-connector.md) | ✅ | Microsoft Semantic Kernel Project | +| Postgres | Use [JDBC](./jdbc-connector.md) | | Microsoft Semantic Kernel Project | +| [Redis](./redis-connector.md) | ✅ | ✅ | Microsoft Semantic Kernel Project | +| SQLite | Use [JDBC](./jdbc-connector.md) | ✅ | Microsoft Semantic Kernel Project | +| [Volatile (In-Memory)](./volatile-connector.md) | ✅ | N/A | Microsoft Semantic Kernel Project | ::: zone-end diff --git a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/mongodb-connector.md b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/mongodb-connector.md index 6c406610..8671e1a3 100644 --- a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/mongodb-connector.md +++ b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/mongodb-connector.md @@ -13,25 +13,45 @@ ms.service: semantic-kernel > [!WARNING] > The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release. -::: zone pivot="programming-language-csharp" - ## Overview The MongoDB Vector Store connector can be used to access and manage data in MongoDB. The connector has the following characteristics. -| Feature Area | Support | -|-----------------------------------|----------------------------------------------------------------------------------------------------------------------------------| -| Collection maps to | MongoDB Collection + Index | -| Supported key property types | string | -| Supported data property types |
    • string
    • int
    • long
    • double
    • float
    • decimal
    • bool
    • DateTime
    • *and enumerables of each of these types*
    | -| Supported vector property types |
    • ReadOnlyMemory\
    • ReadOnlyMemory\
    | -| Supported index types | N/A | -| Supported distance functions |
    • CosineSimilarity
    • DotProductSimilarity
    • EuclideanDistance
    | -| Supported filter clauses |
    • EqualTo
    | -| Supports multiple vectors in a record | Yes | -| IsFilterable supported? | Yes | -| IsFullTextSearchable supported? | No | -| StoragePropertyName supported? | No, use BsonElementAttribute instead. [See here for more info.](#data-mapping) | +::: zone pivot="programming-language-csharp" + +| Feature Area | Support | +| ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | MongoDB Collection + Index | +| Supported key property types | string | +| Supported data property types |
    • string
    • int
    • long
    • double
    • float
    • decimal
    • bool
    • DateTime
    • *and enumerables of each of these types*
    | +| Supported vector property types |
    • ReadOnlyMemory\
    • ReadOnlyMemory\
    | +| Supported index types | N/A | +| Supported distance functions |
    • CosineSimilarity
    • DotProductSimilarity
    • EuclideanDistance
    | +| Supported filter clauses |
    • EqualTo
    | +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | No | +| StoragePropertyName supported? | No, use BsonElementAttribute instead. [See here for more info.](#data-mapping) | +::: zone-end +::: zone pivot="programming-language-python" + +| Feature Area | Support | +| ------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | MongoDB Collection + Index | +| Supported key property types | string | +| Supported data property types |
    • string
    • int
    • long
    • double
    • float
    • decimal
    • bool
    • DateTime
    • *and iterables of each of these types*
    | +| Supported vector property types |
    • list[float]
    • list[int]
    • ndarray
    | +| Supported index types |
    • Hnsw
    • IvfFlat
    | +| Supported distance functions |
    • CosineDistance
    • DotProductSimilarity
    • EuclideanDistance
    | +| Supported filter clauses |
    • EqualTo
    • AnyTagsEqualTo
    | +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | No | +::: zone-end +::: zone pivot="programming-language-java" +More info coming soon. +::: zone-end +::: zone pivot="programming-language-csharp" ## Getting started @@ -136,9 +156,52 @@ public class Hotel ::: zone-end ::: zone pivot="programming-language-python" -## Coming soon +## Getting started -More info coming soon. +Add the MongoDB Atlas Vector Store dependencies to your environment. It needs the `pymongo` package which is included in the mongo extra: , you need to install with these extras: + +```bash +pip install semantic-kernel[mongo] +``` + +You can then create the vector store. + +```python +from semantic_kernel.connectors.memory.mongodb_atlas import MongoDBAtlasStore + +# If the right environment settings are set, namely MONGODB_ATLAS_CONNECTION_STRING and optionally MONGODB_ATLAS_DATABASE_NAME and MONGODB_ATLAS_INDEX_NAME, this is enough to create the Store: +store = MongoDBAtlasStore() +``` + +Alternatively, you can also pass in your own mongodb client if you want to have more control over the client construction: + +```python +from pymongo import AsyncMongoClient +from semantic_kernel.connectors.memory.mongodb_atlas import MongoDBAtlasStore + +client = AsyncMongoClient(...) +store = MongoDBAtlasStore(mongo_client=client) +``` + +When a client is passed in, Semantic Kernel will not close the connection for you, so you need to ensure to close it, for instance with a `async with` statement. + +You can also create a collection directly, without the store. + +```python +from semantic_kernel.connectors.memory.mongodb_atlas import MongoDBAtlasCollection + +# `hotel` is a class created with the @vectorstoremodel decorator +collection = MongoDBAtlasCollection( + collection_name="my_collection", + data_model_type=hotel +) +``` + +## Serialization + +Since the MongoDB Atlas connector needs a simple dict with the fields corresponding to the index as the input, the serialization is quite easy, it only uses a predetermined key `_id`, so we replace the key of the data model with that if it is not already `_id`. + +For more details on this concept see the [serialization documentation](./../serialization.md). ::: zone-end ::: zone pivot="programming-language-java" diff --git a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/weaviate-connector.md b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/weaviate-connector.md index 52a0178a..b3efc390 100644 --- a/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/weaviate-connector.md +++ b/semantic-kernel/concepts/vector-store-connectors/out-of-the-box-connectors/weaviate-connector.md @@ -13,33 +13,51 @@ ms.service: semantic-kernel > [!WARNING] > The Semantic Kernel Vector Store functionality is in preview, and improvements that require breaking changes may still occur in limited circumstances before release. -::: zone pivot="programming-language-csharp" - ## Overview The Weaviate Vector Store connector can be used to access and manage data in Weaviate. The connector has the following characteristics. -| Feature Area | Support | -|-----------------------------------|----------------------------------------------------------------------------------------------------------------------------------| -| Collection maps to | Weaviate Collection | -| Supported key property types | Guid | -| Supported data property types |
    • string
    • byte
    • short
    • int
    • long
    • double
    • float
    • decimal
    • bool
    • DateTime
    • DateTimeOffset
    • Guid
    • *and enumerables of each of these types*
    | -| Supported vector property types |
    • ReadOnlyMemory\
    • ReadOnlyMemory\
    | -| Supported index types |
    • Hnsw
    • Flat
    • Dynamic
    | -| Supported distance functions |
    • CosineDistance
    • NegativeDotProductSimilarity
    • EuclideanSquaredDistance
    • Hamming
    • ManhattanDistance
    | -| Supported filter clauses |
    • AnyTagEqualTo
    • EqualTo
    | -| Supports multiple vectors in a record | Yes | -| IsFilterable supported? | Yes | -| IsFullTextSearchable supported? | Yes | -| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) | +::: zone pivot="programming-language-csharp" + +| Feature Area | Support | +| ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | Weaviate Collection | +| Supported key property types | Guid | +| Supported data property types |
    • string
    • byte
    • short
    • int
    • long
    • double
    • float
    • decimal
    • bool
    • DateTime
    • DateTimeOffset
    • Guid
    • *and enumerables of each of these types*
    | +| Supported vector property types |
    • ReadOnlyMemory\
    • ReadOnlyMemory\
    | +| Supported index types |
    • Hnsw
    • Flat
    • Dynamic
    | +| Supported distance functions |
    • CosineDistance
    • NegativeDotProductSimilarity
    • EuclideanSquaredDistance
    • Hamming
    • ManhattanDistance
    | +| Supported filter clauses |
    • AnyTagEqualTo
    • EqualTo
    | +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | Yes | +| StoragePropertyName supported? | No, use `JsonSerializerOptions` and `JsonPropertyNameAttribute` instead. [See here for more info.](#data-mapping) | +::: zone-end +::: zone pivot="programming-language-python" +| Feature Area | Support | +| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Collection maps to | Weaviate Collection | +| Supported key property types | Guid | +| Supported data property types |
    • string
    • byte
    • short
    • int
    • long
    • double
    • float
    • decimal
    • bool
    • *and iterables of each of these types*
    | +| Supported vector property types |
    • list[float]
    • list[int]
    • ndarray
    | +| Supported index types |
    • Hnsw
    • Flat
    • Dynamic
    | +| Supported distance functions |
    • CosineDistance
    • NegativeDotProductSimilarity
    • EuclideanSquaredDistance
    • Hamming
    • ManhattanDistance
    | +| Supported filter clauses |
    • AnyTagEqualTo
    • EqualTo
    | +| Supports multiple vectors in a record | Yes | +| IsFilterable supported? | Yes | +| IsFullTextSearchable supported? | Yes | +::: zone-end +::: zone pivot="programming-language-java" +Coming soon. +::: zone-end ## Limitations Notable Weaviate connector functionality limitations. -| Feature Area | Workaround | -|------------------------------------------------------------------------| -----------------------------------------------------------------------------------------------| -| Using the 'vector' property for single vector objects is not supported | Use of the 'vectors' property is supported instead. | +| Feature Area | Workaround | +| ---------------------------------------------------------------------- | --------------------------------------------------- | +| Using the 'vector' property for single vector objects is not supported | Use of the 'vectors' property is supported instead. | > [!WARNING] > Weaviate requires collection names to start with an upper case letter. If you do not provide a collection name with an upper case letter, Weaviate will return an error when you try and create your collection. The error that you will see is `Cannot query field "mycollection" on type "GetObjectsObj". Did you mean "Mycollection"?` where `mycollection` is your collection name. In this example, if you change your collection name to `Mycollection` instead, this will fix the error. @@ -181,9 +199,64 @@ public class Hotel ::: zone-end ::: zone pivot="programming-language-python" -## Coming soon +## Getting Started -More info coming soon. +Add the Weaviate Vector Store connector dependencies to your project. + +```bash +pip install semantic-kernel[weaviate] +``` + +You can then create the vector store, it uses environment settings to connect: + +For using Weaviate Cloud: + +- url: WEAVIATE_URL +- api_key: WEAVIATE_API_KEY + +For using Weaviate Local (i.e. Weaviate in a Docker container): + +- local_host: WEAVIATE_LOCAL_HOST +- local_port: WEAVIATE_LOCAL_PORT +- local_grpc_port: WEAVIATE_LOCAL_GRPC_PORT + +If you want to use embedded: + +- use_embed: WEAVIATE_USE_EMBED + +These should be set exclusively, so only one set of the above is present, otherwise it will raise an exception. + +```python +from semantic_kernel.connectors.memory.weaviate import WeaviateStore + +store = WeaviateStore() +``` + +Alternatively, you can also pass in your own mongodb client if you want to have more control over the client construction: + +```python +import weaviate +from semantic_kernel.connectors.memory.weaviate import WeaviateStore + +client = weaviate.WeaviateAsyncClient(...) +store = WeaviateStore(async_client=client) +``` + +You can also create a collection directly, without the store. + +```python +from semantic_kernel.connectors.memory.weaviate import WeaviateCollection + +# `hotel` is a class created with the @vectorstoremodel decorator +collection = WeaviateCollection( + collection_name="my_collection", + data_model_type=hotel +) +``` + +## Serialization + +The Weaviate client returns it's own objects which are parsed and turned into dicts in the regular flow, for more details on this concept see the [serialization documentation](./../serialization.md). ::: zone-end ::: zone pivot="programming-language-java"