-
Notifications
You must be signed in to change notification settings - Fork 2.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Couchbase support [VectorDB] #2179
base: main
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please add steps to set up couchbase at the top of the file?
raise ImportError("`couchbase` not installed. Please install using `pip install couchbase`") | ||
|
||
|
||
class CouchbaseFTS(VectorDb): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
class CouchbaseFTS(VectorDb): | |
class Couchbase(VectorDb): |
What do you think about this change?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While the Couchbase name is more concise, I believe retaining CouchbaseSearch (changed from CouchbaseFTS in the latest commit) is preferable as we're planning to support vector search through multiple Couchbase services (both FTS and the indexing service). The more specific naming clearly indicates which implementation is being used and will help avoid confusion when we add additional Couchbase vector search capabilities in the future.
def insert(self, documents: List[Document], filters: Optional[Dict[str, Any]] = None) -> None: | ||
""" | ||
Insert documents into the Couchbase bucket. Fails if any document already exists. | ||
|
||
Args: | ||
documents: List of documents to insert | ||
filters: Optional filters (not used) | ||
""" | ||
logger.info(f"Inserting {len(documents)} documents") | ||
|
||
docs_to_insert: Dict[str, Any] = {} | ||
for document in documents: | ||
try: | ||
doc_data = self.prepare_doc(document) | ||
docs_to_insert[doc_data["_id"]] = doc_data | ||
del doc_data["_id"] | ||
except Exception as e: | ||
logger.error(f"Error preparing document '{document.name}': {e}") | ||
|
||
if docs_to_insert: | ||
try: | ||
result = self._collection.insert_multi(docs_to_insert) | ||
if result.all_ok: | ||
logger.info(f"Inserted {len(docs_to_insert)} documents successfully") | ||
else: | ||
logger.warning(f"Bulk write error while inserting documents: {result.exceptions}") | ||
except Exception as e: | ||
logger.error(f"Error during bulk insert: {e}") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here. Why not use filters and insert it with documents?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
Thanks for adding the tests! There are conflicts with the main branch that need resolution. |
# Conflicts: # libs/agno/pyproject.toml
@manthanguptaa Resolved the conflicts. |
Description
This PR integrates the Agno framework with Couchbase for vector search
Type of change
Please check the options that are relevant:
Checklist
./scripts/format.sh
and./scripts/validate.sh
to ensure code is formatted and linted.Additional Notes
You need to have a running Couchbase cluster with a created bucket.