New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: improve (re)insertion speed #80

Merged

lsorber merged 1 commit into main from ls-faster-reinsert

Jan 6, 2025

Member

lsorber commented Jan 6, 2025

Changes:

Exit early when inserting a document that is already in the database (fixes Detecting and Preventing Unnecessary Embedding Recomputation in RAGLite #75).
Delay insertion commit to the end so that a document, its chunks, and chunk embeddings are committed as a single transaction.
Automatically cascade the deletion of chunks, chunk embeddings, and evals when their parent documents or chunks are deleted.

lsorber self-assigned this


          fix: improve (re)insertion speed

8689a5b

lsorber force-pushed the ls-faster-reinsert branch from 0436bf3 to 8689a5b Compare

January 6, 2025 13:00

lsorber merged commit 620e556 into main

3 checks passed

lsorber deleted the ls-faster-reinsert branch

January 6, 2025 13:38

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet