SentenceTransformer local_files_only env var #95
Conversation
|
To avoid any confusion in the future about your contribution to Weaviate, we work with a Contributor License Agreement. If you agree, you can simply add a comment to this PR that you agree with the CLA so that we can merge. |
I agree with the CLA. |
vectorizer.py
Outdated
| @@ -83,9 +84,17 @@ class SentenceTransformerVectorizer: | |||
| cuda_core: str | |||
|
|
|||
| def __init__(self, model_path: str, model_name: str, cuda_core: str): | |||
| local_files_only = os.getenv("LOCAL_FILES_ONLY", "False").lower() in ( | |||
There was a problem hiding this comment.
can you move this logic that gets this setting to config.py? you can follow how we introduced TRUST_REMOTE_CODE setting
|
Thank you for your contribution @Merk0ff ! I have added 1 comment to your PR also you need to resolve in conflicts for |
In the current setup, building a custom Docker image with a sentence-transformers model results in an error when running the image in an environment without access to Hugging Face's model hub:
No sentence-transformers model found with name sentence-transformers/distiluse-base-multilingual-cased-v2. Creating a new one with mean pooling.This behavior is undesirable, so a solution was implemented by adding local_files_only=True during SentenceTransformer initialization. To further enhance configuration flexibility, two new environment variables have been introduced:
LOCAL_FILES_ONLY: Enables local file-only loading for offline environments.
MODEL_DIRECTORY: Prepares for future support of model storage in a Persistent Volume Claim (PVC).
These additions improve the robustness of the model loading process, particularly in offline or isolated deployment scenarios.