Conversation
There was a problem hiding this comment.
Will this model work just for classification or for captioning as well? From the OpenCLIP models available, only a small fraction worked for captioning.
Can we add a description of the available models on sparsezoo somewhere? Either in a read me or docstring? Should make it clear how/if this differs from OpenCLIP
|
@dsikka it is just made for image/text retrieval and zero-shot image classification. I left in the pytest.skip for the captioning test as a result. |
dsikka
left a comment
dsikka
left a comment
There was a problem hiding this comment.
A few small comments otherwise LGTM
| @mock_engine(rng_seed=0) | ||
| def test_visual_clip(engine, visual_input): | ||
| from deepsparse import Pipeline | ||
| from deepsparse.legacy import Pipeline |
| @mock_engine(rng_seed=0) | ||
| def test_text_clip(engine, text_input): | ||
| from deepsparse import Pipeline | ||
| from deepsparse.legacy import Pipeline |
| def model_folder(): | ||
| from huggingface_hub import snapshot_download | ||
|
|
||
| model_id = "neuralmagic/CLIP-ViT-B-32-256x256-DataComp-s34B-b86K-quant-ds" |
There was a problem hiding this comment.
Could we add a quick comment/note indicating that this model is not from OpenCLIP and only used for zero-shot classification?
|
Per the main README announcement, DeepSparse is being deprecated by June 2, 2025. Closing the PR as work has been suspended; thank you for the inputs and support! |
4 participants
Since we've made a quantized CLIP (https://huggingface.co/neuralmagic/CLIP-ViT-B-32-256x256-DataComp-s34B-b86K-quant-ds), let's use it to test our pipeline!