Skip to content

Conversation

@michel-aractingi
Copy link
Collaborator

What this does

Refactors generate_embeddings.py to use dataset tools. Specifically, modify_features to add the image and language embeddings and remove videos in one call. Dataset tools also writes parquet files in a more optimized way.

How it was tested

Tested by running generate embeddings script on this dataset aractingi/utokyo_embeddings

I ran

python src/lerobot/datasets/generating_embeddings/generate_embeddings.py \
        --repo-id lerobot/utokyo_xarm_bimanual \
        --output-repo-id aractingi/utokyo_embeddings \
        --image-encoder dinov2_vitb14 \
        --language-encoder minilm-l12 \
        --remove-videos \
        --push-to-hub

Then :

python src/lerobot/datasets/generating_embeddings/validate_embeddings.py \
        --original-repo-id lerobot/utokyo_xarm_bimanual \
        --embeddings-repo-id aractingi/utokyo_xarm_bimanual_embeddings \
        --image-encoder dinov2_vitb14 \
        --language-encoder minilm-l12 \
        --num-samples 10

Both passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant