Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate using cffi to speed up git lfs interface #242

Open
blester125 opened this issue Apr 24, 2024 · 0 comments
Open

Investigate using cffi to speed up git lfs interface #242

blester125 opened this issue Apr 24, 2024 · 0 comments
Labels
Speed Up Working towards decreasing the latency of git theta commands.

Comments

@blester125
Copy link
Collaborator

Currently we store a serialized parameter group with git lfs by making a subprocess call. This means that process needs to be spun up and the serialized data is passed via a pipe.

Look if using the c foreign function interface in python can let us call the git lfs filters (written in go) directly, avoiding spin up and piping. Example https://blog.kchung.co/faster-python-with-go-shared-objects/

This will most likely make packaging a bit harder, but maybe remove the need to install git lfs?

@blester125 blester125 added the Speed Up Working towards decreasing the latency of git theta commands. label Apr 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Speed Up Working towards decreasing the latency of git theta commands.
Projects
None yet
Development

No branches or pull requests

1 participant