[end2ends] push repos to Hugging Face Hub (and git clone from there) to ensure it works

[julien-c]

Hi 👋

I've had success pushing then pulling (git clone https://huggingface.co/julien-c/git-theta-small-pytorch) a small pytorch file, but i've had trouble pushing a slightly larger safetensors checkpoint (e.g. my repo https://huggingface.co/julien-c/git-theta-safetensors is broken imo)

I haven't dived into git-theta enough for now, but I think it could be very useful to push and pull to/from the HF Hub in end2ends as a way to guarantee it works well.

I realize it's a test of the Hub as much as a test of this library (and it wouldn't need to run on every commit probably), but would be very useful to have!

[blester125]

I think this would make sense to add, probably would make sense for it to get run when a tag is pushed to the repo (pushing a tag creates a new release so we probably want to double check the release can interact with HF Hub before cutting it). I assume we'd need to create a HF hub user for git-theta and then include the creds as GitHub Secrets

w.r.t. your safetensors repo, I looked at it and it seems like the lfs objects never got pushed to the remote. When git-theta runs git lfs smudge to fetch the serialized parameter it would have an error like Smudge error: Error downloading (this error was suppressed because we weren't checking the returncode, #252 updates how it is git lfs smudge so the error message is more obvious)

When you push a commit to the remote, git-theta is supposed to find all the git-lfs objects that have been generated since the last push and push them to the lfs remote before the actual commit it pushed to the git remote. I'm not sure what could have caused that to not happen, @nkandpa2 would probably have a better idea. I don't think we currently have a tool to manually push lfs objects, but that would probably be helpful for cases like this.