Skip to content

gitea doctor consistently reports orphaned LFS #34039

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
eeyrjmr opened this issue Mar 27, 2025 · 5 comments · Fixed by #34061
Closed

gitea doctor consistently reports orphaned LFS #34039

eeyrjmr opened this issue Mar 27, 2025 · 5 comments · Fixed by #34061
Labels
Milestone

Comments

@eeyrjmr
Copy link
Contributor

eeyrjmr commented Mar 27, 2025

Description

Recently LFS has started to be used to manage binary data.
Gitea doctor check --fix --all removed all LFS files. These were restored from backup. Closer inspection of gitea doctor is showing that all LFS data is orphaned

Image

However, the equivalent view from the repo in question appears to indicate all is ok

Image

All clones and interactions are consistent but the check appears to be incorrectly flagging an issue.

Gitea Version

1.23.6

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

2.49.0

Operating System

Windows

How are you running Gitea?

Windows service

Database

SQLite

@eeyrjmr
Copy link
Contributor Author

eeyrjmr commented Mar 27, 2025

I am trying to understand what passes as an LFS orphan and I wonder if it is also associated with: #34035

@eeyrjmr
Copy link
Contributor Author

eeyrjmr commented Mar 28, 2025

This might be a windows subtly but how LFS is done on windows (be it gitea or git-lfs) means the hash filename is actually packed into the directory

Image

Image

Looking in the BARE repositories the gitea-repositories/$ORG/$REPO.git/lfs/objects/ae/b3/ is empty but that makes sense if gitea then manages the RAW data, via pointers, thus the total hash vs dir/dir/partial_hash might be the issue because if gitea doctor was trying to find the file lfs/ae/b3/aeb4d3fbb... but is finding lfs/ae/b3/d3fbb... it would explain why doctor says all is orphan

https://github.com/git-lfs/git-lfs/wiki/Tutorial

Image

I need to test on a linux machine but the git-lfs documentation implies the full filename aligns with the hash/point and not split across the directories.

@wxiaoguang
Copy link
Contributor

thus the total hash vs dir/dir/partial_hash might be the issue because if gitea doctor was trying to find the file lfs/ae/b3/aeb4d3fbb... but is finding lfs/ae/b3/d3fbb..

It seems to be a serious bug if so .....

@eeyrjmr
Copy link
Contributor Author

eeyrjmr commented Mar 28, 2025

I think this is a gitea issue :(
git-lfs/git-lfs#5845 and this is a git-lfs closed issue showing the directory structure is as expected. At least gitea can serve the lfs files just the doctor is expecting consistency :(

I will cross-check on linux later while try to figure out how gitea and doctor works out the pointers for lfs.

I have also tried gogit and nogogit for windows as I saw there are specific functions in gitea for gogit and nogogit. Same issue. I will however make a test repo to see if the gogit then makes a valid structure

@eeyrjmr
Copy link
Contributor Author

eeyrjmr commented Mar 28, 2025

checked on linux... the structure is the same as windows in that the directory name is part of the overall pointer (separate discussion) but the gitea doctor check --run storages is showing all is ok

so this would appear to be windows specific and how gitea doctor determines orphens

GiteaBot added a commit to GiteaBot/gitea that referenced this issue Mar 30, 2025
…o-gitea#34061)

The doctor storage check reconstructs the lfs oid by producing a string
where the path separator is stripped
ab/dc/efg -> abdcefg. Windows however uses a backslash and thus the
ReplaceAll call doesn't produce the correct oid resulting in all lfs
objects being classed as orphaned.
This PR allows this to be more OS agnostic.

Closes go-gitea#34039

---------

Co-authored-by: Giteabot <[email protected]>
@lunny lunny added this to the 1.23.7 milestone Mar 30, 2025
wxiaoguang pushed a commit that referenced this issue Mar 30, 2025
…34061) (#34070)

Backport #34061 by eeyrjmr

The doctor storage check reconstructs the lfs oid by producing a string
where the path separator is stripped
ab/dc/efg -> abdcefg. Windows however uses a backslash and thus the
ReplaceAll call doesn't produce the correct oid resulting in all lfs
objects being classed as orphaned.
This PR allows this to be more OS agnostic.

Closes #34039

Co-authored-by: JonRB <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants