-
Notifications
You must be signed in to change notification settings - Fork 83
feat(handler): add erofs filesystem & decompression handler #1163
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
11bb317
to
27366dc
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Copilot reviewed 3 out of 9 changed files in this pull request and generated 2 comments.
Files not reviewed (6)
- install-deps.sh: Language not supported
- overlay.nix: Language not supported
- package.nix: Language not supported
- shell.nix: Language not supported
- tests/integration/filesystem/android/erofs/input/foo.erofs.img: Language not supported
- tests/integration/filesystem/android/erofs/output/foo.erofs.img_extract/bar.txt: Language not supported
@rxpha3l make sure you pull from your remote with |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code is clean. Nice catch on fsck.erofs
being able to extract. Copilot's comments are relevant, you can take care of it.
tests/integration/filesystem/android/erofs/__input__/foo.erofs.img
Outdated
Show resolved
Hide resolved
@rxpha3l please make sure to also update the documentation with information about EROFS. |
Instead of adding patches on top of unlob package available on nixpkgs, we keep our own package.nix version here so that it's easy to add new third party dependencies that new handlers may require. It also makes it easier to upstream it since we just have to submit the package.nix file to nixpkgs.
ee660fe
to
70b29d6
Compare
EROFS is a high-performance read-only filesystem originally developed by Huawei and now used extensively in AOSP. It offers features like compression (LZ4, LZ4HC, LZMA, ZLIB, ZSTD), inline data, and reduced storage overhead, making it ideal for read-only system images. Create erofs file : > mkfs.erofs -z[compressor, level (optional)] foo.erofs.img foo/ Extract erofs file: > fsck.erofs --extract=foo_extracted/ foo.erofs.img Key notes: - First 1024 bytes are reserved for the boot loader - Header starts at 0x400 with E0F5E1E2 - Header can be more than 128 bytes if extra slots provided - File format in little endian - There is no standarized file extension. The official documentation uses ".erofs.img" - Checksum changes from version to version HEADER / SUPERBLOCK: > 4 bytes : Magic > 4 bytes : Checksum > 4 bytes : Feature > 1 byte : Block size in bit shift > 1 byte : Superblock extension > 2 bytes : Root NID > 8 bytes : Total valid i-node > 8 bytes : Built time in timestamp > 4 bytes : Nanoseconds (ns) component of timestamp > 4 bytes : Total block count > 4 bytes : Start block addr of metadata area > 4 bytes : Start block of shared xattr area > 16 bytes : UUID for volume > 16 bytes : Filesystem label > 4 bytes : Incompatible feature flag for kernel > 2 bytes : Available compression algorithm > 2 bytes : LZ4 max distance > 2 bytes : Extra devices > 2 bytes : Start address of external device table > 1 byte : Directory block size 2^blkszbits+dirblkbits > 1 byte : Total number of long xattr name prefixes > 4 bytes : Start address of long xattr prefixes > 8 bytes : NID of the special packed inode. > 1 byte : xattr filter reserved > 23 bytes : Reserved Unblob structs the header until feature_incompact. The rest is labeled as "reserved". Unblob parses the end offset by multiplying the block size in bit shift with the block count. Since the checksum is not consistent troughout versions, unblob matches on block count (min 1), build time, build time in ns, bit shift (min 9) and printable volume name. [Sources] https://elixir.bootlin.com/linux/v6.14-rc6/source/fs/erofs/erofs_fs.h https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/erofs/erofs_fs.h?h=v6.6 https://erofs.docs.kernel.org/en/latest/core_ondisk.html https://www.kernel.org/doc/html/latest/filesystems/erofs.html
feat(handler): add erofs filesystem & decompression handler
EROFS is a high-performance read-only filesystem originally developed by Huawei and now used extensively in AOSP. It offers features like compression (LZ4, LZMA), inline data, and reduced
storage overhead, making it ideal for read-only system images.
Create erofs file :
Extract erofs file:
Key notes:
uses ".erofs.img"
Header format :
[Sources]
https://elixir.bootlin.com/linux/v6.14-rc6/source/fs/erofs/erofs_fs.h
https://web.git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/erofs/erofs_fs.h?h=v6.6
https://erofs.docs.kernel.org/en/latest/core_ondisk.html
https://www.kernel.org/doc/html/latest/filesystems/erofs.html