feat(vfs): classify NTFS special files in both Linux encodings - #15
Merged
Conversation
A Linux driver writes non-regular files to NTFS in one of two forms, and
this reader recognizes neither beyond the plain IntxLNK symlink:
Interix (ntfs-3g default) IntxCHR\0 / IntxBLK\0 + major/minor
WSL (-o special_files=wsl) LX_SYMLINK 0xA000001D, AF_UNIX 0x80000023,
LX_FIFO 0x80000024, LX_CHR 0x80000025,
LX_BLK 0x80000026
ntfs_all_node_types.zip is one volume mounted twice by ntfs-3g 2022.10.3,
once per mode, so the same five nodes appear in both encodings. It is a
superset of the tree tiny.zip carries, so existing assertions keep their
ground truth.
Two cases are asserted as File deliberately: the Interix encoding stores
a FIFO as a zero-length $DATA and a socket as a one-byte $DATA with no
magic, so nothing on disk distinguishes them from ordinary files
(libntfs-3g/dir.c, whose own comment reads "FIFO or regular file"). That
is a format limitation, and File is the honest reading.
Requires forensic-vfs 0.9 for the widened NodeKind; without it the test
fails to compile rather than failing for its own reason.
RED: IntxCHR classifies as File, expected CharDevice; LX_SYMLINK
classifies as File, expected Symlink.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Classification asked one question — "does this record decode to a link
target?" — so a character device, block device, FIFO and socket all
surfaced as NodeKind::File, and WSL-encoded symlinks were missed
entirely because their tag is 0xA000001D, not 0xA000000C.
Adds the tag and magic vocabulary, and a `special_kind` that reports the
node type a record states:
WSL reparse tags LX_SYMLINK 0xA000001D -> Symlink
AF_UNIX 0x80000023 -> Socket
LX_FIFO 0x80000024 -> Fifo
LX_CHR 0x80000025 -> CharDevice
LX_BLK 0x80000026 -> BlockDevice
classic tags SYMLINK / MOUNT_POINT -> Symlink
Interix $DATA IntxLNK -> Symlink, IntxCHR -> CharDevice,
IntxBLK -> BlockDevice
A reparse tag this reader does not model (dedup pointer, cloud
placeholder, vendor tag) returns None rather than a node type: those
records are not POSIX nodes, and reporting one would state something the
volume did not.
Two types are deliberately NOT inferred. ntfs-3g's Interix mode stores a
FIFO as a zero-length $DATA and a socket as a one-byte $DATA, neither
carrying a magic — libntfs-3g/dir.c's own comment reads "FIFO or regular
file". Nothing on disk separates them from ordinary files, so guessing
from size would fabricate an observation. They read as File, and the
limitation is asserted in the tests rather than left implicit.
is_symlink_record is replaced by special_kind_of_record; read_link still
uses reparse_target, since a target and a type are different questions.
535 passed, 0 failed (--features vfs); fmt and clippy -D warnings clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Adds ntfs_windows_reparse.zip: relative and absolute file symlinks, relative and absolute directory symlinks, a junction and a hard-link control, all authored by mklink on Windows 11 (10.0.26200.8875), with Windows own `fsutil reparsepoint query` dump as the answer key. Two of the three tests are REGRESSION coverage, not TDD: the Flags-offset defect they exercise was fixed earlier from a synthetic RED built to the MS-FSCC layout. What the real artifact adds is a sharper control — Windows stores the print name first, so reverting the fix yields "xttarget.t", a plausible-looking filename, where a zero-offset synthetic fixture yields an obvious "\0\0…". The third test is a genuine RED for behavior this reader does not yet have: a directory symlink and a junction set the MFT directory flag, so classification returns Dir and the reparse point is never seen. A consumer walking the tree then recurses into a junction — the loop and duplicate-data hazard that Windows own `dir` avoids by showing <JUNCTION> and <SYMLINKD> rather than <DIR>. RED: rel_dirlink classifies as Dir, expected Symlink. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A directory symlink (mklink /D) and a junction (mklink /J) both set the MFT directory flag while redirecting elsewhere. Classification tested that flag first, so both surfaced as NodeKind::Dir and the reparse point was never reported. A consumer walking the tree then descends into the junction: at best it duplicates the target subtree, at worst it loops. Windows own `dir` avoids exactly this by showing <JUNCTION> and <SYMLINKD> rather than <DIR>, and read_link already returns the substitute name — the kind was the only part still hiding it. Check the reparse point first in both classification sites; the directory flag remains the answer for a plain directory. 539 passed, 0 failed (--features vfs) — no existing assertion moved; fmt and clippy -D warnings clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Per-file entries to the fleet standard: source, verbatim generator command, what each encoding records, size + MD5, ground truth, tier, and the consuming test. Commits the Windows generator script and the fsutil answer key next to the fixture, so the artifact declares its own origin rather than relying on a sentence in a README — the answer key is the evidence that neither the bytes nor the expected values were authored here. Records two things that cost real time to find: the Interix encoding cannot express a FIFO or a socket distinguishably (dir.c: "FIFO or regular file"), and rootless podman cannot mknod a device even with --privileged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
cargo-vet runs with --locked, so a Cargo.toml whose forensic-vfs requirement had moved to 0.9 while Cargo.lock still pinned 0.7 fails before it audits anything: error: cannot update the lock file ... because --locked was passed Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
cargo vet --locked verifies a [[trusted]] entry against the publisher record cached in imports.lock. That record still named 0.7.0, so 0.9.0 read as unvetted even though the trust entry covers it: forensic-vfs:0.9.0 missing ["safe-to-deploy"] The trust entry itself is unchanged and in date (h4x0r, user-id 347968, 2026-07-07 to 2027-08-01). This is version churn, not a new dependency: the diff adds and removes no [[publisher.*]] section, only version fields within existing ones, so nothing new is entering the graph unaudited. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Classification asked one question — "does this record decode to a link target?" — so every character device, block device, FIFO and socket surfaced as
NodeKind::File, and WSL-encoded symlinks were missed entirely because their tag is0xA000001D, not0xA000000C.What changed
A
special_kindthat reports the node type a record actually states, in both encodings a Linux driver writes:NodeKindLX_SYMLINK 0xA000001DSymlinkAF_UNIX 0x80000023SocketLX_FIFO 0x80000024FifoLX_CHR 0x80000025CharDeviceLX_BLK 0x80000026BlockDeviceSYMLINK/MOUNT_POINTSymlinkIntxLNK/IntxCHR/IntxBLKSymlink/CharDevice/BlockDeviceAn unmodelled reparse tag (dedup pointer, cloud placeholder, vendor tag) returns
Nonerather than a node type — those records are not POSIX nodes and reporting one would state something the volume didn't.A directory reparse point is now reported as the link it is
Second, separate fix. A directory symlink (
mklink /D) and a junction (mklink /J) both set the MFT directory flag while redirecting elsewhere. Classification tested that flag first, so both read asDirand the reparse point was invisible.A consumer walking the tree then descends into the junction — duplicating the target subtree at best, looping at worst. Windows' own
diravoids exactly this by showing<JUNCTION>and<SYMLINKD>rather than<DIR>, andread_linkalready returned the substitute name; the kind was the only part still hiding it.Two types are deliberately not inferred
ntfs-3g's Interix mode stores a FIFO as a zero-length
$DATAand a socket as a one-byte$DATA, neither carrying a magic —libntfs-3g/dir.c's own comment reads "FIFO or regular file." Nothing on disk separates them from ordinary files, so guessing from size would fabricate an observation. They read asFile, and the limitation is asserted in the tests rather than left implicit.Fixtures
ntfs_all_node_types.zip— one volume mounted twice by ntfs-3g (Interix, then-o special_files=wsl), so the same five nodes appear in both forms. Superset oftiny.zip's tree. Tier 2.ntfs_windows_reparse.zip— classic reparse points authored bymklinkon Windows 11 10.0.26200.8875, with Windows' ownfsutil reparsepoint querydump committed alongside as the answer key. No Linux tool can produce these tags. Tier 2 at the strong end: neither the bytes nor the expected values were authored here, but the scenario was chosen, so it is not Tier 1.Microsoft's own output confirms both halves of the earlier
Flagsfix —rel_link.txthas length0x34= 12 + 20 + 20 withFlags=1;junctionhas0x3c= 8 + names with noFlagsfield.Negative control worth noting: reverting the
data + 12base makes the Windows test fail with"xttarget.t". Windows stores the print name first, so the bug yields a plausible-looking filename — where a zero-offset synthetic fixture yields obvious NUL-prefixed garbage. That is precisely why the real artifact earns its place.Also
Adds the missing
.gitattributes. This repo commits binary fixtures with no-textguard, and git only scans the first 8000 bytes for a NUL — anything past that window is treated as text and rewritten on a runner withcore.autocrlf=true(the default onwindows-latest).Requires
forensic-vfs 0.9for the widenedNodeKind(#[non_exhaustive], so existing matches keep compiling).RED → GREEN as separate commits for both behavior changes; the Windows fixture's other two tests are labelled regression coverage, not TDD, since the defect they exercise was fixed earlier from a synthetic RED. 539 tests pass with
--features vfs;fmtandclippy -D warningsclean.