Skip to content

Commit 92c05a4

Browse files
authored
fix(task): use shared memory for fspy ipc (#234)
Implemented a shared memory IPC channel for fspy to improve performance and reliability when tracking file system accesses. The channel implementation is cross-platform. In this PR it's only adopted in macOS/Linux version of `fspy`​. Windows version of `fspy`​ will switch to it in a separate PR. ### Why make this change? The previous IPC implementation uses **file descriptor inheritance** to pass down the IPC channel. It has the risk of descriptors being closed in the child processes. Some defensive logic was implemented to handle it, including: - Increase the fd number to avoid Node.js closing small fds: https://github.com/voidzero-dev/vite-plus/blob/540f597f50d049d444a09356da7c929b6da10b22/crates/fspy/src/unix/mod.rs#L156 - Intercept `posix_spawn`​ to avoid being closed by `POSIX_SPAWN_CLOEXEC_DEFAULT`​: https://github.com/voidzero-dev/vite-plus/blob/540f597f50d049d444a09356da7c929b6da10b22/crates/fspy_preload_unix/src/client/mod.rs#L208 It apparently missed some cases, and the IPC fd still got closed, leading to `bad descriptor`​ error in https://github.com/voidzero-dev/vibe-dashboard/actions/runs/18532354385/job/52818262992. ### What changed? - Created a new shared memory-based IPC channel implementation for fspy that replaces the previous Unix socket approach. - The shared memory is looked up **based on names instead of descriptors**, making it impossible to be unexpectedly closed in child processes. - Proper file locking is added to ensure safe access to shared memory - Writing to the IPC channel **becomes much faster as it now only does some atomic memory operations** and doesn't trigger any syscalls. - Modified the semantics of fs access tracking: - The previous implementation automatically waits for the root target process and all its descendants to exit due to the nature of file descriptor inheritance. - The new implementation requires waiting `accesses_future` after the root target process has exited. If descendants are spawned after the root target process has exited, their fs accesses will be missed. <!-- CURSOR_SUMMARY --> --- > [!NOTE] > Replaces Unix fspy IPC with a lock-backed shared-memory channel and updates payloads, readers, and callers; adds test utils and adjusts env/TLS dependencies. > > - **IPC (core)**: > - Introduces shared-memory IPC (`fspy_shared::ipc::channel`) with `Sender`/`Receiver` and lock-guarded `ShmReader`/`ShmWriter` for zero-syscall writes. > - **fspy (Unix/macOS)**: > - Switches from FD/Unix-socket IPC to shared memory; payload now carries `ChannelConf` instead of an FD. > - Refactors `SpyInner::init_in`, fixture handling, and path injection; reads frames via shared memory; lazy-locks receiver after child exit. > - **Preload (Unix)**: > - Client sends `PathAccess` via shared memory; removes POSIX spawn FD-inheritance handling; minor argv handling fix. > - **Windows**: > - Keeps pipe IPC; small fix to `NativeStr` construction and a temporary task to avoid blocking. > - **API/Semantics & Callers**: > - `accesses_future` must be awaited after child exit; updates examples, tests, e2e, and `vite_task` execution flow accordingly. > - Adds `fspy_test_utils` crate for cross-process test helpers. > - **Shared types**: > - Enhances `NativeStr`/`NativeString` for zero-copy (ANSI/wide) and cross-platform encoding; updates spawn payload/env usage. > - **Dependencies**: > - Adds `shared_memory`, `memmap2`, `uuid`, `tracing`; tweaks `nix` features; switches `reqwest` to `native-tls-vendored` (adds `openssl-src`). > - **Misc**: > - Cleans up obsolete files and adjusts test steps (platform filters). > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 8f35015. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY -->
1 parent e687c62 commit 92c05a4

31 files changed

Lines changed: 1284 additions & 456 deletions

File tree

Cargo.lock

Lines changed: 120 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ fspy_preload_windows = { path = "crates/fspy_preload_windows", artifact = "cdyli
7373
fspy_seccomp_unotify = { path = "crates/fspy_seccomp_unotify" }
7474
fspy_shared = { path = "crates/fspy_shared" }
7575
fspy_shared_unix = { path = "crates/fspy_shared_unix" }
76+
fspy_test_utils = { path = "crates/fspy_test_utils" }
7677
futures = "0.3.31"
7778
futures-core = "0.3.31"
7879
futures-util = "0.3.31"
@@ -109,6 +110,7 @@ serde_yml = "0.0.12"
109110
serial_test = "3.2.0"
110111
sha1 = "0.10.6"
111112
sha2 = "0.10.9"
113+
shared_memory = "0.12.4"
112114
shell-escape = "0.1.5"
113115
slab = "0.4.9"
114116
smallvec = { version = "2.0.0-alpha.11", features = ["std"] }

bench/Cargo.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ version = "0.1.0"
44
edition = "2024"
55

66
[dependencies]
7-
vite_task = { workspace = true }
87
vite_path = { workspace = true }
8+
vite_task = { workspace = true }
99

1010
[dev-dependencies]
1111
criterion = { workspace = true }

crates/fspy/Cargo.toml

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ allocator-api2 = { workspace = true, features = ["alloc"] }
99
bincode = { workspace = true }
1010
bstr = { workspace = true, default-features = false }
1111
bumpalo = { workspace = true }
12+
const_format = { workspace = true, features = ["fmt"] }
1213
fspy_shared = { workspace = true }
1314
futures-util = { workspace = true }
1415
libc = { workspace = true }
@@ -18,6 +19,7 @@ slab = { workspace = true }
1819
tempfile = { workspace = true }
1920
tokio = { workspace = true, features = ["net", "process", "io-util", "sync"] }
2021
which = { workspace = true }
22+
xxhash-rust = { workspace = true }
2123

2224
[target.'cfg(target_os = "linux")'.dependencies]
2325
arrayvec = { workspace = true }
@@ -39,10 +41,6 @@ passfd = { git = "https://github.com/polachok/passfd", features = ["async"] }
3941
[target.'cfg(target_os = "macos")'.dependencies]
4042
phf = { workspace = true }
4143

42-
[target.'cfg(any(target_os = "macos", windows))'.dependencies]
43-
const_format = { workspace = true, features = ["fmt"] }
44-
xxhash-rust = { workspace = true }
45-
4644
[target.'cfg(target_os = "windows")'.dependencies]
4745
fspy_detours_sys = { workspace = true }
4846
fspy_preload_windows = { workspace = true }

crates/fspy/examples/cli.rs

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@ async fn main() -> io::Result<()> {
2323

2424
let TrackedChild { mut tokio_child, accesses_future } = command.spawn().await?;
2525

26+
let output = tokio_child.wait().await?;
27+
2628
let accesses = accesses_future.await?;
2729

2830
let mut path_count = 0usize;
@@ -47,7 +49,6 @@ async fn main() -> io::Result<()> {
4749
}
4850
csv_writer.flush().await?;
4951

50-
let output = tokio_child.wait().await?;
5152
eprintln!("\nfspy: {path_count} paths accessed. {output}");
5253
Ok(())
5354
}

crates/fspy/src/fixture.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ macro_rules! fixture {
2626
pub use fixture;
2727

2828
impl Fixture {
29+
#[cfg(not(target_os = "linux"))]
2930
pub const fn new(name: &'static str, content: &'static [u8], hash: &'static str) -> Self {
3031
Self { name, content, hash }
3132
}

crates/fspy/src/lib.rs

Lines changed: 5 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,7 @@
11
#![cfg_attr(target_os = "windows", feature(windows_process_extensions_main_thread_handle))]
22
#![feature(once_cell_try)]
33

4-
// Windows and macOS both need to persist the injected DLL/shared library somewhere in the filesystem.
5-
// Linux doesn't need this. Instead we use `memfd_create` to create an in-memory shared library.
6-
#[cfg(not(target_os = "linux"))]
4+
// Persist the injected DLL/shared library somewhere in the filesystem.
75
mod fixture;
86

97
#[cfg(unix)]
@@ -17,9 +15,7 @@ mod os_impl;
1715
mod arena;
1816
mod command;
1917

20-
#[cfg(not(target_os = "linux"))]
21-
use std::{env::temp_dir, fs::create_dir};
22-
use std::{ffi::OsStr, io, sync::OnceLock};
18+
use std::{env::temp_dir, ffi::OsStr, fs::create_dir, io, sync::OnceLock};
2319

2420
pub use command::Command;
2521
pub use fspy_shared::ipc::{AccessMode, PathAccess};
@@ -30,23 +26,19 @@ use tokio::process::Child;
3026

3127
pub struct TrackedChild {
3228
pub tokio_child: Child,
33-
pub accesses_future: BoxFuture<'static, io::Result<os_impl::PathAccessIterable>>,
29+
/// This future lazily locks the IPC channel when it's polled.
30+
/// Do not `await` it until the child process has exited.
31+
pub accesses_future: BoxFuture<'static, io::Result<PathAccessIterable>>,
3432
}
3533

3634
pub struct Spy(SpyInner);
3735
impl Spy {
38-
#[cfg(not(target_os = "linux"))]
3936
pub fn new() -> io::Result<Self> {
4037
let tmp_dir = temp_dir().join("fspy");
4138
let _ = create_dir(&tmp_dir);
4239
Ok(Self(SpyInner::init_in(&tmp_dir)?))
4340
}
4441

45-
#[cfg(target_os = "linux")]
46-
pub fn new() -> io::Result<Self> {
47-
Ok(Self(SpyInner::init()?))
48-
}
49-
5042
pub fn global() -> io::Result<&'static Self> {
5143
static GLOBAL_SPY: OnceLock<Spy> = OnceLock::new();
5244
GLOBAL_SPY.get_or_try_init(Self::new)

0 commit comments

Comments
 (0)