LifeViz

Windows 11-ready WPF visualization of a 3D-stacked Game of Life grid. The UI stays minimalist-16:9 canvas, no chrome, with controls centered in the right-click context menu plus a dedicated Scene Editor for source stack workflows-and it supports tapping into open desktop windows, webcams, and media files as live depth sources.

Development

dotnet build
dotnet run

The new Rider solution (lifeviz.sln) includes a "lifeviz: Run App" configuration so IDE runs mirror dotnet run.

For runtime validation without launching your saved scene manually:

dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-benchmark
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-handoff
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-rgb-threshold
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-frequency-hue
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test passthrough-underlay-only
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test simulation-reactive-mappings
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test simulation-reactive-persistence
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test simulation-reactive-legacy-migration
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test simulation-reactive-removal
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test simulation-reactive-editor-isolation
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test pixel-sort-editor-roundtrip
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test no-sim-group-renders-composite
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test sim-group-removal-clears-runtime
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test disabled-sim-group-renders-composite
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test sim-group-stack-order
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test sim-group-inline-hue
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test sim-group-inline-presentation
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-pixel-sort
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test sim-group-pixel-sort-color
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-injection-mode
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-file-injection-mode C:\path\to\video.mp4
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-sim
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-source
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test source-reset
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test gpu-render
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-mainloop
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-mainloop-sim-group
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-240
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-480
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-rgb-240
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-rgb-480
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-file-240 C:\path\to\video.mp4
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-file-480 C:\path\to\video.mp4
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-file-rgb-240 C:\path\to\video.mp4
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-file-rgb-480 C:\path\to\video.mp4
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-visible
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-fullscreen
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-bisect
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-720
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-visible-720
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-fullscreen-720
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-presets
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-visible-presets
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-fullscreen-presets
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test profile-current-scene-interaction
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test pacing-current-scene-visible-presets
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test pacing-current-scene-fullscreen-presets
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test pacing-current-scene-interaction
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test pacing-current-scene-overlay-fullscreen-144
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test pacing-current-scene-suite
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test frame-pump-thread-safety
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test shutdown
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test startup
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --smoke-test all

The live renderer now keeps source-layer sampling pixel-sharp by default on the GPU path too; the source compositor uses point sampling rather than linear filtering unless a future explicit smoothing control is added.

Inline Sim Group scenes still use the shared-GPU presentation path, and the renderer now logs a throttled warning if that inline GPU handoff falls back to the CPU-present path so live flicker reports can be diagnosed from the real app log instead of by guesswork.

For visible normal-startup diagnostics against your real saved scene instead of smoke-mode startup:

dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --diagnostic-test profile-current-scene-visible-144
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --diagnostic-test profile-current-scene-visible-480
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --diagnostic-test profile-current-scene-visible-720
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --diagnostic-test profile-current-scene-interaction
dotnet bin\Debug\net9.0-windows-sbx\lifeviz.dll --diagnostic-test profile-current-scene-soak-720

gpu-benchmark reports GPU sim/source timings for the current synthetic workload so you can see whether readback is still dominating. gpu-handoff validates that a GPU-built composite can inject directly into the GPU simulation path without a CPU composite readback. gpu-rgb-threshold specifically validates the RGB composite-injection threshold path against a pure-white source, which catches GPU handoff regressions where the compositor texture is sampled incorrectly or left bound as a render target. gpu-passthrough-signed-model verifies that passthrough uses the signed additive/subtractive composition model whenever a passthrough baseline is present, including the shared-GPU underlay path, so layer opacity changes cannot incorrectly fade the underlay. passthrough-underlay-only now drives the real frame tick and verifies that with zero active simulation layers, source capture/composite refresh still happens and passthrough still presents the source composite instead of falling to black. gpu-frequency-hue validates that per-layer frequency-driven hue shift numerically changes the resolved hue handed to presentation while the raw RGB simulation buffer itself stays unchanged in the live GPU path. simulation-reactive-mappings validates the modular per-layer mapping system itself, including each mapping row's input Min / Max threshold window normalization, against known audio inputs even when the legacy global audio-reactivity master toggle is off. pixel-sort-reactive-cell-size validates the Cell Width / Cell Height reactive outputs against a live GPU Pixel Sort layer and fails unless the runtime cell size actually changes while the sorted output still preserves the source histogram. simulation-reactive-persistence round-trips those mappings through both the scene-project serializer and the app-config normalization path so per-layer reactivity does not silently disappear on save/load. simulation-reactive-legacy-migration verifies that old whole-scene Level -> Framerate and Level -> Life Opacity config flags are migrated into per-layer mappings and then cleared, so the app no longer relies on the split global path. simulation-reactive-removal verifies that removing a mapping really returns the layer to its base state instead of leaving stale reactive output behind. simulation-reactive-editor-isolation verifies both that newly added simulation layers start with an empty reactive list instead of inheriting the selected layer's mappings and that the live sim-group Selected Layer editor still exposes the Reactive Mappings UI for the selected child layer. pixel-sort-editor-roundtrip specifically verifies that a Pixel Sort layer survives the Scene Editor add/apply/refresh path with its layer type and cell-size settings intact, so editor rebinding cannot silently downgrade it back to Life Sim. no-sim-group-renders-composite verifies that when there are no scene Sim Group layers, the app presents the source composite directly instead of leaving a blank frame or a hidden runtime sim stack. sim-group-removal-clears-runtime verifies that removing the last scene Sim Group actually clears the runtime simulation stack instead of leaving a hidden stack running. disabled-sim-group-renders-composite verifies that even when a Sim Group still exists in the scene, disabling all of its child sim layers falls back to the source composite instead of leaving stale simulation output onscreen. sim-group-stack-order now verifies both that moving the same sim group to a different position in the main scene stack changes the resolved output and that the resolved inline scene composite stays GPU-backed. sim-group-inline-hue verifies that the inline sim-group compositor still applies per-layer RGB hue shift while remaining GPU-backed, so moving sim groups into the main stack does not silently bypass the live hue controls. sim-group-inline-presentation now uses the minimal repro stack for the flicker bug: rainbow background, inline Sim Group with Pixel Sort, and a static portrait-style source above it at 240p. It now has a dynamic phase that requires the presented output to keep advancing with the evolving scene, a static phase that fails unless the presented frame stays bit-stable for the whole dwell window, a redraw-pressure phase that hammers chrome/UI invalidation while manual redraws and frame submits keep running, a real file-backed 480p hover phase, and an exact-repro phase that uses the copied rainbow JPG plus converted Mona Lisa PNG from Assets/SmokeRepro. That exact phase keeps the three-layer stack static at 480p, fails unless both the resolved inline composite and the presented frame stay bit-stable through hover pressure, and now also asserts that the inline scene is being handed to presentation through the dedicated GPU snapshot ring instead of directly through a live source-compositor shared texture. gpu-pixel-sort validates the new GPU pixel-sort layer backend directly by injecting a live GPU composite, resolving the cell sort on-GPU, and failing unless the resolved output surface actually differs from the input pattern while preserving the pixel histogram. sim-group-pixel-sort-color pushes a Pixel Sort layer through the real inline sim-group renderer and fails unless the resolved group output still preserves the source histogram instead of blowing out colors by re-adding the underlay or sampling the wrong GPU texture format. gpu-injection-mode validates that Threshold, Random Pulse, and Pulse Width Modulation still produce distinct densities on a mid-gray source in the GPU composite-injection path. gpu-file-injection-mode <video> runs that same density comparison against the first decoded frame of a real file source, which is useful when a report only reproduces on actual media. gpu-sim validates the D3D11 simulation backend in both Naive Grayscale and RGB Channel Bins. gpu-source drives the real BuildCompositeFrame path through MainWindow with synthetic sources and fails unless the GPU source compositor actually runs. Static file-backed images no longer opt into the native-source fast path; they stay on the exact CPU-downscaled BGRA path so hover/input redraws cannot destabilize them and they keep the default pixel-sharp look. gpu-render launches a hidden MainWindow and fails unless the real GPU composite pipeline initializes through the app render path. profile-mainloop runs the real frame loop on a hidden synthetic scene, exports a JSON timing report under %LOCALAPPDATA%\lifeviz\profiles, and logs the highest-cost stages plus UI dispatcher latency samples and frame-gap spike counters (>25ms, >33ms, >50ms). profile-mainloop-sim-group runs that same profiler against a synthetic scene with an embedded Sim Group, so inline-stack compositing cost can be compared directly against the plain-source baseline. Profile and pacing smokes no longer request CPU fallback color buffers from GPU simulation layers, and the sim-group profiling path now also suppresses intermediate sim-group readback, so those timings stay close to the live shared-texture renderer instead of the correctness-validation path. profile-240 / profile-480 do the same at fixed grayscale resolutions, while profile-rgb-240 / profile-rgb-480 pin the reference layer to RGB Channel Bins so renderer changes can be measured against the formerly slow path directly. profile-file-240 / profile-file-480 run that same profiler against a real file-backed source instead of synthetic buffers, and profile-file-rgb-240 / profile-file-rgb-480 do the same with the reference layer pinned to RGB Channel Bins; pass the media path as the third argument (or set LIFEVIZ_SMOKE_VIDEO). Those file-backed smokes now fail if a file source stops publishing fresh frame tokens, which catches non-exact video handoff regressions automatically. profile-current-scene loads the persisted user scene/config and profiles it headlessly, while profile-current-scene-visible does the same in a visible window so visible-window pacing regressions can be measured without hand-copying overlay text. profile-current-scene-bisect runs the same real-scene visible profile several times from fresh startup variants (baseline, no-audio, no-video, no-sim-groups, and first-static-only) so config-dependent regressions can be narrowed to audio loopback, video decode, inline sim groups, or source complexity instead of being treated as one opaque “current config” problem. These current-scene profile smokes now wait through an explicit warmup period before they start measuring, and then fail on settled playback freshness (capture_*_frame_age_ms, capture_*_fresh_frame_ratio) instead of trusting startup-noisy averages. profile-current-scene-fullscreen does the same after forcing the main output window into fullscreen and now also fails if the render host collapses into the old tiny centered/top-centered rectangle instead of occupying the expected fullscreen display rect. profile-current-scene-<144|240|480|720|1080|1440|2160>, profile-current-scene-visible-<...>, and profile-current-scene-fullscreen-<...> force the real saved scene to that exact preset height before profiling, and the profile-current-scene-presets / profile-current-scene-visible-presets / profile-current-scene-fullscreen-presets suite targets walk the full preset ladder up to 2160p using a short per-preset dwell so the full sweep is practical to run regularly. profile-current-scene-interaction now opens the Scene Editor, verifies that the main window does not stay throttled while the editor is open, cycles the root context menu, and fails unless pacing stays near the pre-interaction baseline after both operations. current-scene-hover-presentation loads the saved scene, applies hover-style chrome invalidation pressure while the live frame pump is running, and compares presented-frame delta statistics against a baseline window. When the saved scene contains inline Sim Group layers, it now also asserts that the stable GPU snapshot ring was actually exercised during the run. pacing-current-scene-visible-presets and pacing-current-scene-fullscreen-presets take the real saved scene, pin it to a stable 60 fps budget, walk the realtime preset ladder (144/240/480), and fail on explicit underrun thresholds (avg/p95/p99 frame gap plus >25ms, >33ms, >50ms ratios) instead of just exporting timing data. pacing-current-scene-interaction does the same around the 480p interaction case with the Scene Editor and root context menu. pacing-current-scene-overlay-fullscreen-144 turns on Show FPS in fullscreen at 144p and fails if the overlay itself reintroduces pacing regressions. pacing-current-scene-suite runs the visible preset ladder plus the interaction case in one go. frame-pump-thread-safety opens the real Scene Editor, hits the frame-pump interaction helper from a worker thread, and fails on the exact cross-thread Window.IsActive regression that crashed the app. In smoke-test mode, profile reports are written under bin\Debug\net9.0-windows-sbx\profiles\ so they stay local to the test run. shutdown now opens the Scene Editor and then closes the real main window, failing if shutdown teardown captures any exception or if the owned editor-close path throws. startup launches a minimal WPF window in smoke-test mode without loading the persisted project, media sources, ffmpeg decode, or audio loopback so UI/render initialization can be tested in isolation. all runs gpu-sim plus the combined GPU handoff/passthrough/source/render UI smoke suite. --diagnostic-test ... is the complement to smoke tests: it launches the normal saved-scene startup path instead of App.IsSmokeTestMode, writes reports under the build output's profiles\ folder, and is intended for live-only divergences where smoke mode looks clean. The current targets cover visible preset runs, fullscreen preset runs, interaction profiling, and a longer 720p soak. Settled profile validation now also checks presentation_draw_fps, so a run can fail when the frame loop looks healthy but the actual presented cadence falls behind. In the running app, Show FPS now reports Present, Loop, and Sim separately, and the root context menu includes Export Live Profile... to capture the exact live session without restarting into the saved scene. Long-running media workers and ffmpeg child processes now run at below-normal priority so LifeViz yields more cleanly to other realtime apps and USB/audio stacks under load. The root context menu now includes Performance controls for live contention tuning: Low Contention Mode lowers LifeViz's own process/frame-pump priority, stops requesting the global 1 ms multimedia timer, and pushes ffmpeg decode processes toward idle/background scheduling; Decoder Threads lets you cap ffmpeg decode fanout (Auto, 1, 2, 4); and Video Decode FPS lets you cap file/video decode output to 30 or 15 fps when you need LifeViz to back off while sharing a machine with DJ/controller software. Those settings apply to existing video/sequence sessions immediately. dimensions now covers the real Scene Editor height dropdown in both Live Mode and deferred Apply mode with the reference simulation layer forced to RGB Channel Bins, and fails unless every simulation layer plus the presentation surface resize to match.

For a quick local ClickOnce install/update smoke test:

.\install.ps1

By default, install.ps1 now runs Install-ClickOnce.ps1 directly against the fresh publish output (more reliable for in-place updates). Use -BundleInstaller if you explicitly want a generated single-file lifeviz_installer.exe.

Live Window Injection

Right-click the scene and use Sources to stack multiple windows, OBS-style:

• Add entries via Sources > Add Window Source, Add Webcam Source, Add File Source, Add Video Sequence, or Add Layer Group (checked items are already in the stack). File sources accept static images (PNG/JPG/BMP/WEBP), GIFs, and videos; animated files loop. Video sources always loop, and video sequences build a playlist that advances on end and loops back to the first clip. Video layers expose Restart, Pause/Play, and Scrub controls so you can seek directly to a position. Layer groups have their own submenu stack; they composite their children internally and blend as a single layer, using the first child to determine the group's aspect ratio. The top entry is the primary: it drives the canvas aspect ratio (unless the aspect ratio lock is enabled). Make Primary, Move Up/Down, or Remove/Remove All to resequence quickly; clearing all sources drops back to the 16:9 default.
• Video, video-sequence, and YouTube layers include a Play Audio toggle (default off) plus a per-layer Audio Volume control. Enable audio per source when you want that layer's soundtrack routed to your default output device, then trim that source independently. Sources without an audio stream stay silent and log a warning instead of continuously retrying playback.
• Audio playback is lifecycle-managed per source: muting a layer, removing it, or closing the app now shuts down that source's in-app audio pipeline.
• Audio playback startup is debounced per source so rapid state changes do not spawn duplicate decode/playback pipelines.
• Enabling Play Audio now seeks to the current video playback position (best effort), so toggling audio on mid-playback stays close to visual timing instead of always starting at 0:00.
• Audio decode is clocked in real-time to avoid fast decode churn/restarts that can cause stuttery playback artifacts.
• Each source exposes a wallpaper-style fit mode (Fill default, plus Fit/Stretch/Center/Tile/Span) that controls how the layer scales into the frame.
• Each source has its own blend mode applied during compositing (Additive default; Normal, Multiply, Screen, Overlay, Lighten, Darken, Subtractive). Normal respects per-pixel transparency in sources like PNGs, and can optionally key out a background color (default black) with an adjustable range.
• Scene Editor... opens a dedicated two-pane source manager with a draggable nested tree on the left and tabbed controls on the right. The main scene tree now supports a Sim Group layer type directly alongside source layers and layer groups. When a Sim Group is selected, its nested simulation-layer tree and per-layer simulation settings live on the Selected Layer tab instead of in a separate global simulation-stack editor. The secondary simulation tab now focuses on project-wide controls (Height, Depth, Framerate, and Global Life Opacity). Sim groups define ordered chaining: the first child samples the scene-stack surface that exists at that exact point in the main layer order, and each later child samples the previous child output. After the group resolves its own sim layers, lower scene layers composite on top of that fully rendered group output, so moving a sim group up or down the main stack now changes both its input and everything below it. On the live GPU path, that inline sim-group resolution now stays GPU-backed end-to-end: source layers append onto the current scene composite on the shared D3D11 device, sim-group children resolve into an offscreen GPU composite surface, and the final scene composite is presented from that GPU surface without CPU readback except for recording/smoke validation paths. New sim groups now start empty, so you explicitly choose whether the group contains Life Sim, Pixel Sort, or a mix. Inside the sim-group editor, the add controls are now Add Life Sim and Add Pixel Sort Layer. Life Sim keeps the Conway settings (Input, Injection, Life Mode, Binning, Injection Noise, threshold window). Pixel Sort is a GPU-only per-frame cell sort layer with its own Cell Width and Cell Height sliders; 1x1 is effectively passthrough, and larger cells target a full top-left-to-bottom-right sort order inside each cell while preserving the sampled pixel shades. Pixel Sort is treated as a full-frame effect layer, so once it has baked the upstream scene into its own output the sim-group compositor does not add the original underlay back on top again. Both layer types keep common Blend, Opacity, hue, and Reactive Mappings controls. Each reactive mapping now includes an audio input, an output, an amount, and per-mapping input Min / Max threshold sliders. The incoming audio value is normalized through that Min / Max window before it drives the output. For Pixel Sort, Cell Width / Cell Height mapping amounts are additive deltas from the base cell size, shown as +N px, and capped at 50 px so the effect stays in the useful range for this filter. New simulation layers start with an empty reactive list instead of inheriting the selected layer's mappings, and cleared mappings now also clear the old hidden legacy hue-reactivity field on live apply/save so deleting the last mapping really removes the effect. Height is preset-driven (144, 240, 480, 720, 1080, 1440, 2160) in both the main menu and Scene Editor, and the Scene Editor dropdown now uses an explicit dark selection/hover style so preset rows stay readable while you scrub through them. The editor includes an App Controls... button that opens the full main context menu at the editor location for global settings parity. For video-capable layers it adds a scrub slider with live time readout so you can seek playback position. The Scene Editor header also provides global source-audio controls: Master Audio (toggle) and a master volume slider that scale/mute all video/YouTube/sequence audio without changing per-layer settings. Enable Live Mode for immediate updates, or turn it off and use Apply to batch changes. The editor includes Save... and Load... buttons to export/import .lifevizlayers.json scene projects; these files now include sim groups inside the main scene stack, per-group simulation-layer trees, source stack, per-layer reactive mapping lists, and core simulation/render project settings (height, depth, framerate, global life opacity, passthrough/composite blend, invert composite).
• Legacy global simulation config is no longer honored on normal startup. Simulation now only comes from real scene-stack Sim Group layers, and loading an old legacy-only config clears that legacy path on the next save.
• When there are no scene Sim Group layers at all, or when every child layer inside the existing sim groups is disabled, LifeViz presents the composited source stack directly instead of falling back to a hidden/default simulation stack, stale simulation frame, or a blank output.
• Clearing all scene sources no longer forces Passthrough Underlay off, so re-adding sources preserves the previous underlay display mode instead of coming back black until you manually re-enable passthrough.
• Each source can stack animations (Zoom In, Translate, Rotate, Beat Shake, Audio Granular, Fade, DVD Bounce) synced to the global animation BPM, with forward or reverse loops plus expanded speed steps (1/8x–8x) and per-animation cycle lengths for long fades; Beat Shake responds to detected audio beats when a device is selected (falls back to BPM if not) and includes an intensity slider (speed/cycle do not affect its amplitude), Audio Granular now uses a gated/compressed response curve with a per-layer 3-band EQ (Low/Mid/High) and a wider intensity range up to 1000% (with a stronger neutral 100/100/100 EQ baseline), and DVD Bounce exposes a size control. DVD Bounce translation is now snapped to whole pixels on the point-sampled render path to reduce shimmer/flicker on moving photographic layers without turning layer smoothing back on by default. Use Animation BPM > Sync to Audio BPM to beat-match all animations, and Audio Source > None to clear the input.
• Audio Reactivity adds configurable simulation modulation from the selected audio source: simulation layers now own the canonical modular reactive-mapping system in the Scene Editor. Supported per-layer inputs are Level, Low, Mid, High, overall Frequency, Low Frequency, Mid Frequency, and High Frequency. Supported per-layer outputs are Opacity, Framerate, Hue Shift, Hue Speed, Injection Noise, Threshold Min, Threshold Max, Cell Width, and Cell Height. Every mapping row now also has input Min / Max thresholds, and the incoming audio value is normalized through that window before the output amount is applied. For Pixel Sort, Cell Width / Cell Height amounts are additive deltas from the base value, displayed as +N px, and capped at 50 px. Band-energy signals are normalized logarithmically to 0..1 inside their own frequency bins, and band-frequency signals are log-normalized within those bins as well so the mapping feels musically sensible instead of linear in Hz. Per-layer mappings are keyed off the selected audio device and do not depend on the legacy global audio-reactivity master toggle. The context menu now keeps only the genuinely global audio actions: input gain, level seeding, and beat seeding. Old whole-scene Level -> Framerate / Level -> Life Opacity settings are migrated forward into per-layer mappings automatically. Audio Source includes output devices (including System Output (Default) via WASAPI loopback) plus input devices.
• Context menu responsiveness: the Sources and Audio Source submenus now refresh device lists when those submenus are opened, instead of doing that work every time the root menu opens.
• Audio level normalization is tuned for typical program material instead of near-full-scale samples, so loopback audio reaches a much larger portion of the 0-100% range and opacity/framerate reactions read more strongly at normal listening levels.
• Audio level percent shown in the FPS overlay now uses that rectified short-window max envelope directly, so it tracks the absolute waveform peaks per bucket instead of averaging them down.
• Video file sources are decoded with ffmpeg; if frames render blank, LifeViz will auto-transcode to H.264 using ffmpeg (cached under %LOCALAPPDATA%\\lifeviz\\video-cache). While transcoding, the layer is temporarily skipped (so it won't blank out the stack). Install ffmpeg on your PATH or transcode manually if needed.
• Composite Blend controls shader-based passthrough fallback mixing (Additive default; Normal is transparency-aware).
• Passthrough Underlay shows the composite behind the simulation.
• The root Simulation Layers submenu now exposes simulation-layer management plus a global master Life Opacity control. Per-layer life mode, binning mode, injection noise, and layer opacity now live in the Scene Editor's Simulation Layers panel.
• Life Opacity now attenuates simulation output only; passthrough underlay remains visible at full strength even at 0% life opacity.
• If reactive opacity drives every simulation layer to 0% on a frame, the GPU renderer now keeps presenting passthrough as an underlay-only frame instead of dropping out to a black fallback frame.
• Source capture/composite refresh is now decoupled from simulation stepping, so passthrough and file/video underlay keep updating every render tick even when there are no enabled simulation layers or when a layer is stepping more slowly than the render loop.
• Shared GPU sim/underlay presentation now performs one synchronized producer flush at the final presentation handoff instead of per-layer flushes. That keeps the live shared-texture path visually current without going back to the old worst-case flush-per-layer behavior.
• Fullscreen toggle lives in the context menu and persists; it now sizes to the active monitor bounds, stays topmost, and covers the taskbar.
• The main output window now uses custom chrome instead of the native Windows title bar. Dragging the background anywhere in the output window now moves the window, double-clicking that background toggles fullscreen, and resizing from the left/right/bottom hit zones stays inside LifeViz pointer handling instead of entering the OS move/resize loop. The top chrome buttons expose the root menu (...), Scene Editor (SE), minimize, fullscreen, and close. The main render loop no longer self-throttles just because the root context menu or Scene Editor is open; throttling is now reserved for actual native move/resize loops.
• Frame pacing is now driven by a dedicated high-resolution scheduler thread rather than System.Threading.Timer. The scheduler posts at most one pending frame callback to the WPF dispatcher at a time, which substantially reduces baseline underruns and timer-queue jitter even before any mouse interaction occurs.
• Entering or exiting fullscreen now explicitly resets frame-pump cadence, so the scheduler does not carry stale timing state across a display-mode transition.
• Custom resize now owns aspect enforcement directly while you drag, so the window no longer snaps back after release. The generic WPF SizeChanged aspect snap no longer fights the chrome resize path.
• Update to Latest Release... pulls down the newest GitHub release installer and runs it to upgrade the current installation in place (LifeViz closes while the installer runs).
• Capture uses DPI-correct window bounds (via DWM) so the full surface is normalized even for PiP/scaled windows, and the composited buffer feeds the injection path (per-layer threshold windows + noise + life/binning modes) on every tick.
• Source downscale now uses small supersampled filtered sampling before injection to reduce visible vertical/horizontal banding in live imagery. The CPU compositor keeps cheap aligned copies for normal cases and only uses filtered remapping when needed, while the final life grid still presents with nearest-neighbor scaling so cells stay crisp.
• Renderer migration now includes a real GPU presentation/composite path. MainWindow talks to an IRenderBackend; the default path attempts a GpuRenderBackend first, hosting a D3D11-backed DrawingSurface (Vortice.Wpf), uploading the final simulation frame plus passthrough underlay into GPU textures, and resolving passthrough blend modes in a fullscreen shader pass before presentation. Source capture compositing now also prefers a D3D11 path: source frames are uploaded as textures, blended on a GPU render target with fit modes/transforms/keying/blend modes preserved, then the final composite is read back once for injection. CPU source compositing remains as automatic fallback/migration scaffolding, but simulation itself is now GPU-only and startup fails if the D3D11 simulation backend cannot initialize.
• The live render surface is now treated as display-only by WPF. The Viewbox host, fallback Image, and GPU DrawingSurface are non-hit-testable/non-focusable so ordinary mouse hover over the window does not route interaction through the render surface itself.
• The final render host no longer routes the GPU/D3D surface through a WPF Viewbox. The fallback Image and GPU DrawingSurface now scale with Stretch=Uniform inside a plain black host instead, which removes a large-window/fullscreen WPF scaling cost that could make the underlay video look like a slideshow even when simulation timing was fine.
• Simulation layers now sit behind an ISimulationBackend seam, and the default backend is now GpuSimulationBackend when possible. Both Naive Grayscale and RGB Channel Bins now run their injection, Conway stepping, depth-history shifting, and color-buffer generation in D3D11 compute shaders. RGB layers can inject directly from the final GPU composite too, including direct/inverse input mapping, threshold/random pulse/PWM injection modes, per-layer threshold windows, injection noise, and hue-rotated channel mapping.
• Runtime smoke tests are now first-class: --smoke-test gpu-benchmark profiles the current GPU sim/source stack, --smoke-test gpu-handoff verifies direct GPU composite-to-sim injection without CPU composite readback, --smoke-test gpu-sim exercises the GPU simulation backend directly, --smoke-test gpu-source verifies that MainWindow really uses the D3D11 source compositor for BuildCompositeFrame, --smoke-test gpu-render verifies that MainWindow really brings up the D3D11 presentation composite path, --smoke-test profile-mainloop captures full frame-stage timing plus UI dispatcher latency samples and frame-gap spike counters from the real frame loop, --smoke-test shutdown verifies that the real window can tear down cleanly even when the Scene Editor is open, and --smoke-test startup opens the app in an isolated smoke-test mode that skips persisted scene/config startup so renderer/UI regressions can be caught without spinning up user media pipelines.
• File-backed smoke profiling is now supported: --smoke-test profile-file-240 <video> and profile-file-480 <video> run the real frame profiler against an actual media file source, which is the right way to measure decode/playback regressions that synthetic scenes cannot reproduce. Those smokes now fail if the file source stops advancing fresh frame tokens, so a frozen underlay video is caught automatically instead of only by eye.
• Audio analysis is now demand-driven: waveform history, bass/mid/high frequency extraction, and 30-second debug traces only run when the FPS/debug overlay is visible or when a live feature actually needs spectrum data (for example Audio Granular or a per-layer reactive mapping that consumes frequency/band inputs).
• When spectrum analysis is required, FFT/spectrum extraction is now rate-limited to about 60 Hz instead of running on every audio callback. That keeps Audio Granular and per-layer frequency mappings responsive without letting the audio thread dominate low-resolution scenes.
• File-video playback no longer hands the UI a single latest-frame slot. VideoSession now keeps a short decoded-frame queue plus buffer pool, which smooths over brief UI/render stalls instead of immediately turning them into visible dropped frames.
• Long-lived file-video raw decode, sequence-advance, sequence-probe, and source-audio decode workers now run on dedicated long-running background threads instead of the shared threadpool. That keeps blocking media loops from starving unrelated work during long sessions.
• File-video decode resolution is now adaptive. When the render path does not need native source pixels, LifeViz restarts ffmpeg at a process resolution sized to the current simulation grid instead of always pushing native-sized raw frames through the app. That specifically targets the "smooth sim, slideshow video" failure mode on low-resolution scenes with HD sources.
• File-video decode no longer emits periodic per-frame heartbeat logs during normal playback, and tiny file-video downscales now stay single-threaded instead of spawning Parallel.For work. That reduces long-run threadpool churn and low-resolution playback collapse in file-backed scenes.
• On the GPU source-composite path, low-resolution scenes no longer prefer native file/video frames by default. For grids below roughly 640x360, LifeViz uses the already-downscaled file frame instead of hauling full native video frames through the compositor, which substantially reduces CPU/memory pressure for dense low-res multi-video scenes.
• Native-source preference is now media-aware. Static file sources can still switch back to native frames at moderate sizes, but video file sources and video sequences stay on adapted/direct decoder output at every preset height. For the live multi-video scene, switching video back to native frames was the point where 1080p+ freshness collapsed into slideshow behavior.
• File-video downscale now has direct-copy and centered crop/pad fast paths for the common low-resolution adapted-video cases (source == target, vertical crop, horizontal crop, centered letterbox/pillarbox). The profiler also records delivered file-frame freshness (capture_file_fresh_frame_ratio) and age (capture_file_frame_age_ms) so stale underlay video can be diagnosed directly instead of inferred from render timing.
• When a video source does not need native source pixels and its fit mode is Fill, Fit, or Stretch, LifeViz asks ffmpeg to emit exact target-sized raw frames directly only when that does not require decoder-side upscaling past the native source size. That preserves the low-resolution freshness win without turning 1080p+ and 4K runs into giant decoder-side upscale workloads.
• File-video sessions now also publish their current decoder-sized frame directly into the compositor path and let the GPU scale it to the final composite size. That removes the CPU resample worker from the normal live video path even when the decoder is using an adaptive/native-sized frame instead of exact target output, which is the key to making 1080p+ and 4K profiling credible. The capture path now preserves those decoder-sized buffers all the way through CaptureFrame() instead of discarding them for not matching the requested target size, which fixes the freeze that could appear when a video source switched from exact-target decode to adaptive/native decode.
• When a file-video pipeline is restarted for a new target/process size, LifeViz now holds the last good decoded frame until the new worker publishes replacement frames. That prevents the old immediate black flicker during height changes such as 480p -> 1080p.
• File-video frames now carry their true published buffer dimensions through the capture path even when the decoder is not emitting exact target-sized output. Without that metadata fix, the compositor could treat a native/adaptive video frame as if it were already target-sized and appear to freeze or smear only on the non-exact video path.
• Current-scene profiling now records per-source file freshness metrics as well as the aggregate ones. That makes it possible to tell when one file source is freezing while another keeps updating, which is exactly the failure mode that aggregate capture_file_frame_age_ms can miss.
• File-video sessions now defer starting their ffmpeg decode worker until the first real capture request. That removes the old probe-time native-worker startup followed by an immediate restart into the actual process shape, which was a bad fit for mixed-resolution scenes and short looping clips.
• Video-sequence layers keep the last good frame visible while the next clip session is opened asynchronously. Sequence handoff is still a cold open today, but the underlay no longer drops to black just because the next clip has not published its first frame yet.
• Sequence clips now cache probe metadata ahead of time, so an advance no longer has to pay the full synchronous duration/dimension probe cost during the live handoff.
• The GPU final-composite path no longer forces redundant producer-device flushes before every present. That was the main cause of the render-side 720p stalls after the sequence path was fixed, because it converted otherwise cheap queued GPU work into a synchronous wait on every frame.
• Simulation injection now lazily rebuilds a CPU-readable composite only when GPU injection cannot satisfy a layer on that frame, so turning passthrough/recording off no longer risks an empty CPU fallback composite and lost injection.
• Final simulation-layer compositing now prefers a dedicated GPU presentation pass when not recording, so the old CPU per-pixel layer blend is mostly a fallback path instead of the normal render hot loop.
• Naive-grayscale GPU simulation layers now expose their rendered color textures to the presentation backend through shared D3D11 resources, so the normal GPU presentation path can sample those layers directly without a per-frame CPU FillColorBuffer() readback.
• On the normal GPU live path, passthrough underlay now rides the shared GPU composite surface directly instead of forcing a CPU composite readback just to feed final presentation. CPU composite readback is still required for recording and for any simulation layer that falls back to CPU mask injection on that frame.
• The main frame loop now uses a coalesced one-shot background timer that schedules the next UI-thread frame callback at the current target cadence instead of spamming 1ms dispatcher ticks. The dispatcher callback itself now runs at background priority, and cadence state is explicitly reset after context-menu/native move-loop interaction and after simulation-height changes so temporary UI throttling does not leave the renderer in a degraded pacing phase.
• The main output window no longer relies on native title-bar dragging for its normal shell interaction. Custom chrome/manual resize keeps ordinary window moves out of the OS move loop, so render pacing is not forced to compete with native non-client dragging in the same way. Title-bar/menu pacing smoke coverage is now focused on the custom root-menu interaction path.
• Double-clicking the output surface now toggles fullscreen in both directions, so you can enter or leave fullscreen without using the context menu.
• Naive-grayscale GPU simulation now shares the same D3D11 device/context as the GPU source compositor, so grayscale layers can inject straight from the final GPU composite texture without rebuilding CPU masks first.
• The legacy CPU simulation fallback has been removed. GpuSimulationBackend is now the only simulation backend, and the old CPU-oriented runtime fallback path no longer exists inside simulation startup or stepping.
• Simulation output is now a user-managed stack of simulation layers and sim groups. Top-level leaf layers still inject from the final source composite. A sim group chains its children in order: the first child samples the group's incoming surface, each later child samples the previous child output, and the group publishes its last child as the downstream surface. Catch-up steps after the first injected pass stay step-only, so source injection still happens once per render frame.
• Injection now modifies the current simulation generation before the normal Conway step instead of inserting an extra duplicate history frame, which keeps gliders and ships visually crisp rather than fattening their trails.
• Default layer set is two entries: Positive (Direct + Additive) and Negative (Inverse + Subtractive), with Negative above Positive by default.
• In Subtractive mode, simulation layers subtract from the current composited simulation pixel. With passthrough off, subtractive-only stacks start from a white baseline (subtractive identity). For additive+subtractive-only mixed stacks, composition now uses a neutral gray baseline (50% normalization bias) so both layers remain visible instead of washing each other out; other mixed blend combinations keep black baseline. With passthrough on, subtraction uses the passthrough-backed pixel baseline.
• Mixed Additive/Subtractive simulation stacks now accumulate in an order-neutral way (values clamp at the end of layer composition, not after each add/sub step), so swapping direct/inverse layer order does not bias the result.
• With passthrough enabled and compatible buffer dimensions, LifeViz now composites the source underlay into the render buffer first, then applies simulation layers in stack order on top. This preserves subtractive-layer detail inside bright passthrough regions (instead of washing out under additive underlay blending).
• Passthrough compositing now applies simulation as a delta over the underlay baseline, so enabling passthrough does not globally weaken life contrast.
• For additive/subtractive-only simulation stacks, passthrough now composes additive and subtractive families separately over the underlay, which prevents multi-layer collapse when both families are enabled.
• Capture buffers are pooled and only the downscaled composite is retained, eliminating GC spikes from per-frame allocations.
• Webcam sources stream via WinRT MediaCapture; clearing sources or closing the app releases the camera. Cameras retry initialization once and wait longer for first frames before being removed.
• Framerate lock: choose 15 / 30 / 60 / 144 fps from the context menu to match capture needs or ease CPU/GPU load. That menu now controls render/presentation cadence directly; Level -> Framerate modulates simulation speed against that target instead of throttling the whole app.
• Startup is now guarded by a recovery flag. If the previous launch dies before the main window finishes initializing, the next launch falls back to a safe windowed startup (<=480p, <=60 fps, fullscreen off, Show FPS off, Level -> Framerate off) instead of immediately reloading the bad persisted fullscreen/fps combination.
• Show FPS now includes reactive diagnostics, separate Render fps and Sim sps readouts, live Steps/frame, frame-pacing stats (avg, p95, and frame-gap spike counts), and a live Work: breakdown (frame, composite, inject, sim, render ms averages over recent frames). Steps/frame is the actual catch-up count executed on the current render tick, so it immediately distinguishes "presentation is slow" from "simulation is jumping multiple generations per draw." It also draws two bottom-of-screen debug panels on a fixed 30-second window. The upper timing panel shows orange = rolling frame-gap history, red dashed = current frame budget, and amber vertical bars for individual frame-budget underruns/overruns so missed-present bursts are easy to spot. The lower audio panel shows white = rolling waveform bucket range (min/max per history slice), yellow = rolling rectified max-envelope history, cyan = rolling bass-energy history, magenta = rolling dominant-frequency history on a logarithmic musical-note scale, blue = bass-band dominant frequency, green = mid-band dominant frequency, and red = high-band dominant frequency. New data enters on the right and scrolls left. Yellow and cyan share the waveform zero baseline so you can compare fit directly, while the timing traces use a full-height 0..50 ms scale so pacing spikes are visible even when average simulation throughput still looks acceptable. Audio graph histories now come straight from AudioBeatDetector's rolling history buffers instead of being re-sampled into UI-side queues, and the overlay is decimated to a capped sample count with split timing/audio refresh cadences so diagnostics stay readable without dominating visible rendering.
• Capture threshold window: adjustable min/max sliders (with optional invert) in the context menu; this updates all simulation layers at once. Each simulation layer also has its own injection mode plus threshold min/max + invert controls in Scene Editor. In Threshold mode the window is a boolean gate (inside-window alive, or outside-window alive when inverted). In Random Pulse and Pulse Width modes the same per-layer window shapes injection intensity/probability (including invert), so min/max still strongly affect behavior.
• Injection noise: adjustable slider (0-1) that randomly skips cell injection per pixel to introduce controlled noise.
• RGB Channel Bins hue is now per simulation layer instead of global. Each layer exposes Hue Offset, animated Hue Speed, and reactive mappings in the Scene Editor. A Frequency -> Hue Shift mapping uses the same log-note-scaled dominant frequency shown by the magenta debug trace, so audio pitch can drive palette changes per layer. The same per-layer hue is used for RGB channel injection mapping and final RGB presentation.
• Built-in recording: Start Recording writes to %UserProfile%\\Videos\\LifeViz using a pixel-perfect integer upscale to the nearest HD height (720/1080/1440/2160 when divisible). Use Recording Quality to pick Lossless (FFV1 in MKV, compressed), Crisp (H.264 in MP4, Windows Media Player compatible), Uncompressed (AVI, huge files), or H.264 tiers (High/Balanced/Compact); encoding favors quality-based VBR with bitrate caps so pixel lines stay crisp, and a taskbar overlay appears while active. Lossless and crisp recording use ffmpeg on PATH.

Configuration

• Settings persist to %AppData%\lifeviz\config.json after the app finishes loading (height/rows, depth, framerate, blend/composite toggles, full simulation-layer stack settings including per-layer thresholds, opacity, per-layer RGB hue offset/speed, per-layer reactive mapping lists, passthrough, audio reactivity controls, etc.) and restore on next launch. The window keeps the current aspect ratio and can be resized from the corner grip; use height presets or fullscreen to change simulation resolution without letterboxing.
• The source stack is restored too: window sources are matched by title, webcams by device id/name, file sources by path (video sequences restore their ordered path list), keeping order (including nested layer groups) plus per-layer blend mode, fit mode, opacity, video-audio toggle, video-audio volume, mirror, and keying settings (enable/color/range) when the inputs are available. The global source-audio master toggle/volume are also restored.
• YouTube sources resolve asynchronously on launch so the UI can come up even if the stream lookup is slow; the layer starts once the stream URL is resolved (check logs for failures).
• Aspect ratio lock state persists (default lock ratio is 16:9).
• Fullscreen preference is remembered and re-applied on launch using the active monitor bounds so the taskbar stays hidden.

Packaging & Deployment

ClickOnce remains the primary distribution path. Packaging requires the full .NET Framework MSBuild that ships with Visual Studio/Build Tools:

# Developer PowerShell where msbuild.exe is available
msbuild lifeviz.csproj `
  /t:Publish `
  /p:PublishProfile=Properties\PublishProfiles\WinClickOnce.pubxml `
  /p:Configuration=Release

The repo now bundles three helper scripts:

• Publish-Installer.ps1 - resolves MSBuild.exe, runs the publish target, and writes manifests + installer assets into bin/Release/net9.0-windows/publish/.
• deploy.ps1 - builds, publishes with an auto-generated version (so ClickOnce always sees an update), then launches the lifeviz.application manifest to trigger an in-place update of the installed app.
• Publish-GitHubRelease.ps1 - builds, publishes a ClickOnce payload, bundles it into a single lifeviz_installer.exe (self-extracting + auto-install), and creates a GitHub release that uploads that one exe (requires gh CLI authenticated to your repo). It asks a quick vibe check (tiny tweak / glow-up / new era) and auto-bumps the version/tag for you-no need to invent numbers.
• Install-ClickOnce.ps1 - bundled alongside published payloads; stages the ClickOnce files to %LOCALAPPDATA%\lifeviz-clickonce, clears the old ClickOnce cache, and launches the manifest from that stable path so future installs/updates don't break when the zip is extracted to a new folder.
• Rider users: run the lifeviz: Publish Installer (MSBuild.exe) configuration (stored in .run/), which shells out to Publish-Installer.ps1 so full MSBuild is used. The auto-generated Rider publish config uses dotnet msbuild and will fail with MSB4803.
• If you do publish via dotnet msbuild (e.g., Rider's generated publish config), the project automatically disables the ClickOnce bootstrapper so the build succeeds; use the MSBuild.exe-backed scripts to produce the optional setup.exe bootstrapper.

Artifacts:

• Application Files/lifeviz_<version>/... - versioned payload.
• lifeviz.application - ClickOnce manifest; launching it after deploy.ps1 applies the latest build.
• setup.exe - optional bootstrapper for clean machines (installs prerequisites + shortcuts). Only needed for first-time installs.

To push a Windows release to GitHub:

gh auth login # one-time
.\Publish-GitHubRelease.ps1 -NotesPath release-notes.md

Use -Draft to stage without publishing. The script reuses Publish-Installer.ps1 to generate assets and emits a single lifeviz_installer.exe into artifacts/github-release/; the GitHub release uploads just that executable.

NOTE: .NET CLI alone cannot produce ClickOnce installers (MSB4803). Always use the full MSBuild toolchain, either directly (msbuild) or through the scripts above.

Wiki

All technical details (rendering pipeline, controls, install flow) live in /wiki. Update the relevant page whenever you change the app; see agents.md for documentation expectations.