Windows/Linux: decide and measure the libmpv software-texture rendering path

[Serph91P]

Problem

The exact current upstream/dev routing differs from the earlier local-checkout assumption and is now verified directly from the Git object:

• macOS uses MediaKitDesktopAdapter.
• Windows and Linux use DesktopLibmpvBackend.
• app_router.dart deliberately moved Windows/Linux away from media_kit because Flutter 3.38 can leave media_kit without the platform-thread EGL context required by its hardware render path, causing software texture upload. The source references media-kit issue #1404.
• The selected native Windows/Linux backend sets hwdec=auto-safe, but its libmpv render context uses MPV_RENDER_API_TYPE_SW and writes RGBA frames into a Flutter PixelBufferTexture.
• The output buffer is fixed at 1280x720. Each buffer is 3,686,400 bytes before Flutter/GPU/decoder allocations.
• Linux coalesces frame notifications with frame_pending. The equivalent Windows path appears to post every libmpv update callback to the Flutter platform dispatcher.
• The production orchestrator currently uses _UnavailableTranscodeGateway and advertises serverTranscodeAvailable: false, so a desktop decode/render failure does not currently reach a real server-transcode fallback. Related fallback work is tracked by Production playback never wires the server-transcode fallback #110.

This makes CPU-side frame transfer, Flutter texture upload, fixed output resolution, and Windows callback queue pressure plausible performance risks. They are evidence-backed source hypotheses, not yet measured device benchmarks.

Maintainer decision required

No source work starts until the maintainer chooses a direction.

Option A: Measurement-first, then bounded renderer remediation

1. Add exact runtime telemetry and a deterministic performance harness.
2. Establish Windows and Linux baselines for the current software render path.
3. Apply one bounded renderer remediation only when the baseline proves the bottleneck.
4. Prefer a native child HWND on Windows and a separately tested Wayland/X11 native-surface strategy on Linux when direct GPU output is approved.

Option B: Immediate native GPU/native-surface migration

Replace the software render context and Flutter pixel-buffer transport directly, while preserving the shared PlayerAdapter contract. This has higher implementation, packaging, lifecycle, compositor, and driver risk and needs separate Windows, Linux Wayland, and Linux X11 acceptance.

Option C: Retain software rendering

Keep the current compatibility path and implement only diagnostics, bounded callback handling, packaged-runtime checks, and explicit performance warnings. This avoids a renderer rewrite but retains the CPU copy and fixed-resolution trade-offs.

Proposed scope after approval

• Query and normalize libmpv runtime properties required to classify decoder, render path, frame drops, A/V sync, and cache state.
• Report hwdec-current, active decoder/codec, input dimensions and frame rate, rendered/dropped frames, cache duration/state, buffering state, renderer type, output dimensions, and terminal renderer errors.
• Record whether video frames remain in a GPU-native path or are downloaded/copied through system memory.
• Add bounded diagnostics sampling that cannot create high-frequency MethodChannel traffic, UI rebuilds, or log spam.
• Measure and, if approved, coalesce Windows frame-dispatch work so outstanding callbacks remain bounded.
• Keep live TV, VOD, series, seeking, speed, track selection, subtitles, EOF, and disposal behavior unchanged.
• Treat server-transcode unavailability truthfully. Do not claim fallback coverage while _UnavailableTranscodeGateway is active.
• Add release-runtime checks that prove required libmpv libraries are packaged for Windows and Linux.
• If native-surface rendering is selected, preserve a documented compatibility fallback and test Windows, Linux Wayland, and Linux X11 separately.

Required performance harness

• Deterministic local HTTP/HLS fixtures with versioned 720p, 1080p, and 4K profiles, H.264/H.265, AAC/AC3, segmented live streams, controlled segment delay, and an unsupported-codec case.
• No real provider URLs, headers, tokens, or credentials.
• Packaged Windows and Linux applications, not only flutter run.
• Cold start, 20 to 50 channel switches, buffer stall/recovery, background/foreground, multiview where supported, and a soak test.
• Timestamps for open, native load, file loaded, first decoded frame, first presented frame, playing, buffering start/end, and dispose.
• Metrics for frame/update callback rate, outstanding Windows dispatches, Flutter frame timing, decoded/dropped frames, CPU/GPU, RSS, cache state, network rate, and native handles.
• Repeated baseline and candidate runs with exact app commit, package hash, OS, GPU, driver, display refresh rate, codec, resolution, and frame rate.

Non-goals

• No Android or Apple changes.
• No changes to PR feat: Migrate macOS build to MPV to align desktop builds #209.
• No HDR correctness claim. HDR remains in Add verified HDR playback support on all supported platforms #168.
• No assumption that gpu-next is faster. Quality filters, tone mapping, and shaders remain capability- and benchmark-gated.
• No global low-latency or fixed FPS override.
• No deletion of pre-existing backend code outside the selected renderer scope.
• No private provider stream, credential, header, token, channel name, or personal media metadata in logs, fixtures, telemetry, or issue comments.

Acceptance criteria

• The maintainer has selected Option A, B, C, or another explicit direction in Standardize playback on mpv where feasible across supported platforms #167.
• Windows and Linux diagnostics show active backend, render API, hardware/software decoder decision, codec, dimensions/frame rate, rendered/dropped frames, A/V sync, cache health, output dimensions, and categorized errors.
• Hardware decode with software frame transfer is not mislabeled as a fully GPU-native path.
• Diagnostics sampling and frame dispatch are bounded, teardown-safe, and tested against late events, reopen/dispose races, and callback bursts.
• The UI remains backend-agnostic. Widgets do not import native libmpv classes.
• Current seeking, speed, tracks, subtitles, EOF, multiview where supported, and disposal contracts remain green.
• Packaged Linux and Windows builds contain required libmpv runtime files and fail closed when required assets are absent.
• New behavior is developed test-first with explicit RED and GREEN evidence.
• dart format --output=show --set-exit-if-changed ., flutter analyze, plain flutter test, serial flutter test --concurrency=1, Linux native tests/build, and Windows native tests/build pass where the exact runner is available.
• Android and Apple source paths are unchanged.
• Test packages are labeled with the exact source commit and SHA-256 checksums.
• Real-runtime results compare the established baseline and candidate for startup, channel changes, drops, cache/rebuffer behavior, CPU/GPU, memory, visible judder, and lifecycle stability.
• Any native-surface migration is accepted separately on Windows, Linux Wayland, and Linux X11. Unsupported targets remain explicit blockers or use the documented compatibility path.

Threat model and mandatory gates

The adapter processes untrusted network media, playlists, containers, codecs, subtitles, redirects, and headers through native libraries. Diagnostic property collection must not leak URLs, headers, tokens, channel names, or personal media metadata. Native callbacks and timers must not outlive the player or mutate disposed Flutter state. Native runtime packaging must use verified dependencies and immutable artifact identities.

Required gates:

1. Independent exact-head functional/code review.
2. Exact-head security review covering untrusted media/native library boundaries, credential redaction, callback lifetime, process/memory bounds, dependency trust, and packaged runtime integrity.
3. One remediation round maximum for reproducible in-scope P0/P1/P2 findings.

Relationships

• Parent initiative: Standardize playback on mpv where feasible across supported platforms #167
• Server-transcode fallback: Production playback never wires the server-transcode fallback #110
• HDR validation: Add verified HDR playback support on all supported platforms #168
• Apple work and PR feat: Migrate macOS build to MPV to align desktop builds #209 are explicitly outside this issue.