fix(graphics): support high-dpi direct file frames

[ogulcancelik]

Summary

• advertise a separate 400 MiB limit for primary-layer direct RGBA file frames
• keep inline, BGRA, named-layer, and fallback frames capped at 16 MiB
• upload oversized frames without placement while a pane is hidden or a redraw is in flight, then replay placement after terminal acknowledgement
• preserve small-frame inline fallback and close oversized streams without producer acknowledgement on terminal rejection
• update the socket API schema/docs and add high-DPI and tab-switch regression coverage

Testing

• just check
• just bench-render-scale (50 active panes: 1.40x median versus one pane)
• linked Ghostty test for file upload followed by placement
• independent regression and test review: no remaining findings

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 0e515034-e5f5-4590-b811-d4b141f37e1c

📥 Commits

Reviewing files that changed from the base of the PR and between 82807f4 and 06d0fd5.

📒 Files selected for processing (1)

• docs/next/api/herdr-api.schema.json

🚧 Files skipped from review as they are similar to previous changes (1)

• docs/next/api/herdr-api.schema.json

---

📝 Walkthrough

Walkthrough

Pane graphics now expose and enforce a separate 400 MiB direct-file limit for eligible primary-layer RGBA frames. Kitty control validation checks payload dimensions and profiles. Hidden placement, fallback, cache cleanup, and oversized-transfer behavior are covered.

Changes

Pane graphics direct-transfer flow

Layer / File(s)	Summary
API limits and response contract src/api/schema/..., docs/next/api/herdr-api.schema.json, src/app/api/pane_graphics.rs	The response exposes file_frame_direct_max_bytes. The direct-file limit is 400 MiB.
Frame limit selection and leasing src/app/api/pane_graphics.rs	Eligible primary-layer RGBA frames use the direct-file limit. Other frames use the standard stream limit. Tests cover advertised limits and fullscreen retina files.
Kitty control validation and file preparation src/client/direct_graphics.rs, src/client/mod.rs, src/kitty_graphics.rs, src/ghostty/mod.rs	Validation checks control keys, action profiles, dimensions, RGBA length, and placement rules. Direct-file preparation supports upload-only commands and deferred placement.
Fallback and cache state handling src/server/headless/pane_graphics.rs	Inline fallback rejects oversized leases. Failed direct transfers clear cached pane layers before fallback.
Transfer-path integration coverage src/server/headless/tests/pane_graphics.rs, docs/next/website/src/content/docs/socket-api.mdx	Tests and documentation cover hidden uploads, placement replay, redraw uploads, fallback limits, and oversized terminal errors.

Estimated code review effort: 4 (Complex) | ~45 minutes

Mergeability Score: ⚪ Minimal · up to 06d0f

The change expands support for high-DPI direct file frames while preserving existing frame limits and fallback behavior; no actionable merge-blocking risk remains beyond normal checks and review.

Sequence Diagram(s)

sequenceDiagram
  participant PaneGraphicsAPI
  participant KittyGraphics
  participant DirectGraphicsValidator
  participant HeadlessPaneGraphics
  participant ProducerResponse
  PaneGraphicsAPI->>KittyGraphics: Prepare eligible direct file
  KittyGraphics->>DirectGraphicsValidator: Validate control and RGBA length
  DirectGraphicsValidator-->>KittyGraphics: Accept transmit or display profile
  KittyGraphics->>HeadlessPaneGraphics: Send direct file
  HeadlessPaneGraphics-->>ProducerResponse: Acknowledge successful transfer
  HeadlessPaneGraphics->>HeadlessPaneGraphics: Clear cache after failed transfer
  HeadlessPaneGraphics-->>ProducerResponse: Disconnect oversized terminal transfer

Loading

Possibly related PRs

• herdrdev/herdr#2523: Introduced the direct pane-frame streaming path extended by this change.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly identifies the graphics fix and directly reflects the PR’s primary change for high-DPI direct file frames.
Description check	✅ Passed	The description accurately summarizes the direct-frame limits, fallback behavior, API updates, and regression coverage.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch shared-pane-terminal-browser

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Greptile Summary

The PR extends direct Kitty file transport to support high-DPI primary-layer RGBA frames while retaining the existing inline fallback limit.

• Advertises separate 16 MiB fallback and 400 MiB direct-file limits through the API.
• Adds upload-only transfers when placement must wait for pane visibility or redraw completion.
• Tightens client-side validation of direct graphics controls and dimensions.
• Adds regression coverage for delayed placement, rejection, redraw, and tab-switch behavior.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains in the eligible follow-up review scope.

No blocking failure remains.

Important Files Changed

Filename	Overview
src/app/api/pane_graphics.rs	Adds the direct-file capability limit and permits larger primary-layer RGBA file frames only when direct transport is available.
src/client/direct_graphics.rs	Validates direct graphics commands against the expected image dimensions and the supported upload or upload-and-placement profiles.
src/client/mod.rs	Applies direct-control validation only after validating the transferred file.
src/kitty_graphics.rs	Separates upload-only command generation from upload-and-placement generation so placement can be deferred.
src/server/headless/pane_graphics.rs	Coordinates direct transfers, inline fallback, delayed placement, acknowledgement, and rejection cleanup.
src/server/headless/tests/pane_graphics.rs	Adds coverage for high-DPI transfers, hidden panes, redraw timing, delayed placement, and oversized rejection.
src/api/schema/response.rs	Adds the optional direct-file byte limit to the public capability response.
docs/next/api/herdr-api.schema.json	Updates the generated API schema for the new direct-file capability field.

Sequence Diagram

sequenceDiagram
    participant Producer
    participant Server
    participant Client
    participant Terminal
    Producer->>Server: Direct RGBA file frame
    alt Placement currently allowed
        Server->>Client: "GraphicsFile a=T (upload and place)"
    else Pane hidden or redraw active
        Server->>Client: "GraphicsFile a=t (upload only)"
    end
    Client->>Terminal: Validated direct-file command
    Terminal-->>Client: Transfer acknowledgement
    Client-->>Server: GraphicsTransmissionResult
    Server-->>Producer: Frame acknowledgement
    opt Placement was deferred
        Server->>Client: "Replay a=p after visibility/redraw"
        Client->>Terminal: Place resident image
    end

Loading

_{Reviews (5): Last reviewed commit: "Merge branch 'master' into shared-pane-t..." | Re-trigger Greptile}