Skip to content

feat: ground plane (ground plane step 3) - #22

Merged
provos merged 2 commits into
mainfrom
feat/ground-plane
Sep 25, 2026
Merged

provos merged 2 commits into
mainfrom
feat/ground-plane

Conversation

@provos

@provos provos commented Sep 25, 2026 •

Copy link
Copy Markdown
Owner

Step 3 of the ground-plane plan. Any slice can now be the scene's horizontal ground plane. The default camera still reproduces the original image.

Geometry

  • Ground: the horizontal plane at camera height that the bottom image row meets at camera.groundNear. It extends out to maxDistance.
  • Backdrop: the band of ground rows between that far edge and the horizon goes on an upright backdrop at maxDistance. This keeps the scene bounded for 3D tools, and every ground pixel keeps a place. My first version ran the ground out to 100× the max distance, which was unusable in a 3D viewer.
  • scene.py: one planar layer per slice (cards, ground, backdrop). Any view is rendered by mapping each layer's source quad onto its projection, which is an exact homography for a plane.
  • Draw order: the ground goes first, then the cards far to near. I tried per-pixel depth sorting first, but it lets the ground cover card pixels that reach below the card's ground contact. A foreground cliff is the typical case: those pixels covered the ground in the original photo, so the default view no longer matched. Standing cards never overlap the ground in front of them, so drawing the ground first is both correct and identity-preserving.
  • Callers: camera navigation and the animation now use the scene renderer.

glTF

  • Ground mesh: a grid of back-projected image points with exact UVs. Rows are spaced geometrically from the horizon, which keeps the texture within 0.15 px at 12 MP; evenly spaced rows drifted up to 10 px.
  • Backdrop: exported as a second primitive of the same mesh.
  • Node position: the ground node sits beyond every card, so renderers that sort transparent objects by node position (three.js, Blender) draw it first, matching the preview.

API

  • PUT /projects/{id}/slices/{index}/ground with {isGround}. Only one slice can be the ground; it returns 409 when the horizon isn't above the bottom edge.
  • SliceView.isGround.
  • camera.groundNear and horizonRow in settings.
  • The flag is persisted as ground_plane_slice.
  • Settings that would hide the ground while one is marked are rejected before anything changes.

Validation

test_ground_plane.py (18 tests) covers:

  • the ground is horizontal and bounded, with the backdrop upright on its far edge and a seamless row split between them;
  • the reference view reproduces the image at pitch 0, +9 and −5;
  • the ground stays behind the cards regardless of list order;
  • the horizon stays fixed under camera moves;
  • every exported vertex reprojects onto its UV, and the texture stays registered between vertices;
  • persistence and the API rules.

Real-image check on LightHouse.jpg:

  • Segmentation: SAM selections for the lighthouse, the cliff and the ocean (as the ground), with the sky as everything else.
  • Pitch: 9.84° from the ocean's horizon.
  • Default view: matches a plain 2D stack of the same slices to within 0.6 grey levels. The remaining differences from the original photo are mask seams.
  • glTF export: reprojects to within 1.4e-7.

Totals: pytest 438, vitest 147, Playwright 45/45.

The UI (horizon line, Ground toggle, ground-distance slider) comes in step 4.

🤖 Generated with Claude Code

- A slice can be the scene's horizontal ground plane (at most one): the
  plane at camera height that the bottom image row meets at
  camera.ground_near, out to the max distance. Ground rows beyond that,
  up to the horizon, go on an upright backdrop at the max distance, so
  the scene stays bounded and every ground pixel keeps a place.
- scene.py builds one planar layer per slice (cards, ground, backdrop)
  and renders any view by mapping each layer's source quad onto its
  projection (an exact homography per plane). The ground is drawn first,
  then cards far to near: standing cards never overlap the ground in
  front of them, and a card pixel below its ground contact covered the
  ground in the original, so the reference camera reproduces the image.
  Camera navigation and the animation use it.
- glTF: the ground exports as a grid of back-projected image points with
  exact UVs, rows spaced geometrically from the horizon (texture within
  0.15 px at 12 MP), plus the backdrop as a second primitive; its node
  sits beyond every card so renderers that sort transparent objects by
  node position draw it first, like the preview.
- API: PUT .../slices/{index}/ground {isGround}; SliceView.isGround;
  camera.groundNear and horizonRow in settings; the flag persists as
  ground_plane_slice. Settings that would hide the ground while one is
  marked are rejected before anything changes.

Tests: ground geometry (horizontal, bounded, backdrop upright on its far
edge, seamless row split), reference-view identity with a ground at three
pitches, ground behind cards regardless of list order, horizon fixed under
camera moves, exported ground reprojection and texture registration
between vertices, persistence, and the API rules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 25, 2026 19:28

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The new ground-plane mutation can return a misleading “no ground in view; lower pitch” error message even when the actual failure is unrelated to pitch (e.g., invalid groundNear/maxDistance constraint).

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Medium severity

Open (1)
What changed in this PR

Adds “ground plane” support to Parallax Maker’s 2.5D scene model so any slice can be treated as a horizontal ground plane (plus far-edge backdrop), while preserving the reference camera’s ability to reproduce the original image. This extends the renderer/export pipeline and exposes the feature via the HTTP API and generated frontend schema/types.

Changes:

  • Introduces a plane-based scene renderer (scene.py) that composites ground/backdrop then cards far-to-near using exact planar homographies.
  • Adds ground-plane support end-to-end: camera model (ground_near + ground_height), persistence (ground_plane_slice), API (PUT .../slices/{index}/ground), navigation/animation rendering, and glTF export (bounded ground mesh + backdrop primitive).
  • Adds comprehensive tests for geometry/rendering/export/API/persistence and updates existing glTF/API tests for new primitives/fields.
File Description
parallax_maker/​test_ground_plane.py New comprehensive test suite for ground-plane geometry/rendering/export/API/persistence.
parallax_maker/​test_gltf_reprojection.py Updates reprojection helper to validate all mesh primitives (ground adds another primitive).
parallax_maker/​test_api_project_export.py Updates settings persistence test to account for groundNear and horizonRow in responses.
parallax_maker/​slice.py Replaces prior ground “card” hack with scene.ground_layers() geometry.
parallax_maker/​slice_editing_services.py Adds SetGroundPlane command + service method enforcing single ground slice.
parallax_maker/​segmentation.py Allows animation rendering to be supplied via an injected renderer callback.
parallax_maker/​scene.py New planar-layer scene model + homography renderer + helpers for ground/backdrop.
parallax_maker/​project_services.py Adds ground_near settings support and validates camera settings when a ground slice is marked.
parallax_maker/​gltf.py Adds bounded ground/backdrop export mesh generation; supports non-square grid indexing; refactors card creation.
parallax_maker/​export_services.py Switches animation rendering to the new scene renderer for ground-aware animation frames.
parallax_maker/​controller.py Persists/restores ground_plane_slice in project JSON.
parallax_maker/​camera.py Adds ground_near, plane backprojection helper, and ground_height() computation/validation.
parallax_maker/​camera_services.py Switches camera navigation preview rendering to the new scene renderer.
parallax_maker/​api/​slice_editing.py Adds PUT /projects/{id}/slices/{index}/ground endpoint and logging/revision behavior.
parallax_maker/​api/​schemas.py Adds SliceView.isGround, camera.groundNear, camera.horizonRow, and request schema for ground endpoint.
parallax_maker/​api/​projects.py Exposes isGround, groundNear, and horizonRow in ProjectView.
parallax_maker/​api/​project.py Wires groundNear through settings update endpoint into ProjectService.
frontend/​src/​lib/​api/​schema.json Updates generated JSON schema with ground-plane request/response fields.
frontend/​src/​lib/​api/​generated.ts Updates generated TS types for ground-plane fields and endpoint request type.
docs/​svelte-migration/​ARCHITECTURE.md Documents the new ground-plane API route and settings field (groundNear).

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +471 to +478
if command.is_ground:
width, height = state.imgData.size
try:
state.camera.ground_height(width, height)
except ValueError as error:
raise SliceEditingNotReady(
f"no ground in view: {error}; lower the camera pitch"
) from None
…stance)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@provos
provos merged commit e07c6d3 into main Sep 25, 2026
1 check passed
@provos
provos deleted the feat/ground-plane branch September 25, 2026 19:42
provos added a commit that referenced this pull request Sep 26, 2026
* refactor: remove dead code and stale docs left by the ground-plane feature

- slice.py: drop ImageSlice._dimension_at_depth, unused since create_card
  switched to cam.backproject_to_depth/scene.ground_layers; hoist the
  now-non-circular `from .scene import ground_layers` import to module
  level (also in gltf.py's ground_grids).
- camera_services.py: fix docstrings still naming the removed
  segmentation.render_view path; the module now renders via
  scene.render_state_view.
- export_services.py: drop the camera_rotation argument to
  render_image_sequence in render_animation - render_image_sequence only
  consults it when no render= callback is given, and this call always
  supplies one, so the value was silently ignored.
- controller.py: use next() instead of building a full list to find the
  (at most one) ground-plane slice index in AppState.to_json.

No behavior change; full suite (438 tests) still passes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(gltf): replace hand-rolled quaternion_from_matrix with scipy

quaternion_from_matrix (added for the pitched camera-node rotation) was a
22-line, four-branch reimplementation of a standard rotation-matrix-to-
quaternion conversion. scipy is already a core dependency, and
Rotation.from_matrix(...).as_quat() returns the same (x, y, z, w),
scalar-last convention this function documents. Verified against the old
implementation over 20000 random rotations plus the axis-aligned edge
cases each branch handled (max abs component error ~4e-16); the two
differ only by an occasional global sign flip, which represents the same
rotation and leaves every consumer (camera_node_rotation, the
reprojection tests) unaffected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(camera_pitch): reuse scene.project instead of a local reimplementation

test_camera_pitch.py hand-rolled a pinhole-projection helper (_project)
that duplicated scene.project's formula (the two are numerically
identical - verified exactly on a sample of points). test_ground_plane.py
already imports scene.project for the same purpose; do the same here so
there's one projection helper instead of two, and use it directly in the
one call site. Same assertions, same tolerances, same 14 tests passing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* refactor(gltf): factor the shared grid/UV build out of card_grid and ground_grids

Both functions built the same (meshgrid -> row-major points -> normalized
UVs) sequence before back-projecting it differently (to a depth plane vs.
to each ground/backdrop layer's plane). Pull that shared step into
_grid_points_and_uvs so each caller only states what's specific to it.
No behavior change - same points, same UVs, same test results.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants