feat: redesign phase 4 — Segment Anything first slices, selection bar, rest-of-image slice

docs/svelte-migration/ARCHITECTURE.md

@@ -106,6 +106,8 @@ type SliceView = {
   image: AssetRef;          // raw RGBA slice
   thumbnail: AssetRef;      // checkerboard composite for display
   mask: AssetRef | null;    // RGBA (r, 0, 0, r); null when no mask is saved
+  isGround: boolean;        // the horizontal ground plane, not a vertical card
+  isRest: boolean;          // the "rest of image" slice (at most one); see `slices/create` below
 };
 type InpaintingView = {
   model: string; strength: number; guidanceScale: number;
@@ -185,10 +187,10 @@ same project-lock `_mutation_guard` rather than as a background job.
 
 | Method & path | Body | Result |
 | --- | --- | --- |
-| `POST /api/v1/projects/{id}/slices/create` | `{}` | `200 ProjectView & {changed: true}` (sync); `create_slice` - a new slice from the current mask (or an empty transparent one with none), appended, selected, and saved (JSON + the new slice's own image). |
-| `DELETE /api/v1/projects/{id}/slices/{index}` | – | `200 ProjectView & {changed: true}` (sync); `delete_slice` - out-of-range `index` is `400 invalid_request`; the selection is always cleared. |
-| `POST /api/v1/projects/{id}/slices/{index}/add-mask` | `{}` | `200 ProjectView & {changed: true}` (sync); `add_mask_to_slice`. `index` must equal the project's current `selectedSlice` (`400 invalid_request` otherwise, since the service itself has no index parameter - it always operates on the selection, exactly like Dash). |
-| `POST /api/v1/projects/{id}/slices/{index}/remove-mask` | `{}` | Same shape and `index`-must-match-selection rule as add-mask, calling `remove_mask_from_slice`. |
+| `POST /api/v1/projects/{id}/slices/create` | `{}` | `200 ProjectView & {changed: true}` (sync); `create_slice` - a new slice from the current mask (or an empty transparent one with none), appended, selected, and saved (JSON + the new slice's own image). When Segment Anything cuts the *first* object slice out of a project with no slices yet, a farthest "rest of image" slice is also created (`SliceView.isRest`): the input image with alpha `255 - object.alpha`, depth 0, logged as `"Created image_slice_N with the rest of the image"`. The selection stays on the object slice even though the rest slice's depth-0 insertion can shift its index. If a rest slice already exists, its alpha is instead reduced by the new object's footprint (`rest.alpha = min(rest.alpha, 255 - object.alpha)`), saved as a new (undoable) version. A project split by depth first (slices exist, none is the rest) behaves exactly as before - no rest slice is added. An empty create (no mask) never creates or changes the rest slice. Persisted as `rest_slice` in the project JSON, the same way `ground_plane_slice` is. |
+| `DELETE /api/v1/projects/{id}/slices/{index}` | – | `200 ProjectView & {changed: true}` (sync); `delete_slice` - out-of-range `index` is `400 invalid_request`; the selection is always cleared. When a rest slice exists and an *object* slice is deleted, pixels the deleted slice covered that no remaining (non-rest) slice covers either get their rest alpha back to 255, with RGB refilled from the input image only where the rest slice was fully transparent; every other rest pixel - including any inpainting done directly on it - is left alone, and the rest slice is saved as a new version. Deleting the rest slice itself removes it like any other slice; none is recreated automatically. |
+| `POST /api/v1/projects/{id}/slices/{index}/add-mask` | `{}` | `200 ProjectView & {changed: true}` (sync); `add_mask_to_slice`. `index` must equal the project's current `selectedSlice` (`400 invalid_request` otherwise, since the service itself has no index parameter - it always operates on the selection, exactly like Dash). If a rest slice exists and the edited slice is an object slice, the rest slice's alpha is reduced by the added area the same way `slices/create` does, and saved as a new version. |
+| `POST /api/v1/projects/{id}/slices/{index}/remove-mask` | `{}` | Same shape and `index`-must-match-selection rule as add-mask, calling `remove_mask_from_slice`. If a rest slice exists and the edited slice is an object slice, wherever the mask lowered the object's alpha and no other (non-rest) slice still covers the pixel, the rest alpha rises to `255 - object.alpha` (so a feathered removal restores only what it removed), the same way slice deletion does. |
 | `POST /api/v1/projects/{id}/clipboard/copy` | `{}` | `200 ProjectView & {changed: true}` (sync); `copy_to_clipboard` - requires a mask (`409 not_ready` otherwise); no save, no display change (matches Dash exactly: `copy_to_clipboard` has no `STORE_UPDATE_SLICE` output). |
 | `POST /api/v1/projects/{id}/clipboard/paste` | `{}` | `200 ProjectView & {changed: true}` (sync); `paste_clipboard` - requires both a populated clipboard and a selected slice (`409 not_ready` otherwise); the clipboard survives the paste. |
 | `POST /api/v1/projects/{id}/slices/balance` | `{}` | `200 ProjectView & {changed}` (sync); `balance_slices` - `changed: false` (not an error) with zero slices, matching `WorkflowUnchanged`'s pattern. Implements the *fixed*, intended behavior; the equivalent Dash button is currently unreachable (500s) due to an unrelated, pre-existing `webui.py` bug - see PARITY.md "Known quirks". |

e2e/drivers/svelte.ts

@@ -108,6 +108,14 @@ export class SvelteDriver implements UiDriver {
     }
   }
 
+  /** Split by depth is a collapsible section of the Slices panel. */
+  private async openSplitByDepth(): Promise<void> {
+    const toggle = this.page.getByTestId('split-toggle');
+    if (!(await toggle.isVisible())) await this.openTab('Segmentation');
+    if ((await toggle.getAttribute('aria-expanded')) !== 'true') await toggle.click();
+    await expect(toggle).toHaveAttribute('aria-expanded', 'true');
+  }
+
   /** Per-slice actions live in the Inspector's header for the selected slice. */
   private async ensureSelected(index: number): Promise<void> {
     const row = this.sliceRow(index);
@@ -189,6 +197,7 @@ export class SvelteDriver implements UiDriver {
   }
 
   async generateSlices(): Promise<void> {
+    await this.openSplitByDepth();
     await this.page.getByTestId('generate-slices').click();
   }
 
@@ -440,6 +449,7 @@ export class SvelteDriver implements UiDriver {
   }
 
   async balanceSlices(): Promise<void> {
+    await this.openSplitByDepth();
     await this.page.getByTestId('balance-slices').click();
   }
 

frontend/src/lib/api/generated.ts

@@ -95,6 +95,7 @@ export type Canundo = boolean;
 export type Depth = number;
 export type Index1 = number;
 export type Isground = boolean;
+export type Isrest = boolean;
 export type Negativeprompt2 = string;
 export type Positiveprompt2 = string;
 export type Version1 = number;
@@ -506,6 +507,7 @@ export interface SliceView {
   image: AssetRef;
   index: Index1;
   isGround?: Isground;
+  isRest?: Isrest;
   mask?: AssetRef | null;
   negativePrompt: Negativeprompt2;
   positivePrompt: Positiveprompt2;

frontend/src/lib/api/schema.json

@@ -1350,6 +1350,11 @@
           "title": "Isground",
           "type": "boolean"
         },
+        "isRest": {
+          "default": false,
+          "title": "Isrest",
+          "type": "boolean"
+        },
         "mask": {
           "anyOf": [
             {

frontend/src/lib/components/canvas/CanvasArea.svelte

@@ -13,6 +13,7 @@
   import ViewModeBar from './ViewModeBar.svelte';
   import ToolOptionsBar from './ToolOptionsBar.svelte';
   import CanvasToolbar from './CanvasToolbar.svelte';
+  import SelectionBar from './SelectionBar.svelte';
 
   let stage: ReturnType<typeof InputImagePanel> | undefined = $state();
 
@@ -40,14 +41,21 @@
     <ToolOptionsBar />
   {/if}
   <div class="stage-wrap">
-    <div class="stage" class:hidden={uiStore.view === '3d'}>
+    <div
+      class="stage"
+      class:hidden={uiStore.view === '3d'}
+      class:with-selection-bar={uiStore.tool === 'segment'}
+    >
       <InputImagePanel bind:this={stage} />
     </div>
     <div class="viewer-3d" class:hidden={uiStore.view !== '3d'} data-testid="viewer-3d">
       <Model3DViewer />
     </div>
     {#if toolsShown}
       <CanvasToolbar />
+      {#if uiStore.tool === 'segment' && uiStore.view === 'input'}
+        <SelectionBar />
+      {/if}
     {/if}
   </div>
 </section>
@@ -82,4 +90,10 @@
     padding-left: 56px;
     padding-right: 56px;
   }
+
+  /* With the Segment tool, keep the selection bar's strip clear so the bar
+     never covers the image (and the image doesn't jump when it appears). */
+  .stage.with-selection-bar {
+    padding-bottom: 60px;
+  }
 </style>

frontend/src/lib/components/canvas/SelectionBar.svelte

@@ -0,0 +1,97 @@
+<script lang="ts">
+  /**
+   * Floats under the image while a Segment Anything selection exists
+   * (docs/redesign/HANDOFF.md §5): segment queued points, add the selection
+   * to the selected slice or remove it from there, or make it a new slice.
+   */
+  import { projectStore } from '../../state/project.svelte';
+  import { isBusy } from '../../state/busy.svelte';
+  import * as workflow from '../../workflow';
+
+  const view = $derived(projectStore.view);
+  const segmentation = $derived(view?.segmentation);
+  const hasMask = $derived(segmentation?.hasMask ?? false);
+  const queued = $derived(segmentation?.multiPointMode ? (segmentation?.queuedPoints.length ?? 0) : 0);
+  const selected = $derived(view?.slices.find((s) => s.index === view.selectedSlice) ?? null);
+  const selectedName = $derived(selected ? `image_slice_${selected.index}` : null);
+</script>
+
+{#if hasMask || queued > 0}
+  <div class="selection-bar" role="toolbar" aria-label="Selection actions" data-testid="selection-bar">
+    <span class="summary">{hasMask ? 'Selection' : `${queued} ${queued === 1 ? 'point' : 'points'}`}</span>
+    {#if queued > 0}
+      <span class="divider" aria-hidden="true"></span>
+      <button
+        type="button"
+        class="btn btn-sm btn-selected"
+        data-testid="selection-commit"
+        disabled={isBusy()}
+        onclick={() => void workflow.commitMultiPoint()}
+      >
+        Segment {queued} {queued === 1 ? 'point' : 'points'}
+      </button>
+    {/if}
+    {#if hasMask}
+      <span class="divider" aria-hidden="true"></span>
+      <button
+        type="button"
+        class="btn btn-sm"
+        data-testid="selection-add"
+        title="Add the selection to the selected slice"
+        disabled={!selected || isBusy()}
+        onclick={() => void workflow.addMaskToSlice()}
+      >
+        Add to {selectedName ?? 'slice'}
+      </button>
+      <button
+        type="button"
+        class="btn btn-sm"
+        data-testid="selection-remove"
+        title="Remove the selection from the selected slice"
+        disabled={!selected || isBusy()}
+        onclick={() => void workflow.removeMaskFromSlice()}
+      >
+        Remove from {selectedName ?? 'slice'}
+      </button>
+      <button
+        type="button"
+        class="btn btn-sm btn-primary"
+        data-testid="selection-new-slice"
+        disabled={isBusy()}
+        onclick={() => void workflow.createSlice()}
+      >
+        New slice
+      </button>
+    {/if}
+  </div>
+{/if}
+
+<style>
+  .selection-bar {
+    position: absolute;
+    bottom: var(--space-4);
+    left: 50%;
+    transform: translateX(-50%);
+    z-index: 6;
+    display: flex;
+    align-items: center;
+    gap: 6px;
+    padding: 6px 6px 6px var(--space-3);
+    background: var(--color-float);
+    border: 1px solid var(--color-border);
+    border-radius: var(--radius-lg);
+    box-shadow: var(--shadow-float);
+    white-space: nowrap;
+  }
+
+  .summary {
+    font-weight: 600;
+  }
+
+  .divider {
+    width: 1px;
+    height: 22px;
+    margin: 0 var(--space-1);
+    background: var(--color-border-strong);
+  }
+</style>