Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,11 @@ No need to explicitly document the telemetry behaviors.
EVERY artifact→chrome message must go through `postArtifactMessage`, the only send path that stamps the current `artifact_load_token`: the chrome's message listener drops any frame message whose token is not the current load's before dispatch, so a raw `parent.postMessage` is silently discarded (a token-less `lavish:uploadAttachment` shipped exactly this break while every mocked harness stayed green - the chrome harness must therefore send test messages verbatim, never patch the token in).
Nothing (Tailwind, DaisyUI, Mermaid, layout safety CSS) is auto-injected into artifacts; agents choose a design direction via the single-sourced `DESIGN_PRIORITY_RULE` (see AXI integration above).
The `lavish-axi design` Mermaid snippet chooses dark or light rendering from the effective artifact page background and re-renders when a page-theme or OS appearance change alters that appearance, so do not hardcode one Mermaid theme.
- For rendered Mermaid SVGs outside `.mermaid` containers, the injected SDK retains dependency-free viewBox pan (drag) and zoom (wheel) in explore mode, then freezes it in annotation mode so a click resolves cleanly to one node instead of panning. It enhances on load and `DOMContentLoaded` and re-runs through a throttled `MutationObserver` because Mermaid renders asynchronously and can re-render. Enhancement touches only the live SVG's `viewBox` and listeners, never the saved artifact, so the diagram still renders identically when opened directly. Node detection, label extraction, and target validation live in `src/mermaid-node.js` so they are unit-testable and shared with the server; `createSdkJs` serializes each exported helper into the SDK as a same-scope `const` (like `deriveQueueKey`), derived from the module's exports, so a helper may reference only its own arguments, browser globals, or its sibling exports.
- For rendered Mermaid SVGs outside `.mermaid` containers, the injected SDK retains dependency-free viewBox pan (drag) and zoom (wheel) in explore mode, then freezes it in annotation mode so a click resolves cleanly to one node instead of panning. It enhances on load and `DOMContentLoaded` and re-runs through a throttled `MutationObserver` because Mermaid renders asynchronously and can re-render. Enhancement touches only the live SVG's `viewBox` and listeners, never the saved artifact, so the diagram still renders identically when opened directly. Node detection, label extraction, and target validation live in `src/mermaid-node.js` so they are unit-testable and shared with the server.
- Any helper `createArtifactSdk` calls must reach the browser through `serializeModuleHelpers` in `createSdkJs`, which turns every export of a shared module (`src/mermaid-node.js`, `src/table-cell.js`) into a same-scope `const`. A module-private function called from the SDK closure compiles fine and only `ReferenceError`s on the first click, so put new helpers in one of those wholesale-serialized modules and export them; a helper may then reference only its own arguments, browser globals, or its sibling exports - never a module-level constant, which is not serialized. Those modules must export functions and nothing else: only functions survive `toString()`, so `serializeModuleHelpers` throws on any other export rather than shipping a `Set` or `RegExp` that would arrive as an empty `{}`. `test/artifact-sdk-bundle.test.js` boots the served bundle and drives a real click, which is what catches an unreachable helper; the module-level unit tests cannot.
- Table-cell annotations attach `src/table-cell.js`'s semantic row/column names as `target` only. The clicked element's own `selector`, `tag`, and `text` keep describing that element, because the on-screen highlight outlines exactly what was clicked. Both coordinates stay silent rather than name a row or column they cannot prove, because a confidently wrong name reads as authoritative and is worse than none: a rowspan is clipped to its own row group, so only one starting in an earlier row of that group (including `rowspan="0"`, which runs to the end of it) makes a row's DOM order stop being its rendered order, and that suppresses the row's positional heading - only a declared `scope="row"` heading survives it - while the column label needs the header row unshifted the same way, plus a row whose colspans sum to the header's and a cell that does not straddle a grouped header.
A grouped header's `<th rowspan="2">` therefore costs the leaf header row its names and leaves every `<tbody>` row nameable.
Spans come from the browser-parsed `rowSpan`/`colSpan` whenever present, because HTML's integer rules stop at the first non-digit and render `rowspan="2x"` as a real two-row span that `Number` reads as `NaN`. Resolving a cell walks its whole table, so `context()` computes the target only under `{ table: true }`, which the annotation card passes and `snapshot()` - which calls `context()` for every element in the document - deliberately does not.
- Annotation handlers ignore native controls (`button`, `input`, `select`, `textarea`, `option`, `label`, `summary`, and editable regions) and their descendants, so they stay interactive without markup; `data-lavish-action` opts custom non-native controls out of annotation and gives them a pointer cursor. Artifact-author guidance for input patterns (`data-lavish-question`, `queueKey`, per-question submits) is owned by the input playbook in `src/playbooks.js`.
- For text annotations, `prompt.selector` is the common ancestor/container selector, not the complete identity. Use the `target` range boundaries and snapshot context to locate the exact selected text.
- For non-whiteboard Mermaid diagram nodes, a click annotates the whole rendered `<g>` node - not the sub-shape under the cursor - and hover highlights the same node. `SessionStore.normalizeTarget` routes these targets through `normalizeMermaidNodeTarget`, which strips them to the fixed `type`/`diagramId`/`nodeId`/`label`/`selector` shape, while text-range and other/legacy targets pass through unchanged.
Expand Down
5 changes: 4 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,10 @@ pnpm link
A plain `lavish-axi <html-file>` after a user-initiated end refuses to reopen the browser and returns guidance instead; pass `--reopen` only when the user asks for further review or something important needs their visual attention.
Agent-initiated ends keep reopening normally, same as before.
`lavish-axi poll`'s `ended` response and the `feedback` response for the final batch before an end both carry `next_step` guidance telling the agent to stop polling and deliver remaining updates in chat instead of reopening.
- **Precise targets** - Text annotations include selected text plus range anchors, so agents are not limited to whole-element selectors.
- **Precise targets** - Text annotations include selected text plus range anchors, and text selections carry those anchors only.
Clicking an element inside a table also carries the cell's visible row and column names alongside the exact CSS locator, so filtered or sorted rows do not make feedback look misdirected.
When merged cells make either name ambiguous, Lavish leaves that name out rather than guessing; an explicit `<th scope="row">` remains authoritative even when a `rowspan` makes the row's position ambiguous.
The CSS locator still points at the exact element you clicked, so an annotation with an omitted name is only less descriptive, never mislabelled.
- **Image attachments** - Attach reference images (PNG, JPEG, WebP) to an annotation by pasting, drag-dropping, or using the annotation card's **Attach image** picker; each shows a thumbnail chip with upload, remove, retry, and error states.
Images are stored under the state dir and the queued prompt carries a server-generated absolute `path` and content-hash `id` (plus mime and dimensions) - never the raw bytes - so `lavish-axi poll` hands the agent a local file path to open.
Limits are `LAVISH_AXI_MAX_ATTACHMENT_BYTES` (default 10 MiB per image), `LAVISH_AXI_MAX_ATTACHMENTS_PER_PROMPT` (default 4), and `LAVISH_AXI_MAX_PROMPT_ATTACHMENT_BYTES` (default 25 MiB per annotation); if any image is missing or any annotation breaches a count or byte cap, the entire send batch is rejected, the queue is preserved, and the reason is surfaced in the composer rather than silently dropping images.
Expand Down
38 changes: 30 additions & 8 deletions src/artifact-sdk.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
/* global CSS, Element, MutationObserver, ResizeObserver, document, getComputedStyle, parent, window */

import * as mermaidHelpers from "./mermaid-node.js";
import { tableCellTarget } from "./table-cell.js";

export const LAVISH_INTERNAL_QUEUE_KEY = "_lavishQueueKey";

Expand Down Expand Up @@ -746,14 +747,22 @@ export function createArtifactSdk(
return parts.join(" > ");
}

function context(el) {
// `table` is opt-in because resolving a cell's row and column walks the whole table, while
// `snapshot()` calls this for every element in the document and reads only uid/tag/text.
function context(el, { table = false } = {}) {
const base = {
uid: uid(el),
selector: selector(el),
tag: (el.tagName || "").toLowerCase(),
text: (el.innerText || el.textContent || "").trim().replace(/\s+/g, " ").slice(0, 240),
};

// Semantic table coordinates are extra context, never a replacement identity: the highlight
// outlines the element the reviewer clicked, so its selector, tag, and text must keep
// describing that exact element rather than being coarsened up to the enclosing cell.
const tableTarget = table ? tableCellTarget(el, selector) : null;
if (tableTarget) base.target = tableTarget;

const mermaidNode = mermaidNodeFrom(el, selector);
if (mermaidNode) {
base.tag = "mermaid-node";
Expand Down Expand Up @@ -2131,7 +2140,7 @@ export function createArtifactSdk(
const root = ensureShadow();
closeCard();

const c = options.context || context(target);
const c = options.context || context(target, { table: true });
activeCardContext = c;
let anchor = target;
if (options.range) {
Expand All @@ -2146,18 +2155,31 @@ export function createArtifactSdk(
const card = document.createElement("div");
card.className = "lavish-annotation-card";
const nodeLabel = c.tag === "mermaid-node" ? c.target?.label || c.text || "" : "";
const isTableCell = c.target?.type === "table-cell";
// The annotation targets the element that was clicked, which inside a table cell is often a
// nested badge or code span. Say "cell" only when the cell itself was clicked; otherwise name
// the clicked element and place it at the cell's coordinates. An unlabelled table names
// nothing, so it falls back to the plain element heading rather than a dangling "cell: ".
const isCellItself = isTableCell && (c.tag === "td" || c.tag === "th");
const tableLabel = isTableCell ? [c.target?.rowLabel, c.target?.columnLabel].filter(Boolean).join(" β†’ ") : "";
const heading =
c.tag === "text"
? "Annotate text"
: c.tag === "mermaid-node"
? "Annotate node" + (nodeLabel ? ": " + escapeAnnotationText(nodeLabel) : "")
: "Annotate &lt;" + c.tag + "&gt;";
: tableLabel
? isCellItself
? "Annotate cell: " + escapeAnnotationText(tableLabel)
: "Annotate &lt;" + c.tag + "&gt; in " + escapeAnnotationText(tableLabel)
: c.tag === "mermaid-node"
? "Annotate node" + (nodeLabel ? ": " + escapeAnnotationText(nodeLabel) : "")
: "Annotate &lt;" + c.tag + "&gt;";
const placeholder =
c.tag === "text"
? "Tell the agent what to change about this text..."
: c.tag === "mermaid-node"
? "Tell the agent what to change about this diagram node..."
: "Tell the agent what to change about this element...";
: isCellItself
? "Tell the agent what to change about this table cell..."
: c.tag === "mermaid-node"
? "Tell the agent what to change about this diagram node..."
: "Tell the agent what to change about this element...";
const sendNowHint = /Mac|iP(hone|ad|od)/.test(navigator.platform) ? "⌘" : "Ctrl";
card.innerHTML =
'<div class="lavish-heading">' +
Expand Down
26 changes: 21 additions & 5 deletions src/chrome-client.js
Original file line number Diff line number Diff line change
Expand Up @@ -277,26 +277,42 @@ function persistQueuedPrompts() {
}
}

function promptTargetLabel(prompt) {
if (prompt?.target?.type === "table-cell") {
const semantic = [prompt.target.rowLabel, prompt.target.columnLabel].filter(Boolean).join(" β†’ ");
if (semantic) return semantic;
}
return String(prompt?.selector || "");
}

function render() {
annotationPills.innerHTML = queued
.map(
(prompt, index) =>
.map((prompt, index) => {
const targetLabel = promptTargetLabel(prompt);
const showLocator = targetLabel && prompt.selector && targetLabel !== prompt.selector;
return (
'<div class="pill-wrap"><div class="pill"><span class="pill-preview">' +
escapeHtml(prompt.prompt || (attachmentCount(prompt) ? "Image annotation" : "")) +
"</span>" +
pillAttachmentsHtml(prompt) +
'<button class="pill-close" type="button" aria-label="Remove queued prompt" data-index="' +
index +
'"><svg width="10" height="10" viewBox="0 0 10 10" fill="none" aria-hidden="true" focusable="false"><path d="M1 1L9 9M9 1L1 9" stroke="currentColor" stroke-width="1.6" stroke-linecap="round"/></svg></button></div><div class="pill-tooltip">' +
(prompt.selector
(targetLabel
? '<div class="tooltip-label">Target</div><div class="pill-tooltip-target">' +
escapeHtml(targetLabel) +
"</div>"
: "") +
(showLocator
? '<div class="tooltip-label">Locator</div><div class="pill-tooltip-target">' +
escapeHtml(prompt.selector) +
"</div>"
: "") +
'<div class="tooltip-label">Prompt</div><div class="pill-tooltip-prompt">' +
escapeHtml(prompt.prompt) +
"</div></div></div>",
)
"</div></div></div>"
);
})
.join("");

for (const button of annotationPills.querySelectorAll(".pill-close")) {
Expand Down
8 changes: 4 additions & 4 deletions src/mermaid-node.js
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@

// Pure Mermaid node-identity helpers shared by the injected artifact SDK and the
// server-side session store. The SDK ships them to the browser by serializing
// each one with `.toString()` (see `createSdkJs`), which drops the surrounding
// module scope β€” so a helper may reference only its own arguments, browser
// globals, or its sibling exports from this module. `createSdkJs` re-declares
// every export here as a same-scope `const` before invoking the SDK, so
// each one with `.toString()` via `serializeModuleHelpers` in `createSdkJs`, which
// drops the surrounding module scope β€” so a helper may reference only its own
// arguments, browser globals, or its sibling exports from this module. `createSdkJs`
// re-declares every export here as a same-scope `const` before invoking the SDK, so
// cross-helper calls (e.g. `mermaidNodeFrom` β†’ `mermaidNodeElement`) resolve in
// the browser exactly as they do here; never close over anything else. Keeping
// the logic here β€” instead of inside the `createArtifactSdk` closure β€” lets us
Expand Down
36 changes: 27 additions & 9 deletions src/server.js
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ import {
serializeLayoutWarnings,
} from "./layout-warnings.js";
import * as mermaidNode from "./mermaid-node.js";
import * as tableCellHelpers from "./table-cell.js";
import { extractMermaidSources, mermaidSourceHash } from "./mermaid-source.js";
import {
isValidDiagramIndex,
Expand Down Expand Up @@ -1889,6 +1890,27 @@ export function createWhiteboardFrameHtml(channelToken = "") {
</html>`;
}

// Serialize every helper a shared module exports as a same-scope const so cross-helper calls
// (e.g. mermaidNodeFrom β†’ mermaidNodeElement) resolve in the browser. Deriving these from the
// module's exports β€” rather than a hand-kept list β€” means adding a helper can never silently
// ReferenceError at runtime.
// Only functions survive `toString()` round-tripping: a Set, Map, or RegExp would serialize to a
// valid-looking `{}` and reach the browser semantically empty, which is far harder to find than
// this throw. A shared module must therefore export nothing but helpers.
function serializeModuleHelpers(module) {
const entries = Object.entries(module);
const unsupported = entries.filter(([, value]) => typeof value !== "function").map(([name]) => name);
if (unsupported.length > 0) {
throw new TypeError(
`Cannot serialize non-function SDK helper export(s) into the artifact bundle: ${unsupported.join(", ")}`,
);
}
return {
declarations: entries.map(([name, fn]) => `const ${name}=${fn.toString()};`).join("\n"),
names: entries.map(([name]) => name),
};
}

/**
* @param {string} key
* @param {number} [artifactRevision]
Expand All @@ -1901,13 +1923,8 @@ export function createSdkJs(
artifactLoadToken = "",
{ maxAttachmentCount, maxAttachmentBytes } = {},
) {
// Serialize every helper exported by mermaid-node.js as a same-scope const so
// cross-helper calls (e.g. mermaidNodeFrom β†’ mermaidNodeElement) resolve in the
// browser. Deriving this from the module's exports β€” rather than a hand-kept
// list β€” means adding a helper can never silently ReferenceError at runtime.
const mermaidHelperEntries = Object.entries(mermaidNode).filter(([, value]) => typeof value === "function");
const mermaidHelperDecls = mermaidHelperEntries.map(([name, fn]) => `const ${name}=${fn.toString()};`).join("\n");
const mermaidHelperKeys = mermaidHelperEntries.map(([name]) => name).join(", ");
const mermaidHelperSource = serializeModuleHelpers(mermaidNode);
const tableHelperSource = serializeModuleHelpers(tableCellHelpers);
const revisionNumber = Number(artifactRevision);
const revision = Number.isFinite(revisionNumber) && revisionNumber >= 0 ? Math.trunc(revisionNumber) : 0;
const loadToken = String(artifactLoadToken || "").slice(0, 200);
Expand Down Expand Up @@ -1937,8 +1954,9 @@ const classifyAttachmentBatch=${classifyAttachmentBatch.toString()};
const partitionDroppedFiles=${partitionDroppedFiles.toString()};
const isTrustedAttachmentResult=${isTrustedAttachmentResult.toString()};
const deriveAttachmentNoticeState=${deriveAttachmentNoticeState.toString()};
${mermaidHelperDecls}
const mermaidHelpers={ ${mermaidHelperKeys} };
${mermaidHelperSource.declarations}
const mermaidHelpers={ ${mermaidHelperSource.names.join(", ")} };
${tableHelperSource.declarations}
(${createArtifactSdk.toString()})(deriveQueueKey, isNativeInteractiveControl, mermaidHelpers, artifactRevision, artifactLoadToken, key, ${JSON.stringify(sdkOptions)});
})();`;
}
Expand Down
Loading
Loading