Skip to content

add a2ui message rendering#5838

Merged
dnbrwstr merged 25 commits into
developfrom
db/a2ui-v1
May 19, 2026
Merged

add a2ui message rendering#5838
dnbrwstr merged 25 commits into
developfrom
db/a2ui-v1

Conversation

@dnbrwstr
Copy link
Copy Markdown
Contributor

@dnbrwstr dnbrwstr commented May 12, 2026

Summary

Adds a first pass at A2UI rendering in messages. This keeps the client presentation-only for now, with validation around the small supported subset before we render it.

Changes

  • Add A2UI blob parsing and validation for a v0.9-style component payload.
  • Render Text, Row, Column, Card, Divider, and Button blocks in post content.
  • Wire supported button events to send visible message text from the draft context.
  • Add Cosmos fixtures for weather, approval cards, basic components, and stacked examples.
  • Release @tloncorp/api@0.0.8 with the A2UI types and validation helpers.

How did I test?

Tested on web + device

Comment thread packages/api/package.json
{
"name": "@tloncorp/api",
"version": "0.0.6",
"version": "0.0.8",
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0.0.7 was published but the version bump wasn't committed

@dnbrwstr dnbrwstr marked this pull request as ready for review May 13, 2026 23:21
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review to trigger a review and subscribe this PR to future pushes, or @claude review once for a one-time review.

Tip: disable this comment in your organization's Code Review settings.

@dnbrwstr
Copy link
Copy Markdown
Contributor Author

@claude review

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dfd87c1ad4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/api/src/client/a2ui.ts
Comment thread packages/app/ui/components/ChatMessage/StaticChatMessage.tsx Outdated
Comment thread packages/app/ui/components/PostContent/A2UIBlock.tsx
Comment thread packages/api/src/client/a2ui.ts
Comment thread packages/app/ui/components/PostContent/A2UIBlock.tsx
Comment thread packages/api/src/client/a2ui.ts
Comment thread packages/api/src/client/a2ui.ts
Comment thread packages/api/src/client/a2ui.ts
Comment thread packages/api/src/client/a2ui.ts
Comment thread packages/app/ui/components/PostContent/A2UIBlock.tsx
Comment thread packages/app/ui/components/PostContent/A2UIBlock.tsx
@patosullivan
Copy link
Copy Markdown
Member

Two things stand out:

  1. What happens if a2ui cards render in some place where the user shouldn't be able to send anything? Like search results or read-only channels. It looks like in those cases the actions still appear as enabled.

  2. How can the user be sure that an a2ui button does what it's supposed to? It seems like someone could label a button anything they want and then assign any action they want, so a bad actor could send a DM with a card that says "Refresh" (or whatever) but it actually maps to some destructive action. I'm thinking we could surface the actual text of the action somewhere (if we want to fix this right now).

@dnbrwstr
Copy link
Copy Markdown
Contributor Author

Two things stand out:

  1. What happens if a2ui cards render in some place where the user shouldn't be able to send anything? Like search results or read-only channels. It looks like in those cases the actions still appear as enabled.

Addressed this by making rendering conditional on ability to start a draft in context.

  1. How can the user be sure that an a2ui button does what it's supposed to? It seems like someone could label a button anything they want and then assign any action they want, so a bad actor could send a DM with a card that says "Refresh" (or whatever) but it actually maps to some destructive action. I'm thinking we could surface the actual text of the action somewhere (if we want to fix this right now).

Addressed this by only rendering the blocks in dms. Right now, the only thing buttons can do is send a message with given text in the same context, which is innocuous in a dm. We can consider a more elaborate solution later.

Copy link
Copy Markdown
Member

@patosullivan patosullivan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm!

Copy link
Copy Markdown
Member

@arthyn arthyn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, do we want to update docs/post-blobs.md? Is the "update surface" message intended to update previous surfaces in the message history or is it just on a per message basis? would love to see more on how that's supposed to work

Comment thread packages/app/ui/components/PostContent/A2UIBlock.tsx Outdated
Comment thread packages/app/ui/components/ChatMessage/StaticChatMessage.tsx
Comment on lines +122 to +126
const canRenderA2UI = isDmChannelId(post.channelId);
const canHandleA2UIAction =
canRenderA2UI &&
!!draftInputContext &&
draftInputContext.canStartDraft !== false;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The DM-only render gate at StaticChatMessage.tsx:122 uses isDmChannelId(post.channelId) only, which matches 1:1 DMs (~-prefixed) but excludes group/multi-DMs (0v-prefixed) — so a2ui cards posted in group DMs are silently filtered to their fallback text, contradicting the documented "render only in direct messages" policy and your stated send-message safety rationale, both of which apply equivalently to multi-DMs. Combined with the activity/reference paths that render the card regardless, a multi-DM a2ui post is visible in activity previews but hidden in the conversation itself. Fix: widen to isDmChannelId(post.channelId) || isGroupDmChannelId(post.channelId) (the established convention used at 6 other callsites in packages/api), or tighten the docs to say "1:1 DMs only" if the narrower scope is intentional.

Extended reasoning...

What's wrong

In packages/app/ui/components/ChatMessage/StaticChatMessage.tsx:122:

const canRenderA2UI = isDmChannelId(post.channelId);

isDmChannelId (packages/api/src/client/apiUtils.ts:69-71) returns channelId.startsWith('~') — matching 1:1 DMs only. Group/multi-DMs use a separate 0v-prefixed id detected by the sibling helper isGroupDmChannelId (apiUtils.ts:77-79). The two helpers are mutually exclusive.

Established convention

Every other "is DM-of-any-kind" check in the API package writes both helpers together as a union:

  • packages/api/src/client/postsApi.ts:229, 314, 684, 812, 1110
  • packages/api/src/client/utils.ts:394

That's 6 callsites consistently using isDmChannelId(channelId) || isGroupDmChannelId(channelId) when they mean "any DM channel". This PR's gate is the lone exception.

How it manifests

canRenderA2UI is used in two places (StaticChatMessage.tsx:130-143 and 216) to:

  1. useMemo-filter a2ui blocks out of postContent and lastEditPostContent — so the recipient sees only the inline fallback story text.
  2. Disable onA2UIAction via canHandleA2UIAction.

For a multi-DM with channelId like 0v…, both consequences fire: the card is stripped from the message, and even if it weren't, button actions would be inert.

Why the docs and PR rationale don't justify the narrower scope

  • docs/post-blobs.md (added in this PR): "rendering policy: A2UI blocks render only in direct messages for now" — plural and generic.
  • Your reply to @patosullivan (2026-05-18T18:15:49Z): "Addressed this by only rendering the blocks in dms. Right now, the only thing buttons can do is send a message with given text in the same context, which is innocuous in a dm." — speaks generically about DMs.

The safety argument — "the button can only send a message in the same context" — applies identically to multi-DMs. sendPostFromDraft posts to draftInputContext.channel.id, whatever its type. A group DM's sender and recipient set are knowable in the same way a 1:1 DM's are. There is no rationale in the PR or docs that distinguishes 1:1 from group DMs.

Step-by-step proof

  1. Alice posts an a2ui blob to a multi-DM (channelId 0v…) shared with Bob and Carol.
  2. validateBlobEntry accepts the payload, convertContent emits an A2UIBlockData, the post is stored with both a story (inline fallback text) and the a2ui block.
  3. Bob opens the multi-DM. StaticChatMessage renders post with channelId = 0v….
  4. Line 122: canRenderA2UI = isDmChannelId('0v…')'0v…'.startsWith('~')false.
  5. Line 132-136: content = postContent.filter((block) => block.type !== 'a2ui') — the a2ui block is stripped.
  6. ChatContentRenderer renders only the fallback paragraph text. Bob sees "Weather: 22903 is 57F and partly cloudy." instead of the weather card.
  7. Meanwhile, the activity feed entry and any quoted reference of this post render through paths that bypass StaticChatMessage, so the same post's a2ui card does appear in Bob's activity preview / quoted snippet. Bob now has a UX inconsistency: card visible in activity, hidden in chat.

Impact

Not a security or correctness bug — the recipient sees the inline fallback text in the multi-DM, no crash, no data loss. The current behavior is strictly safer than the docs imply. But:

  1. It contradicts the documented "render only in direct messages" policy.
  2. It produces an observable UX inconsistency with the activity/reference paths (which render a2ui unconditionally).
  3. It breaks the codebase convention used at 6 sibling callsites in packages/api.
  4. The author's own safety rationale doesn't distinguish 1:1 from group DMs.

Severity

nit — real codebase-convention bug, one-line fix, no security impact because the failure mode is conservative (hide the card). Filing for fixability rather than blocking.

How to fix

Either widen the gate (recommended, matches docs + convention):

import { isDmChannelId, isGroupDmChannelId } from '@tloncorp/api/client';
// ...
const canRenderA2UI =
  isDmChannelId(post.channelId) || isGroupDmChannelId(post.channelId);

Or tighten the docs to "1:1 DMs only" if the narrower scope is intentional. The validator/postContent code is shared across all DM types so widening the gate is the lower-friction option.

Comment on lines +148 to +161
const handleButtonPress = useCallback(
(component: A2UI.Button) => {
const fallbackText =
component.action.event.context?.text ??
getComponentText(components.get(component.child), components);

if (!fallbackText.trim()) {
return;
}

context.onA2UIAction?.(component.action, fallbackText);
},
[components, context]
);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 A2UI button onPress at A2UIBlock.tsx:148-161 fires context.onA2UIAction?.(...) synchronously and drops the returned promise. The handler (handleA2UIAction in StaticChatMessage.tsx:95-121) is async and awaits draftInputContext.sendPostFromDraft(...), so any rejection becomes an unhandled promise rejection. The dominant network-failure mode is bounded (the optimistic post in _sendPost is marked deliveryStatus: failed, giving the user visible feedback), so impact is limited to pre-send throw paths (e.g. db.getChannel, sync.handleAddPost). One-line fix: attach .catch(logger.error) at the call site, or wrap handleA2UIAction in try/catch — matching the existing pattern at handleRetryPressed in the same file and at BareChatInput/index.tsx:457-465.

Extended reasoning...

What is wrong

handleButtonPress (A2UIBlock.tsx:148-161) is a synchronous useCallback that fires the action and ignores the returned Promise:

const handleButtonPress = useCallback(
  (component: A2UI.Button) => {
    const fallbackText = ...;
    if (!fallbackText.trim()) return;
    context.onA2UIAction?.(component.action, fallbackText);
  },
  [components, context]
);

In real usage, context.onA2UIAction is bound to handleA2UIAction (StaticChatMessage.tsx:95-121), which is async and awaits draftInputContext.sendPostFromDraft(...). When the inner call rejects, handleA2UIAction returns a rejected Promise that handleButtonPress discards — an unhandled rejection.

Step-by-step proof

  1. User taps a Refresh/Allow/etc. button on an A2UI card in a DM.
  2. handleButtonPress(component) invokes context.onA2UIAction?.(component.action, fallbackText). Because the call is not awaited and no .catch is attached, the returned Promise is unobserved.
  3. handleA2UIAction enters its await draftInputContext.sendPostFromDraft({...}) path. In production this resolves to finalizeAndSendPost_sendPost (postActions.ts:141-385).
  4. _sendPost does most of its work inside a try/catch starting at ~line 270 that marks the optimistic post as deliveryStatus: failed on send-time errors. But the pre-try setup phase (db.getChannel, db.getContact, db.buildPost, sync.handleAddPost at line 266) is outside that try/catch.
  5. If any of those throws (rare but possible — e.g. SQLite transient failure during the optimistic insert), the rejection propagates back through handleA2UIAction and into the dropped Promise. React Native logs an UnhandledPromiseRejectionWarning; production gets nothing user-visible.

Why existing safeguards do not fully cover this

  • _sendPost’s try/catch handles network/send failures by setting deliveryStatus, so those do surface to the user as a failed post in the chat. This is the refutation’s strongest point and it is correct — the “user gets no feedback” framing overstates the realistic case.
  • However, pre-send DB/sync throws bypass that try/catch and propagate.
  • handleRetryPressed in this same file (StaticChatMessage.tsx:87-93) already wraps onPressRetry in try/catch + console.error. BareChatInput/index.tsx:457-465 wraps the analogous sendPostFromDraft call in try/catch with bareChatInputLogger.error. The PR introduces a third caller that breaks that local convention.

Addressing the refutation

The refutation makes several fair points:

  1. ButtonInput.tsx:33 has the identical drop-the-promise pattern. True — and that means this is consistency with one of two existing patterns in the codebase, not a regression. But BareChatInput (the primary text-send path) and handleRetryPressed (the same-file analogue) both use try/catch, so the convention is mixed and the more defensive pattern is the one used in the higher-traffic surfaces.
  2. Errors are shown via the optimistic-post pipeline. True for send-time failures. Pre-send throws are the remaining unhandled window and they are rare.
  3. RN unhandled rejections do not “intermittently crash.” Correct — the bug description’s phrasing was too strong. The real-world consequence is a yellow-box warning in dev and silence in prod, not a crash. Severity adjusted accordingly.
  4. handleRetryPressed has different semantics. Partially fair — retry has no optimistic-pipeline backing — but BareChatInput does have the same optimistic pipeline and still uses try/catch, so the convention argument holds for the closer analogue.

Given the refutation, this is not a blocker. But the fix is one line and removes a known footgun.

How to fix

const handleButtonPress = useCallback(
  (component: A2UI.Button) => {
    const fallbackText = ...;
    if (!fallbackText.trim()) return;
    void Promise.resolve(
      context.onA2UIAction?.(component.action, fallbackText)
    ).catch((e) => {
      // log via existing logger
      console.error("A2UI action failed", e);
    });
  },
  [components, context]
);

Or alternatively wrap the body of handleA2UIAction in StaticChatMessage.tsx in try/catch + console.error, matching handleRetryPressed directly above it.

Comment thread packages/app/fixtures/A2UI.fixture.tsx
@dnbrwstr
Copy link
Copy Markdown
Contributor Author

lgtm, do we want to update docs/post-blobs.md? Is the "update surface" message intended to update previous surfaces in the message history or is it just on a per message basis? would love to see more on how that's supposed to work

Updated docs -- right now it's just per-message, but we'd like to figure out a way to do live updates in the future.

Comment thread packages/app/ui/components/ChatMessage/StaticChatMessage.tsx
Comment thread packages/app/ui/components/PostContent/BlockRenderer.tsx
@dnbrwstr dnbrwstr merged commit 74e99c2 into develop May 19, 2026
11 of 13 checks passed
@dnbrwstr dnbrwstr deleted the db/a2ui-v1 branch May 19, 2026 00:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants