Vuejs For Spatial computing #827
Closed
Amiya167
started this conversation in
RFC Discussions
Replies: 1 comment
-
|
Sorry, the merge request link is incorrect. It's here: vuejs/core#14612 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
Summary
Introduce a spatial computing compilation target for Vue Single-File Components, enabling developers to write Vue SFCs that compile to native SwiftUI views running on Apple visionOS. This is achieved through:
spatialattribute on<script setup>and<template>blockscompiler-spatialpackage that compiles Vue templates into SwiftUI@ViewBuildercoderuntime-spatialpackage that runs Vue's reactivity system inside JavaScriptCore and bridges to SwiftUI via IPCBasic example
Compiles to:
A more advanced example with spatial-specific features:
Motivation
The Spatial Computing Gap
Apple Vision Pro and visionOS represent a new paradigm in computing — spatial computing, where applications exist in three-dimensional space around the user. Currently, building for visionOS requires:
None of these options leverage the existing Vue ecosystem — the components, state management patterns, composables, and developer tooling that millions of developers already know.
Why Vue is Uniquely Positioned
Vue's architecture is already conceptually aligned with spatial computing:
Compiler-first approach: Vue's SFC compiler already transforms templates into render functions. Adding a SwiftUI codegen target is a natural extension of this architecture — the same way
compiler-ssrextendscompiler-dom.Reactivity is runtime-agnostic: Vue's reactivity system (
@vue/reactivity) has no DOM dependency. It can run in JavaScriptCore on visionOS, driving SwiftUI state updates through a bridge.SFC as the unit of composition: A
.vuefile already encapsulates template + logic + style. For spatial computing, this maps perfectly to: SwiftUI View (template) + ViewModel (script) + View Modifiers (style).Existing precedent: Projects like Vue Native (Vue → React Native) and Weex demonstrated that Vue's template syntax can target non-DOM renderers. This RFC proposes a first-party, compiler-level approach rather than a runtime abstraction layer.
The Electron Analogy
Electron succeeded because it gave web developers access to native desktop capabilities through a familiar model: write web code, render in Chromium, communicate with Node.js via IPC.
Vue Spatial follows the same principle:
ipcMain/ipcRendererspatialBridge.main/spatialBridge.rendererBrowserWindowSpatialWindow/SpatialVolume/ImmersiveSpaceThis gives Vue developers native spatial computing capabilities without leaving the Vue mental model.
Constraints
spatialis opt-in per-componentref(),computed(),watch()work identicallyDetailed design
1. SFC Parsing: The
spatialAttribute1.1 Grammar Extension
The
spatialattribute is valid on<script setup>,<template>, and<style>blocks:A component is considered a spatial component if any of its blocks carry the
spatialattribute. In practice, if<script setup spatial>is present, the compiler treats the entire SFC as spatial.1.2 Parser Changes (
compiler-sfc/parse.ts)The
SFCDescriptorinterface gains a new field:In the
parse()function, when processing<script>nodes:1.3 Validation Rules
<script setup spatial>and<script setup>cannot coexist in the same SFC. A spatial component is fully spatial.<script spatial>(withoutsetup) is not supported. Spatial components must use Composition API.<template spatial>without<script setup spatial>is a warning — script determines the compilation target.<template>with<script setup spatial>is an error — ambiguous target.2. Template Compilation:
compiler-spatial2.1 Package Structure
A new package
@vue/compiler-spatialis added alongside existing compiler packages:2.2 Element Mapping
Core element mapping table:
<v-stack>VStackspacing,alignmentprops<h-stack>HStack<z-stack>ZStack<text>Textfont,coloras modifiers<button>Button@tap→ action closure<image>Imagesrc→Image(systemName:)orAsyncImage<sf-symbol>Image(systemName:)<text-field>TextFieldv-model→@Binding<toggle>Toggle<slider>Slider:range→in:<list>Listv-forchild →ForEach<scroll-view>ScrollViewaxisprop<navigation-stack>NavigationStack<tab-view>TabView<label>Label<divider>Divider<spacer>Spacer<group>Group<model3d>Model3D<reality-view>RealityView<spatial-window>WindowGroup<spatial-volume>WindowGroup+.windowStyle(.volumetric)<spatial-immersive>ImmersiveSpace2.3 Directive Compilation
v-if/v-else-if/v-elseCompiles to:
v-forCompiles to:
v-modelCompiles to:
The
vm.binding()method returns a SwiftUIBinding<String>that reads/writes through the IPC bridge.v-showCompiles to:
2.4 Spatial Directives (New)
v-spatial-dragCompiles to:
v-spatial-rotateCompiles to:
v-spatial-look(eye tracking hover)Compiles to:
2.5 Style Compilation
<style spatial>uses a CSS-like syntax that maps to SwiftUI view modifiers:Compiles to a modifier chain:
Style property mapping:
padding.padding()corner-radius.clipShape(RoundedRectangle(...))glass-background.glassBackgroundEffect()opacity.opacity()shadow-radius.shadow(radius:)frame-width/frame-height.frame(width:height:)foreground-color.foregroundStyle()background-color.background()font.font()depth.offset(z:)— spatial depth in pointshover-effect.hoverEffect()2.6 Codegen Output Structure
For each spatial
.vuefile, the compiler produces:The
.bridge.jsondescribes the contract between JS and Swift:{ "component": "HelloVisionOS", "reactiveState": { "greeting": { "type": "String", "default": "Hello from Vue Spatial" }, "count": { "type": "Int", "default": 0 } }, "events": { "tap:button_0": { "handler": "count_increment" } }, "bindings": [], "spatialGestures": [] }3. Runtime Architecture:
runtime-spatial3.1 Dual-Process Model
3.2 SpatialBridge IPC Protocol
The bridge communicates via JSON messages over a synchronous JSContext ↔ Swift channel:
3.3 Swift-Side Bridge Implementation
3.4 JavaScript-Side Runtime
3.5 Application Entry Point
A spatial Vue app's entry is a Swift
@mainstruct that bootstraps both processes:3.6 Lifecycle Mapping
onMountedonAppeartriggers → IPC → JSonUnmountedonDisappeartriggers → IPC → JSonActivatedonDeactivatedonSpatialEnter(new)onSpatialExit(new)3.7 Spatial-Specific Composables
4. Build Pipeline
4.1 Vite Plugin
4.2 End-to-End Build Flow
5. TypeScript Support
5.1 Type Augmentation
5.2 Template Type Checking
Spatial components have their own set of intrinsic elements.
vue-tscis extended to type-check spatial templates using a separate intrinsic element map:Drawbacks
Implementation Complexity
This is the largest compilation target extension in Vue's history.
compiler-spatialis comparable in scope tocompiler-ssr, and the runtime bridge adds an entirely new process model. The implementation spans:Platform Lock-in
This RFC targets Apple visionOS specifically. SwiftUI codegen only runs on Apple platforms. However:
compiler-spatialarchitecture is designed as a pluggable codegen target. Future targets (Android XR / Jetpack Compose Spatial, WebXR) could reuse the same template AST transforms with different output backends.<template spatial>syntax and composable APIs are platform-agnostic. Only the generated code is platform-specific.Two-Language Boundary
The JS ↔ Swift IPC bridge introduces serialization overhead and debugging complexity. Compared to pure SwiftUI:
Mitigations:
requestAnimationFrameequivalent)@MainActorto ensure thread safetyLearning Curve
Developers need to learn:
However, the core Vue knowledge (reactivity, Composition API, SFC structure) transfers directly.
Alternatives
1. Pure WebView on visionOS
Safari on visionOS supports WebXR and can display web content in spatial windows. However:
2. Runtime Abstraction Layer (like Vue Native)
Instead of compile-time codegen, use a runtime that creates SwiftUI views dynamically:
@ViewBuilderis compile-time; dynamic view construction is limited and loses type safetyThe compile-time approach was chosen because SwiftUI is fundamentally a compile-time DSL — its
@ViewBuilderresult builder pattern generates view hierarchies at compile time. A runtime approach would require reimplementing SwiftUI's diffing, which is impractical.3. Separate
.spatialFile FormatInstead of extending
.vuefiles, create a new file format:4. WASM-based Renderer
Compile Vue runtime to WASM running on visionOS:
Adoption strategy
Opt-in, Non-breaking
spatialis purely opt-in. Existing Vue apps are unaffected.compiler-spatialis a separate package — not bundled unless explicitly used.runtime-dom,compiler-dom, or any existing API.Scaffolding
Scaffolds a project with:
vite-plugin-vue-spatialIncremental Adoption in Existing Apps
Spatial components can coexist with DOM components in the same project:
The Vite plugin routes
.vuefiles to the appropriate compiler based on thespatialattribute.Migration from Existing visionOS Apps
For teams with existing SwiftUI visionOS apps, Vue Spatial can be adopted incrementally:
Ecosystem Impact
Unresolved questions
Hot Module Replacement: How to implement HMR for spatial components? The JS side can use standard Vite HMR via JavaScriptCore reloading, but the Swift side requires either Xcode Previews integration or a dynamic view replacement mechanism.
Component Interop: Can a spatial component use a non-spatial child component, and vice versa? The current design assumes a clean boundary, but cross-target composition would be valuable.
Animations: SwiftUI has its own animation system (
.animation(),withAnimation {}). Should Vue's<Transition>and<TransitionGroup>be mapped to SwiftUI animations, or should spatial components use SwiftUI's animation API directly?State Persistence: When a visionOS app is backgrounded and resumed, should Vue component state be automatically persisted and restored? This is important for spatial apps where users may switch between immersive and shared spaces.
Multi-Window State Sharing: visionOS apps can have multiple windows open simultaneously. How should Vue's app-level state (Pinia stores) be shared across windows that each run in their own scene?
Performance Budgets: What is the acceptable latency for IPC round-trips? Initial measurements suggest <1ms for simple patches, but complex state trees may need optimization strategies (diffing, batching, binary protocols instead of JSON).
Testing: How should spatial components be unit tested? Options include a mock SwiftUI renderer (like
runtime-testfor DOM), JavaScriptCore-based test runner, or Xcode XCTest integration.Accessibility: SwiftUI has built-in accessibility support. Should Vue's accessibility directives (
aria-*) be mapped to SwiftUI accessibility modifiers, or should spatial components use SwiftUI's native.accessibilityLabel()etc. directly?Beta Was this translation helpful? Give feedback.
All reactions