UI Toolkit Based VR Tablet System

VR tablet interface built with Unity UI Toolkit and Meta Interaction SDK in under 8 weeks. Demonstrates how a UGUI-based system can be transitioned to a modular, separated architecture with centralised styling driving both 2D panels and 3D elements.

Research Documentation · System Documentation · Studio Presentation

Developed as part of my internship as an Immersive Developer for Mersus Technologies

The goal was recreating the existing interface without needlessly rewriting backend systems, while keeping Meta Interaction SDK elements intact on top of UITK documents. Localisation was a central requirement, integrated from the start.

Features

• Panel-based architecture -- each screen is a self-contained host with its own UXML document and USS styles
• Elements Records pattern -- type-safe C# records decouple UXML element queries from backend logic
• Meta SDK hand tracking -- 3D buttons driven by PokeInteractor and InteractableUnityEventWrapper
• ScriptableObject theming -- TabletStyleSO drives colours, fonts, tweens, and materials for all 3D elements
• Unity Localization -- labels and TMP strings bound through a central string table
• PrimeTween animations -- allocation-free tweens with SO-driven parameters
• CSS custom properties -- core.uss :root variables provide a single place to tweak colours, radii, and motion across all 2D panels

Architecture

Front-End Layers

flowchart TB
    Host[HostBase<br/>manages UIDocument + root]
    Elements[Elements Records<br/>typed UXML references]
    Query[DocumentQueryUtil<br/>Q&lt;T&gt; extension methods]

    Loc[Unity Localization<br/>string table binding]
    Tween[PrimeTween<br/>animations]
    XR[Meta Interaction SDK<br/>hand-tracked 3D buttons]

    Host -->|queries| Query
    Query -->|populates| Elements
    Elements -->|references| UXML[UXML Documents]
    Loc -->|binds labels| Host
    Tween -->|animates| Host
    XR -->|PokeInteractable| Host

Loading

Styling Bridge

flowchart TB
    SO[TabletStyleSO<br/>colours, fonts, tweens, materials]
    USS[USS Stylesheets<br/>2D UI theming]
    StyleCtrl[Style Controllers<br/>3D MaterialPropertyBlock]

    SO -->|drives 2D| USS
    SO -->|drives 3D| StyleCtrl
    USS -->|applies to| UXML[UXML Documents]
    StyleCtrl -->|applies to| Buttons[3D Button Meshes]

Loading

Centralised Styling

The 2D and 3D sides of the tablet UI are styled from the same design intent. core.uss handles everything flat; TabletStyleSO handles everything spatial. Both are swappable without touching code.

Panel Styling

core.uss defines a :root block of CSS custom properties -- backgrounds, borders, text, accents, radii, motion -- that every panel stylesheet inherits. Change a variable in one place, every panel updates. Each panel layers its own USS on top for layout-specific rules.

Button Styling

TabletStyleSO holds the same design tokens for 3D elements -- layer colours, content colours, font mapping, and tween parameters. Style controllers read from this SO and push colours to renderers via MaterialPropertyBlock, avoiding material duplication entirely. Swapping the SO asset re-themes every 3D button with a single assignment.

Localisation

All user-facing text flows through Unity's Localization package. String tables are maintained as CSV, making it straightforward to add languages and share translations.

Project Structure

Assets/
  UI Toolkit/
    Documents/          -- UXML panel layouts
    Styles/             -- USS stylesheets (core + panel-specific)
  Scripts/
    Data/               -- ScriptableObjects (TabletStyleSO, ModuleData, UserData, ThemeSO)
    Interactions/       -- MetaHandJointPokeDriver (XR hand tracking bridge)
    UI/
      Constants/        -- UIElements, UIStyleSheets, UIStyleClasses, TabletShaderIDs, LocalizationKeys
      Controllers/
        Styles/         -- ButtonStyleController, FlatButtonStyleController, TextFieldStyleController, CanvasStyleController
        Tablet/         -- LobbyTabletController
      Elements/         -- Sealed records per panel
      Enums/            -- TabletFontStyle, TabletPanels, TweenType
      Hosts/            -- Panel hosts (Start, MyTraining, Language, GeneralSettings, Module)
      Interactables/
        Buttons/        -- DropDownToggle
      Interfaces/       -- ITweenable
      Tweenables/       -- ButtonGrowTween, PositionTween, ScaleTween
      Utilities/        -- DocumentQueryUtil, VisualElementUtil
      Factories/        -- UIElementFactory
  Scriptable Objects/
    Styles/             -- MainTheme ScriptableObject
    Users/              -- User data assets
  Localization/         -- Unity Localization string tables

Tech Stack

• Unity 6000.2.9f1
• UI Toolkit (runtime UI Documents in world space)
• Meta Interaction SDK (PokeInteractor, InteractableUnityEventWrapper)
• Unity Localization (string table + UITK binding)
• PrimeTween (allocation-free animation)

Demo

Localisation Walkthrough

UI_2D.mov

3D Button Interaction

UI_3D.mp4

---

Note: The front-end layers (Display, Elements, Styling, Localisation) are complete and functional. The Data layer SOs and controller bindings are scaffolded for demonstration.

A full migration would flesh these out with real data pipelines and production-ready event routing.