Skip to content

Latest commit

 

History

History
159 lines (116 loc) · 11.3 KB

README.md

File metadata and controls

159 lines (116 loc) · 11.3 KB

Experimental native Rust SDK

This private crate provides a callable Rust API over the experimental native engine in ../rust-shadow. It does not start a JSONL process. publish = false prevents publication.

Add a Cargo dependency

Create a Rust application with cargo new /tmp/doc-sync-native-example. Add the following to its Cargo.toml. Replace the absolute path with your checkout:

[dependencies]
doc-sync-native-sdk = { path = "/absolute/path/to/doc-sync/experiments/rust-sdk" }

Put the complete program below in the application's src/main.rs, then run cargo run --manifest-path /tmp/doc-sync-native-example/Cargo.toml. This native path does not require an npm build.

Use Document for one actor and one fixed SchemaPreset. Local commands return a Change with typed Op values. The host must send those operations to peers. Use apply_remote for delivery and exact retry. Remote delivery does not add local history or pending changes. Use a distinct actor for each independent author.

use doc_sync_native_sdk::{Attributes, Document, EditOptions, Key, SchemaPreset};

fn main() -> Result<(), doc_sync_native_sdk::Error> {
    let mut alice = Document::new("alice", SchemaPreset::Text)?;
    let mut bob = Document::new("bob", SchemaPreset::Text)?;
    let change = alice.insert_text(
        &Key::root(), "hello", Attributes::new(), EditOptions::default(), 0.0,
    )?;
    bob.apply_remote(&change.ops)?;
    let undo = alice.undo()?;
    bob.apply_remote(&undo.ops)?;
    Ok(())
}

Run the complete two-actor, undo/redo, checkpoint, restart, and duplicate delivery example:

CARGO_TARGET_DIR=output/r4-sdk-target cargo run --offline \
  --manifest-path experiments/rust-sdk/Cargo.toml --example two_actors

Run tests from the repository root:

CARGO_TARGET_DIR=output/r4-sdk-target cargo test --offline \
  --manifest-path experiments/rust-sdk/Cargo.toml

See the Rust guide for a complete selection, checkpoint, exact retry, acknowledgement, and History recipe. See packages and engines for the TypeScript client and browser adapter boundary.

Supported presets

Document keeps these six fixed presets. Its enum is closed, and an instance cannot change its preset. Use the separate RuntimeSchema and RuntimeDocument API below for supported custom schemas.

SDK preset Native schema identity Native capability
Text flow-root-paragraph:boolean-marks:bold,italic:v1 none
Paragraphs flow-root-paragraph:boolean-marks:bold,italic:v1 paragraph-markers-v1
Blocks general-flow-blocks:json-attrs:v1 general-blocks-attrs-v1
Nested nested-flows-tables:json-attrs:v1 nested-flows-tables-v1
Shapes groups-inline-spans:json-attrs:value-marks:v1 groups-inline-spans-v1
Movement movement-logical-storage:json-attrs:value-marks:v1 movement-logical-storage-v1

The native preset validates node types, attributes, marks, boundaries, and moves. Mark::Link requires a preset that supports links. A typed operation can still be invalid for its preset. The engine rejects invalid input.

Runtime schemas

RuntimeSchema::compile(serde_json::Value) compiles and owns a pure JSON tree schema. RuntimeDocument::new(actor, schema) creates its native storage. Custom names are strings supplied at runtime. The native binary does not contain a registry of those names. Compilation checks the complete schema before it creates a replica.

Domain Native runtime rules Explicit limits
Document One root flow, declared text and atom children, custom root/default/type names, scalar attribute defaults and constraints, Boolean marks Nested flows, grids, groups, inline nodes, spans, value marks, structured attributes, and child cardinality rules are unavailable
Graphic Root and nested container nodes, text and atom nodes, declared child membership, custom root/type/attribute/mark names, scalar and one-level field attributes Flow/grid nodes, inline children, value marks, and computed attributes are unavailable
Spreadsheet SpreadsheetSchema::compile plus RuntimeSheet, 1–128 arbitrary declared field names, scalar defaults, type/enum/range/string-length constraints Fixed axes; no formulas, structural axes, or renderer

Unsupported tree rules return UnsupportedSchema at <JSON path>: <reason>. The native string-pattern grammar supports anchored ASCII literals, the declared six/eight-digit hex color patterns, and the control-free bounded string pattern ^[^\\u0000-\\u001f\\u007f]{0,N}$ with N <= 1000000. Other regex syntax fails compilation. No callback or executable schema rule is accepted.

Tree identities use the same canonical JSON SHA-256 as TypeScript schemaVersion. Explicit domain: "document" has the identity of the omitted legacy document domain. UI metadata is included in the fingerprint. Native tree compilation rejects JSON integer values that JavaScript cannot represent exactly, including values in UI metadata. SpreadsheetSchema::identity matches spreadsheetSchemaVersion.

RuntimeDocument provides generic insert, set_attribute, set_field, insert_label_text, and reparent methods for graphics. For the flat Document profile, use insert_block and insert_text. record accepts typed wire operations. model returns JSON because custom schemas define their own names. Both domains use the existing native LWW, deletion, and selective History rules.

RuntimeSheet takes RuntimeSheetOptions {actor, sheet_id, rows, columns, schema}. Patches are BTreeMap<String, RuntimeValue>. get_cell returns stored non-null values. get_cell_value resolves an unwritten or cleared field to its default without creating a register. Custom operations carry the exact schemaVersion. Legacy SparseSheet retains its existing v1 format and field order.

Runtime checkpoints retain the descriptor, its fingerprint, document or sheet state, History, and exact pending queue. Restore recompiles the descriptor and replays the journal before it commits state. Live checkpoint replacement requires the same actor and schema identity. Checkpoints require trusted host storage.

Run the complete custom Graphic peer/checkpoint and Spreadsheet default/checkpoint example from the repository root:

CARGO_TARGET_DIR=output/runtime-quickstart-target cargo +stable run --offline --locked \
  --manifest-path experiments/rust-sdk/Cargo.toml --example runtime_quickstart

Run the independent six-schema differential trace:

node experiments/runtime-schema-native/compare.mjs

The trace builds one native binary and supplies two schemas per domain to one process. It compares normalized Document/Graphic models and History availability. Spreadsheet comparisons include exact operations, pending queues, journals, History, and checkpoints. The receipt lists its exact scope and excluded work. This does not qualify the full arbitrary Document schema surface or a browser, WASM, renderer, transport, or release.

API and ownership

  • insert_text, delete_text, and set_mark allocate fresh own tickets and record own history. record accepts an explicit typed batch for block births, moves, and other supported native operations. It requires fresh own tickets in order. It is not a retry API.
  • next_ticket returns the next available ticket from the observed clock. It does not reserve a ticket. Construct all tickets in a batch with distinct clocks. A text birth occupies one clock per UTF-16 unit.
  • model returns ModelNode, ModelRun, and ModelText. Utf16Text retains isolated surrogate units. Its conversion to a Rust String can fail.
  • capture_anchor, resolve_anchor, capture_selection, and resolve_selection use native stable token anchors. Position indexes follow native model paths. The last text index is a UTF-16 offset. Anchor keys can include engine-owned storage heads.
  • undo and redo return operations for peer delivery. The engine retains actor ownership and foreign edit fences. Each nonempty local result enters pending_changes.
  • acknowledge removes only an exact pending prefix. An invalid acknowledgement leaves the queue intact. A host must call it only after its transport accepts that prefix.

Coupled restart and retry

Persist checkpoint().to_json() as one host storage transaction. It contains the actor, preset, opaque native document/history checkpoint, and exact pending changes. Checkpoint::from_json validates with a fresh native server. Document::restore restores this coupled state. restore_checkpoint replaces a live document only when actor and preset match and all validation succeeds.

After restart, send the stored pending operations unchanged. The peer can receive them more than once. A retry must use apply_remote, including a retry on the source actor. Replaying pending changes does not add history or pending entries. Restoration rejects a pending queue that changes the native document/history when replayed. Pending changes can include undo and redo output.

The host must provide trusted checkpoint storage and transport. Checkpoints have no signature or authentication. These checks do not prove that an external checkpoint came from a trusted author.

Error and proof limits

Native batches are atomic. SDK history commands also stage a fresh native replica before decoding their result. The SDK commits the replica and pending change only after decoding succeeds. Invalid checkpoint replacement leaves the live document intact. Staging uses a full checkpoint restore for each history command; this slice makes no performance claim.

This is an experimental native SDK slice. It provides no full doc.edit compiler, mark inheritance, text split/join compiler, network transport, durable storage driver, browser or editor adapter, WASM package, Node binding, or MCP integration. It is not a released Rust package. The native engine dependency keeps its existing scope and limits.

Independent authoring comparison

Run node --experimental-strip-types experiments/rust-sdk/scripts/compare-typescript.mjs from the repository root. It executes authoring_trace, a Rust example that uses only public SDK authoring methods. The TypeScript oracle independently calls Doc, History, and editing commands. It does not send expected operations, state, or inverses to Rust.

The fixed 20-action trace covers two concurrent actors, explicit italic and empty marks, isolated UTF-16 surrogate units, a foreign same-value mark writer, concurrent deletion of the same unit, undo/redo, checkpoint restart, and exact pending retry twice on source and peer. After each action it compares generated operations, both models, both history availability flags, and both pending queues. It converts TypeScript strings to UTF-16 units, treats omitted empty birth marks as an empty map, and fills omitted empty model containers. It does not compare transport bytes or full internal history snapshot structures.

The script writes raw traces, source hashes, and its PASS/FAIL receipt to output/r4-sdk-evidence/sdk-typescript-receipt.json. This is bounded authoring proof for these declared actions. It does not prove a generic doc.edit compiler or any adapter, network, or production behavior.