[Experiment] Lite SDK - SQLite not included

[stevensJourney]

PowerSync Lite SDK - Experimental Package

Overview

This PR demonstrates an experimental SDK (@powersync/lite-sdk) that provides a lightweight interface to connect to a PowerSync service and interact with its synchronization protocol. This SDK is designed to expose sync bucket and stream operations to agnostic data APIs, such as TanStack DB collections, enabling flexible integration with various data storage solutions.

⚠️ Note: This is currently an experimental package and is still under active development.

Design Philosophy

This implementation is similar to how Raw Tables work in the main PowerSync SDK. However, instead of funneling operations to SQL put and delete statements, the Lite SDK provides direct access to the operations themselves. This allows users to handle the operations however they please - whether that's applying them to TanStack DB collections, custom databases, in-memory stores, or any other storage solution.

Key Features

• Storage-Agnostic Design: Unlike the main PowerSync SDK which uses SQLite, this SDK does not use SQLite. Instead, it provides a storage-agnostic interface that allows you to integrate with any data storage solution.
• Protocol Implementation: The logic here implements the Sync bucket protocol logic found in the PowerSync Rust SQLite core extension, but written in TypeScript.
• Flexible Integration: Designed to work with agnostic data APIs like TanStack DB collections, allowing you to use your preferred storage solution.
• Connection Management: Handles connection management, credential refresh, and automatic retries.
• Bidirectional Sync: Supports bidirectional data synchronization with a PowerSync service.

Architecture

The SDK consists of several key components:

SyncClient

The main interface for synchronizing data with a PowerSync service. It handles:

• Connection management and automatic reconnection
• Bidirectional data synchronization
• Credential refresh via the Connector interface
• Status tracking (connection state, errors, etc.)

BucketStorage

An interface for managing bucket data and synchronization state. The SDK provides:

• MemoryBucketStorageImpl: An in-memory implementation for cases where persistence is not required
• You can implement your own BucketStorage to integrate with your preferred storage solution

SyncOperationsHandler

Receives sync operations from the protocol and allows you to apply them to your external data storage. Each operation contains:

• type: The table/collection name
• id: The row identifier
• op: The operation type (PUT or REMOVE)
• data: The operation data (null for REMOVE operations)

Connector

Provides a way to dynamically fetch credentials for authentication, enabling:

• Token refresh without disconnecting
• Dynamic credential rotation
• Custom authentication logic

Example Usage

import {
  MemoryBucketStorageImpl,
  SyncClientImpl,
  DEFAULT_SYSTEM_DEPENDENCIES,
  type Connector,
  type SyncOperationsHandler
} from '@powersync/lite-sdk';

// Define a connector to fetch credentials
const connector: Connector = {
  fetchCredentials: async () => {
    const tokenResponse = await fetch(`http://localhost:6060/api/auth/token`, {
      method: `GET`,
      headers: {
        'content-type': `application/json`
      }
    });

    if (!tokenResponse.ok) {
      throw new Error(`Failed to fetch token: ${tokenResponse.statusText}`);
    }

    const tokenBody = await tokenResponse.json();
    return {
      endpoint: `http://localhost:8080`,
      token: tokenBody.token
    };
  }
};

// Define a handler to process sync operations
const syncOperationsHandler: SyncOperationsHandler = {
  processOperations: async (operations) => {
    // Process operations and apply them to your external storage
    // For example, update TanStack DB collections, your database, etc.
    for (const operation of operations) {
      if (operation.op === 'PUT') {
        // Insert or update data in your storage
        console.log(`PUT ${operation.type}:${operation.id}`, operation.data);
      } else if (operation.op === 'REMOVE') {
        // Remove data from your storage
        console.log(`REMOVE ${operation.type}:${operation.id}`);
      }
    }
  }
};

// Create system dependencies (default uses browser/Node.js globals)
const systemDependencies = DEFAULT_SYSTEM_DEPENDENCIES();

// Create storage implementation
const storage = new MemoryBucketStorageImpl({
  operationsHandlers: [syncOperationsHandler],
  systemDependencies: systemDependencies
});

// Create and connect the sync client
const syncClient = new SyncClientImpl({
  connectionRetryDelayMs: 1000,
  uploadRetryDelayMs: 1000,
  debugMode: false,
  storage: storage,
  systemDependencies: systemDependencies
});

// Connect to the PowerSync service
await syncClient.connect(connector);

System Dependencies

The SDK requires system-level dependencies for HTTP requests, streams, and cryptographic operations. The DEFAULT_SYSTEM_DEPENDENCIES() function provides a default implementation that uses browser/Node.js globals (fetch, ReadableStream, TextDecoder, crypto).

For custom environments (e.g., React Native), you can provide your own implementation.

Current Limitations

The following features are planned but not yet fully implemented:

1. Sync Status Indicators: The sync status tracking is partially implemented but needs completion. Status updates for connection state, sync progress, and error reporting are not fully functional.

2. CRUD Functionality: Client-side CRUD operations (create, read, update, delete) that need to be uploaded to the server are not yet fully implemented. The ps_crud storage layer exists but is not complete.

3. Sync Streams Support: Support for sync streams is mentioned in the codebase but may need additional implementation work. The checkpoint structure includes streams, but full stream synchronization support may be pending.

Testing

The package includes unit tests demonstrating basic connection functionality. See packages/lite-sdk/tests/powersync-lite.test.ts for examples.

TODOs

• Complete sync status indicator implementation
• CRUD functionality support
• Sync streams support
• Enhanced error handling and recovery

[changeset-bot]

⚠️ No Changeset found

Latest commit: 4a83028

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR