EightBot.Orbit

A comprehensive client-server synchronization framework for .NET applications, providing seamless data sync capabilities between mobile/desktop clients and server-side APIs with built-in Cosmos DB support.

📋 Table of Contents

• Features
• Quick Start
• Installation
• EightBot.Orbit.Client
• EightBot.Orbit.Server
• EightBot.Orbit.Server.Web
• Configuration
• Examples
• API Reference
• Contributing
• License

✨ Features

Client Features (EightBot.Orbit.Client)

• 🔄 Bidirectional Synchronization - Sync local changes with remote data sources
• 💾 Local Caching - Persistent local storage with SQLite backend via TychoDB
• 🔀 Conflict Resolution - Pluggable reconciliation strategies for handling data conflicts
• 📱 Offline Support - Continue working offline with automatic sync when reconnected
• 🏷️ Type Registration - Strongly-typed object registration with custom ID properties
• 📊 Change Tracking - Full audit trail of local modifications
• 🎯 CRUD Operations - Create, Read, Update, Delete with automatic sync queue management
• 🔍 Query Support - Get latest objects, sync history, and filtered results
• 📂 Categorization - Support for categorized object collections

Server Features (EightBot.Orbit.Server + Server.Web)

• 🌐 Auto-Generated APIs - Automatic REST endpoint generation for registered types
• 🗄️ Cosmos DB Integration - Built-in Azure Cosmos DB support with partitioning
• 🔐 Authentication Support - Configurable authentication per endpoint
• 🚀 ASP.NET Core Integration - Seamless integration with existing ASP.NET Core applications
• 📋 Convention-Based Routing - Automatic API route generation
• 🔧 Custom Data Clients - Extensible data access layer via IOrbitDataClient
• ⚡ High Performance - Optimized for large-scale synchronization scenarios

🚀 Quick Start

1. Install Packages

# For client applications
dotnet add package EightBot.Orbit.Client

# For server applications  
dotnet add package EightBot.Orbit.Server
dotnet add package EightBot.Orbit.Server.Web

2. Client Setup

// Register and configure the client
var orbitClient = new OrbitClient(new JsonSerializer())
    .Initialize(FileSystem.AppDataDirectory)
    .AddTypeRegistration<User, string>(x => x.Id)
    .AddTypeRegistration<Post, string>(x => x.Id);

// Populate initial data
var users = await apiClient.GetUsersAsync();
await orbitClient.PopulateCache(users);

3. Server Setup

// In Startup.cs ConfigureServices
services.AddDefaultOrbitSyncCosmosDataClient(cosmosUri, authKey, databaseId, x =>
{
    x.EnsureCollectionAsync<User>(u => u.Id, u => u.CompanyId);
    x.EnsureCollectionAsync<Post>(p => p.Id, p => p.UserId);
});

services.AddOrbitSyncControllers(x =>
{
    x.EnsureSyncController<User>();
    x.EnsureSyncController<Post>(requireAuth: false);
});

📦 Installation

NuGet Packages

Package	Description	Install Command
EightBot.Orbit.Client	Client synchronization library	dotnet add package EightBot.Orbit.Client
EightBot.Orbit.Server	Server-side synchronization core	dotnet add package EightBot.Orbit.Server
EightBot.Orbit.Server.Web	ASP.NET Core web integration	dotnet add package EightBot.Orbit.Server.Web
EightBot.Orbit.Core	Shared models and interfaces	dotnet add package EightBot.Orbit.Core

Requirements

• .NET 6.0 or later
• For server: ASP.NET Core 6.0+
• For Cosmos DB: Azure Cosmos DB account (optional, custom data clients supported)

🔄 EightBot.Orbit.Client

The client library provides comprehensive offline-first synchronization capabilities.

Core Concepts

OrbitClient

The main client class that manages local caching and synchronization:

public class OrbitClient : IDisposable
{
    // Initialize with custom JSON serializer and sync reconciler
    public OrbitClient(IJsonSerializer jsonSerializer, ISyncReconciler syncReconciler = null)
    
    // Fluent configuration methods
    public OrbitClient Initialize(string cacheDirectory)
    public OrbitClient AddTypeRegistration<T, TId>(Expression<Func<T, TId>> idSelector)
}

Setup and Configuration

1. Basic Initialization

var client = new OrbitClient(new SystemTextJsonSerializer())
    .Initialize("/path/to/cache/directory");

2. Type Registration

Register your model types with their unique identifier properties:

client
    .AddTypeRegistration<User, string>(x => x.Username)
    .AddTypeRegistration<Post, int>(x => x.Id)
    .AddTypeRegistration<Comment, Guid>(x => x.CommentId);

3. Dependency Injection Setup

// Register as singleton
services.AddSingleton<OrbitClient>(provider => 
{
    return new OrbitClient(provider.GetService<IJsonSerializer>())
        .Initialize(FileSystem.AppDataDirectory)
        .AddTypeRegistration<User, string>(x => x.Id);
});

Data Population and Synchronization

Initial Cache Population

// Basic population
var users = await apiService.GetUsersAsync();
await client.PopulateCache(users);

// With category
await client.PopulateCache(managers, category: "managers");
await client.PopulateCache(employees, category: "employees");

// Clear existing sync queue
await client.PopulateCache(users, terminateSyncQueueHistory: true);

Server Reconciliation

Handle incoming server updates with conflict resolution:

// Reconcile server changes with local changes
var serverUpdates = await apiService.GetUpdatesAsync(lastSync);
await client.Reconcile(serverUpdates);

The Reconcile method expects ServerSyncInfo<T> objects:

public class ServerSyncInfo<T>
{
    public ServerOperationType Operation { get; set; } // Create, Update, Delete
    public string Id { get; set; }
    public long ModifiedOn { get; set; } // Unix timestamp
    public T Value { get; set; }
}

CRUD Operations

Creating Objects

var newUser = new User { Id = "user123", Name = "John Doe" };
await client.Create(newUser);

Updating Objects

user.Name = "Jane Doe";
await client.Update(user);

Upserting (Create or Update)

// Handles both create and update scenarios
await client.Upsert(user);

Deleting Objects

// Soft delete - marks for deletion in sync queue
await client.Delete(user);

Querying Data

Get All Latest Objects

// Get all current objects (including local modifications)
var allUsers = await client.GetAllLatest<User>();

// Get with category
var managers = await client.GetAllLatest<User>("managers");

Get Specific Objects

// By object instance
var user = await client.GetLatest<User>(existingUser);

// By ID
var user = await client.GetLatest<User>("user123");

// With category
var user = await client.GetLatest<User>("user123", "managers");

Sync Queue Operations

// Get objects pending sync to server
var pendingSync = await client.GetAllLatestSyncQueue<User>();

// Get sync history for specific object
var history = await client.GetSyncHistory<User>("user123");

// Get all sync history
var allHistory = await client.GetSyncHistory<User>();

// Get full chronological history
var fullHistory = await client.GetSyncHistory<User>(SyncType.FullHistory);

Cache Management

Clearing Sync Queue

// Clear specific object from sync queue
await client.TerminateSyncQueueHistory<User>("user123");

// Clear by object instance
await client.TerminateSyncQueueHistory<User>(user);

// Clear entire sync queue for type
await client.TerminateSyncQueueHistory<User>();

Cache Operations

// Drop entire cache for type
await client.DropCache<User>();

// Delete specific cached item
await client.DeleteCacheItem<User>("user123");

// Update cached item
await client.UpsertCacheItem<User>(user);

Conflict Resolution

Implement custom conflict resolution by creating an ISyncReconciler:

public interface ISyncReconciler
{
    T Reconcile<T>(ServerSyncInfo<T> server, ClientSyncInfo<T> client);
}

public class CustomSyncReconciler : ISyncReconciler
{
    public T Reconcile<T>(ServerSyncInfo<T> server, ClientSyncInfo<T> client)
    {
        // Custom conflict resolution logic
        // Return the object that should be kept
        
        if (server.ModifiedOn > client.ModifiedOn)
            return server.Value;
        
        return client.Value;
    }
}

// Use custom reconciler
var client = new OrbitClient(jsonSerializer, new CustomSyncReconciler());

Built-in reconcilers:

• ServerWinsSyncReconciler - Server always wins (default)

🌐 EightBot.Orbit.Server

The server library provides the core synchronization logic and data abstraction.

Core Components

IOrbitDataClient Interface

Implement this interface to integrate with your preferred data store:

public interface IOrbitDataClient
{
    Task<IEnumerable<ServerSyncInfo<T>>> Sync<T>(IEnumerable<ClientSyncInfo<T>> syncables);
}

Custom Data Client Implementation

public class CustomDataClient : IOrbitDataClient
{
    private readonly IMyRepository _repository;
    
    public CustomDataClient(IMyRepository repository)
    {
        _repository = repository;
    }
    
    public async Task<IEnumerable<ServerSyncInfo<T>>> Sync<T>(IEnumerable<ClientSyncInfo<T>> syncables)
    {
        var results = new List<ServerSyncInfo<T>>();
        
        foreach (var item in syncables)
        {
            switch (item.Operation)
            {
                case ClientOperationType.Create:
                    var created = await _repository.CreateAsync(item.Value);
                    results.Add(new ServerSyncInfo<T>
                    {
                        Operation = ServerOperationType.Create,
                        Id = GetId(created),
                        ModifiedOn = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(),
                        Value = created
                    });
                    break;
                    
                case ClientOperationType.Update:
                    var updated = await _repository.UpdateAsync(item.Value);
                    results.Add(new ServerSyncInfo<T>
                    {
                        Operation = ServerOperationType.Update,
                        Id = GetId(updated),
                        ModifiedOn = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(),
                        Value = updated
                    });
                    break;
                    
                case ClientOperationType.Delete:
                    await _repository.DeleteAsync(item.Id);
                    results.Add(new ServerSyncInfo<T>
                    {
                        Operation = ServerOperationType.Delete,
                        Id = item.Id,
                        ModifiedOn = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(),
                        Value = default(T)
                    });
                    break;
            }
        }
        
        return results;
    }
}

🕸️ EightBot.Orbit.Server.Web

The web library provides ASP.NET Core integration with automatic API generation.

Setup and Configuration

1. Cosmos DB Integration

public void ConfigureServices(IServiceCollection services)
{
    // Configure Cosmos DB client
    services.AddDefaultOrbitSyncCosmosDataClient(
        cosmosUri: "https://your-cosmos.documents.azure.com:443/",
        cosmosAuthKey: "your-auth-key",
        cosmosDataBaseId: "your-database-id",
        configureCollections: async x =>
        {
            // Configure collections with ID and partition key expressions
            await x.EnsureCollectionAsync<User>(
                idExpression: u => u.Id,
                partitionExpression: u => u.CompanyId);
                
            await x.EnsureCollectionAsync<Post>(
                idExpression: p => p.Id, 
                partitionExpression: p => p.UserId);
                
            await x.EnsureCollectionAsync<Comment>(
                idExpression: c => c.Id,
                partitionExpression: c => c.PostId);
        });
}

2. Controller Generation

public void ConfigureServices(IServiceCollection services)
{
    // Generate sync controllers
    services.AddOrbitSyncControllers(config =>
    {
        // Require authentication (default: true)
        config.EnsureSyncController<User>();
        
        // Disable authentication
        config.EnsureSyncController<Post>(requireAuthentication: false);
        
        // Multiple types
        config.EnsureSyncController<Comment>();
        config.EnsureSyncController<Tag>(false);
    });
}

3. Custom Data Client Registration

public void ConfigureServices(IServiceCollection services)
{
    // Register custom data client instead of Cosmos DB
    services.AddScoped<IOrbitDataClient, MyCustomDataClient>();
    
    // Then add controllers
    services.AddOrbitSyncControllers(config =>
    {
        config.EnsureSyncController<User>();
    });
}

Generated API Endpoints

For each registered type, the following REST endpoints are automatically created:

POST /api/sync/{TypeName}

Example: For User type, creates POST /api/sync/User

Request Format

[
  {
    "operation": 1, // 1=Create, 2=Update, 3=Delete
    "id": "user123",
    "modifiedTimestamp": 1634567890123,
    "value": {
      "id": "user123",
      "name": "John Doe",
      "email": "[email protected]"
    }
  }
]

Response Format

[
  {
    "operation": 1, // 1=Create, 2=Update, 3=Delete  
    "id": "user123",
    "modifiedOn": 1634567890456,
    "value": {
      "id": "user123",
      "name": "John Doe",
      "email": "[email protected]"
    }
  }
]

Authentication Configuration

Enable Authentication (Default)

config.EnsureSyncController<User>(); // Authentication required

Disable Authentication

config.EnsureSyncController<Post>(false); // No authentication required

Custom Authentication

Implement custom authentication using standard ASP.NET Core patterns:

[Authorize(Policy = "MyCustomPolicy")]
public class CustomSyncController<T> : SyncController<T>
{
    public CustomSyncController(IOrbitDataClient dataClient) : base(dataClient) { }
}

⚙️ Configuration

Client Configuration Options

public class OrbitClientOptions
{
    public string CacheDirectory { get; set; }
    public ISyncReconciler SyncReconciler { get; set; }
    public IJsonSerializer JsonSerializer { get; set; }
    public string PartitionSeparator { get; set; } = "___";
}

Server Configuration Options

Cosmos DB Options

public class CosmosDbOptions
{
    public string ConnectionString { get; set; }
    public string DatabaseId { get; set; }
    public int? ThroughputProvisioning { get; set; }
    public ConsistencyLevel ConsistencyLevel { get; set; } = ConsistencyLevel.Session;
}

Sync Controller Options

public class SyncControllerOptions
{
    public bool RequireAuthentication { get; set; } = true;
    public string RoutePrefix { get; set; } = "api/sync";
    public Type[] RegisteredTypes { get; set; }
}

💡 Examples

Complete Client Implementation

public class UserService
{
    private readonly OrbitClient _orbitClient;
    private readonly IApiClient _apiClient;
    
    public UserService(OrbitClient orbitClient, IApiClient apiClient)
    {
        _orbitClient = orbitClient;
        _apiClient = apiClient;
    }
    
    public async Task<IEnumerable<User>> GetUsersAsync()
    {
        // Get latest local data
        return await _orbitClient.GetAllLatest<User>();
    }
    
    public async Task<User> CreateUserAsync(User user)
    {
        // Create locally
        await _orbitClient.Create(user);
        
        // Sync with server
        await SyncWithServerAsync();
        
        return await _orbitClient.GetLatest<User>(user.Id);
    }
    
    public async Task SyncWithServerAsync()
    {
        try
        {
            // Get pending changes
            var pendingChanges = await _orbitClient.GetAllLatestSyncQueue<User>();
            
            if (pendingChanges.Any())
            {
                // Send to server
                var serverResponse = await _apiClient.SyncUsersAsync(pendingChanges);
                
                // Reconcile with server response
                await _orbitClient.Reconcile(serverResponse);
                
                // Clear synced items from queue
                foreach (var change in pendingChanges)
                {
                    await _orbitClient.TerminateSyncQueueHistory<User>(change.Id);
                }
            }
            
            // Get server updates
            var lastSync = await GetLastSyncTimestamp();
            var serverUpdates = await _apiClient.GetUpdatesAsync(lastSync);
            
            if (serverUpdates.Any())
            {
                await _orbitClient.Reconcile(serverUpdates);
            }
        }
        catch (Exception ex)
        {
            // Handle sync errors (network issues, conflicts, etc.)
            LogError("Sync failed", ex);
        }
    }
}

Complete Server Implementation

// Startup.cs
public class Startup
{
    public void ConfigureServices(IServiceCollection services)
    {
        // Add Cosmos DB
        services.AddDefaultOrbitSyncCosmosDataClient(
            Configuration.GetConnectionString("CosmosDb"),
            Configuration["CosmosDb:AuthKey"], 
            Configuration["CosmosDb:DatabaseId"],
            async collections =>
            {
                await collections.EnsureCollectionAsync<User>(u => u.Id, u => u.TenantId);
                await collections.EnsureCollectionAsync<Post>(p => p.Id, p => p.UserId);
            });
        
        // Add authentication
        services.AddAuthentication("Bearer")
            .AddJwtBearer("Bearer", options => { /* JWT config */ });
        
        // Add sync controllers
        services.AddOrbitSyncControllers(config =>
        {
            config.EnsureSyncController<User>(requireAuthentication: true);
            config.EnsureSyncController<Post>(requireAuthentication: true);
        });
        
        services.AddControllers();
    }
    
    public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        app.UseRouting();
        app.UseAuthentication();
        app.UseAuthorization();
        app.UseEndpoints(endpoints => endpoints.MapControllers());
    }
}

Advanced Reconciliation

public class TimestampBasedReconciler : ISyncReconciler
{
    public T Reconcile<T>(ServerSyncInfo<T> server, ClientSyncInfo<T> client) where T : ITimestamped
    {
        // Handle deletions
        if (server.Operation == ServerOperationType.Delete)
            return default(T);
        
        if (client.Operation == ClientOperationType.Delete)
        {
            // Check if server was modified after client deletion
            if (server.ModifiedOn > client.ModifiedTimestamp)
                return server.Value; // Resurrect the object
            
            return default(T); // Keep deletion
        }
        
        // For creates/updates, use timestamp comparison
        var serverValue = server.Value;
        var clientValue = client.Value;
        
        if (serverValue.LastModified >= clientValue.LastModified)
            return serverValue;
        
        return clientValue;
    }
}

📚 API Reference

OrbitClient Methods

Method	Description	Returns
Initialize(string)	Initialize client with cache directory	OrbitClient
AddTypeRegistration<T,TId>(Expression<Func<T,TId>>)	Register type with ID selector	OrbitClient
PopulateCache<T>(IEnumerable<T>, string, bool)	Populate cache with initial data	Task
Create<T>(T)	Create new object locally	Task
Update<T>(T)	Update existing object locally	Task
Upsert<T>(T)	Create or update object locally	Task
Delete<T>(T)	Mark object for deletion	Task
GetAllLatest<T>(string)	Get all current objects	Task<IEnumerable<T>>
GetLatest<T>(T) / GetLatest<T>(string)	Get specific object	Task<T>
GetAllLatestSyncQueue<T>()	Get pending sync objects	Task<IEnumerable<ClientSyncInfo<T>>>
GetSyncHistory<T>(string, SyncType)	Get object sync history	Task<IEnumerable<ClientSyncInfo<T>>>
Reconcile<T>(IEnumerable<ServerSyncInfo<T>>)	Reconcile server changes	Task
TerminateSyncQueueHistory<T>(string)	Clear sync queue items	Task
DropCache<T>()	Clear entire type cache	Task

Core Types

ClientSyncInfo

public class ClientSyncInfo<T>
{
    public ClientOperationType Operation { get; set; }
    public string Id { get; set; }
    public long ModifiedTimestamp { get; set; }
    public T Value { get; set; }
}

ServerSyncInfo

public class ServerSyncInfo<T>
{
    public ServerOperationType Operation { get; set; }
    public string Id { get; set; }
    public long ModifiedOn { get; set; }
    public T Value { get; set; }
}

Operation Types

public enum ClientOperationType
{
    Create = 1,
    Update = 2,
    Delete = 3
}

public enum ServerOperationType  
{
    Create = 1,
    Update = 2,
    Delete = 3
}

🤝 Contributing

We welcome contributions! Please see our Contributing Guidelines for details.

Development Setup

1. Clone the repository

git clone https://github.com/EightBot/EightBot.Orbit.git
cd EightBot.Orbit

2. Build the solution

dotnet build

3. Run tests

dotnet test

Submitting Changes

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🆘 Support

• 📖 Documentation
• 🐛 Issue Tracker
• 💬 Discussions

---

Built with ❤️ by EightBot