Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,11 @@
npm-debug.log*
yarn-debug.log*
yarn-error.log*

# Visual Studio
.vs/

# Yarn
.yarn/
.yarnrc.yml
yarn.lock
3 changes: 3 additions & 0 deletions .vs/ProjectSettings.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"CurrentProjectSetting": null
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file added .vs/SDK/v17/.wsuo
Binary file not shown.
23 changes: 23 additions & 0 deletions .vs/SDK/v17/DocumentLayout.backup.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"Version": 1,
"WorkspaceRootPath": "C:\\Users\\Sebastian\\source\\repos\\SDK\\",
"Documents": [],
"DocumentGroupContainers": [
{
"Orientation": 0,
"VerticalTabListWidth": 256,
"DocumentGroups": [
{
"DockedWidth": 200,
"SelectedChildIndex": -1,
"Children": [
{
"$type": "Bookmark",
"Name": "ST:0:0:{aa2115a1-9712-457b-9047-dbb71ca2cdd2}"
}
]
}
]
}
]
}
23 changes: 23 additions & 0 deletions .vs/SDK/v17/DocumentLayout.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
{
"Version": 1,
"WorkspaceRootPath": "C:\\Users\\Sebastian\\source\\repos\\SDK\\",
"Documents": [],
"DocumentGroupContainers": [
{
"Orientation": 0,
"VerticalTabListWidth": 256,
"DocumentGroups": [
{
"DockedWidth": 200,
"SelectedChildIndex": -1,
"Children": [
{
"$type": "Bookmark",
"Name": "ST:0:0:{aa2115a1-9712-457b-9047-dbb71ca2cdd2}"
}
]
}
]
}
]
}
6 changes: 6 additions & 0 deletions .vs/VSWorkspaceState.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"ExpandedNodes": [
""
],
"PreviewInSolutionExplorer": false
}
Binary file added .vs/slnx.sqlite
Binary file not shown.
Binary file added .yarn/install-state.gz
Binary file not shown.
1 change: 1 addition & 0 deletions .yarnrc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
nodeLinker: node-modules
37 changes: 37 additions & 0 deletions docs/software/SDK/Baballonia Overview.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
---
title: Overview of Baballonia SDK
sidebar_position: 1
---

# Overview of Baballonia SDK

The Baballonia SDK provides developers with the tools and interfaces necessary to integrate eye-tracking and face-tracking capabilities into applications. It works seamlessly with the Baballonia desktop software, giving access to processed tracking data as well as raw camera streams.

---

## SDK Components

### 1. [`Capture` Class](./Camera%20Stream%20Behavior.mdx)

`Capture` provides the framework that lets developers create their own camera capture sources. By extending the `Capture` class, you define how your camera behaves and how it provides frames to Baballonia.

Baballonia currently supports:

- Standard USB webcams (UVC)
- Varjo eye-tracking cameras
- Android camera streams
- IP cameras
- V4L2 devices on Linux
- Custom or experimental capture sources

---

### 2. [`ICaptureFactory` Interface](./ICaptureFactory.mdx)

Every custom camera backend must provide a corresponding `ICaptureFactory`. A factory tells Baballonia:

- Whether it can connect to a given camera address (`CanConnect`)
- How to construct the correct `Capture` implementation (`Create`)
- What provider name should appear in the UI and logs (`GetProviderName`)

The capture system and the factory work together: a `Capture` implementation requires an `ICaptureFactory`, and a factory has no purpose without its corresponding `Capture` implementation.
125 changes: 125 additions & 0 deletions docs/software/SDK/Camera Stream Behavior.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
---
title: Custom Camera Stream Behavior
sidebar_position: 2
---

# Custom Camera Stream Behavior

The Capture class is the base for every type for camera inputs inside of Baballonia. By extending this class, you can define how your camera behaves and how it provides frames to Baballonia.

---

## Lifecycle of Capture Body

### 1. Capture framework and initialization

This defines the structure for how a capture class interacts with the camera source and provides frames to Baballonia.

```csharp
/// <summary>
/// Sets up the latest capture frame source for the capture backend.
/// This is used when Baballonia wants a new frame from the hardware.
/// </summary>
/// public abstract class Capture(string source, ILogger logger) : IDisposable
````

### 2. Connection Verification

Checks if the specified connection string can be used to open this device.

```csharp
/// <summary>
/// Determines whether the specified connection string can be used to open this capture device.
/// </summary>
/// <param name="connectionString">The connection string representing the camera source.</param>
/// <returns>True if the device can be opened with this connection string; otherwise, false.</returns>
/// public abstract bool CanConnect(string connectionString);
````

### 3. Capture Instance address Setter and Getter

This is the getter and setter for the camera’s source address.

```csharp
/// <summary>
/// Gets or sets the source address from which this Capture instance is currently pulling data.
/// </summary>
/// public string Source { get; set; } = source;
````

### 4. Frame Acquisition

This class receives the frame data in the BGR color space. The value takes ownership of the object and makes it thread-safe.

```csharp
/// <summary>
/// Represents the incoming frame data for this capture source in BGR color space.
/// Acquiring this value transfers ownership of the Mat object to the caller,
/// and the internal reference is set to null. Thread safe.
/// </summary>
/// public Mat? AcquireRawMat()
/// {
/// Mat? result;
/// lock (_rawMatLock)
/// {
/// result = _rawMat;
/// _rawMat = null;
/// }
/// return result;
/// }
````

### 5. Frame Management

This method sets the Mat object that will be provided to the caller when a new frame is requested. It is important to note that after calling this method, the caller should no longer use the Mat object, as ownership has been transferred.

```csharp
/// <summary>
/// Sets the current Mat object that can be acquired by a caller.
/// After calling this method, it is prohibited to use the value object,
/// as ownership has been transferred.
/// </summary>
/// protected void SetRawMat(Mat value)
/// {
/// lock (_rawMatLock)
/// {
/// if (ReferenceEquals(_rawMat, value)) return;
/// _rawMat?.Dispose();
/// _rawMat = value;
/// }
/// }
````

### 6. Capture State Management

This property checks whether the capture source is ready to provide frames to Baballonia. It uses a getter and a protected setter to manage the allowed states of the capture source.

```chsharp
/// <summary>
/// Checks whether the capture source is ready to provide frames to Baballonia.
/// Uses a getter and a protected setter to manage the allowed states of the capture source.
/// </summary>
/// public bool IsReady { get; protected set; } = false;
````

### 7. Start Capture

This method starts the capture process on the source. It is an asynchronous method that returns a boolean indicating whether the capture started successfully. No frames will be provided to Baballonia until this method is called and streaming has begun.

```csharp
/// <summary>
/// Start Capture on this source
/// </summary>
/// public abstract Task<bool> StartCapture();
````

### 8. Stop Capture

This method stops the capture process on the source. It's a method that returns a boolean indicating whether the capture stopped successfully.

```csharp
/// <summary>
/// Stop Capture on this source
/// </summary>
/// public abstract Task<bool> StopCapture();
````
51 changes: 51 additions & 0 deletions docs/software/SDK/ICaptureFactory.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Delete this file, this has already been explained above and can be condensed.

title: ICaptureFactory Interface
sidebar_position: 3
---

# ICaptureFactory Interface

### 1. Method configuration

The configuration of this method allows it to be detected and works in tandem with `Camera`.

```csharp
/// <summary>
/// This public method allows the interface to create camera-specific Capture instances and for it to work in tandem with the Capture class.
/// </summary>
/// public interface ICaptureFactory
````

### 2. Camera Address Creation

This codeblock uses an address for the camera to be discoverable by Baballonia to be used somewhere else.

```csharp
/// <summary>
/// This uses a discoverable address for the camera source for it to be discoverable.
/// </summary>
/// public Capture Create(string address);
````

### 3. Camera Connection Verification

This simply checks if the `Camera` source is connectable.

```csharp
/// <summary>
/// Checks if the camera source is connectable
/// </summary>
/// public bool CanConnect(string address);
````

### 4. Device Name Retrieval

This retrieves the name of the `Camera` provider.

```csharp
/// <summary>
/// Retrieves the name of the Camera provider
/// </summary>
/// public string GetProviderName();
````

Loading