-
Notifications
You must be signed in to change notification settings - Fork 17
Add SDK Overview and Camera Stream Behavior and ICaptureFactory docs #106
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
PhotonLabs-VR
wants to merge
9
commits into
Project-Babble:main
Choose a base branch
from
PhotonLabs-VR:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+307
−14,231
Open
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
448b6f2
Add SDK Overview
PhotonLabs-VR 795021c
Add Camera Stream Behavior and ICaptureFactory docs
PhotonLabs-VR 97289fe
Merge branch 'Project-Babble:main' into main
PhotonLabs-VR a74cb97
Update SDK overview, remove old SDK docs, ignore VS/Yarn files
PhotonLabs-VR e45d01e
Update SDK overview and restore Capture/Factory pages
PhotonLabs-VR 310b0f0
Update SDK overview and restore Capture/Factory pages
PhotonLabs-VR 68da614
Update Camera Stream Behavior documentation
PhotonLabs-VR 940351f
Minor grammatical update
PhotonLabs-VR 7c9df29
Update ICaptureFactory documentation
PhotonLabs-VR File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,3 +18,11 @@ | |
| npm-debug.log* | ||
| yarn-debug.log* | ||
| yarn-error.log* | ||
|
|
||
| # Visual Studio | ||
| .vs/ | ||
|
|
||
| # Yarn | ||
| .yarn/ | ||
| .yarnrc.yml | ||
| yarn.lock | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| { | ||
| "CurrentProjectSetting": null | ||
| } |
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| { | ||
| "Version": 1, | ||
| "WorkspaceRootPath": "C:\\Users\\Sebastian\\source\\repos\\SDK\\", | ||
| "Documents": [], | ||
| "DocumentGroupContainers": [ | ||
| { | ||
| "Orientation": 0, | ||
| "VerticalTabListWidth": 256, | ||
| "DocumentGroups": [ | ||
| { | ||
| "DockedWidth": 200, | ||
| "SelectedChildIndex": -1, | ||
| "Children": [ | ||
| { | ||
| "$type": "Bookmark", | ||
| "Name": "ST:0:0:{aa2115a1-9712-457b-9047-dbb71ca2cdd2}" | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| { | ||
| "Version": 1, | ||
| "WorkspaceRootPath": "C:\\Users\\Sebastian\\source\\repos\\SDK\\", | ||
| "Documents": [], | ||
| "DocumentGroupContainers": [ | ||
| { | ||
| "Orientation": 0, | ||
| "VerticalTabListWidth": 256, | ||
| "DocumentGroups": [ | ||
| { | ||
| "DockedWidth": 200, | ||
| "SelectedChildIndex": -1, | ||
| "Children": [ | ||
| { | ||
| "$type": "Bookmark", | ||
| "Name": "ST:0:0:{aa2115a1-9712-457b-9047-dbb71ca2cdd2}" | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| { | ||
| "ExpandedNodes": [ | ||
| "" | ||
| ], | ||
| "PreviewInSolutionExplorer": false | ||
| } |
Binary file not shown.
Binary file not shown.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| nodeLinker: node-modules |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| --- | ||
| title: Overview of Baballonia SDK | ||
| sidebar_position: 1 | ||
| --- | ||
|
|
||
| # Overview of Baballonia SDK | ||
|
|
||
| The Baballonia SDK provides developers with the tools and interfaces necessary to integrate eye-tracking and face-tracking capabilities into applications. It works seamlessly with the Baballonia desktop software, giving access to processed tracking data as well as raw camera streams. | ||
|
|
||
| --- | ||
|
|
||
| ## SDK Components | ||
|
|
||
| ### 1. [`Capture` Class](./Camera%20Stream%20Behavior.mdx) | ||
|
|
||
| `Capture` provides the framework that lets developers create their own camera capture sources. By extending the `Capture` class, you define how your camera behaves and how it provides frames to Baballonia. | ||
|
|
||
| Baballonia currently supports: | ||
|
|
||
| - Standard USB webcams (UVC) | ||
| - Varjo eye-tracking cameras | ||
| - Android camera streams | ||
| - IP cameras | ||
| - V4L2 devices on Linux | ||
| - Custom or experimental capture sources | ||
|
|
||
| --- | ||
|
|
||
| ### 2. [`ICaptureFactory` Interface](./ICaptureFactory.mdx) | ||
|
|
||
| Every custom camera backend must provide a corresponding `ICaptureFactory`. A factory tells Baballonia: | ||
|
|
||
| - Whether it can connect to a given camera address (`CanConnect`) | ||
| - How to construct the correct `Capture` implementation (`Create`) | ||
| - What provider name should appear in the UI and logs (`GetProviderName`) | ||
|
|
||
| The capture system and the factory work together: a `Capture` implementation requires an `ICaptureFactory`, and a factory has no purpose without its corresponding `Capture` implementation. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| --- | ||
| title: Custom Camera Stream Behavior | ||
| sidebar_position: 2 | ||
| --- | ||
|
|
||
| # Custom Camera Stream Behavior | ||
|
|
||
| The Capture class is the base for every type for camera inputs inside of Baballonia. By extending this class, you can define how your camera behaves and how it provides frames to Baballonia. | ||
|
|
||
| --- | ||
|
|
||
| ## Lifecycle of Capture Body | ||
|
|
||
| ### 1. Capture framework and initialization | ||
|
|
||
| This defines the structure for how a capture class interacts with the camera source and provides frames to Baballonia. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Sets up the latest capture frame source for the capture backend. | ||
| /// This is used when Baballonia wants a new frame from the hardware. | ||
| /// </summary> | ||
| /// public abstract class Capture(string source, ILogger logger) : IDisposable | ||
| ```` | ||
|
|
||
| ### 2. Connection Verification | ||
|
|
||
| Checks if the specified connection string can be used to open this device. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Determines whether the specified connection string can be used to open this capture device. | ||
| /// </summary> | ||
| /// <param name="connectionString">The connection string representing the camera source.</param> | ||
| /// <returns>True if the device can be opened with this connection string; otherwise, false.</returns> | ||
| /// public abstract bool CanConnect(string connectionString); | ||
| ```` | ||
|
|
||
| ### 3. Capture Instance address Setter and Getter | ||
|
|
||
| This is the getter and setter for the camera’s source address. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Gets or sets the source address from which this Capture instance is currently pulling data. | ||
| /// </summary> | ||
| /// public string Source { get; set; } = source; | ||
| ```` | ||
|
|
||
| ### 4. Frame Acquisition | ||
|
|
||
| This class receives the frame data in the BGR color space. The value takes ownership of the object and makes it thread-safe. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Represents the incoming frame data for this capture source in BGR color space. | ||
| /// Acquiring this value transfers ownership of the Mat object to the caller, | ||
| /// and the internal reference is set to null. Thread safe. | ||
| /// </summary> | ||
| /// public Mat? AcquireRawMat() | ||
| /// { | ||
| /// Mat? result; | ||
| /// lock (_rawMatLock) | ||
| /// { | ||
| /// result = _rawMat; | ||
| /// _rawMat = null; | ||
| /// } | ||
| /// return result; | ||
| /// } | ||
| ```` | ||
|
|
||
| ### 5. Frame Management | ||
|
|
||
| This method sets the Mat object that will be provided to the caller when a new frame is requested. It is important to note that after calling this method, the caller should no longer use the Mat object, as ownership has been transferred. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Sets the current Mat object that can be acquired by a caller. | ||
| /// After calling this method, it is prohibited to use the value object, | ||
| /// as ownership has been transferred. | ||
| /// </summary> | ||
| /// protected void SetRawMat(Mat value) | ||
| /// { | ||
| /// lock (_rawMatLock) | ||
| /// { | ||
| /// if (ReferenceEquals(_rawMat, value)) return; | ||
| /// _rawMat?.Dispose(); | ||
| /// _rawMat = value; | ||
| /// } | ||
| /// } | ||
| ```` | ||
|
|
||
| ### 6. Capture State Management | ||
|
|
||
| This property checks whether the capture source is ready to provide frames to Baballonia. It uses a getter and a protected setter to manage the allowed states of the capture source. | ||
|
|
||
| ```chsharp | ||
| /// <summary> | ||
| /// Checks whether the capture source is ready to provide frames to Baballonia. | ||
| /// Uses a getter and a protected setter to manage the allowed states of the capture source. | ||
| /// </summary> | ||
| /// public bool IsReady { get; protected set; } = false; | ||
| ```` | ||
|
|
||
| ### 7. Start Capture | ||
|
|
||
| This method starts the capture process on the source. It is an asynchronous method that returns a boolean indicating whether the capture started successfully. No frames will be provided to Baballonia until this method is called and streaming has begun. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Start Capture on this source | ||
| /// </summary> | ||
| /// public abstract Task<bool> StartCapture(); | ||
| ```` | ||
|
|
||
| ### 8. Stop Capture | ||
|
|
||
| This method stops the capture process on the source. It's a method that returns a boolean indicating whether the capture stopped successfully. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Stop Capture on this source | ||
| /// </summary> | ||
| /// public abstract Task<bool> StopCapture(); | ||
| ```` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| --- | ||
| title: ICaptureFactory Interface | ||
| sidebar_position: 3 | ||
| --- | ||
|
|
||
| # ICaptureFactory Interface | ||
|
|
||
| ### 1. Method configuration | ||
|
|
||
| The configuration of this method allows it to be detected and works in tandem with `Camera`. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// This public method allows the interface to create camera-specific Capture instances and for it to work in tandem with the Capture class. | ||
| /// </summary> | ||
| /// public interface ICaptureFactory | ||
| ```` | ||
|
|
||
| ### 2. Camera Address Creation | ||
|
|
||
| This codeblock uses an address for the camera to be discoverable by Baballonia to be used somewhere else. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// This uses a discoverable address for the camera source for it to be discoverable. | ||
| /// </summary> | ||
| /// public Capture Create(string address); | ||
| ```` | ||
|
|
||
| ### 3. Camera Connection Verification | ||
|
|
||
| This simply checks if the `Camera` source is connectable. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Checks if the camera source is connectable | ||
| /// </summary> | ||
| /// public bool CanConnect(string address); | ||
| ```` | ||
|
|
||
| ### 4. Device Name Retrieval | ||
|
|
||
| This retrieves the name of the `Camera` provider. | ||
|
|
||
| ```csharp | ||
| /// <summary> | ||
| /// Retrieves the name of the Camera provider | ||
| /// </summary> | ||
| /// public string GetProviderName(); | ||
| ```` | ||
|
|
||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Delete this file, this has already been explained above and can be condensed.