Accessing frame data using OpenXR plugin

[trev3d]

When using the Oculus plugin, it's possible to get depth frame descriptors via
XR.Oculus.Utils.GetEnvironmentalDepthFrameDesc(int eye); This is the only method according to the docs. This isn't possible in OpenXR, as the XR.Oculus.Utils class does not exist. EnvironmentDepthManager has a frameDescriptors array and onDepthTextureUpdate event, but these are marked internal and I cannot access them in my scripts without modifying the package.

[TudorJude]

Hey,

When using Unity's OpenXR package for depth, the code path that executes is that in 'DepthProviderOpenXR'. Our implementation is an abstraction over MetaOpenXROcclusionSubsystem. If you go to that class you can do the same thing we do to get the descriptors.
Get the subsystem:
_occlusionSubsystem = loader.GetLoadedSubsystem<XROcclusionSubsystem>() as MetaOpenXROcclusionSubsystem;
And then go to the 'TryGetUpdatedDepthTexture' function and do the same thing we do there:
_occlusionSubsystem.TryGetSwapchainTextureDescriptors(out var swapchainDescriptors)

One thing to note is that in OculusXR Utils, the DepthFrameDescriptor also has some timestamp fields there. Those always return 0, the underlying OpenXR implementation for depth no longer computes those.

Hope this helps.

[trev3d]

Finally got around to trying this. I'm attempting to call MetaOpenXROcclusionSubsystem.TryGetFrame() within Application.onBeforeRender similarly to Meta's depth provider, but I'm running into strange issues.

TryGetFrame() always returns false if I call it after EnvironmentDepthManager.TryFetchDepthTexture() runs. Calling it before returns frame data properly, but also breaks occlusion; it stops working visually and the depth texture is null when I try to access it via Shader.GetGlobalTexture("_EnvironmentDepthTexture");

Seems that MetaOpenXROcclusionSubsystem.TryGetFrame() changes some internal state that I wouldn't expect it to. I can't dig any deeper into the issue since this seems to happen inside NativeApi calls. For now, I need to edit the core SDK package to make EnvironmentDepthManager.frameDescriptors public