Camera/Transparent Background

[Thomas101]

Hi,

We're looking at what kind of experience we can give when a device doesn't support ARKit/ARCore and one of the ideas we've been playing with is displaying the camera feed behind the scene. With this we could, for example, display some points of interest with their position set based on compass orientation/device location etc.

We were thinking of something along the lines of...

return (
  <View>
    <RNCamera />
    <Viro3DSceneNavigator
        initialScene={{
          scene: (props} => {
            return (
              <ViroScene>
                <ViroText text="POI 1" width={2} height={2} position={[0, 0, -2]} />
                <ViroText text="POI 2" width={2} height={2} position={[0, 0, 2]} />
              </ViroScene>
              )
          }
       }} />
  </View>
)

So far we haven't found anything that will allow us to make the scene or scene navigator transparent to allow the camera feed to show through, or if this even the best way of doing this. Is there any way of doing this with the current version of the library? If not, would it be possible to add the camera feed to the scene as background for example?

I know this is easily possible using the ARScene, but we're just looking into ways we can provide a degraded experience for older devices

[achuvm]

Hi @Thomas101,

That's an interesting idea.

Making the scene/scene navigator transparent through React isn't supported on our platform (mainly because we haven't had any requests for it until now). However, this is supported on our ViroCore platform by setting the ViroViewScene's backgroundColor to something transparent (as well as the scene itself).

A possible second issue here would be that using Viro3DSceneNavigator requires the developer to manually move the camera around themselves (setting rotation + position), so the camera won't automatically move when the user moves the device (like in VR/AR). This might also be an impediment to your project.

However, this is a use-case we'd like to support, and we're evaluating a few ways of enabling this in a future release

We do have a few questions:

1. Are you looking to support iOS, Android or both?
2. How important would "auto" moving the camera based on gyroscope/accelerometer be to enabling your use case? (vs. you doing it automatically)
3. Will this be used in a production application, or possibly only for prototyping/testing?

We're looking forward to your answers. Let us know if you have any other concerns!

[Thomas101]

Hi @achuvm thanks for your detailed answer! It's great to hear that it's something that could be possible. I'll answer your questions below, but let me know if you need more info!

1. Are you looking to support iOS, Android or both?

Ideally both. Arguably as device support is more widespread on iOS you could use ARScene to do most of this, but it would be a real pain to create two scenes to do the same thing.

2. How important would "auto" moving the camera based on gyroscope/accelerometer be to enabling your use case? (vs. you doing it automatically)

I think this is a really key feature that's missing from the Viro3DSceneNavigator at the moment. There's definately use cases that wouldn't use it but I think a bunch that would. Having something to make it a configurable option would be great to give more options for building scenes easily.

One of the main reasons for not using an ARScene is device support so if you're creating simple experiences falling back to VR or just 3D makes a lot of sense. Using a VR Scene gives a really great experience when there's the option of using VR goggles, but if this is never suggested to the user or they've never used them before it can be a bit confusing that they have to do things like center the crosshairs to click on an object. This kind of falls into #246 that I opened.

For our use case we'd be looking to use auto rotation or implement it ourselves if needs be. For ease of constructing scenes moveWithHead={true} would be just interstellar!

3. Will this be used in a production application, or possibly only for prototyping/testing?

Prototyping & testing currently but looking towards production. I think the use case that we have here is just one thing that we've come up with and there's a few more ideas we're thinking about. We've played at prototyping something in aframe with the camera behind it and it does work, but performance is a bit hit-and-miss and the entire native->javascript->native->webview stack is a bit of a nightmare! We've fallen in ❤ with what you've managed to do with Viro and think it gives the best experience on native mobile!

[OpenToInnovate]

Hi, has there been progress made on this issue over the last year? I'm in the same boat, I'd like to offer some sort of support for Android users that don't qualify for Arcore. Thanks!

[AleeeKoi]

@Thomas101 @achuvm Hi, so is there a way to make viro ar scene transparent?