Storing ARHitTestResults in ViroARScene parent state?

[MayeulGarreau]

Hi!

I would like to:

• trigger a performARHitTest from another component than ViroARScene by cliking on a simple button;
• store the results in the state of ViroSampleAPP App.js (not in ARHitTestSample.js).

Having studied carrefully the ViroARSample, I then did my best to follow the general React rules on "lifting state up", but I still can't find a way to solve this problem.

Do you have any advice for me?

Thanks in advance for your help!

[dthian]

Hey @MayeulGarreau, the performARHitTest API is actually a property of the ViroARScene and as such needs to be called on a ViroARScene object. If you'd like to have a reference to this control from another part of your code (like the callback of the 2D button), you can simply create a reference to it to call the hit test function:

 <ViroARScene 
ref={(component)=>{this.arSceneRef = component}}  >

_yourJavaScriptFunc(){
  this.arSceneRef.performARHitTest(....);
}

Do let us know if you have problems with the above.

[MayeulGarreau]

Hi @dthian , thanks for your quick and awesome answer!

I've implemented what you said on my code (based on the Hello World example) but when I run app there is an error:

undefined is not an object (evaluating: '_this.arSceneRef.getCameraOrientationAsync').

Maybe the following code may help you to identifies what went wrong?

// In App.js:

_getARNavigator() {
    return (
      <View style={{flex:1}}>
          <ViroARSceneNavigator {...this.state.sharedProps}
          initialScene={{scene: InitialARScene}}
          viroAppProps={this.state.viroAppProps}
          debug={true} />
        <View style={localStyles.button_container}>
          <TouchableHighlight style={localStyles.buttons}
            onPress={this._onCameraARHitTest}
            underlayColor={'#00000000'} >
            <Text>coord</Text>
          </TouchableHighlight>
        </View>
      </View>
    );
  }

_onCameraARHitTest = () => {
      this.arSceneRef.getCameraOrientationAsync().then((orientation)=>{
      this.arSceneRef.performARHitTestWithRay(orientation.forward).then((results)=>{
        this._onARHitTestResults(orientation.position, orientation.forward, results);
      })
    });
}

// In HelloWorldSceneAR.js:

render() {
    return (
      <ViroARScene ref={(component) => {this.arSceneRef = component}} onTrackingUpdated={this._onInitialized}  displayPointCloud={true} onAnchorFound={this._setARPlane}>
        <ViroText text={this.state.text} scale={[.5, .5, .5]} position={[0, 0, -1]} style={styles.helloWorldTextStyle} />
        <ViroARPlane minHeight={.5} minWidth={.5} materials={['planeColor']}>
          <ViroBox position={[0, .25, 0]} scale={[.2, .2, .2]} alignment={"Horizontal"} materials={['boxColor']} />
        </ViroARPlane>
        { this._getCoordinates() }
        { this._drawUI() }
      </ViroARScene>
    );
  }

Thanks again for your help and sorry I'm not really used to references and stuff!

[dthian]

Hey @MayeulGarreau, no worries, it's all good. In your code example above, you have saved a reference to the ViroARScene in HelloWorldSceneAR through 'this'. However, you are also attempting to access it outside of the HelloWorldSceneAR scope from App.js, causing the undefined behavior.

Another approach we can take, is that we could 'tell' HelloWorldSceneAR to perform the hit test, and then in HelloWorldSceneAR define the _onCameraARHitTest() function correctly accesses ViroARScene's 'this' within the correct scope. This also helps separate AR functionality to AR classes, while updating your 2D UI separately.

However, with the current Viro Classes, this doesn't work in this case due to the nature of the Viro Scene Navigator class - The HelloWorldSceneAR may not yet been inflated by the navigator, kind of like React's own SceneNav. As such, we'll have to make a few tweaks to ViroARSceneNavigator:

Within ViroARSceneNavigator, please replace the _renderSceneStackItems() function with this below. Note this effectively enables the referencing of the active scene.

_renderSceneStackItems: function() {
      let views = [];
      var i = 0;
      var sceneDictionary = this.state.sceneDictionary;
      for (var scene in sceneDictionary){
          var Component = sceneDictionary[scene].sceneClass.scene;
          var props = sceneDictionary[scene].sceneClass.passProps;
          
          if (i == this.state.currentSceneIndex){
            views.push((<Component key={'scene' + i} ref={component => {this.activeScene = component}}
            sceneNavigator={this.sceneNavigator} {...props} arSceneNavigator={this.arSceneNavigator} {...props}/>));
          } else {
            views.push((<Component key={'scene' + i} sceneNavigator={this.sceneNavigator} {...props} arSceneNavigator={this.arSceneNavigator} {...props}/>));
          }
          
          i++;
      }
      return views;
  },

Then, within your HelloWorldAR.js file you can perform all the AR items you'd like, and confine them to a helper function to be triggered. Note that we also must bind the callback to our class in the constructor:

triggerCustomUpdate(callback){
    // Do AR related items here (ar hit check, etc).

    // Once it's done fire the callback to notify 2D elements
    // We pass back a string for example purposes.
    callback("result params");
  }
 
constructor() {
    super();
    this.state = {} // Set initial state here
    this.triggerCustomUpdate = this.triggerCustomUpdate.bind(this);
  }

Finally, we can then refer to this triggerCustomUpdate function via the refs we have previously plumbed through the arSceneNavigator: this.nav.activeScene.triggerCustomUpdate. We can then trigger the triggerCustomUpdate() from the click event within App.js:

// Returns the ViroSceneNavigator which will start the VR experience
  _getVRNavigator() {
    return (
      <View style={styles.imageContainer}>
        {/* 3D AR Stuff */}
        <ViroVRSceneNavigator
          ref={(component) => {this.nav = component}}
          {...this.state.sharedProps}
          initialScene={{scene: InitialVRScene}} />

        {/* Mock 2D text to click for triggering triggerCustomUpdate()*/}
        <TouchableHighlight style={{justifyContent:'center',alignItems:'center',alignSelf:'center', position:'absolute'}}
        onPress={()=>{ this.nav.activeScene.triggerCustomUpdate(this._updateUI); }}  >
        <Text style={localStyles.buttonText}>{this.state.testText}</Text>
        </TouchableHighlight>
      </View>
    );
  }

  _updateUI(updateText){
    // You can do whatever you'd like with the passed back text from hello world.
    // For the sake of this demo, we updated the text of the button itself that which was clicked.
    this.setState({testText:updateText});
  }

I hacked a solution together and this appears to work well. This should get you unblocked for now, please let me know if you are experiencing any difficulties with the above.

[MayeulGarreau]

Hey, waaah amazing, thank you so much for this very complete answer! I wouldn’t have been able to handle it alone.

I’m actually implementing it, I’ll keep in touch!

[MayeulGarreau]

Hi @dthian , it's me again, sorry for the double post! Thank you one more time, I worked on the solution you gave and it's perfectly doing the job 👍.

I just have another question regarding the next step of what we talked before:

I now try to keep updated the ViroARScene with the data in App.js state. So I thought that things like...

return <ViroText text={ this.props.arSceneNavigator.viroAppProps.testText } />;

or:

return <ViroText text={ this.props.nav.viroAppProps.testText } />;

... would work, but it's not!

Do you have any ideas?

Thanks again!

[dthian]

Hey @MayeulGarreau, previously the examples were accessing Viro scene data and components, from the context of your App.js. If instead you want to access App.js data, from the context of your App.js, it would be different - you would have to pass data through to your scene. To do that you can use viroAppProps.

How to achieve that should be thoroughly explained in this issue. Let me know if you have any questions or difficulties with the above.

[MayeulGarreau]

Hi, Sorry for the delay of my answer!

I fully implemented it, it works fine. Thanks a lot for your help!

I close the issue :)

[dthian]

Sounds good @MayeulGarreau. Note that you will have to re-apply the patch above if you ever upgrade ViroReact in the future. I've filed a bug internally so that we can consider making a more permanent fix. Just an FYI.

[MayeulGarreau]

Hi @dthian , sorry to bother you again on this previsoulsy closed issue!

I am migrating my app state to Redux at the moment.

I want to connect the component where exists the triggerCustomUpdate function but I have an error:

This error appears when I change export default line from:
export default class myComponentName extends Component {}
to:
export default connect(mapStateToProps)(myComponentName);

The parent component is also connected to Redux. the getHitTestResults function you see in the stacktrace the updateUI function mentionned in one of the previous message of this issue.

Do you know what my mistake is?

Thank you in advance for your help and have a nice day!