Hi everyone! My name is Mathilde and I’m super excited to join the VLA research community.
I’m currently fine-tuning the OpenVLA-OFT model on my own dataset.
My setup consists of a UR5e robot with a Robotiq 2F-140 gripper. I recorded around 100 episodes with the following tasks: pick-and-place apple, pick-and-place banana, and lifting a black/white cup. For each episode, I vary the robot’s initial pose, the object’s position (to pick or lift), and the type and position of the container for the place action (large bowl, small bowl, plate). I do not vary the camera position, as it’s not very clear in the literature whether this is necessary or not. Data is recorded at a fixed frequency of 5 Hz.
The training and validation loss curves look consistent, and I’m now at the evaluation stage on my real setup. I’m trying to reproduce some of my training episodes as a sanity check, and here’s what I observe: the robot consistently moves in the correct direction toward the target object, but once it reaches it, it seems to freeze instead of triggering the grasping action.
Has anyone encountered a similar issue before? If so, I’d really appreciate any insight to help me understand what might be going on.
Thanks in advance! 🙏
Hi everyone! My name is Mathilde and I’m super excited to join the VLA research community.
I’m currently fine-tuning the OpenVLA-OFT model on my own dataset.
My setup consists of a UR5e robot with a Robotiq 2F-140 gripper. I recorded around 100 episodes with the following tasks: pick-and-place apple, pick-and-place banana, and lifting a black/white cup. For each episode, I vary the robot’s initial pose, the object’s position (to pick or lift), and the type and position of the container for the place action (large bowl, small bowl, plate). I do not vary the camera position, as it’s not very clear in the literature whether this is necessary or not. Data is recorded at a fixed frequency of 5 Hz.
The training and validation loss curves look consistent, and I’m now at the evaluation stage on my real setup. I’m trying to reproduce some of my training episodes as a sanity check, and here’s what I observe: the robot consistently moves in the correct direction toward the target object, but once it reaches it, it seems to freeze instead of triggering the grasping action.
Has anyone encountered a similar issue before? If so, I’d really appreciate any insight to help me understand what might be going on.
Thanks in advance! 🙏