Clarification on Hand and Object Mesh Poses in Released Dataset and Possible extension to new objects

[gabrielecaddeo]

Hi,
Thanks for sharing your work! I'm currently exploring the dataset you released and would like to ask a few clarifying questions.

I'm particularly interested in exporting the meshes of the hand and objects in successful grasp configurations. I noticed that in DGN_5k_processed.tar.gz, there are the object meshes used in processed_data, along with scale and pose information in the scene_cfg files. Meanwhile, Dexonomy_GRASP_shadow appears to contain the successful grasp information.

However, I'm not entirely sure how to process these components together to export hand and object meshes with the correct poses in space, so that they appear in contact as in the successful grasps. My understanding is that the object pose and scale should come from DGN_5k_processed.tar.gz, while the hand configuration is provided in Dexonomy_GRASP_shadow.

I noticed that the alleged hand poses (grasp_qpos, pregrasp_qpos, and squeeze_qpos) are of shape (N, 29), but it's unclear why N varies across entries. Additionally, I couldn't find documentation or files related to the hand model used. Could you clarify which hand model is being used and where I can find the corresponding mesh or URDF?

Moreover, I was wondering if your approach could be applied zero-shot to other objects/hands.
Any guidance on how to correctly reconstruct the hand-object scenes would be greatly appreciated.

Thanks in advance!
Gabriele

[JYChen18]

Hi Gabriele,

Thank you for your interest in our work!

To help you visualize the hand-object configurations, we provide a script. The vusd option exports .usd files viewable in usdview, while vobj exports .obj files.

Specifically, the Dexonomy_GRASP_shadow dataset contains scene_path, scene_scale, and grasp_qpos. We use scene_path and scene_scale to retrieve and calculate the corresponding obj_path, obj_pose, and obj_scale from DGN_5k_processed.tar.gz and objaverse_5k_processed.tar.gz. The Shadow Hand mesh and URDF can be found here. All of the above logic is implemented in the script, so you should be able to run it and visualize directly after placing the required files in the correct locations.

In addition, our DexLearn repository contains code for learning from the dataset.

Please note that the released dataset is built for the Shadow Hand, not the Allegro Hand. The arrays with shape (N, 29) contain N successful grasps—this number may vary across objects depending on how many successful grasps were generated. We store all successful grasps for each object in a single file to reduce memory usage and simplify data loading.

Regarding generalization, our approach can be applied to different objects and hands. However, we consider it one-shot rather than zero-shot, since it requires one initial grasp template per hand and grasp type.

Let me know if you have any further questions!

Best,
Jiayi

[gabrielecaddeo]

Hi @JYChen18 ,
Thanks for your prompt answer!

To help you visualize the hand-object configurations, we provide a script. The vusd option exports .usd files viewable in usdview, while vobj exports .obj files. Specifically, the Dexonomy_GRASP_shadow dataset contains scene_path, scene_scale, and grasp_qpos. We use scene_path and scene_scale to retrieve and calculate the corresponding obj_path, obj_pose, and obj_scale from DGN_5k_processed.tar.gz and objaverse_5k_processed.tar.gz. The Shadow Hand mesh and URDF can be found here. All of the above logic is implemented in the script, so you should be able to run it and visualize directly after placing the required files in the correct locations.

Thanks, now it's clearer.

In addition, our DexLearn repository contains code for learning from the dataset.

So, it this the code associated to Dexonomy? It's written it still need to be released.

Regarding generalization, our approach can be applied to different objects and hands. However, we consider it one-shot rather than zero-shot, since it requires one initial grasp template per hand and grasp type

Is there any documentation on how to perform this? I am interested in using different end effectors and different objects.

Thanks,
Gabriele

[JYChen18]

Hi Gabriele,

DexLearn is focused solely on learning from the dataset and is already publicly available for use.

The core contribution of Dexonomy, however, lies in the synthesis pipeline based on sampling and optimization. We're sorry, but we are not permitted to release the key code related to this synthesis pipeline at the moment. Additionally, we are actively working on improving and extending it.

As a temporary alternative, you might consider using our previous work BODex, which is fully open-sourced and supports three common types of dexterous hands and a wide range of objects.

Best,
Jiayi

[gabrielecaddeo]

Hi @JYChen18 ,
Sorry for contacting again. Do you provide grasps for obejcts with texture? Or is there a simple way of obtaining the grasps for textured objects?
Thanks

[JYChen18]

Hi,

The simplest way is to download the raw object meshes, align them with our processed meshes, and then load our synthesized grasps. The raw mesh for DexGraspNet is available at HuggingFace, while the raw mesh for objaverse can be downloaded from the official repository. The reference code for alignment is in here.

As mentioned in #3 , there may be some mismatches between the raw mesh and the grasp poses we provide, but these differences are not expected to greatly impact performance.

[gabrielecaddeo]

Thanks @JYChen18, I am going to try, then I will let you know!