Skip to content

Add video_human_3d_pose_mapper.#976

Open
Qirui-jiao wants to merge 7 commits into
mainfrom
dev/video_human_3d_pose_mapper
Open

Add video_human_3d_pose_mapper.#976
Qirui-jiao wants to merge 7 commits into
mainfrom
dev/video_human_3d_pose_mapper

Conversation

@Qirui-jiao
Copy link
Copy Markdown
Collaborator

Add video_human_3d_pose_mapper: Extract 3D human pose with Human3R (SMPL-X).

@Qirui-jiao Qirui-jiao requested review from HYLcool, markyyds and yxdyc May 10, 2026 08:31
@Qirui-jiao Qirui-jiao added enhancement New feature or request dj:op issues/PRs about some specific OPs labels May 10, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the VideoHuman3DPoseMapper, an operator designed to extract 3D human poses from videos using the Human3R (SMPL-X) model. The implementation includes the core mapper logic, configuration updates, model preparation utilities, and unit tests. Review feedback identifies several improvement opportunities: correcting the default model path and YAML null values in the configuration, adopting loguru for standardized logging, fixing a typo in an error message, and addressing unreachable code and potential resource leaks in video parsing. Additionally, suggestions were made to refine the model setup process to be less disruptive to automated pipelines and to use the huggingface_hub library for more robust model downloads.

Comment thread data_juicer/config/config_all.yaml Outdated
Comment thread data_juicer/config/config_all.yaml Outdated
Comment thread data_juicer/ops/mapper/video_human_3d_pose_mapper.py
Comment thread data_juicer/ops/mapper/video_human_3d_pose_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_human_3d_pose_mapper.py Outdated
Comment thread data_juicer/ops/mapper/video_human_3d_pose_mapper.py
Comment thread data_juicer/utils/model_utils.py
Comment thread data_juicer/utils/model_utils.py Outdated
@Qirui-jiao
Copy link
Copy Markdown
Collaborator Author

/gemini review

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces the VideoHuman3DPoseMapper for extracting 3D human poses using the Human3R (SMPL-X) model, along with model preparation utilities and documentation updates. The review feedback identifies several critical issues, including an uninitialized video_key attribute that causes runtime errors, the need to convert metadata to JSON-serializable lists, and potential race conditions in the model loading logic. Additionally, the reviewer recommended replacing assert statements with robust error handling and extending the mapper to process all videos within a sample instead of only the first one.

Comment thread data_juicer/ops/mapper/video_human_3d_pose_mapper.py
Comment thread data_juicer/ops/mapper/video_human_3d_pose_mapper.py
Comment thread data_juicer/utils/model_utils.py
Comment thread data_juicer/ops/mapper/video_human_3d_pose_mapper.py
Comment thread data_juicer/ops/mapper/video_human_3d_pose_mapper.py
Comment thread data_juicer/utils/model_utils.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dj:op issues/PRs about some specific OPs enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant