DrPose

This repository provides the code for the paper, DrPose: Direct Reward Fine-tuning on Poses

Requirements

DrPose requires a Linux machine with an NVIDIA GPU (CUDA), the uv package manager, and direnv for loading project environment variables. It has been tested on the following setup:

Component	Tested version
OS	Ubuntu 24.04
CUDA	12.8.2
uv	0.11.19
direnv	2.37.1
Python	3.11 (managed automatically by uv)

Other Linux distributions and nearby CUDA versions may work, but are not officially supported.

Getting Started

Installation

1. System packages

Install the required system libraries (needed for headless EGL rendering):

sudo apt-get install -y libegl1

2. Python environment

Install uv:

curl -LsSf https://astral.sh/uv/install.sh | sh

Then set up the environment from the lockfile:

uv sync

3. Environment variables

Install direnv:

sudo apt-get install -y direnv

Set the required variables in .envrc, then allow direnv to load them:

direnv allow

Download pretrained models

Pretrained weights and model assets are fetched with fetch_data.sh. Some downloads require registering on the respective websites:

• Register for SMPL-X at https://smpl-x.is.tue.mpg.de/ and follow the installation instructions there.
• The PyMAF-X assets used by this repository are downloaded from ICON, so register at https://icon.is.tue.mpg.de/.

The script will prompt for your SMPL-X and ICON username/password when needed. A Hugging Face token must also be specified as HF_TOKEN in .envrc to download models from Hugging Face.

For inference/testing, you only need SMPL-X, PSHuman, PyMAF-X, and DrPose:

./fetch_data.sh --smplx --pshuman --pymafx --drpose

For the full pipeline (training, preprocessing, etc.), download everything:

./fetch_data.sh --all

Run ./fetch_data.sh --help to see all available options (--smplx, --pymafx, --era3d, --pshuman, --sam2, --mimo, --drpose).

Run inference

Once the models are downloaded, run the full inference pipeline:

./scripts/infer.sh --input ./examples --model ./data/models/PSHuman-768-6view --unet-model ./data/models/drpose/unet --all

Data preprocessing

Preprocessing builds the datasets used for training. It is only needed if you want to (re)train DrPose — it is not required for inference. First download the full set of assets (SMPL-X, PyMAF-X, and the MIMO animation models):

./fetch_data.sh --all

Two datasets are produced under $DATA_DIR/datasets (DATA_DIR defaults to ./data, set in .envrc):

Dataset	Built by	Contents
human_models	scripts/preprocess/prepare_human_models.sh	Normalized 3D scans + SMPL-X fits + skeleton joints + rendered multi-view RGB/normal images + face-detection scores
human_images	scripts/preprocess/prepare_human_images.sh	Reference humans animated along motion sequences (MIMO) with backgrounds removed

Raw data

The source datasets live under data/raw_data/ and are not downloaded by fetch_data.sh. Four of the five must be downloaded manually from their original sources (registration may be required); RP_Mixamo is the only exception.

Folder	Used by	Source
THuman2.1	human_models (scans + SMPL-X)	https://github.com/ytrock/THuman2.0-Dataset/blob/main/README.md
CustomHumans	human_models (scans + SMPL-X)	https://custom-humans.github.io/
Motion-X	human_images (motion sequences)	https://motion-x-dataset.github.io/
humans100k_jpg	human_images (reference photos)	https://generated.photos/ (humans100k_jpg)
RP_Mixamo	human_models (scans)	— (not part of the manual downloads)

The 3D scan folders (THuman2.1, CustomHumans, RP_Mixamo) follow this layout:

data/raw_data/{dataset}/
├── scan/{subject_id}/{subject_id}.obj
└── smplx/{subject_id}/{mesh_smplx.obj, smplx_param.pkl}

The convenience scripts call python directly, so run them inside the project environment with uv run.

Human models

uv run bash scripts/preprocess/prepare_human_models.sh

This runs three stages in order:

1. prepare_data.py — loads each raw scan from data/raw_data/, normalizes and aligns it, runs SMPL-X, and writes the mesh + joints.npy to data/datasets/human_models/{split}/{dataset}/{subject_id}/.
2. render_images.py — renders turntable orthographic/perspective RGB and normal-map images (Kaolin, GPU) into each subject's images/ folder.
3. compute_face_scores.py — runs face detection (insightface) on the rendered views and writes face_scores.json, used to select front-facing views.

It draws from the THuman2.1, CustomHumans, and RP_Mixamo folders described under Raw data. Subjects and their coordinate conventions are listed in src/drpose/preprocessing/human_models/metadata.json; to add subjects, append entries there (set the SMPL-X paths to null for scan-only subjects).

Human images

uv run bash scripts/preprocess/prepare_human_images.sh

This animates reference human images along motion sequences using the MIMO models (animate.py), then removes the background from the generated frames (rembg.py), writing the result under data/datasets/human_images/.

The inputs come from the humans100k_jpg and Motion-X folders described under Raw data. They are first converted into model inputs by prepare_ref_images.py (humans100k_jpg → data/datasets/ref_imgs) and process_motion_data.py (Motion-X → data/datasets/human_images); both steps are included, commented out, at the top of prepare_human_images.sh — uncomment them on a first run.

Individual stages can also be run directly, e.g.:

uv run python src/drpose/preprocessing/human_models/render_images.py --num-views 24 --img-size 768

License

DrPose is released for non-commercial research and education only. See LICENSE.

This repository vendors code from several upstream projects, including components under non-commercial licenses (PyMAF-X / PaMIR, ECON / ICON, SMPL-X) and a copyleft license (Era3D / AGPL-3.0). These determine the repository-wide non-commercial constraint. Every vendored source, its license, and commercial-use status is cataloged in THIRD_PARTY_LICENSES.md.