cyclo_lab

ISAAC_FFW_RL_reach_play.mp4

Overview

cyclo_lab is a research-oriented repository based on Isaac Lab, designed to enable reinforcement learning (RL) and imitation learning (IL) experiments using Robotis robots in simulation. This project provides simulation environments, configuration tools, and task definitions tailored for Robotis hardware, leveraging NVIDIA Isaac Sim’s powerful GPU-accelerated physics engine and Isaac Lab’s modular RL pipeline.

Important

This repository currently depends on IsaacLab v2.2.0 or higher.

Installation (Docker)

Docker installation provides a consistent environment with all dependencies pre-installed.

Prerequisites:

Docker and Docker Compose installed
NVIDIA Container Toolkit installed
NVIDIA GPU with appropriate drivers

Steps:

Clone cyclo_lab and initialize its direct submodules:

git clone https://github.com/ROBOTIS-GIT/cyclo_lab.git
cd cyclo_lab
git submodule update --init

Build and start the Docker container:
```
./docker/container.sh start
```
Enter the container:
```
./docker/container.sh enter
```

Docker Commands:

./docker/container.sh start - Build and start the container
./docker/container.sh recreate - Recreate the container from the current image
./docker/container.sh enter - Enter the running container
./docker/container.sh stop - Stop the container
./docker/container.sh logs - View container logs
./docker/container.sh clean - Remove container and image

What's included in the Docker image:

Isaac Sim 5.1.0
Isaac Lab v2.3.0 (from third_party submodule)
zenoh_ros2_sdk 0.1.8 (from third_party submodule)
IsaacLab-Arena integration (from third_party submodule)
LeRobot 0.3.3 (in separate virtual environment at ~/lerobot_env)
All required dependencies and configurations

Try examples

Sim2Sim

Reinforcement learning

OMY Reach Task

# Train
python scripts/reinforcement_learning/rsl_rl/train.py --task Cyclo-Reach-OMY-v0 --num_envs=512 --headless

# Play
python scripts/reinforcement_learning/rsl_rl/play.py --task Cyclo-Reach-OMY-v0 --num_envs=16

OMY Lift Task

# Train
python scripts/reinforcement_learning/rsl_rl/train.py --task Cyclo-Lift-Cube-OMY-v0 --num_envs=512 --headless

# Play
python scripts/reinforcement_learning/rsl_rl/play.py --task Cyclo-Lift-Cube-OMY-v0 --num_envs=16

OMY Open drawer Task

# Train
python scripts/reinforcement_learning/rsl_rl/train.py --task Cyclo-Open-Drawer-OMY-v0 --num_envs=512 --headless

# Play
python scripts/reinforcement_learning/rsl_rl/play.py --task Cyclo-Open-Drawer-OMY-v0 --num_envs=16

FFW-BG2 reach Task

# Train
python scripts/reinforcement_learning/rsl_rl/train.py --task Cyclo-Reach-FFW-BG2-v0 --num_envs=512 --headless

# Play
python scripts/reinforcement_learning/rsl_rl/play.py --task Cyclo-Reach-FFW-BG2-v0 --num_envs=16

Imitation learning

If you want to control a SINGLE ROBOT with the keyboard during playback, add --keyboard at the end of the play script.

Key bindings:
=========================== =========================
Command                     Key
=========================== =========================
Toggle gripper (open/close) K      
Move arm along x-axis       W / S   
Move arm along y-axis       A / D
Move arm along z-axis       Q / E
Rotate arm along x-axis     Z / X
Rotate arm along y-axis     T / G
Rotate arm along z-axis     C / V
=========================== =========================

OMY Stack Task (Stack the blocks in the following order: blue → red → green.)

# Teleop and record
python scripts/imitation_learning/isaaclab_recorder/record_demos.py --task Cyclo-Stack-Cube-OMY-IK-Rel-v0 --teleop_device keyboard --dataset_file ./datasets/dataset.hdf5 --num_demos 10

# Annotate
python scripts/imitation_learning/isaaclab_mimic/annotate_demos.py --device cuda --task Cyclo-Stack-Cube-OMY-IK-Rel-Mimic-v0 --auto --input_file ./datasets/dataset.hdf5 --output_file ./datasets/annotated_dataset.hdf5 --headless

# Mimic data
python scripts/imitation_learning/isaaclab_mimic/generate_dataset.py \
--device cuda --num_envs 100 --generation_num_trials 1000 \
--input_file ./datasets/annotated_dataset.hdf5 --output_file ./datasets/generated_dataset.hdf5 --headless

# Train
python scripts/imitation_learning/robomimic/train.py \
--task Cyclo-Stack-Cube-OMY-IK-Rel-v0 --algo bc \
--dataset ./datasets/generated_dataset.hdf5

# Play
python scripts/imitation_learning/robomimic/play.py \
--device cuda --task Cyclo-Stack-Cube-OMY-IK-Rel-v0 --num_rollouts 50 \
--checkpoint /PATH/TO/desired_model_checkpoint.pth

FFW-BG2 Pick and Place Task (Move the red stick into the basket.)

# Teleop and record
python scripts/imitation_learning/isaaclab_recorder/record_demos.py --task Cyclo-PickPlace-FFW-BG2-IK-Rel-v0 --teleop_device keyboard --dataset_file ./datasets/dataset.hdf5 --num_demos 10 --enable_cameras

# Annotate
python scripts/imitation_learning/isaaclab_mimic/annotate_demos.py --device cuda --task Cyclo-PickPlace-FFW-BG2-Mimic-v0 --input_file ./datasets/dataset.hdf5 --output_file ./datasets/annotated_dataset.hdf5 --enable_cameras

# Mimic data
python scripts/imitation_learning/isaaclab_mimic/generate_dataset.py \
--device cuda --num_envs 20 --generation_num_trials 300 \
--input_file ./datasets/annotated_dataset.hdf5 --output_file ./datasets/generated_dataset.hdf5 --enable_cameras --headless

# Train
python scripts/imitation_learning/robomimic/train.py \
--task Cyclo-PickPlace-FFW-BG2-IK-Rel-v0 --algo bc \
--dataset ./datasets/generated_dataset.hdf5

# Play
python scripts/imitation_learning/robomimic/play.py \
--device cuda --task Cyclo-PickPlace-FFW-BG2-IK-Rel-v0  --num_rollouts 50 \
--checkpoint /PATH/TO/desired_model_checkpoint.pth --enable_cameras

Sim2Real

Important

OMY Hardware Setup: To run Sim2Real with the real OMY robot, you need to bring up the robot.

This can be done using ROBOTIS’s open_manipulator repository.

AI WORKER Hardware Setup: To run Sim2Real with the real AI WORKER robot, you need to bring up the robot.

This can be done using ROBOTIS’s ai_worker repository.

The training and inference of the collected dataset should be carried out using physical_ai_tools. This can be done using ROBOTIS’s physical_ai_tools

Bringup

ROBOTIS HAND VR Teleoperation in Isaac Sim Introduction YouTube

Screencast.from.2026-05-29.17-13-39.mp4

Launch AI Worker SH5

# Run SH5 on a ground plane through the Zenoh ROS2 bridge.
python scripts/sim2real/bringup.py \
    --task Cyclo-Bringup-FFW-SH5-v0 \
    --bridge ffw_sh5

# Run SH5 in NVIDIA Simple Warehouse with the operator camera view.
python scripts/sim2real/bringup.py \
    --task Cyclo-Bringup-Simple-Warehouse-FFW-SH5-v0 \
    --bridge ffw_sh5 \
    --camera_view operator \
    --headless

Launch FFW SG2 in the Robotis showroom

# Publish SG2 state and camera topics from the empty 5 m showroom.
python scripts/sim2real/bringup.py \
    --task Cyclo-Real-Showroom-FFW-SG2-v0 \
    --bridge ffw_sg2 \
    --headless \
    --enable_cameras

# Run SG2 in the empty 5 m showroom with the six-panel operator view.
python scripts/sim2real/bringup.py \
    --task Cyclo-Real-Showroom-FFW-SG2-v0 \
    --bridge ffw_sg2 \
    --headless \
    --camera_view operator

Launch FFW SG2 in the IsaacLab-Arena Galileo pick-and-place environment

# Run the upstream Galileo scene with Cyclo Lab's 22D SG2 mobile action.
python scripts/sim2real/bringup.py \
    --task Cyclo-Arena-Galileo-Pick-Place-FFW-SG2-v0 \
    --bridge ffw_sg2 \
    --enable_cameras \
    --headless

Reinforcement learning

OMY Reach Task Introduction YouTube

sim2real.mp4

Run Sim2Real Reach Policy on OMY

# Train
python scripts/reinforcement_learning/rsl_rl/train.py --task Cyclo-Reach-OMY-v0 --num_envs=512 --headless

# Play (You must run rsl_rl play in order to generate the policy file.)
python scripts/reinforcement_learning/rsl_rl/play.py --task Cyclo-Reach-OMY-v0 --num_envs=16

# Sim2Real
python scripts/sim2real/reinforcement_learning/inference/OMY/reach/run_omy_reach.py --model_dir=<2025-07-10_08-47-09>

Replace <2025-07-10_08-47-09> with the actual timestamp folder name under:

logs/rsl_rl/reach_omy/

Imitation learning

OMY Pick and Place Task

Sim2Sim

506326011-26108031-55b0-4809-8d56-1faa495920c1.mp4

Sim2Real

506326055-02cd503e-0f24-4bec-aea2-79010c21f874.mp4

Teleop and record demos

python scripts/sim2real/imitation_learning/recorder/record_demos.py --task=Cyclo-Real-Pick-Place-Bottle-OMY-v0 --robot_type OMY --dataset_file ./datasets/omy_pick_place_task.hdf5 --num_demos 10 --enable_cameras

Mimic generate dataset

# Data convert ee_pose action from joint action
python scripts/sim2real/imitation_learning/mimic/action_data_converter.py --robot_type OMY --input_file ./datasets/omy_pick_place_task.hdf5 --output_file ./datasets/processed_omy_pick_place_task.hdf5 --action_type ik

# Annotate dataset
python scripts/sim2real/imitation_learning/mimic/annotate_demos.py --task Cyclo-Real-Mimic-Pick-Place-Bottle-OMY-v0 --auto --input_file ./datasets/processed_omy_pick_place_task.hdf5 --output_file ./datasets/annotated_dataset.hdf5 --enable_cameras --headless

# Generate dataset
python scripts/sim2real/imitation_learning/mimic/generate_dataset.py --device cuda --num_envs 10 --task Cyclo-Real-Mimic-Pick-Place-Bottle-OMY-v0 --generation_num_trials 500 --input_file ./datasets/annotated_dataset.hdf5 --output_file ./datasets/generated_dataset.hdf5 --enable_cameras --headless

# Data convert joint action from ee_pose action
python scripts/sim2real/imitation_learning/mimic/action_data_converter.py --robot_type OMY --input_file ./datasets/generated_dataset.hdf5 --output_file ./datasets/processed_generated_dataset.hdf5 --action_type joint

Data convert lerobot dataset from IsaacLab hdf dataset

lerobot-python scripts/sim2real/imitation_learning/data_converter/isaaclab2lerobot.py \
    --task=Cyclo-Real-Pick-Place-Bottle-OMY-v0 \
    --robot_type OMY \
    --dataset_file ./datasets/processed_generated_dataset.hdf5

Inference in simulation

python scripts/sim2real/imitation_learning/inference/inference_demos.py --task Cyclo-Real-Pick-Place-Bottle-OMY-v0 --robot_type OMY --enable_cameras

FFW SG2 Pick and Place Task

AIW_SIM2REAL_2.mp4

Teleop and record demos

python scripts/sim2real/imitation_learning/recorder/record_demos.py --task=Cyclo-Real-Pick-Place-FFW-SG2-v0 --robot_type FFW_SG2 --dataset_file ./datasets/ffw_sg2_raw.hdf5 --num_demos 4 --enable_cameras

Mimic generate dataset

# Data convert ee_pose action from joint action
python scripts/sim2real/imitation_learning/mimic/action_data_converter.py --robot_type FFW_SG2 --input_file ./datasets/ffw_sg2_raw.hdf5 --output_file ./datasets/ffw_sg2_ik.hdf5 --action_type ik

# Annotate dataset
python scripts/sim2real/imitation_learning/mimic/annotate_demos.py --task Cyclo-Real-Mimic-Pick-Place-FFW-SG2-v0 --auto --input_file ./datasets/ffw_sg2_ik.hdf5 --output_file ./datasets/ffw_sg2_annotate.hdf5 --enable_cameras --headless

# Generate dataset
python scripts/sim2real/imitation_learning/mimic/generate_dataset.py --device cuda --num_envs 10 --task Cyclo-Real-Mimic-Pick-Place-FFW-SG2-v0 --generation_num_trials 500 --input_file ./datasets/ffw_sg2_annotate.hdf5 --output_file ./datasets/ffw_sg2_generate.hdf5 --enable_cameras --headless

# Data convert joint action from ee_pose action
python scripts/sim2real/imitation_learning/mimic/action_data_converter.py --robot_type FFW_SG2 --input_file ./datasets/ffw_sg2_generate.hdf5 --output_file ./datasets/ffw_sg2_final.hdf5 --action_type joint

Data convert lerobot dataset from IsaacLab hdf dataset

lerobot-python scripts/sim2real/imitation_learning/data_converter/isaaclab2lerobot.py \
    --task=Cyclo-Real-Pick-Place-FFW-SG2-v0 \
    --robot_type FFW_SG2 \
    --dataset_file ./datasets/ffw_sg2_final.hdf5

Inference in simulation

python scripts/sim2real/imitation_learning/inference/inference_demos.py --task Cyclo-Real-Pick-Place-FFW-SG2-v0  --robot_type FFW_SG2 --enable_cameras

License

This repository is licensed under the Apache 2.0 License. See LICENSE for details.

Third-party components

Isaac Lab: BSD-3-Clause License, see LICENSE-IsaacLab

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
docker		docker
scripts		scripts
source/cyclo_lab		source/cyclo_lab
third_party		third_party
.dockerignore		.dockerignore
.flake8		.flake8
.gitattributes		.gitattributes
.gitignore		.gitignore
.gitmodules		.gitmodules
.pre-commit-config.yaml		.pre-commit-config.yaml
CHANGELOG.rst		CHANGELOG.rst
LICENSE		LICENSE
LICENSE-IsaacLab		LICENSE-IsaacLab
README.md		README.md
THIRD_PARTY_LICENSES.md		THIRD_PARTY_LICENSES.md

Navigation Menu

Repository navigation

About

Repository files navigation

cyclo_lab

Overview

Installation (Docker)

Try examples

Sim2Sim

Sim2Real

License

Third-party components

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

cyclo_lab

Overview

Installation (Docker)

Try examples

Sim2Sim

Sim2Real

License

Third-party components

About

Topics

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages