CarryOn equips a frozen vision-language-action (VLA) policy with two connected loops: a planning loop that adapts tasks to the user and scene, and an execution loop that checks actions for contact risk and repairs failures. Visual feedback connects both loops throughout the task.
- 2026-10-25: Code released
- 2026-10-15: Paper released
carryon.mp4
- Planning with persistent, revisable context. User and scene memory support personalized task planning and are updated through dialogue and observation.
- Proactive protection and reactive recovery. The execution loop shields VLA actions from predicted hazardous contact and uses control-based recovery skills to restore task progress after failures.
- Refining recovery across attempts. Offline reflection improves reusable recovery skills from unresolved failures, while the VLA remains frozen.
This repository contains CarryOn and its four non-real-robot experiments:
| Setting | What it evaluates |
|---|---|
| LIBERO Perturbation | Recovery after object displacement and offline reflection |
| LIBERO-Safety | Proactive shielding across VLA backbones |
| LIBERO-Recovery | Recovery from recorded failure states |
| RoboLab | Multimodal requests and memory revision across scenes |
See the reproduction guide for environment setup, model requirements, and experiment commands.
Run the experiments on Linux. Install the shared package from the repository root:
python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[models]"
carryon --versionFollow the commands in REPRODUCING.md for installation, RoboLab, LIBERO-Safety, LIBERO Perturbation, or LIBERO-Recovery. The guide includes small and full runs, recovery with memory generated from the current run, and result analysis.
| Directory | Contents |
|---|---|
src/carryon/ |
Shared planning, memory, execution and recovery components |
experiments/ |
Experiment runners, Shield runtime, configurations and benchmark inputs |
requirements/ |
Dependency versions, installation sources and model downloads |
experiments/*/references/ |
Object reference images used by each recovery experiment |
prompts/ |
Planner, Judge, memory, and Shield prompts |
CarryOn builds on LIBERO, LIBERO-Safety, StarVLA, openpi, OpenVLA-OFT, VLAct, cuRobo, SAM 3 and GraspGen.
@article{wang2026carryon,
title={CarryOn: A Dual-Loop Harness for Personalized and Reliable Robot Assistance},
author={Wang, Zixuan and Liu, Yuqi and Chen, Yuxin and Yang, Sengqiao and Ye, Jinhui and Chen, Pengguang and Lu, Changsheng and Liu, Shu and Yu, Bei and Jia, Jiaya},
journal={arXiv preprint arXiv:2610.21453},
year={2026}
}