- pytorch >= 1.8
- yaml
- easydict
- pyquaternion
- lightning (tested with pytorch_lightning==1.3.8 and torchmetrics==0.5)
- torch-scatter (pip install torch-scatter -f https://data.pyg.org/whl/torch-1.9.0+${CUDA}.html)
- nuScenes-devkit (optional for nuScenes)
- spconv (tested with spconv==2.1.16 and cuda==11.1, pip install spconv-cu111==2.1.16)
- torchsparse (optional for MinkowskiNet and SPVCNN. sudo apt-get install libsparsehash-dev, pip install --upgrade git+https://github.com/mit-han-lab/torchsparse.git@v1.4.0)
- [selective_scan] (cd selective_scan, python setup.py install or pip install -e.)
Please download the files from the SemanticKITTI website and additionally the color data from the Kitti Odometry website. Extract everything into the same folder.
./dataset/
├──
├── ...
└── SemanticKitti/
├──sequences
├── 00/
│ ├── velodyne/
| | ├── 000000.bin
| | ├── 000001.bin
| | └── ...
│ └── labels/
| | ├── 000000.label
| | ├── 000001.label
| | └── ...
| └── image_2/
| | ├── 000000.png
| | ├── 000001.png
| | └── ...
| calib.txt
├── 08/ # for validation
├── 11/ # 11-21 for testing
└── 21/
└── ...
Please download the Full dataset (v1.0) from the NuScenes website with lidarseg and extract it.
./dataset/
├──
├── ...
└── nuscenes/
├──v1.0-trainval
├──v1.0-test
├──samples
├──sweeps
├──maps
├──lidarseg
You can run the training with
cd <root dir of this repo>
python main.py --log_dir TransSeg --config config/TransSeg-nuscenes.yaml --gpu 0,1,2,3
or
bash run_kitti.shThe output will be written to logs/SemanticKITTI/2DPASS_semkitti by default.
cd <root dir of this repo>
python main.py --log_dir TransSeg --config config/TransSeg-nuscenes.yaml --gpu 0,1,2,3
or
bash run_nus.shYou can run the testing with
cd <root dir of this repo>
python main.py --log_dir TransSeg_semkitti --config config/TransSeg-semantickitti.yaml --gpu 0 --test --num_vote 12 --checkpoint <dir for the pytorch checkpoint>Here, num_vote is the number of views for the test-time-augmentation (TTA). We set this value to 12 as default (on a RTX 3090 GPU), and if you use other GPUs with smaller memory, you can choose a smaller value. num_vote=1 denotes there is no TTA used, and will cause about ~2% performance drop.
Code is built based on2DPASS, SPVNAS, Cylinder3D, xMUDA and SPCONV.
@article{zhuang2026transseg,
title={TransSeg: 3-D Semantic Segmentation Based on 2D--3D Trans-Modal Fusion for Autonomous Driving},
author={Zhuang, Mingxi and Wang, Guangming and Wang, Hesheng},
journal={IEEE Transactions on Industrial Informatics},
year={2026},
volume={22},
number={8},
pages={6789-6799},
doi={10.1109/TII.2026.3680762}
}