Skip to content

Commit 4615bd9

Browse files
Bob ChengBob Cheng
Bob Cheng
authored and
Bob Cheng
committed
init
0 parents  commit 4615bd9

File tree

1,181 files changed

+202384
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,181 files changed

+202384
-0
lines changed

‎.DS_Store

8 KB
Binary file not shown.

‎.gitignore

+6
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
vis
2+
data
3+
__pycache__
4+
*.pth
5+
transfusion/mmdet3d_tf
6+
temp*.png

‎DeepInteraction/.DS_Store

6 KB
Binary file not shown.

‎DeepInteraction/.gitignore

+135
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
# Byte-compiled / optimized / DLL files
2+
__pycache__/
3+
*.py[cod]
4+
*$py.class
5+
*.ipynb
6+
7+
# C extensions
8+
*.so
9+
10+
# Distribution / packaging
11+
.Python
12+
build/
13+
develop-eggs/
14+
dist/
15+
downloads/
16+
eggs/
17+
.eggs/
18+
lib/
19+
lib64/
20+
parts/
21+
sdist/
22+
var/
23+
wheels/
24+
*.egg-info/
25+
.installed.cfg
26+
*.egg
27+
MANIFEST
28+
29+
# PyInstaller
30+
# Usually these files are written by a python script from a template
31+
# before PyInstaller builds the exe, so as to inject date/other infos into it.
32+
*.manifest
33+
*.spec
34+
35+
# Installer logs
36+
pip-log.txt
37+
pip-delete-this-directory.txt
38+
39+
# Unit test / coverage reports
40+
htmlcov/
41+
.tox/
42+
.coverage
43+
.coverage.*
44+
.cache
45+
nosetests.xml
46+
coverage.xml
47+
*.cover
48+
.hypothesis/
49+
.pytest_cache/
50+
51+
# Translations
52+
*.mo
53+
*.pot
54+
55+
# Django stuff:
56+
*.log
57+
local_settings.py
58+
db.sqlite3
59+
60+
# Flask stuff:
61+
instance/
62+
.webassets-cache
63+
64+
# Scrapy stuff:
65+
.scrapy
66+
67+
# Sphinx documentation
68+
docs/_build/
69+
70+
# PyBuilder
71+
target/
72+
73+
# Jupyter Notebook
74+
.ipynb_checkpoints
75+
76+
# pyenv
77+
.python-version
78+
79+
# celery beat schedule file
80+
celerybeat-schedule
81+
82+
# SageMath parsed files
83+
*.sage.py
84+
85+
# Environments
86+
.env
87+
.venv
88+
env/
89+
venv/
90+
ENV/
91+
env.bak/
92+
venv.bak/
93+
94+
# Spyder project settings
95+
.spyderproject
96+
.spyproject
97+
98+
# Rope project settings
99+
.ropeproject
100+
101+
# mkdocs documentation
102+
/site
103+
104+
# mypy
105+
.mypy_cache/
106+
107+
# cython generated cpp
108+
data
109+
pretrained
110+
.vscode
111+
.idea
112+
113+
# custom
114+
*.pkl
115+
*.pkl.json
116+
*.log.json
117+
work_dirs/
118+
exps/
119+
*~
120+
mmdet3d/.mim
121+
122+
# Pytorch
123+
*.pth
124+
125+
# demo
126+
*.jpg
127+
*.png
128+
data/s3dis/Stanford3dDataset_v1.2_Aligned_Version/
129+
data/scannet/scans/
130+
data/sunrgbd/OFFICIAL_SUNRGBD/
131+
*.obj
132+
*.ply
133+
134+
# Waymo evaluation
135+
mmdet3d/core/evaluation/waymo_utils/compute_detection_metrics_main

‎DeepInteraction/LICENSE

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2022 Fudan Zhang Vision Group
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎DeepInteraction/README.md

+64
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# DeepInteraction: 3D Object Detection via Modality Interaction
2+
### [Paper](https://arxiv.org/abs/2208.11112)
3+
> [**DeepInteraction: 3D Object Detection via Modality Interaction**](https://arxiv.org/abs/2208.11112),
4+
> Zeyu Yang, Jiaqi Chen, Zhenwei Miao, [Wei Li](https://weivision.github.io/), [Xiatian Zhu](https://xiatian-zhu.github.io), [Li Zhang](https://www.robots.ox.ac.uk/~lz)
5+
6+
## News
7+
8+
- **(2022/6/27)** DeepInteraction-e ranks first on [nuScenes](https://nuscenes.org/object-detection?externalData=all&mapData=all&modalities=Any) among all solutions.
9+
- **(2022/6/26)** DeepInteraction-large ranks first on [nuScenes](https://nuscenes.org/object-detection?externalData=all&mapData=all&modalities=Any) among all non-ensemble solutions.
10+
- **(2022/5/18)** DeepInteraction-base ranks first on [nuScenes](https://nuscenes.org/object-detection?externalData=all&mapData=all&modalities=Any) among all solutions that do not use test-time augmentation and model ensemble.
11+
12+
13+
## Results
14+
15+
### 3D Object Detection (on nuScenes test)
16+
17+
| Model | Modality | mAP | NDS |
18+
| :-------: | :------: | :--: | :--: |
19+
| DeepInteraction-e | C+L | 75.74 | 76.34 |
20+
| DeepInteraction-large | C+L | 74.12 | 75.52 |
21+
| DeepInteraction-base | C+L | 70.78 | 73.43 |
22+
23+
### 3D Object Detection (on nuScenes val)
24+
25+
| Model | Modality | mAP | NDS | Checkpoint |
26+
| :-------: | :------: | :--: | :--: | :--------: |
27+
| [DeepInteraction-base](projects/configs/nuscenes/Fusion_0075_refactor.py) | C+L | 69.85 | 72.63 | [Fusion_0075_refactor.pth](https://drive.google.com/file/d/1M5eUlXZ8HJ--J53y0FoAHn1QpZGowsdc/view?usp=sharing) |
28+
29+
## Get Started
30+
31+
### Environment
32+
This implementation is build upon [mmdetection3d](https://github.com/open-mmlab/mmdetection3d), please follow the steps in [install.md](./install.md) to prepare the environment.
33+
34+
### Data
35+
Please follow the official instructions of mmdetection3d to process the nuScenes dataset.(https://mmdetection3d.readthedocs.io/en/latest/datasets/nuscenes_det.html)
36+
37+
### Pretrained
38+
Downloads the [pretrained backbone weights](https://drive.google.com/file/d/1IaLMcRu4SYTqcD6K1HF5UjfnRICB_IQM/view?usp=sharing) to pretrained/
39+
40+
### Train & Test
41+
```shell
42+
# train with 8 GPUs
43+
tools/dist_train.sh projects/configs/nuscenes/Fusion_0075_refactor.py 8
44+
# test with 8 GPUs
45+
tools/dist_test.sh projects/configs/nuscenes/Fusion_0075_refactor.py ${CHECKPOINT_FILE} 8 --eval=bbox
46+
```
47+
48+
## Acknowledgement
49+
Many thanks to the following open-source projects:
50+
* [mmdetection3d](https://github.com/open-mmlab/mmdetection3d)
51+
* [detectron2](https://github.com/facebookresearch/detectron2)
52+
* [transfusion](https://github.com/XuyangBai/TransFusion)
53+
54+
55+
## Reference
56+
57+
```bibtex
58+
@inproceedings{yang2022deepinteraction,
59+
title={DeepInteraction: 3D Object Detection via Modality Interaction},
60+
author={Yang, Zeyu and Chen, Jiaqi and Miao, Zhenwei and Li, Wei and Zhu, Xiatian and Zhang, Li},
61+
booktitle={NeurIPS},
62+
year={2022}
63+
}
64+
```

‎DeepInteraction/install.md

+36
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
# Installation
2+
## Prerequisite
3+
1. `Python=3.7`
4+
2. `Pytorch=1.9.1`
5+
3. `CUDA=11.1`
6+
7+
### **Step. 1** Install mmcv-full==1.3.18.
8+
```
9+
pip install mmcv-full==1.3.18
10+
```
11+
12+
### **Step. 2** Install mmdetection and mmsegmentation.
13+
```
14+
pip install mmdet==2.14.0
15+
pip install mmsegmentation==0.14.1
16+
```
17+
18+
### **Step. 3** Install mmdetection3d==0.17.1.
19+
```
20+
git clone https://github.com/open-mmlab/mmdetection3d.git
21+
cd mmdetection3d
22+
git checkout v0.17.1
23+
python setup.py install
24+
```
25+
26+
### **Step. 4** Install detectron2.
27+
```
28+
python -m pip install detectron2 -f \
29+
https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/index.html
30+
```
31+
32+
### **Step. 5** Clone deepInteraction.
33+
```
34+
git clone https://github.com/fudan-zvg/DeepInteraction.git
35+
cd DeepInteraction
36+
```

0 commit comments

Comments
 (0)