Skip to content

Commit 54c07cf

Browse files
KaihuaTangKaihuaTang
committed
no message
1 parent e842506 commit 54c07cf

File tree

1,294 files changed

+429100
-2
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,294 files changed

+429100
-2
lines changed

‎README.md

+83-2
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,84 @@
1-
# A General Single-Stage Solution for Long-Tailed Problems
1+
# A Strong Single-Stage Baseline for Long-Tailed Problems
2+
3+
[![LICENSE](https://img.shields.io/badge/license-MIT-green)](https://github.com/KaihuaTang/Scene-Graph-Benchmark.pytorch/blob/master/LICENSE)
4+
[![Python](https://img.shields.io/badge/python-3.7-blue.svg)](https://www.python.org/)
5+
![PyTorch](https://img.shields.io/badge/pytorch-1.6.0-%237732a8)
6+
7+
This project provides a strong single-stage baseline for Long-Tailed Classification (under ImageNet-LT, Long-Tailed CIFAR-10/-100 datasets), Detection, and Instance Segmentation (under LVIS dataset). It is also a PyTorch implementation of the paper [Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect](), which proposes a general solution to remove the bad momentum causal effect for a variety of Long-Tailed Recognition tasks. The codes are organized into three folders:
8+
1. The [classification folder](classification) supports long-tailed classification on ImageNet-LT, Long-Tailed CIFAR-10/CIFAR-100 datasets.
9+
2. The [lvis_old folder (deprecated)](lvis_old) supports long-tailed object detection and instance segmentation on LVIS V0.5 dataset, which is built on top of mmdet V1.1.
10+
3. The latest version of long-tailed detection and instance segmentation is under [lvis1.0 folder](lvis1.0). Since both LVIS V0.5 and mmdet V1.1 are no longer available on their homepages, we have to re-implement our method on [mmdet V2.4](https://github.com/open-mmlab/mmdetection) using [LVIS V1.0 annotations](https://www.lvisdataset.org/dataset).
11+
12+
13+
# Installation
14+
The classification part allows the lower version of the following requirements. However, in detection and instance segmentation (mmdet V2.4), I tested some lower versions of python and pytorch, which are all failed. If you want to try other environments, please check the updates of [mmdetection](https://github.com/open-mmlab/mmdetection).
15+
16+
### Requirements:
17+
- PyTorch >= 1.6.0
18+
- Python >= 3.7.0
19+
- CUDA >= 10.1
20+
- torchvision >= 0.7.0
21+
- gcc version >= 5.4.0
22+
23+
### Step-by-step installation
24+
```bash
25+
conda create -n longtail pip python=3.7 -y
26+
source activate longtail
27+
conda install pytorch torchvision cudatoolkit=10.1 -c pytorch
28+
pip install pyyaml
29+
30+
# download the project
31+
git clone https://github.com/KaihuaTang/Long-Tail-Classification.private.git
32+
cd Long-Tail-Classification.private.git
33+
34+
# the following part is only used to build mmdetection
35+
cd lvis1.0
36+
pip install mmcv-full
37+
pip install mmlvis
38+
pip install -r requirements/build.txt
39+
pip install -v -e . # or "python setup.py develop"
40+
```
41+
### Additional Notes
42+
When we wrote the paper, we are using lvis V0.5 and mmdet V1.1 for our long-tailed instance segmentation experiments, but they've been deprecated by now. If you want to reproduce our results on lvis V0.5, you have to find a way to build mmdet V1.1 environments and use the code in lvis_old folder.
43+
44+
45+
# Datasets
46+
### ImageNet-LT
47+
ImageNet-LT is a long-tailed subset of original ImageNet, you can download the dataset from its [homepage](http://image-net.org/index). After you download the dataset, you need to change the data_root of 'ImageNet' in [./classification/main.py](classification/main.py) file.
48+
49+
### CIFAR-10/-100
50+
When you run the code for the first time, our dataloader will automatically download the CIFAR-10/-100. You need to set the data_root in [./classification/main.py](classification/main.py) to the path where you want to put all CIFAR data.
51+
52+
### LVIS
53+
[Large Vocabulary Instance Segmentation (LVIS)](https://www.lvisdataset.org/) dataset uses the COCO 2017 train, validation, and test image sets. If you have already downloaded the COCO images, you only need to download the LVIS annotations. LVIS val set contains images from COCO 2017 train in addition to the COCO 2017 val split.
54+
55+
You need to put all the annotations and images under ./data/LVIS like this:
56+
```bash
57+
data
58+
|-- LVIS
59+
|--lvis_v1_train.json
60+
|--lvis_v1_val.json
61+
|--images
62+
|--train2017
63+
|--.... (images)
64+
|--test2017
65+
|--.... (images)
66+
|--val2017
67+
|--.... (images)
68+
```
69+
70+
# Getting Started
71+
For long-tailed classification, please go to [\[link\]](classification)
72+
73+
For long-tailed object detection and instance segmentation, please go to [\[link\]](lvis1.0)
74+
75+
76+
# Advantages of the Proposed Method
77+
- Compared with previous state-of-the-art [Decoupling](https://github.com/facebookresearch/classifier-balancing), our method only requires one-stage training.
78+
- Most of the existing methods for long-tailed problems are using data distribution to conduct re-sampling or re-weighting during training, which is based on an inappropriate assumption that you can know the future distribution before you start to learn. Meanwhile, the proposed method doesn't need to know the data distribution during training, we only need to use an average feature for inference after we train the model.
79+
- Our method can be easily transferred to any tasks. We outperform the previous state-of-the-arts [Decoupling](https://arxiv.org/abs/1910.09217), [BBN](https://arxiv.org/abs/1912.02413), [OLTR](https://arxiv.org/abs/1904.05160) in image classification, and we achieve better results than 2019 Winner of LVIS challenge [EQL](https://arxiv.org/abs/2003.05176) in long-tailed object detection and instance segmentation (under the same settings with even fewer GPUs).
80+
81+
82+
83+
284

3-
Release Soon

‎classification/.gitignore

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
logs
2+
*pkl
3+
*pyc
4+
.vscode
5+
*.sh
6+
runs
7+
.vscode

‎classification/README.md

+64
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,64 @@
1+
# Long-Tailed image Classification
2+
3+
This project is built on top of [Decoupling Representation and Classifier for Long-Tailed Recognition (ICLR 2020)](https://github.com/facebookresearch/classifier-balancing). The main body of the proposed Causal-TDE is under [./models/CausalNormClassifier.py](models/CausalNormClassifier.py) and [run_networks.py](run_networks.py)
4+
5+
6+
### Training
7+
For ImageNet_LT:
8+
```bash
9+
python main.py --cfg ./config/ImageNet_LT/causal_norm.yaml
10+
```
11+
12+
For Long-Tailed CIFAR-10 using ResNet32:
13+
```bash
14+
python main.py --cfg ./config/CIFAR10_LT/causal_norm_32.yaml
15+
```
16+
17+
For Long-Tailed CIFAR-10 using ResNext50:
18+
```bash
19+
python main.py --cfg ./config/CIFAR10_LT/causal_norm.yaml
20+
```
21+
22+
For Long-Tailed CIFAR-100 using ResNet32:
23+
```bash
24+
python main.py --cfg ./config/CIFAR100_LT/causal_norm_32.yaml
25+
```
26+
27+
For Long-Tailed CIFAR-100 using ResNext50:
28+
```bash
29+
python main.py --cfg ./config/CIFAR100_LT/causal_norm.yaml
30+
```
31+
32+
If you want to change any hyper-parameter, you can find them in the corresponding yaml config file. **IMPORTANT: if you just want to change the TDE trade-off parameter alpha, you don't need to re-train the model, you can directly use different alphas during testing, because it's not involved in training**.
33+
34+
### Testing
35+
For ImageNet_LT:
36+
```bash
37+
python main.py --cfg ./config/ImageNet_LT/causal_norm.yaml --test --model_dir ./logs/ImageNet_LT/models/your_model
38+
```
39+
40+
For Long-Tailed CIFAR-10 using ResNet32:
41+
```bash
42+
python main.py --cfg ./config/CIFAR10_LT/causal_norm_32.yaml --test --model_dir ./logs/ImageNet_LT/models/your_model
43+
```
44+
45+
For Long-Tailed CIFAR-10 using ResNext50:
46+
```bash
47+
python main.py --cfg ./config/CIFAR10_LT/causal_norm.yaml --test --model_dir ./logs/ImageNet_LT/models/your_model
48+
```
49+
50+
For Long-Tailed CIFAR-100 using ResNet32:
51+
```bash
52+
python main.py --cfg ./config/CIFAR100_LT/causal_norm_32.yaml --test --model_dir ./logs/ImageNet_LT/models/your_model
53+
```
54+
55+
For Long-Tailed CIFAR-100 using ResNext50:
56+
```bash
57+
python main.py --cfg ./config/CIFAR100_LT/causal_norm.yaml --test --model_dir ./logs/ImageNet_LT/models/your_model
58+
```
59+
60+
### Results
61+
62+
![alt text](imagenet-lt.png "from 'Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect'")
63+
64+
![alt text](long-tailed-cifar.png "from 'Long-Tailed Classification by Keeping the Good and Removing the Bad Momentum Causal Effect'")

‎classification/config/CIFAR100_LT/causal_norm.yaml

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# default num_head = 2
2+
criterions:
3+
PerformanceLoss:
4+
def_file: ./loss/SoftmaxLoss.py
5+
loss_params: {}
6+
optim_params: null
7+
weight: 1.0
8+
last: false
9+
# apply incremental pca to remove main components
10+
apply_ipca: false
11+
num_components: 512
12+
model_dir: null
13+
tuning_memory: false
14+
networks:
15+
classifier:
16+
def_file: ./models/CausalNormClassifier.py
17+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
18+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
19+
params: {dataset: CIFAR100_LT, feat_dim: 2048, num_classes: 100, stage1_weights: false, use_effect: true, num_head: 2, tau: 16.0, alpha: 2.0, gamma: 0.03125}
20+
feat_model:
21+
def_file: ./models/ResNext50Feature.py
22+
fix: false
23+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
24+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
25+
params: {dataset: CIFAR100_LT, dropout: null, stage1_weights: false, use_fc: false}
26+
shuffle: false
27+
training_opt:
28+
backbone: resnext50
29+
batch_size: 512
30+
dataset: CIFAR100_LT
31+
display_step: 10
32+
display_grad: False
33+
display_grad_step: 10
34+
feature_dim: 2048
35+
log_dir: ./logs/CIFAR100_LT/models/resnext50_e90_causal_norm_ratio100_head2_scale16_norm32
36+
log_root: /logs/CIFAR100_LT
37+
num_classes: 100
38+
cifar_imb_ratio: 0.01 # 0.01, 0.02, 0.1 for 100, 50, 10
39+
num_epochs: 90
40+
num_workers: 12
41+
open_threshold: 0.1
42+
sampler: null
43+
sub_dir: models

‎classification/config/CIFAR100_LT/causal_norm_32.yaml

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# default num_head = 2
2+
criterions:
3+
PerformanceLoss:
4+
def_file: ./loss/SoftmaxLoss.py
5+
loss_params: {}
6+
optim_params: null
7+
weight: 1.0
8+
last: false
9+
# apply incremental pca to remove main components
10+
apply_ipca: false
11+
num_components: 512
12+
model_dir: null
13+
tuning_memory: false
14+
networks:
15+
classifier:
16+
def_file: ./models/CausalNormClassifier.py
17+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
18+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
19+
params: {dataset: CIFAR100_LT, feat_dim: 128, num_classes: 100, stage1_weights: false, use_effect: true, num_head: 2, tau: 16.0, alpha: 2.0, gamma: 0.03125}
20+
feat_model:
21+
def_file: ./models/ResNet32Feature.py
22+
fix: false
23+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
24+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
25+
params: {dataset: CIFAR100_LT, dropout: null, stage1_weights: false, use_fc: false}
26+
shuffle: false
27+
training_opt:
28+
backbone: resnet32
29+
batch_size: 512
30+
dataset: CIFAR100_LT
31+
display_step: 10
32+
display_grad: False
33+
display_grad_step: 10
34+
feature_dim: 128
35+
log_dir: ./logs/CIFAR100_LT/models/resnet32_e90_causal_norm_ratio100_head2_scale16_norm32
36+
log_root: /logs/CIFAR100_LT
37+
num_classes: 100
38+
cifar_imb_ratio: 0.01 # 0.01, 0.02, 0.1 for 100, 50, 10
39+
num_epochs: 90
40+
num_workers: 12
41+
open_threshold: 0.1
42+
sampler: null
43+
sub_dir: models

‎classification/config/CIFAR100_LT/cls_lws.yaml

+44
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
coslr: true
2+
criterions:
3+
PerformanceLoss:
4+
def_file: ./loss/SoftmaxLoss.py
5+
loss_params: {}
6+
optim_params: null
7+
weight: 1.0
8+
endlr: 0.0
9+
last: false
10+
memory: {centroids: false, init_centroids: false}
11+
model_dir: ./logs/CIFAR100_LT/models/resnet32_uniform_e90_ratio100
12+
networks:
13+
classifier:
14+
def_file: ./models/TauNormClassifier.py
15+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
16+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
17+
params: {dataset: CIFAR100_LT, feat_dim: 128, log_dir: ./logs/CIFAR100_LT/clslearn/lws_uni2ban_ratio100,
18+
num_classes: 100, stage1_weights: false}
19+
feat_model:
20+
def_file: ./models/ResNet32Feature.py
21+
fix: true
22+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
23+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
24+
params: {dataset: CIFAR100_LT, dropout: null, log_dir: ./logs/CIFAR100_LT/clslearn/lws_uni2ban_ratio100,
25+
stage1_weights: false, use_fc: false}
26+
shuffle: true
27+
training_opt:
28+
backbone: resnet32
29+
batch_size: 512
30+
dataset: CIFAR100_LT
31+
display_step: 10
32+
display_grad: False
33+
display_grad_step: 10
34+
feature_dim: 128
35+
log_dir: ./logs/CIFAR100_LT/clslearn/lws_uni2ban_ratio100
36+
log_root: ./logs/CIFAR100_LT
37+
num_classes: 100
38+
cifar_imb_ratio: 0.01 # 0.01, 0.02, 0.1 for 100, 50, 10
39+
num_epochs: 5
40+
num_workers: 12
41+
open_threshold: 0.1
42+
sampler: {def_file: ./data/ClassAwareSampler.py, num_samples_cls: 4, type: ClassAwareSampler}
43+
stage: lws_uni2ban_ratio100
44+
sub_dir: clslearn

‎classification/config/CIFAR100_LT/feat_unifrom.yaml

+44
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
coslr: true
2+
criterions:
3+
PerformanceLoss:
4+
def_file: ./loss/SoftmaxLoss.py
5+
loss_params: {}
6+
optim_params: null
7+
weight: 1.0
8+
endlr: 0.0
9+
last: false
10+
memory: {centroids: false, init_centroids: false}
11+
model_dir: null
12+
networks:
13+
classifier:
14+
def_file: ./models/DotProductClassifier.py
15+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
16+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
17+
params: {dataset: CIFAR100_LT, feat_dim: 128, log_dir: ./logs/CIFAR100_LT/models/resnet32_uniform_e90_ratio100,
18+
num_classes: 100, stage1_weights: false}
19+
feat_model:
20+
def_file: ./models/ResNet32Feature.py
21+
fix: false
22+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0001}
23+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
24+
params: {dataset: CIFAR100_LT, dropout: null, log_dir: ./logs/CIFAR100_LT/models/resnet32_uniform_e90_ratio100,
25+
stage1_weights: false, use_fc: false}
26+
shuffle: false
27+
training_opt:
28+
backbone: resnet32
29+
batch_size: 512
30+
dataset: CIFAR100_LT
31+
display_step: 10
32+
display_grad: False
33+
display_grad_step: 10
34+
feature_dim: 128
35+
log_dir: ./logs/CIFAR100_LT/models/resnet32_uniform_e90_ratio100
36+
log_root: ./logs/CIFAR100_LT
37+
num_classes: 100
38+
cifar_imb_ratio: 0.01 # 0.01, 0.02, 0.1 for 100, 50, 10
39+
num_epochs: 90
40+
num_workers: 12
41+
open_threshold: 0.1
42+
sampler: null
43+
stage: resnet32_uniform_e90_ratio100
44+
sub_dir: models

‎classification/config/CIFAR10_LT/causal_norm.yaml

+43
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,43 @@
1+
# default num_head = 2
2+
criterions:
3+
PerformanceLoss:
4+
def_file: ./loss/SoftmaxLoss.py
5+
loss_params: {}
6+
optim_params: null
7+
weight: 1.0
8+
last: false
9+
# apply incremental pca to remove main components
10+
apply_ipca: false
11+
num_components: 512
12+
model_dir: null
13+
tuning_memory: false
14+
networks:
15+
classifier:
16+
def_file: ./models/CausalNormClassifier.py
17+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
18+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
19+
params: {dataset: CIFAR10_LT, feat_dim: 2048, num_classes: 10, stage1_weights: false, use_effect: true, num_head: 2, tau: 16.0, alpha: 1.0, gamma: 0.03125}
20+
feat_model:
21+
def_file: ./models/ResNext50Feature.py
22+
fix: false
23+
optim_params: {lr: 0.2, momentum: 0.9, weight_decay: 0.0005}
24+
scheduler_params: {coslr: true, endlr: 0.0, gamma: 0.1, step_size: 30}
25+
params: {dataset: CIFAR10_LT, dropout: null, stage1_weights: false, use_fc: false}
26+
shuffle: false
27+
training_opt:
28+
backbone: resnext50
29+
batch_size: 512
30+
dataset: CIFAR10_LT
31+
display_step: 10
32+
display_grad: False
33+
display_grad_step: 10
34+
feature_dim: 2048
35+
log_dir: ./logs/CIFAR10_LT/models/resnext50_e90_causal_norm_ratio100_head2_scale16_norm32
36+
log_root: /logs/CIFAR10_LT
37+
num_classes: 10
38+
cifar_imb_ratio: 0.01 # 0.01, 0.02, 0.1 for 100, 50, 10
39+
num_epochs: 90
40+
num_workers: 12
41+
open_threshold: 0.1
42+
sampler: null
43+
sub_dir: models

0 commit comments

Comments
 (0)