Skip to content

Latest commit

 

History

History
143 lines (115 loc) · 5.79 KB

README.md

File metadata and controls

143 lines (115 loc) · 5.79 KB

Solution for 2023 LOW-POWER COMPUTER VISION CHALLENGE

The solution of 2nd place (most accurate, speed within the top five) of Segmentation track in 2023 Low-Power Computer Vision Challenge (LPCVC).

Description

The model submitted for the 2023 LPCVC and implementation code for training and inference.

  • Task: Semantic Segmentation
  • Algorithm: TopFormer, Channel-wise Knowledge Distillation

Our submission

  • file solution.pyz
  • Submitted at 2023-08-02 23:19:59 EST
  • Accuracy 55.421%
  • Latency 15ms
  • Perfomance Score 36.792

Methodology

training

  • Our model selection is based on Topformer and makes some modifications upon it.

    For this competition, we modified Topformer_tiny as follows:

    1. Reduce the input resolution to 288;
    2. The number of dynamic pyramid layers in the model is reduced from 9 to 8;
    3. The number of transformer blocks is reduced from 4 to 3.

  • Then, We use Topformer_base as the teacher model to perform knowledge distillation on the modified Topformer_tiny.

post-process

  • We counted the minimum number of pixels in all labels for each type of pixel in the data set, that is, if a certain type appears, how many pixels should it occupy at least.

  • We count mutually exclusive classes, that is, those classes that cannot appear in the same label.

    With these statistics, we post-process the prediction results:

    1. If a certain class is predicted, but the number of pixels it occupies is less than the minimum value it should appear, reset this class to the background class;
    2. If there are mutually exclusive classes in the prediction result, the class with a small number of pixels in the mutually exclusive class is reset to the background class.

Model

Model Input Image Size Accuracy Latency Description Download
Topformer_base 512x512 61.1% 70ms teacher model Download Link
Topformer_tiny_modified 288x288 54.5% 15ms undistilled student model Download Link
Topformer_tiny_modified 288x288 55.4% 15ms distilled student model Download Link
Topformer_tiny_encoder - - - backbone Download Link

Training Environment

Hardware

  • 6x A40 32GB

Software

  • Ubuntu

Main Python Packages

  • python 3.8.8
  • pytorch 1.8.0
  • mmcv-full==1.4.0
  • mmsegmentation==0.19.0
  • mmrazor==0.3.1
  • mmcls==0.19.0
  • mmdeploy==0.14.0

Installation

pip install -U openmim
mim install 'mmengine==0.7.3'
mim install "mmcv-full==1.4.0"
pip install mmsegmentation==0.19.0
pip install mmrazor==0.3.1
pip install mmcls==0.19.0
pip install mmdeploy==0.14.0

under the path of the source program

pip install -v -e .

Training

Step 1: Dataset path

Set the dataset path in local_configs/_base_/datasets/LPCVC.py->data_root.

Set the dataset path in local_configs/_base_/datasets/LPCVC_distill.py->data_root.

Step 2: Train

Train your model by by following command:

python tools_mmseg/train.py local_configs/topformer/topformer_tiny_288x288_160k_2x8_ade20k.py --work-dir <path-to-save-checkpoints>

Step 3: Select the best model as the student model

Select the model with the highest accuracy from all checkpoints and use it as the student model by following command:

python tools_mmseg/mytest.py local_configs/topformer/topformer_tiny_288x288_160k_2x8_ade20k.py --checkpoint <checkpoint-path> --eval mDice

Step 4: Knowledge distillation

Modify the distillation configuration file: local_configs/distill/cwd_seg_topformer_512b_distill_288t.py->student_checkpoint.

Then, run the following command:

python tools_mmraz/mmseg/train_mmseg.py local_configs/distill/cwd_seg_topformer_512b_distill_288t.py --work-dir <path-to-save-checkpoints>

Select the model with the highest accuracy from all checkpoints by following command:

python tools_mmraz/mmseg/mytest_mmseg.py local_configs/distill/cwd_seg_topformer_512b_distill_288t.py --checkpoint <checkpoint-path> --eval mDice

Step 5: Convert to ONNX

Modify the path of the best model into the python file: split_mmrazor_pth.py->cls_model_path.

Then, run split_mmrazor_pth.py, You can get a new pth file. Convert this pth file to onnx file by following command:

python tools_mmdep/deploy.py local_configs/deploy/segmentation_onnxruntime_static-288x288.py local_configs/topformer/topformer_tiny_288x288_160k_2x8_ade20k.py <checkpoint-path> <dummy-data-path> --work-dir <path-to-save-onnx>

Inference

Goto inference readme for inference detail.

License

Apache License 2.0