Skip to content

๐Ÿš€ Code for SLRNet: Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation

License

Notifications You must be signed in to change notification settings

DesertsP/SLRNet

Folders and files

NameName
Last commit message
Last commit date

Latest commit

ย 

History

5 Commits
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 
ย 

Repository files navigation

Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation

License Framework

This is the official implementation of Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation, arXiv, IJCV 2022.

This repository contains the code for SLRNet, which is a unified framework that can be well generalized to learn a label-efficient segmentation model in various weakly and semi-supervised settings. The key component of our approach is the Cross-View Low-Rank (CVLR) module that decompose the multi-view representations via the collective matrix factorization. We provide scripts for Pascal VOC and COCO datasets. Moreover, the SLRNet ranked 2nd in the WSSS Track of CVPR 2021 Learning from Limited and Imperfect Data (L2ID) Challenge.

Setup

  1. Minimum requirements. This project was developed with Python 3.7, PyTorch 1.x. The training requires at least two Titan XP GPUs (12 GB memory each).

  2. Setup your Python environment. Install python dependencies in requirements.txt

    pip install -r requirements.txt
    
  3. Download Datasets.

    Download Pascal VOC data from:

    • VOC: Training/Validation (2GB .tar file)

    • SBD: Training (1.4GB .tgz file)

      Convert SBD data using tools/convert_coco.py.

      Link to the data:

      ln -s <your_path_to_voc> <project>/data/voc
      ln -s <your_path_to_sbd> <project>/data/sbd
      

      Make sure that the first directory in data/voc is VOCdevkit; the first directory in data/sbd is benchmark_RELEASE.

    Download COCO data from:

  4. Download pre-trained backbones.

    Backbone Initial Weights Comment
    WideResNet38 ilsvrc-cls_rna-a1_cls1000_ep-0001.pth (402M) Converted from mxnet
    ResNet101 resnet101-5d3b4d8f.pth PyTorch official

Pre-trained Model

For testing, we provide our checkpoints on Pascal VOC dataset:

Setting Backbone Val Link
Weakly-sup. w/ image-level label WideResNet38 67.2 (w/ CRF) link
Weakly-sup. w/ image-level label (Two-stage) ResNet101 69.3 (w/o CRF)
Semi-sup. w/ image-level label WideResNet38 75.1 (w/o CRF) link
Semi-sup. w/o image-level label WideResNet38 72.4 (w/o CRF) link

Weakly Supervised Semantic Segmentation

Training

Train the weakly supervised model:

python tools/train_voc_wss.py --config experiments/slrnet_voc_wss_v100x2.yaml --run custom_experiment_run_id

For the COCO or L2ID dataset, please refer to the relevant scripts/configs with the suffix "_coco".

Inference and Evaluation

python tools/infer_voc.py \
       --config experiments/slrnet_voc_wss_v100x2.yaml \
       --checkpoint path/to/checkpoint20.pth.tar \
       --use_cls_label 0 \
       --output outputs/wss_prediction \
       --data_list voc12/val.txt \
       --data_root path/to/VOCdevkit/VOC2012 \
       --fp_cut 0.3 \
       --bg_pow 3

Two-stage Training

Generate pseudo labels with following parameters:

...
       --data_list voc12/train_aug.txt \
       --use_cls_label 1 \
...

Then we train the DeepLabV3+ (w/ ResNet101 backbone) network implemented by mmsegmentation

Semi-Supervised Semantic Segmentation with Pixel-level and Image-level Labeled Data

Training

python tools/train_voc_semi.py --config experiments/slrnet_voc_semi_w_cls_v100x2.yaml --run custom_experiment_run_id

Inference and Evaluation

python tools/infer_voc.py \
       --config experiments/slrnet_voc_semi_w_cls_v100x2.yaml \
       --checkpoint path/to/checkpoint28.pth.tar \
       --use_cls_label 0 \
       --output outputs/semi_val_prediction \
       --data_list voc12/val.txt \
       --data_root ../VOCdevkit/VOC2012 \
       --fp_cut 0.3 \
       --bg_pow 1 \
       --apply_crf 0 \
       --verbose 0

Semi-Supervised Semantic Segmentation with Pixel-level Labeled and Unlabeled Data

Training

python tools/train_voc_semi.py --config experiments/slrnet_voc_semi_wo_cls.yaml --run custom_experiment_run_id

Inference and Evaluation

python tools/infer_voc.py \
       --config experiments/slrnet_voc_semi_wo_cls.yaml \
       --checkpoint path/to/checkpoint32.pth.tar \
       --use_cls_label 0 \
       --output outputs/semi_val_prediction \
       --data_list voc12/val.txt \
       --data_root ../VOCdevkit/VOC2012 \
       --fp_cut 0.3 \
       --bg_pow 1 \
       --apply_crf 0 \
       --verbose 0

Acknowledgements

We thank Nikita Araslanov, and Jiwoon Ahn for their great work that helped in the early stages of this project.

Citation

We hope that you find this work useful. If you would like to acknowledge us, please, use the following citation:

@article{pan2022learning,
  title={Learning Self-supervised Low-Rank Network for Single-Stage Weakly and Semi-supervised Semantic Segmentation},
  author={Pan, Junwen and Zhu, Pengfei and Zhang, Kaihua and Cao, Bing and Wang, Yu and Zhang, Dingwen and Han, Junwei and Hu, Qinghua},
  journal={International Journal of Computer Vision},
  pages={1--15},
  year={2022},
  publisher={Springer}
}

Contact

Junwen Pan [email protected]

About

๐Ÿš€ Code for SLRNet: Learning Self-Supervised Low-Rank Network for Single-Stage Weakly and Semi-Supervised Semantic Segmentation

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages