Mechanical Engineering Science

A Survey of Remote Sensing Image Segmentation Based on Deep Learning

SUNShibo (School of Information science and Technology, Shijiazhuang Tiedao University), ZHANGYunzuo (School of Information science and Technology, Shijiazhuang Tiedao University)

Abstract


Remote sensing image segmentation has a wide range of applications in land cover classification, urban building recognition, crop monitoring, and other fields. In recent years, with the booming development of deep learning, remote sensing image segmentation models based on deep learning have gradually emerged and produced a large number of scientific research achievements. This article is based on deep learning and reviews the latest achievements in remote sensing image segmentation, exploring future development directions. Firstly, the basic concepts, characteristics, classification, tasks, and commonly used datasets of remote sensing images are presented. Secondly, the segmentation models based on deep learning were classified and summarized, and the principles, characteristics, and applications of various models were presented. Then, the key technologies involved in deep learning remote sensing image segmentation were introduced. Finally, the future development direction and application prospects of remote sensing image segmentation were discussed. This article reviews the latest research achievements in remote sensing image segmentation from the perspective of deep learning, which can provide reference and inspiration for the research of remote sensing image segmentation.

Keywords


Remote sensing image segmentation; Deep learning; Split tasks; Model classification; Key technology

Full Text:

PDF

References


L. Huang, B. Jiang. Deep Learning-based Semantic Segmentation of Remote Sensing Images: A Survey [J]. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., 2023:1-28.

Q. An, Z. Pan. DRBox-v2: An Improved Detector With Rotatable Boxes for Target Detection in SAR Images [J]. IEEE Trans. Geosci. Remote Sens., 2019, 57(11): 8333-8349.

S. Li, W. Song. Deep Learning for Hyperspectral Image Classification: An Overview [J]. IEEE Trans. Geosci. Remote Sens., 2019, 57(9): 6690-6709.

R. Shang, M. Liu. Region-Level SAR Image Segmentation Based on Edge Feature and Label Assistance [J]. IEEE Trans. Geosci. Remote Sens., 202260: 1–16.

F. Gaudfrin, O. Pujol. A New Lidar Technique Based on Supercontinuum Laser Sources for Aerosol Soundings: Simulations and Measurements—The PERFALIS Code and the COLIBRIS Instrument [J]. IEEE Trans. Geosci. Remote Sens., 2023, 61: 1-21.

A. Ma, J. Wang. FactSeg: Foreground Activation-Driven Small Object Semantic Seg-mentation in Large-Scale Remote Sensing Imagery [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-16.

J.-H. Kim, Y. Hwang. GAN-Based Synthetic Data Augmentation for Infrared Small Target Detection [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-12.

N. Wang, B. Li. Ship Detection in Spaceborne Infrared Image Based on Lightweight CNN and Multisource Feature Cascade Decision [J]. IEEE Trans. Geosci. Remote Sens., 2021, 59(5): 4324-4339.

J. Kang, et al. DisOptNet: Distilling Semantic Knowledge From Optical Images for Weather-Independent Building Segmentation [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-15.

K. Heidler, L. Mou. HED-UNet: Combined Segmentation and Edge Detection for Monitoring the Antarctic Coastline [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-14.

Z. Lu, et al. An Iterative Classification and Semantic Segmentation Network for Old Landslide Detection Using High-Resolution Remote Sensing Images [J]. IEEE Trans. Geosci. Remote Sens., 2023, 61: 1-13.

DiResNet: Direction-Aware Residual Network for Road Extraction in VHR Remote Sensing Images.

Y. Wei, S. Ji. Scribble-Based Weakly Supervised Deep Learning for Road Surface Extraction From Remote Sensing Images [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-12.

K. Simonyan, A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition [J]. arXiv, 2015.

K. He, X. Zhang. Deep Residual Learning for Image Recognition [J]. arXiv, 2015.

C. Szegedy, et al. Going deeper with convolutions [C]. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE: Boston, MA, USA.

C. Zhang, W. Jiang. Transformer and CNN Hybrid Deep Neural Network for Semantic Segmentation of Very-High-Resolution Remote Sensing Imagery [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-20.

O. Tasar, S.L. Happy. ColorMapGAN: Unsupervised Domain Adaptation for Semantic Segmentation Using Color Mapping Generative Adversarial Networks [J]. IEEE Trans. Geosci. Remote Sens., 2020, 58(10): 7178-7193.

L. Zhang, M. Lan. Stagewise Unsupervised Domain Adaptation With Adversarial Self-Training for Road Segmentation of Remote-Sensing Images [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-13.

R. Strudel, R. Garcia. Segmenter: Transformer for Semantic Segmentation [C]. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE: Montreal, QC, Canada.

Z. Liu, et al. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows [C]. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE: Montreal, QC, Canada.

C. Zhang, L. Wang. SwinSUNet: Pure Transformer Network for Remote Sensing Image Change Detection [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-13.

Y. Zhu, S. Newsam. DenseNet for dense flow [C]. In 2017 IEEE International Conference on Image Processing (ICIP), IEEE: Beijing.

R. Li, et al. Multiattention Network for Semantic Segmentation of Fine-Resolution Remote Sensing Images [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-13.

K. Cha, J. Seo. A Billion-scale Foundation Model for Remote Sensing Images [J]. arXiv, 2023.

S. Woo, et al. ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders [J]. arXiv, 2023.

X. Ding, X. Zhang. Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs.

P. Isola, J.-Y. Zhu. Image-to-Image Translation with Conditional Adversarial Networks [J]. arXiv, 2018.

Y. Li, T. Shi. SPGAN-DA: Semantic-Preserved Generative Adversarial Network for Domain Adaptive Remote Sensing Image Semantic Segmentation [J]. IEEE Trans. Geosci. Remote Sens., 2023, 61: 1-17.

L. Zhou, H. Yu. PU-GAN: A One-Step 2-D InSAR Phase Unwrapping Based on Conditional Generative Adversarial Network [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-10.

Q. Hou, C.-Z. Lu. Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition [J]. arXiv, 2022.

Z. Liu, et al. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows [C]. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE: Montreal, QC, Canada.

D. Muhtar, X. Zhang. CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understanding [J]. IEEE Trans. Geosci. Remote Sens., 2023, 61: 1-17.

H. Zhang, et al. DGCC-EB: Deep Global Context Construction With an Enabled Boundary for Land Use Mapping of CSMA [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-15.

L. Ding, H. Tang. LANet: Local Attention Embedding to Improve the Semantic Segmentation of Remote Sensing Images [J]. IEEE Trans. Geosci. Remote Sens., 2021, 59(1): 426-435.

L.-C. Chen, G. Papandreou. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs [J]. arXiv, 2017.

M. Yang, K. Yu. DenseASPP for Semantic Segmentation in Street Scenes [C]. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, IEEE: Salt Lake City, UT, USA.

Q. Zhao, J. Liu. Semantic Segmentation With Attention Mechanism for Remote Sensing Images [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-13.

O. Ronneberger, P. Fischer. U-Net: Convolutional Networks for Biomedical Image Segmentation [J]. arXiv, 2015.

G. Lin, A. Milan. RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation [C]. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE: Honolulu, HI.

J. Fu, X. Sun. An Anchor-Free Method Based on Feature Balancing and Refinement Network for Multiscale Ship Detection in SAR Images [J]. IEEE Trans. Geosci. Remote Sens., 2021, 59(2): 1331-1344.

C. Peng, K. Zhang. Cross Fusion Net: A Fast Semantic Segmentation Network for Small-Scale Semantic Information Capturing in Aerial Scenes [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-13.

J. Zheng, A. Shao. Remote Sensing Semantic Segmentation via Boundary Supervi-sion-Aided Multiscale Channelwise Cross Attention Network [J]. IEEE Trans. Geosci. Remote Sens., 2023, 61: 1-14.

A. Li, L. Jiao. Multitask Semantic Boundary Awareness Network for Remote Sensing Image Segmentation [J]. IEEE Trans. Geosci. Remote Sens., 2022, 60: 1-14.

V. Badrinarayanan, A. Kendall. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation [J]. IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39(12): 2481-2495.

W. Liu, F. Su. Bispace Domain Adaptation Network for Remotely Sensed Semantic Segmentation [J]. IEEE Trans. Geosci. Remote Sens., 2020: 1-11.

M. Tang, K. Georgiou. Semantic Segmentation in Aerial Imagery Using Multi-level Contrastive Learning with Local Consistency [C]. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), IEEE: Waikoloa, HI, USA.

H. Xue, C. Liu. DANet: Divergent Activation for Weakly Supervised Object Localization [C]. In 2019 IEEE/CVF International Conference on Computer Vision (ICCV), IEEE: Seoul, Korea (South).

J. Hu, L. Shen. Squeeze-and-Excitation Networks [J]. arXiv, 2019.

S. Zhang, et al. Attention Guided Network for Retinal Image Segmentation [J]. arXiv, 2019.

H. Zhao, J. Shi. Pyramid Scene Parsing Network [J]. arXiv, 2017.

T.-Y. Lin, P. Dollár. Feature Pyramid Networks for Object Detection [J]. arXiv, 2017.

K. He, G. Gkioxari. Mask R-CNN [J]. arXiv, 2018.

Z. Zheng, Y. Zhong. Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery.

B. Pan, X. Xu. DSSNet: A Simple Dilated Semantic Segmentation Network for Hyperspectral Imagery Classification [J]. IEEE Geosci. Remote Sens. Lett., 2020, 17(11): 1968-1972.

H. Noh, S. Hong. Learning Deconvolution Network for Semantic Segmentation [J]. 2015.

K. He, H. Fan. Momentum Contrast for Unsupervised Visual Representation Learning [J]. arXiv, 2020.

T. Chen, S. Kornblith. A Simple Framework for Contrastive Learning of Visual Representations [J]. arXiv, 2020.

J.-B. Grill. Bootstrap your own latent: A new approach to self-supervised Learning [J]. 2020.




DOI: https://doi.org/10.33142/mes.v7i2.17492

Refbacks

  • There are currently no refbacks.


Copyright (c) 2025 Shibo SUN, Yunzuo ZHANG

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.