A Bridge Neural Network-Based Optical-SAR Image Joint Intelligent Interpretation Framework
Open Access
- 1 January 2021
- journal article
- Published by American Association for the Advancement of Science (AAAS) in Space: Science & Technology
Abstract
The current interpretation technology of remote sensing images is mainly focused on single-modal data, which cannot fully utilize the complementary and correlated information of multimodal data with heterogeneous characteristics, especially for synthetic aperture radar (SAR) data and optical imagery. To solve this problem, we propose a bridge neural network- (BNN-) based optical-SAR image joint intelligent interpretation framework, optimizing the feature correlation between optical and SAR images through optical-SAR matching tasks. It adopts BNN to effectively improve the capability of common feature extraction of optical and SAR images and thus improving the accuracy and application scenarios of specific intelligent interpretation tasks for optical-SAR/SAR/optical images. Specifically, BNN projects optical and SAR images into a common feature space and mines their correlation through pair matching. Further, to deeply exploit the correlation between optical and SAR images and ensure the great representation learning ability of BNN, we build the QXS-SAROPT dataset containing 20,000 pairs of perfectly aligned optical-SAR image patches with diverse scenes of high resolutions. Experimental results on optical-to-SAR crossmodal object detection demonstrate the effectiveness and superiority of our framework. In particular, based on the QXS-SAROPT dataset, our framework can achieve up to 96 high accuracy on four benchmark SAR ship detection datasets.Keywords
Funding Information
- National Natural Science Foundation of China (61702520)
- Beijing Nova Program (Z191100001119129)
This publication has 30 references indexed in Scilit:
- Deep Learning for Remote Sensing Data: A Technical Tutorial on the State of the ArtIEEE Geoscience and Remote Sensing Magazine, 2016
- Learning to compare image patches via convolutional neural networksPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2015
- ImageNet Large Scale Visual Recognition ChallengeInternational Journal of Computer Vision, 2015
- Deep learning in neural networks: An overviewNeural Networks, 2015
- Speeding up Convolutional Neural Networks with Low Rank ExpansionsPublished by British Machine Vision Association and Society for Pattern Recognition ,2014
- Deep convolutional neural networks for LVCSRPublished by Institute of Electrical and Electronics Engineers (IEEE) ,2013
- Estimation of Mutual Information: A SurveyLecture Notes in Computer Science, 2009
- Compressive Radar Imaging2007 IEEE Radar Conference, 2007
- Model compressionPublished by Association for Computing Machinery (ACM) ,2006
- Distinctive Image Features from Scale-Invariant KeypointsInternational Journal of Computer Vision, 2004