(Publisher of Peer Reviewed Open Access Journals)

International Journal of Advanced Technology and Engineering Exploration (IJATEE)

ISSN (Print):2394-5443    ISSN (Online):2394-7454
Volume-10 Issue-101 April-2023
Full-Text PDF
Paper Title : Feature based image registration using CNN features for satellite images having varying illumination level
Author Name : Laukikkumar K. Patel and Manish I. Patel
Abstract :

Many times, meaningful information is derived from the image fusion. In applications like change detection, cancer growth detection, etc., there is a need of alignment of two or more images first. In the image registration process, two images are geometrically aligned, which is an important pre-processing step in the fields such as remote sensing, medical, etc. This paper focuses specifically on satellite images, which are commonly used in applications like change detection, weather forecasting, and growth monitoring. In these applications, image registration is a crucial step, and the accuracy of the registration process is essential. However, there are various challenges to image registration, one of them is illumination change in multi-sensor, multi-spectral satellite images. To address this challenge, paper proposes a feature based approach, where feature detection using speeded up robust feature (SURF) and descriptor from modified visual geometry group (VGG16) convolutional neural network (CNN) structure are used. The descriptors are generated from the initial convolutional layers of the modified VGG16 structure for each key point detected by SURF. The main goal of this approach is to reduce incorrect matches, which in turn improves image registration. Results of this experiment demonstrate 20% to 40% of significant improvement in correct match rate (CMR), which in turn improves image registration by the proposed approach, as compared to the method where only the original SURF is used for feature detection and descriptor generation. Therefore, it is found that, the use of CNN features as descriptor with SURF as feature detector provides improved results in terms of CMR and thus improves image registration compared to the taken method in comparison. This shows that the use of learned feature as descriptor has potential to improve the image registration.

Keywords : Image registration, Speeded up robust feature (SURF), Convolutional neural network (CNN), VGG16, Correct match rate (CMR).
Cite this article : Patel LK, Patel MI. Feature based image registration using CNN features for satellite images having varying illumination level. International Journal of Advanced Technology and Engineering Exploration. 2023; 10(101):440-457. DOI:10.19101/IJATEE.2022.10100139.
References :
[1]Brown LG. A survey of image registration techniques. ACM Computing Surveys. 1992; 24(4):325-76.
[Crossref] [Google Scholar]
[2]Zitova B, Flusser J. Image registration methods: a survey. Image and vision computing. 2003; 21(11):977-1000.
[Crossref] [Google Scholar]
[3]Patel MI, Thakar VK, Shah SK. Image registration of satellite images with varying illumination level using HOG descriptor based SURF. Procedia Computer Science. 2016; 93:382-8.
[Crossref] [Google Scholar]
[4]Lowe DG. Distinctive image features from scale-invariant key points. International Journal of Computer Vision. 2004; 60:91-110.
[Crossref] [Google Scholar]
[5]Bay H, Tuytelaars T, Van GL. Surf: speeded up robust features. Lecture Notes in Computer Science. 2006; 3951:404-17.
[Crossref] [Google Scholar]
[6]Rublee E, Rabaud V, Konolige K, Bradski G. ORB: an efficient alternative to SIFT or SURF. In 2011 international conference on computer vision 2011 (pp. 2564-71). IEEE.
[Crossref] [Google Scholar]
[7]El-gayar MM, Soliman H. A comparative study of image low level feature extraction algorithms. Egyptian Informatics Journal. 2013; 14(2):175-81.
[Crossref] [Google Scholar]
[8]Joshi K, Patel MI. Recent advances in local feature detector and descriptor: a literature survey. International Journal of Multimedia Information Retrieval. 2020; 9(4):231-47.
[Crossref] [Google Scholar]
[9]Sreeja G, Saraniya O. A comparative study on image registration techniques for SAR images. In 5th international conference on advanced computing & communication systems 2019 (pp. 947-53). IEEE.
[Crossref] [Google Scholar]
[10]Mistry D, Banerjee A. Comparison of feature detection and matching approaches: SIFT and SURF. GRD Journals-Global Research and Development Journal for Engineering. 2017; 2(4):7-13.
[Google Scholar]
[11]Kai W, Bo C, Lu M, Song X. Multi-source remote sensing image registration based on normalized SURF algorithm. In international conference on computer science and electronics engineering 2012 (pp. 373-7). IEEE.
[Crossref] [Google Scholar]
[12]Huiqing Z, Lin G. Image registration research based on SUSAN-SURF algorithm. In the 26th Chinese control and decision conference 2014 (pp. 5292-6). IEEE.
[Crossref] [Google Scholar]
[13]Zhao D, Yang Y, Ji Z, Hu X. Rapid multimodality registration based on MM-SURF. Neurocomputing. 2014; 131:87-97.
[Crossref] [Google Scholar]
[14]Zhu XX, Tuia D, Mou L, Xia GS, Zhang L, Xu F. Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geoscience and Remote Sensing Magazine. 2017; 5(4):8-36.
[Crossref] [Google Scholar]
[15]Ma L, Liu Y, Zhang X, Ye Y, Yin G, Johnson BA. Deep learning in remote sensing applications: a meta-analysis and review. ISPRS Journal of Photogrammetry and Remote Sensing. 2019; 152:166-77.
[Crossref] [Google Scholar]
[16]Kuppala K, Banda S, Barige TR. An overview of deep learning methods for image registration with focus on feature-based approaches. International Journal of Image and Data Fusion. 2020; 11(2):113-35.
[Crossref] [Google Scholar]
[17]Liu X, Han F, Ghazali KH, Mohamed II, Zhao Y. A review of convolutional neural networks in remote sensing image. In proceedings of 8th international conference on software and computer applications 2019 (pp. 263-7).
[Crossref] [Google Scholar]
[18]Ye F, Su Y, Xiao H, Zhao X, Min W. Remote sensing image registration using convolutional neural network features. IEEE Geoscience and Remote Sensing Letters. 2018; 15(2):232-6.
[Crossref] [Google Scholar]
[19]Marmanis D, Datcu M, Esch T, Stilla U. Deep learning earth observation classification using imagenet pretrained networks. IEEE Geoscience and Remote Sensing Letters. 2015; 13(1):105-9.
[Crossref] [Google Scholar]
[20]Zhou W, Newsam S, Li C, Shao Z. Learning low dimensional convolutional neural networks for high-resolution remote sensing image retrieval. Remote Sensing. 2017; 9(5):1-20.
[Crossref] [Google Scholar]
[21]Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. Communications of the ACM. 2017; 60(6):84-90.
[Crossref] [Google Scholar]
[22]Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. ICLR 2014 (pp.1-14).
[Google Scholar]
[23]Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In proceedings of the IEEE conference on computer vision and pattern recognition 2015 (pp. 1-9).
[Google Scholar]
[24]Shah U, Mistry D, Banerjee A. Image registration of multiview satellite images using best features points detection and matching methods from SURF, SIFT and PCA-SIFT. Journal of Emerging Technologies and Innovative Research. 2014; 1(1):1-13.
[Crossref] [Google Scholar]
[25]Bouchiha R, Besbes K. Automatic remote-sensing image registration using SURF. International Journal of Computer Theory and Engineering. 2013; 5(1):406-10.
[Google Scholar]
[26]Al-najjar H. UAV and lidar image registration: a SURF-based approach for ground control points selection. ISPRS Geospatial Week. 2019:413-18.
[Google Scholar]
[27]Aarathi MR, Raju J. Influence of different descriptors to enhance image registration techniques using FREAK: case study. In 1st international conference on innovations in information and communication technology 2019 (pp. 1-6). IEEE.
[Crossref] [Google Scholar]
[28]Sheng Z, Peihua L, Yuli L, Mingsi Q, Changgang J, Meng Z. Image registration method based on optimized SURF algorithm. American Journal of Optics and Photonics. 2019; 7(4):63-9.
[Google Scholar]
[29]Liao F, Chen Y, Chen Y, Lu Y. SAR image registration based on optimized Ransac algorithm with mixed feature extraction. In IGARSS international geoscience and remote sensing symposium 2020 (pp. 1153-6). IEEE.
[Crossref] [Google Scholar]
[30]Li X, He H, Huang C, Shi Y. PCB Image registration based on improved SURF algorithm. In international conference on image processing, computer vision and machine learning 2022 (pp. 76-9). IEEE.
[Crossref] [Google Scholar]
[31]Wu Y, Wang Z. Remote sensing image registration algorithm based on improved surf in wavelet domain. Journal of Tianjin University Science and Technology. 2017; 50(10): 1084-92.
[Crossref] [Google Scholar]
[32]Zhongjun W, Yanfeng C. Image registration algorithm using SURF feature and local cross-correlation information. Infrared and Laser Engineering. 2022; 51(6):1-6.
[Crossref] [Google Scholar]
[33]Zhao X, Li H, Wang P, Jing L. An image registration method using deep residual network features for multisource high-resolution remote sensing images. Remote Sensing. 2021; 13(17):1-25.
[Crossref] [Google Scholar]
[34]Yang Z, Dan T, Yang Y. Multi-temporal remote sensing image registration using deep convolutional features. IEEE Access. 2018; 6:38544-55.
[Crossref] [Google Scholar]
[35]Ma W, Zhang J, Wu Y, Jiao L, Zhu H, Zhao W. A novel two-step registration method for remote sensing images based on deep and local features. IEEE Transactions on Geoscience and Remote Sensing. 2019; 57(7):4834-43.
[Crossref] [Google Scholar]
[36]Li C, You Y, Cao J, Zhou W. MoatNet: registration for multi-temporal optical remote sensing images using deep convolutional features. In international geoscience and remote sensing symposium IGARSS 2021 (pp. 2154-7). IEEE.
[Crossref] [Google Scholar]
[37]Ying C, Guoqing L, Hengshi C. Multi-temporal remote sensing image registration based on multi-layer feature fusion of deep residual network. In international conference on intelligent informatics and biomedical sciences 2019 (pp. 363-7). IEEE.
[Crossref] [Google Scholar]
[38]Zeng L, Du Y, Lin H, Wang J, Yin J, Yang J. A novel region-based image registration method for multisource remote sensing images via CNN. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2020; 14:1821-31.
[Crossref] [Google Scholar]
[39]Zou B, Li H, Zhang L. Self-supervised SAR image registration with SAR-superpoint and transformation aggregation. IEEE Transactions on Geoscience and Remote Sensing. 2022; 61:1-15.
[Crossref] [Google Scholar]
[40]IbaƱez D, Fernandez-beltran R, Pla F. FloU-Net: an optical flow network for multimodal self-supervised image registration. IEEE Geoscience and Remote Sensing Letters. 2023; 20:1-5.
[Crossref] [Google Scholar]
[41]Li L, Han L, Ding M, Liu Z, Cao H. Remote sensing image registration based on deep learning regression model. IEEE Geoscience and Remote Sensing Letters. 2020; 19:1-5.
[Crossref] [Google Scholar]
[42]https://docs.opencv.org/4.x/dc/dc3/tutorial_py_matcher.html. Accessed 24 August 2022.
[43]Zuliani M. Computational methods for automatic image registration. University of California, Santa Barbara; 2007:1-24.
[Google Scholar]
[44] http://bhuvan.nrsc.gov.in. Accessed 20 August 2022.
[45]Gong M, Zhao S, Jiao L, Tian D, Wang S. A novel coarse-to-fine scheme for automatic image registration based on SIFT and mutual information. IEEE Transactions on Geoscience and Remote Sensing. 2013; 52(7):4328-38.
[Crossref] [Google Scholar]
[46]http://www.landinfo.com/GalSatResComp.htm, Accessed 20 August 2022.