IMAGE FUSION TECHNIQUES USING POLARIZATION INFORMATION AND INFRARED INTENSITY

Vitalii Artym, Oleh M. Krupych

Abstract


Background. Fusion of polarization images with conventional thermal infrared images combines complementary physical information to improve scene interpretation. Infrared intensity imaging captures thermal emission and emissivity patterns. In contrast, polarization parameters, including the degree and angle of linear polarization, encode surface microstructure, material composition, and geometric orientation that are inaccessible to intensity-only sensing. Their fusion enhances spatial detail, target discrimination, and robustness under degraded visibility conditions, reflecting a broader shift toward multimodal perception.

Materials and Methods. This study provides an analytical review of polarization–infrared fusion techniques. The development of methods is examined from classical linear transforms, multiresolution decomposition, and energy-based decision rules to adaptive hybrid frameworks. Particular attention is given to deep learning architectures and physics-informed models that incorporate polarization formation principles and radiometric constraints. The review situates fusion progress within advances in sensor technologies and computational imaging.

Results and Discussion. The analysis indicates that the incorporation of polarization information substantially enhances fused image quality compared with conventional intensity-based techniques. Classical algorithms provide stable and interpretable performance but exhibit limited adaptability to complex environments. Learning-based and hybrid strategies demonstrate superior contrast preservation, structural detail enhancement, and target detectability, especially in thermally heterogeneous or visually degraded scenarios. Persistent challenges include limited annotated datasets, polarization-induced noise sensitivity, and reduced physical interpretability of deep neural representations.

Conclusion. Polarization–infrared fusion represents a transition from mono-modality to multi-modal observation with the system-oriented perception integrating physical modeling with data-driven learning. Future research should emphasize physically reasoned neural design, standardized evaluation protocols, and improved interpretability to support reliable real-world deployment.

Keywords: image fusion, polarization imaging, infrared imaging, multimodal data, deep learning.


Full Text:

PDF

References


[1] Ma, W., Wang, K., Li, J., Yang, S. X., Song, L., & Li, Q. (2023). Infrared and visible image fusion technology and application: A review. Sensors, 23(2), 599. https://doi.org/10.3390/s23020599

[2] Kolobrodov, V. G. (2024). Energy model of optical system of polarimetric thermal imager. Visnyk NTUU KPI Seriia - Radiotekhnika Radioaparatobuduvannia, 95, 47–53 (in Ukrainian). https://doi.org/10.20535/RADAP.2024.95.47-53

[3] Guo, F., Zhu, J., Huang, L., Li, F., Zhang, N., Deng, J., Li, H., Zhang, X., Zhao, Y., & Jiang, H. (2024). Multi-dimensional fusion of spectral and polarimetric images followed by pseudo-color algorithm integration and mapping in HSI space. Remote Sensing, 16(7), 1119. https://doi.org/10.3390/rs16071119

[4] A.-J. Fresnel (1821). Mémoire sur les modifications que la réflexion imprime à la lumière polarisée. Annales de Chimie et de Physique, 17, 45-176. https://doi.org/10.5281/ZENODO.5060346

[5] Stokes, G. G. (1851). On the composition and resolution of streams of polarized light from different sources. Transactions of the Cambridge Philosophical Society, 9, 399.
https://doi.org/10.1017/CBO9780511702266.010

[6] Born M, Wolf E, Bhatia AB, et al. (1999). Principles of Optics: Electromagnetic Theory of Propagation, Interference and Diffraction of Light. 7th ed. Cambridge University Press. https://doi.org/10.1017/CBO9781139644181

[7] Herschel, W. (1800). II. Experiments on the refrangibility of the invisible rays of the sun. The Philosophical Magazine, 8(29), 9-15. https://doi.org/10.1080/14786440008562602

[8] Planck, M. (1901). On the law of distribution of energy in the normal spectrum. Annalen der physik, 4, 553-563.
https://doi.org/10.1002/andp.19013090310

[9] Jones, R. C. (1941). A new calculus for the treatment of optical systems I. Description and discussion of the calculus. Journal of the Optical Society of America, 31(7), 488-493. https://doi.org/10.1364/JOSA.31.000488

[10] Rogalski, A. (2003). Infrared detectors: status and trends. Progress in quantum electronics, 27(2-3), 59-210.
https://doi.org/10.1016/S0079-6727(02)00024-1

[11] Rogalski, A. (2010). Infrared Detectors (2nd ed.). CRC Press. https://doi.org/10.1201/b10319

[12] Fossum, E. R. (1997). CMOS image sensors: Electronic camera-on-a-chip. IEEE transactions on electron devices, 44(10), 1689-1698. https://doi.org/10.1109/16.628824

[13] Gruev, V., Ortu, A., Yang, Z., Van der Spiegel, J., & Engheta, N. (2007, September). High-resolution integrated image sensor with polymer micropolarization array. In Frontiers in Optics (p. FTuS6). Optica Publishing Group. https://doi.org/10.1364/FIO.2007.FTuS6

[14] Burt, P. J., & Adelson, E. H. (1983). The Laplacian Pyramid as a Compact Image Code. IEEE Transactions on Communications, 31(4), 532-540. https://doi.org/10.1109/TCOM.1983.1095851

[15] Mallat, S. G. (2002). A theory for multiresolution signal decomposition: the wavelet representation. IEEE transactions on pattern analysis and machine intelligence, 11(7), 674-693. https://doi.org/10.1109/34.192463

[16] Piella, G. (2003). A general framework for multiresolution image fusion: from pixels to regions. Information fusion, 4(4), 259-280. https://doi.org/10.1016/S1566-2535(03)00046-0

[17] Tyo, J. S., Goldstein, D. L., Chenault, D. B., & Shaw, J. A. (2006). Review of passive imaging polarimetry for remote sensing applications. Applied optics, 45(22), 5453-5469. https://doi.org/10.1364/AO.45.005453

[18] Li, H., & Wu, X. J. (2018). DenseFuse: A fusion approach to infrared and visible images. IEEE transactions on image processing, 28(5), 2614-2623. https://doi.org/10.1109/TIP.2018.2887342

[19] aswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30, 5998-6008.
https://doi.org/10.48550/arXiv.1706.03762

[20] Yang, J., Fu, X., Hu, Y., Huang, Y., Ding, X., & Paisley, J. (2017). PanNet: A deep network architecture for pan-sharpening. In Proceedings of the IEEE international conference on computer vision (pp. 5449-5457). https://doi.org/10.1109/ICCV.2017.193

[21] Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004). Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), 600-612. https://doi.org/10.1109/TIP.2003.819861

[22] Zhang, Q., & Guo, B. L. (2009). Multifocus image fusion using the nonsubsampled contourlet transform. Signal processing, 89(7), 1334-1346. https://doi.org/10.1016/j.sigpro.2009.01.012

[23] Pratt, W. K. (2007). Digital image processing: PIKS Scientific inside (Vol. 4). Hoboken, New Jersey: Wiley-Interscience.
https://doi.org/10.1002/0470097434

[24] Yue, Z., & Li, F. M. (2014, February). An infrared polarization image fusion algorithm based on oriented Laplacian pyramid. In Selected Papers from Conferences of the Photoelectronic Technology Committee of the Chinese Society of Astronautics: Optical Imaging, Remote Sensing, and Laser-Matter Interaction 2013 (Vol. 9142, pp. 60-70). SPIE. https://doi.org/10.1117/12.2054074

[25] Zhang, J., Zhang, Y., & Shi, Z. (2018). Long-wave infrared polarization feature extraction and image fusion based on the orthogonality difference method. Journal of Electronic Imaging, 27(2), 023021-023021. https://doi.org/10.1117/1.JEI.27.2.023021

[26] Cunha, A. L., Zhou, J., & Do, M. N. (2006). The nonsubsampled contourlet transform: Theory, design, and applications. IEEE Transactions on Image Processing, 15(10), 3089–3101. https://doi.org/10.1109/TIP.2006.877507

[27] Khaustov, Ya. Ye., Khaustov, D. Ye., Lychkovskyy, E., Ryzhov, Ye., & Nastishin, Yu. A. (2019). Image fusion for a target sightseeing system of armored vehicles. Milit. Techn. Collect., 21, 28-37.
https://doi.org/10.33577/2312-4458.21.2019.28-37

[28] Khaustov, Ya. Ye., Khaustov, D. Ye., Ryzhov, Ye., Lychkovskyy, E., & Nastishin, Yu. A. (2020). Fusion of visible and infrared images via complex function. Milit. Techn. Collect., 22, 20-31.
https://doi.org/10.33577/2312-4458.22.2020.20-31

[29] Khaustov, Ya. Ye., Khaustov, D. Ye., Hryvachevskyi, A. P., Ryzhov, Ye., Lychkovskyy, E., Prudyus, I. N., & Nastishin, Yu. A. (2021). Complex function as a template for image fusion. Results in Optics, 2, 100038. https://doi.org/10.1016/j.rio.2020.100038

[30] Khaustov, D. Ye., Khaustov, Ya. Ye., Ryzhov, Ye., Lychkovskyy, E., Vlokh, R., & Nastishin, Yu. A. (2021). Jones formalism for image fusion. Ukr. J. Phys. Opt., 22(3), 165-180. https://doi.org/10.3116/16091833/22/3/165/2021

[31] Khaustov, D. Ye.,Kyrychuk, O. A., Khaustov, Ya. Ye., Stakh, Ya. Ye., Zhyrna O. V., & Nastishin, Yu. A. (2023). The Measure of Target Saliency for Target-Oriented Image Fusion. Collection of scientific papers of the State Research Institute for Testing and Certification of Armaments and Military Equipment, 3(17), 122-136. https://doi.org/10.37701/dndivsovt.17.2023.15

[32] Khaustov, D., Khaustov, Ya., Ryzhov, Ye., Burashnikov, O., Lychkovskyy, E., & Nastishin, Yu. (2021). Dynamic fusion of images from the visible and infrared channels of sightseeing system by complex matrix formalism. Milit. Techn. Collect., 25, 29-37. https://doi.org/10.33577/2312-4458.25.2021.29-37

[33] Khaustov D. Ye., Kyrychuk О. A., Stakh T. M., Khaustov Ya. Ye., Burashnikov O. O., Ryzhov Ye., Vlokh R., & Nastishin Yu. A. (2023). Complex-scalar and complex-vector approaches for express target-oriented image fusion. Ukr. J. Phys. Opt., 24(1), 62-82. https://doi.org/10.3116/16091833/24/1/62/2023

[34] Li, K., Qi, M., Zhuang, S., Liu, Y., & Gao, J. (2023). Noise-aware infrared polarization image fusion based on salient prior with attention-guided filtering network. Optics Express, 31(16), 25781-25796. https://doi.org/10.1364/OE.492954

[35] Duan, J., Zhang, H., Liu, J., Gao, M., Cheng, C., & Chen, G. (2023). A dual-weighted polarization image fusion method based on quality assessment and attention mechanisms. Frontiers in Physics, 11, 1214206. https://doi.org/10.3389/fphy.2023.1214206

[36] Jagalingam, P., & Hegde, A. V. (2015). A review of quality metrics for fused image. Aquatic Procedia, 4, 133-142.
https://doi.org/10.1016/j.aqpro.2015.02.019

[37] Kaur, H., Koundal, D., & Kadyan, V. (2021). Image Fusion Techniques: A Survey. Archives of computational methods in Engineering, 28(7), 4425-4447. https://doi.org/10.1007/s11831-021-09540-7

[38] Jin, X., Jiang, Q., Yao, S., Zhou, D., Nie, R., Hai, J., & He, K. (2017). A survey of infrared and visual image fusion methods. Infrared Physics & Technology, 85, 478-501. https://doi.org/10.1016/j.infrared.2017.07.010

[39] Zhang, H., Xu, H., Tian, X., Jiang, J., & Ma, J. (2021). Image fusion meets deep learning: A survey and perspective. Information Fusion, 76, 323-336.
https://doi.org/10.1016/j.inffus.2021.06.008

[40] Sheikh, H. R., & Bovik, A. C. (2006). Image information and visual quality. IEEE Transactions on Image Processing, 15(2), 430-444. https://doi.org/10.1109/TIP.2005.859378

[41] Zhang, Z., & Blum, R. S. (1999). A categorization of multiscale-decomposition-based image fusion schemes with a performance study for a digital camera application. Proceedings of the IEEE, 87(8), 1315-1326.
https://doi.org/10.1109/5.775414

[42] Wang, L., Dou, J., Qin, P., Lin, S., Gao, Y., Wang, R., & Zhang, J. (2021). Multimodal medical image fusion based on nonsubsampled shearlet transform and convolutional sparse representation. Multimedia Tools and Applications, 80(30), 36401-36421. https://doi.org/10.1007/s11042-021-11379-w

[43] Li, H., Manjunath, B. S., & Mitra, S. K. (1995). Multisensor image fusion using the wavelet transform. Graphical models and image processing, 57(3), 235-245.
https://doi.org/10.1006/gmip.1995.1022

[44] Li, S., Kang, X., & Hu, J. (2013). Image fusion with guided filtering. IEEE Transactions on Image Processing, 22(7), 2864-2875. https://doi.org/10.1109/TIP.2013.2244222

[45] Wald, L. (1999). Some terms of reference in data fusion. IEEE Transactions on Geoscience and Remote Sensing, 37(3), 1190-1193. https://doi.org/10.1109/36.763269

[46] Ma, J., Ma, Y., & Li, C. (2019). Infrared and visible image fusion methods and applications: A survey. Information fusion, 45, 153-178. https://doi.org/10.1016/j.inffus.2018.02.004

[47] Xu, H., Ma, J., Jiang, J., Guo, X., & Ling, H. (2020). U2Fusion: A unified unsupervised image fusion network. IEEE transactions on pattern analysis and machine intelligence, 44(1), 502-518. https://doi.org/10.1109/TPAMI.2020.3012548

[48] Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378, 686-707. https://doi.org/10.1016/j.jcp.2018.10.045

[49] Goldstein, D.H. (2011). Polarized Light (3rd ed.). CRC Press. https://doi.org/10.1201/b10436

[50] Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013). Vision meets robotics: The KITTI dataset. The International Journal of Robotics Research, 32(11), 1231-1237. https://doi.org/10.1177/0278364913491297

[51] Tuchin, V. V. (2015). Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnostics, Third Edition. Society of Photo-Optical Instrumentation Engineers (SPIE). https://doi.org/10.1117/3.1003040

[52] Drgoňa, J., Tuor, A. R., Chandan, V., & Vrabie, D. L. (2021). Physics-constrained deep learning of multi-zone building thermal dynamics. Energy and Buildings, 243, 110992. https://doi.org/10.1016/j.enbuild.2021.110992

[53] Kendall, A., & Gal, Y. (2017). What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision? arXiv preprint arXiv:1703.04977. https://doi.org/10.48550/arXiv.1703.04977




DOI: http://dx.doi.org/10.30970/eli.34.1

Refbacks

  • There are currently no refbacks.