针对现有的红外和可见光图像融合方法难以在抑制红外热噪声的同时保留可见光纹理细节的问题,提出一种基于共生纹理感知的加权全变分(CT-WTV)模型。首先,对红外图像进行灰度离散化以压缩统计状态,进而计算全局灰度共生矩阵(GLCM)以捕获二阶相关性;然后,通过构建共生概率逆投影,将离散的二阶纹理特征转化为连续的空间自适应权重,在保留纹理的同时平滑高频热噪声;最后,采用交替方向乘子法(ADMM)对非光滑泛函进行解耦,原始残差和对偶残差呈现快速收敛性和数值稳定性。实验结果表明,CT-WTV在TNO和RoadScene数据集上能够较好地兼顾红外目标显著性、可见光纹理细节和热噪声抑制,融合结果在定性和定量对比中均表现出较好的结构保持能力和视觉稳定性。该结果验证了共生纹理感知权重引入全变分融合模型的有效性。
Abstract
[Objective] Infrared and visible image fusion aims to generate a single image in which thermal targets remain salient while visible-light textures are preserved. Achieving this balance remains challenging in variational fusion models. Most existing approaches rely on first-order gradient regularization, although identical gradient responses may correspond to true edges, weak textures, or thermal noise. Strong regularization tends to smooth fine visible details, whereas weak regularization permits infrared noise to propagate into the fused image. To address this limitation, this study proposes a co-occurrence texture-aware weighted total variation (CT-WTV) model. The primary objective is to incorporate second-order texture statistics into an interpretable variational framework, allowing local structural consistency to guide smoothing and detail preservation. [Methods] The proposed method first applied gray-level discretization to the infrared image to reduce the statistical complexity required for texture modeling. A gray-level co-occurrence matrix (GLCM) was then constructed from the discretized image to characterize second-order correlations between neighboring gray-level pairs. Unlike gradient operators, which only describe local intensity differences, the GLCM captured stable spatial relationships between neighboring gray levels. The discrete co-occurrence statistics were subsequently projected back to the image domain through a co-occurrence probability back-projection operator, producing a continuous measure of texture consistency at each spatial location. A nonlinear mapping function converted this measure into a spatially adaptive regularization weight. Consequently, the statistical texture prior was integrated with spatially varying variational regularization, allowing the smoothing strength to depend on structural consistency rather than solely on gradient magnitude. The model reduced smoothing in structurally consistent regions to preserve target boundaries and visible textures while increasing smoothing in noise-dominated regions to suppress thermal noise. The resulting nonsmooth weighted total variation model was solved using the alternating direction method of multipliers (ADMM), and fast Fourier transform (FFT) acceleration was employed to solve the reconstruction subproblem efficiently. [Results] Experiments conducted on the public TNO and RoadScene datasets demonstrated the effectiveness of CT-WTV compared with eight state-of-the-art fusion methods. The visual results exhibited clearer infrared targets, better visible contours and texture details, more natural color transitions in RoadScene scenes, and fewer grain-like artifacts in homogeneous infrared backgrounds. Quantitative evaluations indicate competitive performance on both datasets, particularly in terms of information preservation and structural fidelity. These findings suggest that the co-occurrence texture weighting mechanism improves the discrimination between meaningful structural features and noise-like high-frequency fluctuations. The ablation study further confirms this mechanism. The observed improvements do not arise from a single empirical adjustment but from the combined effects of co-occurrence texture modeling, multidirectional structural characterization, and nonlinear mapping from texture consistency to regularization strength. The integration of these components enables CT-WTV to achieve a better balance among infrared target preservation, visible texture reconstruction, and background noise suppression. [Conclusions] The results demonstrate that incorporating co-occurrence texture information can enhance weighted total variation for infrared and visible image fusion. By transforming second-order texture statistics into a continuous spatial regularization field, CT-WTV reduces texture loss and staircase artifacts while effectively suppressing thermal noise. The ADMM- and FFT-based optimization framework also provides stable numerical performance for the proposed variational model. Overall, the proposed method offers an interpretable, training-free fusion framework that achieves a practical balance among target saliency, texture preservation, noise suppression, and visual quality.
关键词
图像融合 /
红外和可见光图像 /
全变分 /
共生统计 /
交替方向乘子法
Key words
image fusion /
infrared and visible image /
total variation /
co-occurrence statistics /
alternating direction method of multipliers
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
参考文献
[1] 毛士艺, 赵巍. 多传感器图像融合技术综述[J]. 北京航空航天大学学报, 2002, 28(5): 512-518. MAO S Y, ZHAO W. Comments on multisensor image fusion techniques [J]. Journal of Beijing University of Aeronautics and Astronautics, 2002, 28(5): 512-518. (in Chinese)
[2] 李扬, 杨海涛, 孔卓, 等. 像素级红外与可见光图像融合方法综述[J]. 计算机工程与应用, 2022, 58(14): 40-50. LI Y, YANG H T, KONG Z, et al. Review of pixel-level infrared and visible image fusion methods[J]. Computer Engineering and Applications, 2022, 58(14): 40-50. (in Chinese)
[3] 杨艳春, 李小苗, 党建武, 等. 基于GEMD与改进PCNN的红外与可见光图像融合[J]. 北京航空航天大学学报, 2023, 49(9): 2317-2329. YANG Y C, LI X M, DANG J W, et al. Infrared and visible image fusion based on GEMD and improved PCNN[J]. Journal of Beijing University of Aeronautics and Astronautics, 2023, 49(9): 2317-2329. (in Chinese)
[4] 沈英, 黄春红, 黄峰, 等. 红外与可见光图像融合技术的研究进展[J]. 红外与激光工程, 2021, 50(9): 20200467. SHEN Y, HUANG C H, HUANG F, et al. Research progress of infrared and visible image fusion technology[J]. Infrared and Laser Engineering, 2021, 50(9): 20200467. (in Chinese)
[5] MA J Y, MA Y, LI C. Infrared and visible image fusion methods and applications: A survey[J]. Information fusion, 2019, 45: 153-178.
[6] YAO J X, ZHAO Y Q, BU Y Y, et al. Laplacian Pyramid fusion network with hierarchical guidance for infrared and visible image fusion[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(9): 4630-4644.
[7] AGHAMALEKI J A, GHORBANI A. Image fusion using dual tree discrete wavelet transform and weights optimization[J]. The Visual Computer, 2023, 39(3): 1181-1191.
[8] XIE Q Y, MA L, GUO Z Q, et al. Infrared and visible image fusion based on NSST and phase consistency adaptive DUAL channel PCNN[J]. Infrared Physics & Technology, 2023, 131: 104659.
[9] 吴一非, 杨瑞, 吕其深, 等. 红外与可见光图像融合: 统计分析, 深度学习方法和未来展望[J]. 激光与光电子学进展, 2024, 61(14): 1400004. WU Y F, YANG R, LÜ Q S, et al. Infrared and visible image fusion: Statistical analysis, deep learning approaches and future prospects[J]. Laser & Optoelectronics Progress, 2024, 61(14): 1400004. (in Chinese)
[10] 李淑慧, 蔡伟, 王鑫, 等. 深度学习框架下的红外与可见光图像融合方法综述[J]. 计算机工程与应用, 2025, 61(9): 25-40. LI S H, CAI W, WANG X, et al. Review of infrared and visible image fusion methods in deep learning frameworks[J]. Computer Engineering and Applications, 2025, 61(9): 25-40. (in Chinese)
[11] ZHAO Z X, BAI H W, ZHANG J S, et al. CDDFuse: Correlation- driven dual-branch feature decomposition for multi-modality image fusion [C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, Canada: IEEE Computer Society, 2023: 5906-5916.
[12] XIE X Y, CUI Y W, TAN T, et al. FusionMamba: Dynamic feature enhancement for multimodal image fusion with mamba [J]. Visual Intelligence, 2024, 2(1): 37.
[13] MA J Y, CHEN C, LI C, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31: 100-109.
[14] XU L, YAN Q, XIA Y, et al. Structure extraction from texture via relative total variation[J]. ACM Transactions on Graphics, 2012, 31(6): 1-10.
[15] MA T, DU L L, LIU X H, et al. Infrared and visible image fusion via total variation and alternating direction method of multipliers [C]//20217th International Conference on Computer and Communications (ICCC). Chengdu, China: IEEE, 2021: 736-741.
[16] LI M M, LI B Z. A novel weighted total variation model for image denoising[J]. IET Image Processing, 2021, 15(12): 2749-2760.
[17] YAN L, CAO J, CHENG Y, et al. Infrared and visible image fusion via L0 decomposition and intensity mask[J]. IEEE Photonics Journal, 2019, 11(6): 1-11.
[18] NIE R C, MA C Z, CAO J D, et al. A total variation with joint norms for infrared and visible image fusion[J]. IEEE Transactions on Multimedia, 2022, 24: 1460-1472.
[19] RUDIN L I, OSHER S, FATEMI E. Nonlinear total variation based noise removal algorithms[J]. Physica D Nonlinear Phenomena, 1992, 60(1-4): 259-268.
[20] TOET A. The TNO multiband image data collection[J]. Data in Brief, 2017, 15: 249-251.
[21] XU H, MA J Y, JIANG J J, et al. U2Fusion: A unified unsupervised image fusion network[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(1): 502-518.
[22] XIANG W H, SHEN J J, ZHANG L, et al. Infrared and visual image fusion based on a local-extrema-driven image filter[J]. Sensors, 2024, 24(7): 2271.
[23] LI H, WU X J. Infrared and visible image fusion using latent low-rank representation[EB/OL]. (2022-01-29) [2025-12-15]. https:// arxiv.org/abs/1804.08992.
[24] CHEN J, LI X J, WU K L. Infrared and visible image fusion based on relative total variation decomposition[J]. Infrared Physics & Technology, 2022, 123: 104112.
[25] TANG H J, LIU G, QIAN Y, et al. EgeFusion: Towards edge gradient enhancement in infrared and visible image fusion with multi-scale transform[J]. IEEE Transactions on Computational Imaging, 2024, 10: 385-398.
[26] WANG Z S, ZHANG Z Q, QI W Q, et al. FreqGAN: Infrared and visible image fusion via unified frequency adversarial learning[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2025, 35(1): 728-740.
[27] YANG G, LI J, LEI H X, et al. A multi-scale information integration framework for infrared and visible image fusion[J]. Neurocomputing, 2024, 600: 128116.
[28] LI H, WU X J. CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach[J]. Information Fusion, 2024, 103: 102147.
[29] XIAO G B, TANG Z M, GUO H L, et al. FAFusion: Learning for infrared and visible image fusion via frequency awareness[J]. IEEE Transactions on Instrumentation and Measurement, 2024, 73: 1-11.
基金
国家自然科学基金重大项目(61790550)