陈施宇, 金鑫, 习修良, 江倩, 邵鑫凤. 基于Transformer和生成对抗网络的多聚焦图像融合[J]. 云南大学学报(自然科学版). doi: 10.7540/j.ynu.20230364
引用本文: 陈施宇, 金鑫, 习修良, 江倩, 邵鑫凤. 基于Transformer和生成对抗网络的多聚焦图像融合[J]. 云南大学学报(自然科学版). doi: 10.7540/j.ynu.20230364
CHEN Shiyu, JIN Xin, XI Xiuliang, JIANG Qian, SHAO Xinfeng. Multi-focus image fusion based on transformer and generative adversarial networks[J]. Journal of Yunnan University: Natural Sciences Edition. DOI: 10.7540/j.ynu.20230364
Citation: CHEN Shiyu, JIN Xin, XI Xiuliang, JIANG Qian, SHAO Xinfeng. Multi-focus image fusion based on transformer and generative adversarial networks[J]. Journal of Yunnan University: Natural Sciences Edition. DOI: 10.7540/j.ynu.20230364

基于Transformer和生成对抗网络的多聚焦图像融合

Multi-focus image fusion based on transformer and generative adversarial networks

  • 摘要: 针对多聚焦图像融合任务中聚焦与离焦边界区域处理不平滑的问题,基于Transformer的全局特征提取能力,提出了一种包含双判别器的生成对抗网络方案,生成器以端到端的形式完成多聚焦图像融合任务. 借助Transformer获取全局依赖性和低频空间细节,通过跨域的交叉注意力机制帮助生成器模型的双分支达到信息交互的效果,获取另一通道上的冗余信息和互补信息. 模型在学习过程中结合图像的全局信息进行参数更新,从而克服上述问题,且尽可能保留多聚焦图像中聚焦区域的有效信息. 通过对比实验表明,所提方法可行且具备竞争力.

     

    Abstract: In response to the issue of the uneven handling of the focused and defocused border regions in the multi-focus image fusion task, this study proposes a generative adversarial network (GAN) approach with dual discriminators based on the global feature extraction capability of Transformer. The generator completes the multi-focus image fusion task in an end-to-end manner. This approach leverages the Transformer to obtain global dependencies and low-frequency spatial details, and utilizes a cross-domain cross-attention mechanism to facilitate information interaction between the dual branches of the generator model, obtaining redundant and complementary information from the other channel. The model combines global image information for parameter updates during the learning process, overcoming the aforementioned issue while preserving effective information from the focused regions in the multi-focus image as much as possible. Comparative experiments demonstrate the feasibility and competitiveness of the proposed method.

     

/

返回文章
返回