论文标题
PanFormer：基于变压器的板折叠模型
PanFormer: a Transformer Based Model for Pan-sharpening
论文作者
论文摘要
pan-sharpening旨在从低分辨率（LR）多光谱（MS）图像及其相应的Panchrosic（PAN）图像中产生高分辨率（HR）多光谱（MS）图像。受到最近深度学习社区的新时尚的启发，我们提出了一种基于变压器的新型模型，用于泛滥。我们探讨了变压器在图像特征提取和融合中的潜力。随着视觉变压器的成功开发，我们设计了一个具有自我注意力的两流网络，以从PAN和MS模式中提取特定于模式的特征，并应用跨意义模块以合并光谱和空间特征。泛滥的图像是由增强的融合功能产生的。关于GaOfen-2和Worldview-3图像的广泛实验表明，我们的基于变压器的模型取得了令人印象深刻的结果，并且胜过许多基于CNN的方法，这表明将变压器引入泛墨件任务的巨大潜力。代码可在https://github.com/zhysora/panformer上找到。
Pan-sharpening aims at producing a high-resolution (HR) multi-spectral (MS) image from a low-resolution (LR) multi-spectral (MS) image and its corresponding panchromatic (PAN) image acquired by a same satellite. Inspired by a new fashion in recent deep learning community, we propose a novel Transformer based model for pan-sharpening. We explore the potential of Transformer in image feature extraction and fusion. Following the successful development of vision transformers, we design a two-stream network with the self-attention to extract the modality-specific features from the PAN and MS modalities and apply a cross-attention module to merge the spectral and spatial features. The pan-sharpened image is produced from the enhanced fused features. Extensive experiments on GaoFen-2 and WorldView-3 images demonstrate that our Transformer based model achieves impressive results and outperforms many existing CNN based methods, which shows the great potential of introducing Transformer to the pan-sharpening task. Codes are available at https://github.com/zhysora/PanFormer.
