论文标题
通过解决方式不平衡问题来改善多光谱的行人检测
Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems
论文作者
论文摘要
多光谱的行人检测能够通过利用色热的方式来适应不足的照明条件。另一方面，仍然缺乏有关如何有效融合这两种方式的深入见解。与传统的行人检测相比，我们发现多光谱的行人检测遇到了模式失衡问题，这将阻碍双模式网络的优化过程并降低检测器的性能。受这一观察的启发，我们提出了模式平衡网络（MBNET），该网络以更加灵活和平衡的方式促进了优化过程。首先，我们设计了一种新型的差异方式意识融合（DMAF）模块，以使两种模态相互补充。其次，照明意识到功能对齐模块根据照明条件选择互补功能，并适应两个模态特征。广泛的实验结果表明，MBNET在具有挑战性的KAIST和CVC-14多光谱行人数据集方面，MBNET优于最先进的实验结果。代码可在https://github.com/calayzhou/mbnet上找到。
Multispectral pedestrian detection is capable of adapting to insufficient illumination conditions by leveraging color-thermal modalities. On the other hand, it is still lacking of in-depth insights on how to fuse the two modalities effectively. Compared with traditional pedestrian detection, we find multispectral pedestrian detection suffers from modality imbalance problems which will hinder the optimization process of dual-modality network and depress the performance of detector. Inspired by this observation, we propose Modality Balance Network (MBNet) which facilitates the optimization process in a much more flexible and balanced manner. Firstly, we design a novel Differential Modality Aware Fusion (DMAF) module to make the two modalities complement each other. Secondly, an illumination aware feature alignment module selects complementary features according to the illumination conditions and aligns the two modality features adaptively. Extensive experimental results demonstrate MBNet outperforms the state-of-the-arts on both the challenging KAIST and CVC-14 multispectral pedestrian datasets in terms of the accuracy and the computational efficiency. Code is available at https://github.com/CalayZhou/MBNet.
