论文标题
关于鉴定视觉变压器偏见的预处理的解释
Explanation on Pretraining Bias of Finetuned Vision Transformer
论文作者
论文摘要
随着预处理模型的微调数量的增加，了解预验证模型的偏见至关重要。但是，几乎没有分析变压器体系结构的工具，注意图的解释仍然具有挑战性。为了解决可解释性，我们提出了输入 - 贡献和注意力评分矢量（IAV），该载体衡量了注意力图和输入 - 互动之间的相似性，并显示了可解释的注意力模式的一般趋势。我们从经验上解释了受监督和无监督预定的VIT模型的预处理偏见，并表明VIT中的每个人都在分类的决策方面具有特定的一致性。我们表明，注意图的概括，鲁棒性和熵不是训练训练类型的特性。另一方面，IAV趋势可以分开预训练类型。
As the number of fine tuning of pretrained models increased, understanding the bias of pretrained model is essential. However, there is little tool to analyse transformer architecture and the interpretation of the attention maps is still challenging. To tackle the interpretability, we propose Input-Attribution and Attention Score Vector (IAV) which measures the similarity between attention map and input-attribution and shows the general trend of interpretable attention patterns. We empirically explain the pretraining bias of supervised and unsupervised pretrained ViT models, and show that each head in ViT has a specific range of agreement on the decision of the classification. We show that generalization, robustness and entropy of attention maps are not property of pretraining types. On the other hand, IAV trend can separate the pretraining types.
