论文标题
弱监督语义细分的自我监督特定图像特定原型探索
Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation
论文作者
论文摘要
基于图像级标签的弱监督语义细分（WSSS）由于注释成本较低而引起了很多关注。现有方法通常依赖于衡量图像像素与分类器重量之间的相关性的类激活映射（CAM）。但是，分类器仅关注判别区域，同时忽略每个图像中的其他有用信息，从而导致不完整的本地化图。为了解决这个问题，我们提出了一个由特定图像特定原型探索（IPE）和一般特异性一致性（GSC）损失组成的自我监督图像特定原型探索（SIPE）。具体而言，IPE针对每个图像量身定制原型，以捕获我们的图像特异性凸轮（IS-CAM）的完整区域，该区域通过两个顺序步骤实现。此外，还提出了GSC来构建一般CAM和我们的特定IS-CAM的一致性，这进一步优化了特征表示，并赋予了原型探索的自校正能力。广泛的实验是在Pascal VOC 2012和MS Coco 2014细分基准上进行的，结果表明我们的Sipe仅使用图像级标签实现了新的最先进的性能。该代码可在https://github.com/chenqi1126/sipe上找到。
Weakly Supervised Semantic Segmentation (WSSS) based on image-level labels has attracted much attention due to low annotation costs. Existing methods often rely on Class Activation Mapping (CAM) that measures the correlation between image pixels and classifier weight. However, the classifier focuses only on the discriminative regions while ignoring other useful information in each image, resulting in incomplete localization maps. To address this issue, we propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Specifically, IPE tailors prototypes for every image to capture complete regions, formed our Image-Specific CAM (IS-CAM), which is realized by two sequential steps. In addition, GSC is proposed to construct the consistency of general CAM and our specific IS-CAM, which further optimizes the feature representation and empowers a self-correction ability of prototype exploration. Extensive experiments are conducted on PASCAL VOC 2012 and MS COCO 2014 segmentation benchmark and results show our SIPE achieves new state-of-the-art performance using only image-level labels. The code is available at https://github.com/chenqi1126/SIPE.
