论文标题
PointCMC：跨模式多尺度对应关系学习，以了解点云理解
PointCMC: Cross-Modal Multi-Scale Correspondences Learning for Point Cloud Understanding
论文作者
论文摘要
一些自我监督的跨模式学习方法最近证明了图像信号的潜力增强点云表示。但是，这仍然是一个问题，即如何以自我监督的方式直接建模跨模式的本地和全球对应关系。为了解决它，我们提出了PointCMC，这是一种新型的跨模式方法，用于模拟跨模态的多尺度对应，以进行自我监督点云表示学习。特别是，PointCMC由：（1）局部到本地（L2L）模块，该模块通过优化的跨模式的局部几何特征来学习本地对应关系，（（2）局部到全球（L2G）模块，旨在通过局部和全局跨度A的局部和全局歧视（3）G2G（3）G2GLOB（3）g2 gl lob的局部和全局特征来学习局部和全局功能之间的对应关系（g2 g。点云和图像之间的全局对比损失，以学习高级语义对应关系。广泛的实验结果表明，我们的方法在各种下游任务（例如3D对象分类和分割）中的表现优于现有的最新方法。接受后将公开提供代码。
Some self-supervised cross-modal learning approaches have recently demonstrated the potential of image signals for enhancing point cloud representation. However, it remains a question on how to directly model cross-modal local and global correspondences in a self-supervised fashion. To solve it, we proposed PointCMC, a novel cross-modal method to model multi-scale correspondences across modalities for self-supervised point cloud representation learning. In particular, PointCMC is composed of: (1) a local-to-local (L2L) module that learns local correspondences through optimized cross-modal local geometric features, (2) a local-to-global (L2G) module that aims to learn the correspondences between local and global features across modalities via local-global discrimination, and (3) a global-to-global (G2G) module, which leverages auxiliary global contrastive loss between the point cloud and image to learn high-level semantic correspondences. Extensive experiment results show that our approach outperforms existing state-of-the-art methods in various downstream tasks such as 3D object classification and segmentation. Code will be made publicly available upon acceptance.
