论文标题
MVTN：学习多视图转换以进行3D理解
MVTN: Learning Multi-View Transformations for 3D Understanding
论文作者
论文摘要
多视图投影技术已表明自己在识别3D形状的表现方面非常有效。这些方法涉及学习如何结合多个观点的信息。但是，从中获得这些视图的相机视点通常是针对所有形状固定的。为了克服当前多视图技术的静态性质，我们建议学习这些观点。具体而言，我们介绍了多视图转换网络（MVTN），该网络使用可区分的渲染来确定3D形状识别的最佳视点。结果，可以使用任何用于3D形状分类的多视图网络端到端训练MVTN。我们将MVTN集成到一种新型的自适应多视图管道中，该管道能够呈现3D网格和点云。我们的方法证明了在3D分类中的最新性能，并在几个基准上进行了形状检索（ModelNet40，Scanobjectnn，Shapenet Core55）。进一步的分析表明，与其他方法相比，我们的方法表现出改善的牢固性。我们还研究了MVTN的其他方面，例如2D预处理及其用于分割的使用。为了支持该领域的进一步研究，我们发布了Mvtorch，这是一个使用多视图预测的Pytorch库，用于3D理解和生成。
Multi-view projection techniques have shown themselves to be highly effective in achieving top-performing results in the recognition of 3D shapes. These methods involve learning how to combine information from multiple view-points. However, the camera view-points from which these views are obtained are often fixed for all shapes. To overcome the static nature of current multi-view techniques, we propose learning these view-points. Specifically, we introduce the Multi-View Transformation Network (MVTN), which uses differentiable rendering to determine optimal view-points for 3D shape recognition. As a result, MVTN can be trained end-to-end with any multi-view network for 3D shape classification. We integrate MVTN into a novel adaptive multi-view pipeline that is capable of rendering both 3D meshes and point clouds. Our approach demonstrates state-of-the-art performance in 3D classification and shape retrieval on several benchmarks (ModelNet40, ScanObjectNN, ShapeNet Core55). Further analysis indicates that our approach exhibits improved robustness to occlusion compared to other methods. We also investigate additional aspects of MVTN, such as 2D pretraining and its use for segmentation. To support further research in this area, we have released MVTorch, a PyTorch library for 3D understanding and generation using multi-view projections.
