论文标题
GRASPNERF：基于多视图的6-DOF GRASP检测用于使用可推广的NERF的透明和镜面对象
GraspNeRF: Multiview-based 6-DoF Grasp Detection for Transparent and Specular Objects Using Generalizable NeRF
论文作者
论文摘要
在这项工作中，我们解决了透明和镜面对象的6-DOF GRASP检测，这在基于视觉的机器人系统中是一个重要但具有挑战性的问题，这是由于深度摄像机在感知其几何形状方面的失败。我们首次提出了一个基于多视RGB的6-DOF GRASP检测网络Graspnerf，该网络利用可概括的神经辐射场（NERF）来实现杂物中的材料 - 敏捷对象。与现有的基于NERF的3-DOF GRASP检测方法相比，依赖于密集捕获的输入图像和耗时的人均优化，我们的系统可以实时实时执行零摄像机NERF结构，并可靠地检测到6-DOF GRASP。拟议的框架以端到端的方式共同学习了可概括的nerf和抓住检测，从而优化了场景表示构造的握把。对于训练数据，我们生成了一个大规模的逼真的域名域的合成数据集，可以在混乱的桌面场景中抓住，以直接传输到现实世界。我们在合成和现实世界环境中进行的广泛实验表明，我们的方法在实时保留的同时，在所有实验中都显着优于所有基准。可以在https://pku-epic.github.io/graspnerf上找到项目页面
In this work, we tackle 6-DoF grasp detection for transparent and specular objects, which is an important yet challenging problem in vision-based robotic systems, due to the failure of depth cameras in sensing their geometry. We, for the first time, propose a multiview RGB-based 6-DoF grasp detection network, GraspNeRF, that leverages the generalizable neural radiance field (NeRF) to achieve material-agnostic object grasping in clutter. Compared to the existing NeRF-based 3-DoF grasp detection methods that rely on densely captured input images and time-consuming per-scene optimization, our system can perform zero-shot NeRF construction with sparse RGB inputs and reliably detect 6-DoF grasps, both in real-time. The proposed framework jointly learns generalizable NeRF and grasp detection in an end-to-end manner, optimizing the scene representation construction for the grasping. For training data, we generate a large-scale photorealistic domain-randomized synthetic dataset of grasping in cluttered tabletop scenes that enables direct transfer to the real world. Our extensive experiments in synthetic and real-world environments demonstrate that our method significantly outperforms all the baselines in all the experiments while remaining in real-time. Project page can be found at https://pku-epic.github.io/GraspNeRF
