论文标题
Fino-net：一个深层多模式传感器融合框架操纵故障检测
FINO-Net: A Deep Multimodal Sensor Fusion Framework for Manipulation Failure Detection
论文作者
论文摘要
在非结构化环境中为服务机器人的安全操纵是一个具有挑战性的问题。需要一个故障检测系统来监视和检测意外结果。我们提出了Fino-Net，这是一种新型的基于多模式传感器融合的深层神经网络，以检测和识别操纵失败。我们还介绍了一个多模式数据集，其中包含229个用百特机器人记录的现实世界操纵数据。我们的网络结合了RGB，深度和音频读数，以有效地检测和分类失败。结果表明，将RGB与深度和音频方式融合可显着提高性能。 Fino-net在我们的新型数据集上实现了98.60％的检测和87.31％的分类精度。代码和数据可在https://github.com/ardai/fino-net上公开获取。
Safe manipulation in unstructured environments for service robots is a challenging problem. A failure detection system is needed to monitor and detect unintended outcomes. We propose FINO-Net, a novel multimodal sensor fusion based deep neural network to detect and identify manipulation failures. We also introduce a multimodal dataset, containing 229 real-world manipulation data recorded with a Baxter robot. Our network combines RGB, depth and audio readings to effectively detect and classify failures. Results indicate that fusing RGB with depth and audio modalities significantly improves the performance. FINO-Net achieves 98.60% detection and 87.31% classification accuracy on our novel dataset. Code and data are publicly available at https://github.com/ardai/fino-net.
