论文标题
Meccano：一种用于人类行为类似工业领域的人类行为的多式联运数据集
MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain
论文作者
论文摘要
可穿戴摄像机可以从用户的角度获取图像和视频。可以处理这些数据以了解人类的行为。尽管人类行为分析已经在第三人称视野中进行了彻底的研究，但仍在以自我为中心的环境中，尤其是在工业场景中进行了研究。为了鼓励在这一领域的研究，我们介绍了Meccano，这是一个以自我为中心视频的多式模式数据集，可以在类似工业的环境中研究人类的行为理解。多模式的特征是存在与自定义耳机同时获得的凝视信号，深度图和RGB视频的存在。该数据集已在从第一人称视角的人类行为理解的背景下明确标记为基本任务，例如识别和预测人类对象的相互作用。使用MECCANO数据集，我们探索了五个不同的任务，包括1）动作识别，2）活动对象检测和识别，3）中心的人类对象互动检测，4）动作预期和5）下一步活动对象检测。我们提出了一个基准，旨在在类似工业的情况下研究人类行为，该方案表明，所研究的任务和所考虑的方案对于最先进的算法具有挑战性。为了支持该领域的研究，我们将在https://iplab.dmi.unict.it/meccano/上公开发布数据集。
Wearable cameras allow to acquire images and videos from the user's perspective. These data can be processed to understand humans behavior. Despite human behavior analysis has been thoroughly investigated in third person vision, it is still understudied in egocentric settings and in particular in industrial scenarios. To encourage research in this field, we present MECCANO, a multimodal dataset of egocentric videos to study humans behavior understanding in industrial-like settings. The multimodality is characterized by the presence of gaze signals, depth maps and RGB videos acquired simultaneously with a custom headset. The dataset has been explicitly labeled for fundamental tasks in the context of human behavior understanding from a first person view, such as recognizing and anticipating human-object interactions. With the MECCANO dataset, we explored five different tasks including 1) Action Recognition, 2) Active Objects Detection and Recognition, 3) Egocentric Human-Objects Interaction Detection, 4) Action Anticipation and 5) Next-Active Objects Detection. We propose a benchmark aimed to study human behavior in the considered industrial-like scenario which demonstrates that the investigated tasks and the considered scenario are challenging for state-of-the-art algorithms. To support research in this field, we publicy release the dataset at https://iplab.dmi.unict.it/MECCANO/.
