论文标题
SIM2REAL深度学习方法，用于将图像从多个车辆安装的摄像机转换为伯德眼景中的语义分段图像
A Sim2Real Deep Learning Approach for the Transformation of Images from Multiple Vehicle-Mounted Cameras to a Semantically Segmented Image in Bird's Eye View
论文作者
论文摘要
准确的环境感知对于自动驾驶至关重要。当使用单眼摄像机时，环境中元素的距离估计构成了一个重大挑战。当将摄像机的视角转换为鸟类视图（BEV）时，可以更容易地估算距离。对于平面表面，反视角映射（IPM）可以准确地将图像转换为BEV。这种转换使三维对象（例如车辆和脆弱的道路使用者）扭曲了，因此很难估计其相对于传感器的位置。本文介绍了一种从多个车辆安装的摄像机中获得校正后的360°BEV图像的方法。校正后的BEV图像分为语义类别，包括遮挡区域的预测。神经网络方法不依赖手动标记的数据，而是在合成数据集上训练的，以使其可以很好地概括到现实世界数据。通过使用语义分割的图像作为输入，我们减少了模拟和现实世界数据之间的现实差距，并能够证明我们的方法可以成功地应用于现实世界中。对合成数据进行的广泛实验证明了我们方法与IPM相比。源代码和数据集可从https://github.com/ika-rwth-aachen/cam2bev获得
Accurate environment perception is essential for automated driving. When using monocular cameras, the distance estimation of elements in the environment poses a major challenge. Distances can be more easily estimated when the camera perspective is transformed to a bird's eye view (BEV). For flat surfaces, Inverse Perspective Mapping (IPM) can accurately transform images to a BEV. Three-dimensional objects such as vehicles and vulnerable road users are distorted by this transformation making it difficult to estimate their position relative to the sensor. This paper describes a methodology to obtain a corrected 360° BEV image given images from multiple vehicle-mounted cameras. The corrected BEV image is segmented into semantic classes and includes a prediction of occluded areas. The neural network approach does not rely on manually labeled data, but is trained on a synthetic dataset in such a way that it generalizes well to real-world data. By using semantically segmented images as input, we reduce the reality gap between simulated and real-world data and are able to show that our method can be successfully applied in the real world. Extensive experiments conducted on the synthetic data demonstrate the superiority of our approach compared to IPM. Source code and datasets are available at https://github.com/ika-rwth-aachen/Cam2BEV
