论文标题

Bachgan:从显着对象布局中的高分辨率图像合成

BachGAN: High-Resolution Image Synthesis from Salient Object Layout

论文作者

Li, Yandong, Cheng, Yu, Gan, Zhe, Yu, Licheng, Wang, Liqiang, Liu, Jingjing

论文摘要

我们提出了一项新的任务,以实现图像生成的更实际应用 - 来自显着对象布局的高质量图像综合。这种新设置允许用户仅提供显着对象的布局(即前景边界框和类别),并让模型以发明的背景和匹配的前景来完成图纸。这项新任务的两个主要挑战是:(i)如何生成细粒细节和逼真的纹理而不分割地图输入; (ii)如何创建背景并将其无缝编织到独立对象中。为了解决这个问题,我们提出了背景幻觉生成对抗网络(Bachgan),该网络首先通过背景检索模块从大型候选池中选择一组分割图,然后通过背景融合模块编码这些候选布局以幻觉,以适合给定对象。通过动态生成幻觉的背景表示形式,我们的模型可以合成具有照片真实前景和积分背景的高分辨率图像。在CityScapes和ADE20K数据集上进行的实验证明了Bachgan比现有方法的优势,该方法在生成图像的视觉保真度和输出图像和输入布局之间的视觉效果上衡量。

We propose a new task towards more practical application for image generation - high-quality image synthesis from salient object layout. This new setting allows users to provide the layout of salient objects only (i.e., foreground bounding boxes and categories), and lets the model complete the drawing with an invented background and a matching foreground. Two main challenges spring from this new task: (i) how to generate fine-grained details and realistic textures without segmentation map input; and (ii) how to create a background and weave it seamlessly into standalone objects. To tackle this, we propose Background Hallucination Generative Adversarial Network (BachGAN), which first selects a set of segmentation maps from a large candidate pool via a background retrieval module, then encodes these candidate layouts via a background fusion module to hallucinate a suitable background for the given objects. By generating the hallucinated background representation dynamically, our model can synthesize high-resolution images with both photo-realistic foreground and integral background. Experiments on Cityscapes and ADE20K datasets demonstrate the advantage of BachGAN over existing methods, measured on both visual fidelity of generated images and visual alignment between output images and input layouts.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源