论文标题
通过示例指导的视频预测
Video Prediction via Example Guidance
论文作者
论文摘要
在视频预测任务中，一个主要挑战是捕获未来内容和动态的多模式性质。在这项工作中，我们提出了一个简单而有效的框架，可以有效地预测合理的未来状态。关键的见解是，在训练池的曲目中，可以用类似的序列近似序列的潜在分布，即专家示例。通过将新颖的优化方案进一步纳入训练程序，可以从从检索到的示例中构建的分布中有效地进行合理的预测。同时，我们的方法可以与现有的随机预测模型无缝集成。在定量和定性方面的全面实验中，可以观察到显着的增强。我们还展示了预测看不见类的运动的概括能力，即在训练阶段没有访问相应数据的情况下。
In video prediction tasks, one major challenge is to capture the multi-modal nature of future contents and dynamics. In this work, we propose a simple yet effective framework that can efficiently predict plausible future states. The key insight is that the potential distribution of a sequence could be approximated with analogous ones in a repertoire of training pool, namely, expert examples. By further incorporating a novel optimization scheme into the training procedure, plausible predictions can be sampled efficiently from distribution constructed from the retrieved examples. Meanwhile, our method could be seamlessly integrated with existing stochastic predictive models; significant enhancement is observed with comprehensive experiments in both quantitative and qualitative aspects. We also demonstrate the generalization ability to predict the motion of unseen class, i.e., without access to corresponding data during training phase.
