论文标题
部分可观测时空混沌系统的无模型预测
Deepening Neural Networks Implicitly and Locally via Recurrent Attention Strategy
论文作者
论文摘要
越来越多的经验和理论证据表明，加深的神经网络可以有效地改善其在合适的训练环境下的表现。但是，加深神经网络的骨干将不可避免地增加计算和参数大小。为了减轻这些问题，我们提出了一种简单的重复注意策略（RAS），该策略暗中通过局部参数共享使用轻量级注意模块来隐含地增加神经网络的深度。在三个广泛使用的基准数据集上进行的广泛实验表明，RAS可以稍微添加参数大小和计算，以提高神经网络的性能，从而对其他现有众所周知的注意力模块进行良好的表现。
More and more empirical and theoretical evidence shows that deepening neural networks can effectively improve their performance under suitable training settings. However, deepening the backbone of neural networks will inevitably and significantly increase computation and parameter size. To mitigate these problems, we propose a simple-yet-effective Recurrent Attention Strategy (RAS), which implicitly increases the depth of neural networks with lightweight attention modules by local parameter sharing. The extensive experiments on three widely-used benchmark datasets demonstrate that RAS can improve the performance of neural networks at a slight addition of parameter size and computation, performing favorably against other existing well-known attention modules.
