Bayesian Attention Modules

方法与贡献

由归一化的可重参数化随机变量构造注意力权重分布，并使用贝叶斯正则化学习。

把概率注意力发展为带有数据依赖先验的模块化随机权重。

结合主题阅读

把注意力建模为隐变量：学习注意力的概率分布 →

引用

Xinjie Fan, Shujian Zhang, Bo Chen, Mingyuan Zhou. Bayesian Attention Modules. NeurIPS 2020.

@inproceedings{fan2020bayesian,
  title = {{Bayesian Attention Modules}},
  author = {Fan, Xinjie and Zhang, Shujian and Chen, Bo and Zhou, Mingyuan},
  year = {2020},
  booktitle = {NeurIPS 2020},
  url = {https://arxiv.org/abs/2010.10604},
  eprint = {2010.10604},
  archivePrefix = {arXiv}
}