# Bayesian Attention Modules

Xinjie Fan, Shujian Zhang, Bo Chen, Mingyuan Zhou. NeurIPS 2020 (2020).

Builds distributions over attention weights from normalized reparameterizable variables and learns them with Bayesian regularization.

Extends probabilistic attention to modular stochastic weights with data-dependent priors.

[Original paper](https://arxiv.org/abs/2010.10604)

## Citation

```bibtex
@inproceedings{fan2020bayesian,
  title = {{Bayesian Attention Modules}},
  author = {Fan, Xinjie and Zhang, Shujian and Chen, Bo and Zhou, Mingyuan},
  year = {2020},
  booktitle = {NeurIPS 2020},
  url = {https://arxiv.org/abs/2010.10604},
  eprint = {2010.10604},
  archivePrefix = {arXiv}
}
```