Energy-Based Diffusion Language Models for Text Generation

Method and contribution

Adds a full-sequence residual energy to each diffusion denoising distribution, with autoregressive or NCE parameterizations and parallel importance sampling.

Directly cites Deng et al. (2020), carrying residual correction and partition estimation into diffusion language modeling.

Read in context

Energy-based text generation: residual models, resampling and diffusion →

Citation

Minkai Xu, Tomas Geffner, Karsten Kreis, Weili Nie, Yilun Xu, Jure Leskovec, Stefano Ermon, Arash Vahdat. Energy-Based Diffusion Language Models for Text Generation. ICLR 2025.

@inproceedings{edlm,
  title = {{Energy-Based Diffusion Language Models for Text Generation}},
  author = {Xu, Minkai and Geffner, Tomas and Kreis, Karsten and Nie, Weili and Xu, Yilun and Leskovec, Jure and Ermon, Stefano and Vahdat, Arash},
  year = {2025},
  booktitle = {ICLR 2025},
  url = {https://arxiv.org/html/2410.21357v4},
  eprint = {2410.21357},
  archivePrefix = {arXiv}
}