Breaking the Factorization Barrier in Diffusion Language Models

Method and contribution

CoDD composes factorized diffusion predictions with a tractable probabilistic-circuit prior to model joint token dependencies.

Shares structured generation’s separation of parallel neural scoring from joint probabilistic inference.

Read in context

Structured Decoding: Higher-Order CRFs, Diffusion Language Models, and Candidate Lattices →

Citation

Ian Li, Zilei Shao, Benjie Wang, Rose Yu, Guy Van den Broeck, Anji Liu. Breaking the Factorization Barrier in Diffusion Language Models. ICML 2026.

@inproceedings{li2026factorization,
  title = {{Breaking the Factorization Barrier in Diffusion Language Models}},
  author = {Li, Ian and Shao, Zilei and Wang, Benjie and Yu, Rose and Van den Broeck, Guy and Liu, Anji},
  year = {2026},
  booktitle = {ICML 2026},
  url = {https://proceedings.mlr.press/v306/li26fd.html},
  eprint = {2603.00045},
  archivePrefix = {arXiv}
}