Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation

Method and contribution

FSA-CDM combines fine-grained cross-modal sequence alignment, contextual modeling, and a contrast-augmented diffusion objective for markup-to-image generation.

A subsequent markup-to-image method that builds on the task and four datasets introduced by Deng, Kojima, and Rush, addressing alignment and contrastive feedback.

Read in context

Learning to render LaTeX, SMILES, HTML and music notation →

Citation

Guojin Zhong, Jin Yuan, Pan Wang, Kailun Yang, Weili Guan, Zhiyong Li. Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation. ACM Multimedia 2023.

@inproceedings{zhong2023contrastaugmented,
  title = {{Contrast-augmented Diffusion Model with Fine-grained Sequence Alignment for Markup-to-Image Generation}},
  author = {Zhong, Guojin and Yuan, Jin and Wang, Pan and Yang, Kailun and Guan, Weili and Li, Zhiyong},
  year = {2023},
  booktitle = {ACM Multimedia 2023},
  url = {https://arxiv.org/abs/2308.01147},
  eprint = {2308.01147},
  archivePrefix = {arXiv}
}