Learning Neural Trans-Dimensional Random Field Language Models with Noise-Contrastive Estimation

Method and contribution

Uses an exponentially tilted reference distribution and noise-contrastive estimation for neural random-field language models.

Earlier residual formulation and NCE training explicitly discussed in the 2020 paper; its evaluation centers on speech-recognition rescoring.

Read in context

Energy-based text generation: residual models, resampling and diffusion →

Citation

Bin Wang, Zhijian Ou. Learning Neural Trans-Dimensional Random Field Language Models with Noise-Contrastive Estimation. ICASSP 2018.

@inproceedings{neuraltrfnce,
  title = {{Learning Neural Trans-Dimensional Random Field Language Models with Noise-Contrastive Estimation}},
  author = {Wang, Bin and Ou, Zhijian},
  year = {2018},
  booktitle = {ICASSP 2018},
  url = {https://arxiv.org/abs/1710.10739},
  eprint = {1710.10739},
  archivePrefix = {arXiv}
}