Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor

Sangwon Yu, Changmin Lee, Hojin Lee, Sungroh Yoon


Abstract
Controlled text generation, aiming to ensure that language models produce text containing only the desired domain or corpus attributes, is immensely crucial in the practical application of language models. Existing methods, however, are inapplicable to black-box models or suffer a significant trade-off between control and fluency in text generation. This paper introduces the Score-based Progressive Editor (ScoPE), a novel approach designed to overcome these issues. ScoPE modifies the context at the token level during the generation process of a backbone language model. This modification guides the subsequent text to naturally include the target attributes. To facilitate this process, ScoPE employs a training objective that maximizes a target score, comprehensively considering both control and fluency. Experimental results on diverse controlled generation tasks demonstrate that ScoPE can effectively regulate the attributes of the generated text while effectively utilizing the capability of the backbone large language models.
Anthology ID:
2024.acl-long.767
Volume:
Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
Month:
August
Year:
2024
Address:
Bangkok, Thailand
Editors:
Lun-Wei Ku, Andre Martins, Vivek Srikumar
Venue:
ACL
SIG:
Publisher:
Association for Computational Linguistics
Note:
Pages:
14215–14237
Language:
URL:
https://aclanthology.org/2024.acl-long.767
DOI:
10.18653/v1/2024.acl-long.767
Bibkey:
Cite (ACL):
Sangwon Yu, Changmin Lee, Hojin Lee, and Sungroh Yoon. 2024. Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14215–14237, Bangkok, Thailand. Association for Computational Linguistics.
Cite (Informal):
Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor (Yu et al., ACL 2024)
Copy Citation:
PDF:
https://aclanthology.org/2024.acl-long.767.pdf