Royal Bank of Canada
SYSTEMS AND METHODS FOR DIVERSE KEYPHRASE GENERATION WITH NEURAL UNLIKELIHOOD TRAINING

Abstract:

Computer implemented methods and systems are provided for generating diverse key phrases while maintaining competitive output quality. A system for training a sequence to sequence (S2S) machine learning model is proposed where neural unlikelihood objective approaches are used at (1) a target token level to discourage the generation of repeating tokens, and (2) a copy token level to avoid copying repetitive tokens from the source text. K-step ahead token prediction approaches are also proposed as an additional mechanism to augment the approach to further enhance the overall diversity of key phrase outputs.