International Business Machines Corporation
Leveraging class information to initialize a neural network language model

Abstract:

Methods and systems for language processing includes initializing a word embedding matrix based on pre-determined word classes, such that matrix entries associated with a class of which a word is a member are initialized to a non-zero value and other entries are initialized to zero. A neural network is trained based on the initialized word embedding matrix to generate a neural network language model. A language processing task is performed using the neural network language model.