Royal Bank of Canada
SYSTEM AND METHOD FOR CONTROLLABLE MACHINE TEXT GENERATION ARCHITECTURE

Abstract:

A computer system and method for machine text generation is provided. The system comprises at least one processor and a memory storing instructions which when executed by the processor configure the processor to perform the method. The method comprises processing an original sentence structure through an encoder neural network to decompose the original sentence structure into an original semantics component and an original syntax component, processing the original syntax component through a syntax VAE to receive a syntax mean vector and a syntax covariance matrix, obtaining a sampled syntax value from a syntax Gaussian posterior parameterized by the syntax mean vector and the syntax covariance matrix, processing the original semantics component through a semantics VAE to receive a semantics mean vector and a semantics covariance matrix, obtaining a sampled semantics vector from the Gaussian semantics posterior parameterized by the semantics mean vector and the semantics covariance matrix, and processing the sampled syntax vector and the sampled semantics vector through a decoder neural network to compose a new sentence. The semantics mean vector is constrained to a simplex associated with a semantics Gaussian posterior and learned during training of a sentence model.