Adobe Inc.
SOCIAL BIAS MITIGATION IN TEXTUAL MODELS

Abstract:

A system for generating text using a trained language model comprises an encoder that includes a debiased language model that penalizes generated text based on an equalization loss that quantifies first and second probabilities of respective first and second tokens occurring at a first point in the generated text. The first and second tokens define respective first and second groups of people. The system further comprises a decoder configured to generate text using the debiased language model. The decoder is further configured to penalize the generated text based on a bias penalization loss that quantifies respective probabilities of the first and second tokens co-occurring with a generated word. The encoder and decoder are trained to produce the generated text using a task-specific training corpus.