Royal Bank of Canada
SYSTEM AND METHOD FOR HETEROGENEOUS MULTI-TASK LEARNING WITH EXPERT DIVERSITY

Abstract:

A computer system and method for training a heterogeneous multi-task learning network is provided. The system comprises at least one processor and a memory storing instructions which when executed by the processor configure the processor to perform the method. The method comprises assigning expert models to each task, processing training input for each task, and storing a final set of weights. For each task, weights in the expert models and in gate parameters are initialized, training inputs are provided to the network, a loss is determined following a forward pass over the network, and losses are back propagated and weights are updated for the experts and the gates. At least one task is assigned one exclusive expert model and at least one shared expert model accessible by the plurality of tasks.