Royal Bank of Canada
SYSTEM AND METHOD FOR MACHINE LEARNING WITH LONG-RANGE DEPENDENCY

Abstract:

A system, electronic device and method for improved neural network training are provided. The improved system is adapted for tracking long range dependence in sequential data during training, and includes bootstrapping a lower bound on the mutual information (MI) over groups of variables (segments or sentences) and subsequently applying the bound to encourage high MI.