Nuance Communications, Inc.
Neural Network with Embedded Filter Layer

Abstract:

A neural network embodiment comprises an input layer, an output layer and a filter layer. Each unit of the filter layer is configured to receive a filter layer input from a single preceding unit via a respective filter layer input connection. Each filter layer input connection is coupled to a different single preceding unit. The filter layer is configured to incentivize the neural network to learn to produce a target output from the output layer for a given input to the input layer while simultaneously learning weights for each filter layer input connection. The weights learned cause the filter layer to reduce a number of filter layer units that pass respective filter layer inputs as non-zero values. When applied as an initial internal layer between an input layer and an output layer, the filter layer incentivizes the neural network to learn which neural network input features to discard to produce the target output.