Xilinx, Inc.
System and method for implementing neural networks in integrated circuits

Abstract:

A neural network system includes an input layer, one or more hidden layers, and an output layer. The input layer receives a training set including a sequence of batches and provides to its following layer output activations associated with the sequence of batches respectively. A first hidden layer receives, from its preceding layer, a first input activation associated with a first batch, receive a first input gradient associated with a second batch preceding the first batch, and provide, to its following layer a first output activation associated with the first batch based on the first input activation and first input gradient. The first and second batches have a delay factor associated with at least two batches. The output layer receives, from its preceding layer, a second input activation, and provide, to its preceding layer, a first output gradient based on the second input activation and the first training set.