International Business Machines Corporation
FIXED, RANDOM, RECURRENT MATRICES FOR INCREASED DIMENSIONALITY IN NEURAL NETWORKS

Abstract:

A method of operating a neural network. The input layer of the network may have n input nodes connected to output nodes via a hidden layer. The hidden layer may include m hidden nodes. The n input nodes may connect to a subset of k nodes of the m hidden nodes via respective synaptic connections, to which training weights are associated, which form an n.times.k input matrix W.sub.in, whereas a subset of m-k nodes of the hidden layer are not connected by any node of the input layer. Running the network may include performing a first matrix vector multiplication between the input matrix W.sub.in and a vector of values obtained in output of the input nodes and a second matrix vector multiplication between a fixed matrix W.sub.rec of fixed weights and a vector of values obtained in output of the m nodes of the hidden layer.