QUALCOMM Incorporated
Learning a truncation rank of singular value decomposed matrices representing weight tensors in neural networks

Abstract:

An apparatus for learning a rank of an artificial neural network is configured to decompose a weight tensor into a first weight tensor and a second weight tensor. A set of rank selection parameters are applied to the first weight tensor and the second weight tensor to truncate the rank of the first weight tensor and the second weight tensor. The set of rank selection parameters are updated simultaneously with the weight tensors by averaging updates calculated for each rank selection parameter of the set of rank selection parameters.