NVIDIA Corporation
PRUNING NEURAL NETWORKS THAT INCLUDE ELEMENT-WISE OPERATIONS

Abstract:

Input layers of an element-wise operation in a neural network can be pruned such that the shape (e.g., the height, the width, and the depth) of the pruned layers matches. A pruning engine identifies all of the input layers into the element-wise operation. For each set of corresponding neurons in the input layers, the pruning engine equalizes the metrics associated with the neurons to generate an equalized metric associated with the set. The pruning engine prunes the input layers based on the equalized metrics generated for each unique set of corresponding neurons.