International Business Machines Corporation
TIME ESTIMATOR FOR DEEP LEARNING ARCHITECTURE

Abstract:

A method for optimizing a neural network architecture by estimating an inference time for each operator in the neural network architecture is provided. The method may include determining a benchmark time for at least one single-path architecture out of a plurality of single-path architectures associated with the neural network by sampling the at least one single-path architecture from the neural network, wherein the at least one single-path architecture comprises one or more operators. The method may further include, based on the benchmark time for the at least one single-path architecture, determining an estimated inference time for an operator, wherein determining the estimated inference time for the operator comprises, applying an operator function, wherein the operator function comprises a function based on a difference between the benchmark time associated with the at least one single-path architecture and the estimated latency of the neural network.