Advanced Micro Devices, Inc.
OPTIMIZING INFERENCE FOR DEEP-LEARNING NEURAL NETWORKS IN A HETEROGENEOUS SYSTEM

Abstract:

Systems, methods, and devices for deploying an artificial neural network (ANN). Candidate ANNs are generated for performing an inference task based on specifications of a target inference device. Trained ANNs are generated by training the candidate ANNs to perform the inference task on an inference device conforming to the specifications. Characteristics describing the trained ANNs performance of the inference task on a device conforming to the specifications are determined. Profiles that reflect the characteristics of each trained ANN are stored. The stored profiles are queried based on requirements of an application to select an ANN from among the trained ANNs. The selected ANN is deployed on an inference device conforming to the target inference device specifications. Input data is communicated to the deployed ANN from the application. An output is generated using the deployed ANN, and the output is communicated to the application.