Apple Inc.
COMPILING MODELS FOR DEDICATED HARDWARE

Abstract:

The subject technology runs a compiled neural network (NN) model on a particular processor with multiple priority queues for executing different processes, the compiled NN model being assigned to a particular priority queue, and the compiled NN model includes context switch instructions that were previously inserted into a neural network (NN) model from which the compiled NN model was compiled. The subject technology determines that a particular context switch instruction has been executed by the particular processor. The subject technology determines that a different process is waiting to be executed, the different process being assigned to a different priority queue and the different process being a higher priority process than the running compiled NN model. In response to executing the particular context switch instruction, the subject technology performs a context switch to the different process assigned to the different priority queue when the different process is waiting to be executed.