Texas Instruments Incorporated
Parametric Power-Of-2 Clipping Activations for Quantization for Convolutional Neural Networks

Abstract:

In described examples of a method for quantizing data for a convolutional neural network (CNN) is provided. A set of data is received and quantized the using a power-of-2 parametric activation (PACT2) function. The PACT2 function arranges the set of data as a histogram and discards a portion of the data corresponding to a tail of the histogram to form a remaining set of data. A clipping value is determined by expanding the remaining set of data to a nearest power of two value. The set of data is then quantized using the clipping value. With PACT2, a model can be quantized either using post training quantization or using quantization aware training. PACT2 helps a quantized model to achieve close accuracy compared to the corresponding floating-point model.