Intel Corporation
Generalized Activations Function for Machine Learning

Abstract:

The present disclosure provides a machine learning model where each activation node within the model has an adaptive activation function defined in terms of an input and a hyperparameter of the model. Accordingly, each activation node can have a separate of distinct activation function, based on the adaptive activation function where the hyperparameter for each activation node is trained during overall training of the model. Furthermore, the present disclosure provides that a set of adaptive activation functions can be provided for each activation node such that a spike train of activations can be generated.