International Business Machines Corporation
Machine learning in heterogeneous processing systems

Abstract:

Computer-implemented methods are provided for implementing training of a machine learning model in a heterogeneous processing system comprising a host computer operatively interconnected with an accelerator unit. The training includes a stochastic optimization process for optimizing a function of a training data matrix X, having data elements X.sub.i,j with row coordinates i=1 to n and column coordinates j=1 to m, and a model vector w having elements w.sub.j. For successive batches of the training data, defined by respective subsets of one of the row coordinates and column coordinates, random numbers associated with respective coordinates in a current batch b are generated in the host computer and sent to the accelerator unit. In parallel with generating the random numbers for batch b, batch b is copied from the host computer to the accelerator unit.