International Business Machines Corporation
Distributed Adversarial Training for Robust Deep Neural Networks

Abstract:

Scalable distributed adversarial training techniques for robust deep neural networks are provided. In one aspect, a method for adversarial training of a deep neural network-based model by distributed computing machines M includes, by distributed computing machines M: obtaining adversarial perturbation-modified training examples for samples in a local dataset D.sup.(i); computing gradients of a local cost function f.sub.i with respect to parameters .theta. of the deep neural network-based model using the adversarial perturbation-modified training examples; transmitting the gradients of the local cost function f.sub.i to a server which aggregates the gradients of the local cost function f.sub.i and transmits an aggregated gradient to the distributed computing machines M; and updating the parameters .theta. of the deep neural network-based model stored at each of the distributed computing machines M based on the aggregated gradient received from the server. A method for distributed adversarial training of a deep neural network-based model by the server is also provided.