Royal Bank of Canada
Reinforcement-based system and method for detecting system vulnerabilities

Abstract:

A system and method for testing system vulnerabilities are provided. The method may include: training, by a processor, a machine learning model and agent to generate a payload to test vulnerabilities in the application by: selecting one or more input data from the action space to form an action data; electronically providing the action data as an input to the application; monitoring utilization of one or more system resources in response to the action data being inputted to the application; determining a score of utilization based on a result of the monitoring; determining a corresponding reward score for the action data based on the score of utilization; and identifying one or more of the action data to form a pool of candidate data for the application.