International Business Machines Corporation
Computer-implemented event detection using sonification

Abstract:

Computer-implemented event detection includes obtaining, at one or more processors, multimedia data including multiple frames of video data and corresponding audio data. The processor(s) process the multiple frames to detect at least one object and to track the object(s) between two or more of the frames. The processor(s) generate sonification audio data representing a position of the object(s) in the two or more frames, movement of the object(s), or both the position and the movement of object(s). The processor(s) generate combined audio data including the audio data and the sonification audio data. The processor(s) generate one or more feature vectors representing the combined audio data and provide the feature vector(s) as input to a trained event classifier to detect an event represented in the multimedia data.