International Business Machines Corporation
VISUAL QUESTION GENERATION WITH ANSWER-AWARENESS AND REGION-REFERENCE

Abstract:

A computer-implemented method for visual question generation includes training an alignment module to analyze an image, an answer hint, and a visual hint with respect to the image. A k-nearest neighbors (KNN) graph is constructed by performing an aligned embedding for each region of the image. A node embedding component is generated by using a graph embedding component of the KNN graph. A visual question is generated by sequence decoding each image and graph of the image.