Adobe Inc.
REFERRING IMAGE SEGMENTATION

Abstract:

A method, apparatus, and non-transitory computer readable medium for referring image segmentation are described. Embodiments of the method, apparatus, and non-transitory computer readable medium may extract an image feature vector from an input image, extract a plurality of language feature vectors for a referral expression, wherein each of the plurality of language feature vectors comprises a different number of dimensions, combine each of the language feature vectors with the image feature vector using a fusion module to produce a plurality of self-attention vectors, combine the plurality of self-attention vectors to produce a multi-modal feature vector, and decode the multi-modal feature vector to produce an image mask indicating a portion of the input image corresponding to the referral expression.