International Business Machines Corporation
Automated novel concept extraction in natural language processing

Abstract:

From a natural language document using a natural language concept analyzer, a set of natural language input concepts is extracted. Using a query generation model, a query corresponding to the set of natural language input concepts is generated. From a set of natural language results using the natural language concept analyzer, a set of natural language output concepts is extracted, a result in the set of natural language results comprising a portion of narrative text within a natural language corpus, the result identified by searching the natural language corpus using the query. Using the set of natural language input concepts and the set of natural language output concepts, a novelty concept is scored, the scored novelty concept comprising a degree to which a natural language input concept in the set of natural language input concepts is external to a boundary defined by the set of natural language output concepts.