International Business Machines Corporation
Building pre-trained contextual embeddings for programming languages using specialized vocabulary

Abstract:

A method, a computer system, and a computer program product for building pre-trained contextual embeddings is provided. Embodiments of the present invention may include collecting programming code. Embodiments of the present invention may include loading and preparing the programming code using a specialized programming language keywords-based vocabulary. Embodiments of the present invention may include creating contextual embeddings for the programming code. Embodiments of the present invention may include storing the contextual embeddings.