Amazon.com, Inc.
Natural language dialog scoring

Abstract:

Techniques for generating a prompt coverage score, which measures an extent to which data output to a user during a dialog is repetitive and monotonous, are described. User input data and system output, corresponding to a dialog exchange between a user and a skill, may be determined. A portion of the system output data, corresponding to a system prompt representing default output data, may be determined. A first number, representing possible variants of the prompt, may be determined along with a second number, representing variants of the prompt output during the dialog exchange. A prompt coverage score may be determined based on the first and second numbers.