Analog Devices, Inc.
MULTI-MODAL FAR FIELD USER INTERFACES AND VISION-ASSISTED AUDIO PROCESSING

Abstract:

Far field devices typically rely on audio only for enabling user interaction and involve only audio processing. Adding a vision-based modality can greatly improve the user interface of far field devices to make them more natural to the user. For instance, users can look at the device to interact with it rather than having to repeatedly utter a wakeword. Vision can also be used to assist audio processing, such as to improve the beamformer. For instance, vision can be used for direction of arrival estimation. Combining vision and audio can greatly enhance the user interface and performance of far field devices.