Meta Platforms, Inc.
Audio selection based on user engagement

Abstract:

In one embodiment, a method includes receiving audio input during an audio-video communication session. The audio input is generated by a first sound source within an environment and a second sound source within the environment. The method includes receiving video input depicting the first sound source and the second sound source in the environment. The method includes identifying the first sound source and the second sound source using the audio input and the video input. The method includes predicting a first engagement metric for the first sound source and a second engagement metric for the second sound source based on the identifying. The method includes processing the audio input to generate an audio output signal based on a comparison of the first engagement metric and the second engagement metric. The method includes providing the audio output signal to a computing device associated with the audio-video communication session.