The Impact of Synchronized Visual and Auditory Attention on Human Perception

The cocktail party problem shows the remarkable human ability to selectively attend to and recognize one source of auditory input in a noisy environment. However, individuals may struggle to identify a speaker’s voice when they are unfamiliar with the speakers and don’t have a clear visual focus, which results in less visual information. This raises the question: How can visual information aid in extracting information from a speaker’s voice? This study explores how synchronized visual and auditory attention impact human perception in scenarios involving two speakers. Using Tobii Glasses 3 to track participants’ eye movements and pupil diameters, combined with questionnaire responses, we explore how these factors influence speech comprehension. Our results demonstrate that participants achieve higher accuracy in speech comprehension when they focus their gaze on the speaker they are listening to, compared to scenarios where visual attention is divided between speakers or where they rely solely on auditory cues. These findings highlight the effectiveness of synchronizing visual and auditory attention in improving the acquisition and processing of information.