The McGurk Effect:Does Age Matter?
Most people assume that they see
and hear the world through two distinctly separate sensory channels; however,
recent research indicates that some neurons in the brain respond to aural and
visual stimuli simultaneously. Individuals experience the McGurk Effect when
viewing a video of a person mouthing a syllable dubbed with the audio of a different
The objectives of this project were to compare the awareness of aural and visual roles in speech perception and the occurrence and perceptions of the McGurk Effect among different age groups. Prior to testing, subjects were asked if they thought their eyes help them hear. Two hundred eighty subjects were tested using three syllable pairs (aural BA-visual GA, aural MA-visual KA, and aural PA-visual FA). Twenty males and twenty females in each of seven age groups from early childhood through late adulthood individually viewed three videos of the researcher mouthing a syllable dubbed with a different aural syllable, and responded with what was perceived. Each also listened to the audio only and told what was heard.
A total of 1480 responses were analyzed using a Factorial Analysis of Variance of Proportions. The following conclusions were drawn: 1.) There are differences between age groups in prior knowledge of aural and visual roles in speech perception; 2.) The occurrence of the McGurk Effect varies depending on the pairing of syllables, and 3.) Variation exists among age groups in reported perception of a specific aural-visual stimulus.
Most people assume that they see
and hear the world through two distinctly separate sensory channels. Vision
involves detecting variations in wavelengths of light energy, and hearing responds
to changes in air pressure produced by vibration (1). For many years most neuroscientists
thought that different regions of the brains cerebral cortex independently
receive and process visual and auditory information. However, evidence shows
that this may not be the case. Recent research leads some to believe that multisensory
neurons in the brain respond to light radiation and acoustic pressure waves
Psychologist Thomas A. Stoffregen has recently proposed a major revision of perceptual theory. He believes that even though some brain cells respond strongly to a single type of sensory stimulation, they may also react to input from several systems (2).
One significant example is speech perception which depends upon what is seen and heard.
The McGurk Effect, named after a psychologist at the University of Surrey in England, was discovered during developmental speech perception studies in 1976 (3). It is caused by dubbing the video of a person saying a syllable with the audio of a different syllable. The syllable most subjects say they hear is a blending of auditory and visual syllables, although sometimes they report hearing a visually dominant syllable (4).
Because of the limits on technology in 1976, scientists could not explain the cause of the McGurk Effect. Now research of the brain, including brain scans, has helped in uncovering reasons for sensory-mixing occurrences. The McGurk Effect occurs even when the observer is aware of the audiovisual discrepancy (4). It provides evidence that the brain treats visual speech information as if it is auditory speech (5). It appears that humans automatically integrate visual speech into what is heard.
The importance of audiovisual speech perception has been shown by research. If the face of a speaker is seen, it can significantly aid in understanding a low quality speech signal. Also, visual speech information helps a person understand speech that is complicated or has a foreign accent. In addition, both visual and auditory speech information facilitates speech development in young children (4). The McGurk Effect has been utilized in audiovisual speech integration and speech reading (3).
Other studies have shown the connection between vision and hearing. Brain scans indicate cochlear implants improve hearing and vision in persons with hearing loss who had previously used lip reading. Sounds of syllables and words cause stronger neural responses in auditory stimulation representing potential dangers activate separate areas of the brains cortex for touch, sight, and sound in addition to three other interconnected, cross-sensory regions of the cortex (2).
Also, the amygdala, an almond-shaped brain region below the cortex, shows increased stimulation when viewing a scared-looking face with a fearful voice. It does not happen when the emotions in the voice and face do not match. This sensory-blending research was done by Raymond J. Dolan of University College London and reported in 2001 in Proceedings of the National Academy of Sciences (2).
The objectives of this project were to compare the awareness, occurrence, and perceptions of the McGurk Effect among different age groups. The null hypotheses were that there would be no significant differences among age groups related to: 1) prior knowledge of aural and visual roles in speech perception; 2) occurrence of the McGurk Effect with different aural-visual syllable pairs; and 3) reported perceptions of a specific aural-visual stimulus.
The videos used in testing were
created with the researcher mouthing syllables while another person spoke different
syllables aloud. The first video consisted of aural BA, visual GA; the second
video was aural MA, visual KA; and the third video was aural PA, visual FA.
The videos were loaded onto a laptop computer for testing convenience. Signed
consent forms were obtained for each subject before testing.
During testing, each subject sat facing the laptop computer screen. Each participant was asked if they thought their eyes and ears work together to determine what they hear, and the response was recorded. The subject was then informed that there would be three separate videos, each would be played twice, and to pay close attention. After the subject viewed the first video (containing simultaneous aural and visual stimuli) twice, the syllable he or she perceived was recorded. The procedure was repeated for the second and third videos.
Then the subject listened to the aural stimuli from the same three videos with eyes closed, and the syllable he or she perceived each time was recorded.
Two hundred eighty subjects were tested, twenty males and twenty females in each of the following seven age groups: Early Childhood (ages 3-8); Later Childhood (ages 9-12); Adolescence (ages 13-18); Young Adult (ages 19-29); The Thirties (ages 30-39); Middle Age (ages 40-55); and Late Adulthood (ages 55-75). A total of 1,480 responses were analyzed.
RESULTS & DISCUSSION
In response to the question Do
you think your eyes help you hear? as age increased the number of
people who said yes increased. Only two females out of the forty children tested
in the youngest group said yes. However, 90% of middle age males and 95% of
the oldest males indicated before testing that their eyes help them hear, possibly
due to hearing loss. (Graph 8).
A Factorial Analysis of Variance of Proportions was calculated to determine the effects gender, stimulus, and age on the results of each trial (Table 1). In trial one aural BA, visual GAsignificance was found (p<.05) for stimulus. Even though almost every subject in all groups perceived a syllable other than BA when viewing the clip, the youngest children and the older males also inaccurately perceived BA with their eyes closed more than half the time (Graph 1 & Graph 7). When the McGurk Effect occurred a variety of syllables were perceived. The most common responses were SA, DA, and THA. The most obvious difference was present in the female group where the younger females heard THA, while the older females heard SA (Graphs 9 and Graph 15).
In trial twoaural MA, visual KAthe McGurk Effect was consistent across age and gender groups, and the majority of subjects accurately perceived MA in the aural test. Statistics showed significance only in the stimulus factor. Even 87% of the subjects perceived the syllable LA instead of MA in the aural-visual test (Graph 16-22).
Trial threeaural PA, visual FAwas the syllable pair with the greatest discrepancy between what the subjects saw and heard. Many people commented on the obvious difference and were hesitant in deciding. In the aural test, 94% heard the correct syllable PA. Thirty-four percent correctly heard PA while they were watching the video of the researcher mouthing FA, showing the McGurk Effect was not present. Sixty-three percent perceived FA, which was actually what they were seeingbut also not the McGurk Effect. Only three percent perceived a syllable other than what was seen or heard. Significance was found due to gender, stimulus, and age. This was due to the visually dominant stimulus, not the McGurk Effect. Younger children studying phonics have a tendency to read lips, as well as older people who are dependent upon lip reading due to loss of hearing. Younger females and older males tend to lip-read the most.
The following conclusions have been
drawn based on the results of this study: