First, let me introduce you to The Great Language Game. Created by “data scientist” Lars Yencken, this online challenge presents you with a series of audio clips that you try to identify from the list of language names given below each one. The better you do, the further you go, and the lists get longer. It’s a little bit of easy. Of course that’s French and not Mandarin! It’s a little bit of impossible. That’s really the name of a language? And it’s a lot of fun.

Go ahead and give it a try. It doesn’t take long. I’ll wait for you.

I’m thinking I could get a higher score if I could see the people speaking. It would really be great if they were wearing their national dress (think Miss Universe pageant). OK, that’s a bit much, but it might help a little even if I could see just their mouths. Of course, being familiar with the languages at hand is the biggest factor. But it make sense that looking at the way people speak should help us distinguish one language from another. You don’t need to have a PhD to figure that out. Actually, you don’t even have to be in preschool.

Kids See the Darndest Things

Janet Werker, director of the University of British Columbia’s Infant Studies Centre, has long been studying babies’ abilities to recognize different languages. Her research shows that infants as young as seven months old can hear the difference between languages by listening to grammar patterns. Werker was also part of a team, led by one of her students, Whitney Weikum, showing that even-younger babies can see the difference.

The group of researchers showed four-month-olds—from English-speaking households—silent videos of bilingual speakers using English or French or switching between the two. The babies were more interested and watched the videos longer when the speakers alternated languages.

When they tested six-month-old infants, some from monolingual homes and some from English-and-French-speaking homes, the results were the same. But when they showed the videos to eight-month-old babies, only the ones from bilingual homes continued to be able to distinguish the languages.

According to the researchers, this suggests that older “monolingual” babies lose their sensitivity to visual cues in language recognition because they no longer need them. “Bilingual” children, on the other hand, keep their ability longer, because they still need it as they learn two languages.

So if we lose that skill with age, how can those of us getting along in years work on regaining it? Welcome the lip-reading computer.

And Computers See Even More

If I had a computer that could read lips, I’d just sit back and watch it in awe. But five years ago a team of scientists at the University of East Anglia’s School of Computing Sciences couldn’t leave well-enough alone and gave their lip-reading computer the ability to recognize languages. Stephen Cox, one of the team’s leaders, says that their work is “the first scientific confirmation of something we already intuitively suspected—that when people speak different languages, they use different mouth shapes in different sequences.”

The group, using “statistical modeling,” studied the mouth movements of 23 bilingual and trilingual speakers. The result is a technology that can distinguish between a range of languages—from the similar to the widely different—including English, French, German, Arabic, Mandarin, Cantonese, Italian, Polish, and Russian.

So it seems that with practice, and a little technological help, we should be able to see the difference between spoken languages even if we can’t hear them. Why, we might get to the point where our looking-without-listening skills could rival our ability to listen with our eyes closed.

But We Don’t Need Our Eyes to Hear, Right?

Well, maybe it’s time to revisit The Great Language Game for a reminder of how imprecise our listening skills can be. Or if that isn’t sufficiently humbling, take a look at the following video on the “McGurk Effect.” Also called the “McGurk Illusion,” the phenomenon was discovered by accident as Harry McGurk and his lab assistant, John MacDonald, both of the University of Surrey, were studying how infants develop their perception of speech.

It’s bad enough when you need help to identify languages, but simple, basic letter sounds? I give up.

(Whitney Weikum, et al., “Babies Able to Tell through Visual Cues when Speakers Switch Languages,” ScienceDaily, May 25, 2007; “Lip-Reading Computers Can Detect Different Languages,” University of East Anglia, April 22, 2009;”The McGurk Effect: Hearing Lips and Seeing Voices,” Haskins Laboratories)

[photo: “Hear No Evil,” by McBeths Photography, used under a Creative Commons license]