Tag: bilingual

Don’t Know Your Malayalam from your Malay? Try Listening with Your Eyes

5337782718_129aee6b88_z

First, let me introduce you to The Great Language Game. Created by “data scientist” Lars Yencken, this online challenge presents you with a series of audio clips that you try to identify from the list of language names given below each one. The better you do, the further you go, and the lists get longer. It’s a little bit of easy. Of course that’s French and not Mandarin! It’s a little bit of impossible. That’s really the name of a language? And it’s a lot of fun.

Go ahead and give it a try. It doesn’t take long. I’ll wait for you.

I’m thinking I could get a higher score if I could see the people speaking. It would really be great if they were wearing their national dress (think Miss Universe pageant). OK, that’s a bit much, but it might help a little even if I could see just their mouths. Of course, being familiar with the languages at hand is the biggest factor. But it make sense that looking at the way people speak should help us distinguish one language from another. You don’t need to have a PhD to figure that out. Actually, you don’t even have to be in preschool.

Kids See the Darndest Things

Janet Werker, director of the University of British Columbia’s Infant Studies Centre, has long been studying babies’ abilities to recognize different languages. Her research shows that infants as young as seven months old can hear the difference between languages by listening to grammar patterns. Werker was also part of a team, led by one of her students, Whitney Weikum, showing that even-younger babies can see the difference.

The group of researchers showed four-month-olds—from English-speaking households—silent videos of bilingual speakers using English or French or switching between the two. The babies were more interested and watched the videos longer when the speakers alternated languages.

When they tested six-month-old infants, some from monolingual homes and some from English-and-French-speaking homes, the results were the same. But when they showed the videos to eight-month-old babies, only the ones from bilingual homes continued to be able to distinguish the languages.

According to the researchers, this suggests that older “monolingual” babies lose their sensitivity to visual cues in language recognition because they no longer need them. “Bilingual” children, on the other hand, keep their ability longer, because they still need it as they learn two languages.

So if we lose that skill with age, how can those of us getting along in years work on regaining it? Welcome the lip-reading computer.

And Computers See Even More

If I had a computer that could read lips, I’d just sit back and watch it in awe. But five years ago a team of scientists at the University of East Anglia’s School of Computing Sciences couldn’t leave well-enough alone and gave their lip-reading computer the ability to recognize languages. Stephen Cox, one of the team’s leaders, says that their work is “the first scientific confirmation of something we already intuitively suspected—that when people speak different languages, they use different mouth shapes in different sequences.”

The group, using “statistical modeling,” studied the mouth movements of 23 bilingual and trilingual speakers. The result is a technology that can distinguish between a range of languages—from the similar to the widely different—including English, French, German, Arabic, Mandarin, Cantonese, Italian, Polish, and Russian.

So it seems that with practice, and a little technological help, we should be able to see the difference between spoken languages even if we can’t hear them. Why, we might get to the point where our looking-without-listening skills could rival our ability to listen with our eyes closed.

But We Don’t Need Our Eyes to Hear, Right?

Well, maybe it’s time to revisit The Great Language Game for a reminder of how imprecise our listening skills can be. Or if that isn’t sufficiently humbling, take a look at the following video on the “McGurk Effect.” Also called the “McGurk Illusion,” the phenomenon was discovered by accident as Harry McGurk and his lab assistant, John MacDonald, both of the University of Surrey, were studying how infants develop their perception of speech.

It’s bad enough when you need help to identify languages, but simple, basic letter sounds? I give up.

(Whitney Weikum, et al., “Babies Able to Tell through Visual Cues when Speakers Switch Languages,” ScienceDaily, May 25, 2007; “Lip-Reading Computers Can Detect Different Languages,” University of East Anglia, April 22, 2009;”The McGurk Effect: Hearing Lips and Seeing Voices,” Haskins Laboratories)

[photo: “Hear No Evil,” by McBeths Photography, used under a Creative Commons license]

Advertisement

Monolinguals Unite: You Can Translate, Too

613445810_2249c2d193

Luis von Ahn, computer-science professor at Carnegie Mellon University, has a goal. It’s to translate the entire Web into every major language, for free. Sound impossible? Not to von Ahn. But he does see two obstacles: not enough bilinguals and not enough translator motivation.

So when it comes to translation, what can turn those obstacles from mountains into molehills? Von Ahn is working on an answer, and so is Chang Hu.

It Takes a Crowd

The Guatemalan-born von Ahn is best known for helping to invent CAPTCHAs. If you don’t know what a CAPTCHA is, it’s that image of distorted letters you see on a lot of Website forms. You’re required to type in those letters to prove that you’re a human, which keeps computer programs from fooling the system.

As he told the crowd at a TEDx Talk in 2011 (embedded below), Von Ahn estimates that each day, about 200 million CAPTCHAs are typed around the globe. With every CAPTCHA taking about 10 seconds to key in, that’s around 500,000 hours a day. Von Ahn wondered how he could redeem this “wasted” time and came up with reCAPTCHA.

Now owned by Google, reCAPTCHA replaces the often random characters of a CAPTCHA with actual words from books that are being digitized. The reason this is a good thing is because the text-scanning software used to digitize printed text can’t recognize every word, especially when dealing with books over 50 years old. But these hard-for-computers-to-read words aren’t hard for human’s at all. So when you’re typing in a CAPTCHA on one of over 350,000 sites using reCAPTCHA—including Facebook, Twitter, and Ticketmaster—you’re helping digitize books.

So what does this have to do with translation? Well, another of von Ahn’s projects, based on the same kind of crowd-sourced “human computing” as reCAPTCHA, is Duolingo. It’s a free language-learning site, currently teaching six languages. What makes Duolingo unique is that while you’re learning a language, you’re joining 10 million other users in translating text on the Web, because the phrases used by Duolingo come from real Websites.

For instance, after you learn some basic Spanish vocabulary, you’ll be able to test your skills by translating simple phrases to and from Spanish. And as you do so, you’ll be helping translate some English Websites into Spanish, or vice versa. Success earns you “skill points,” unlocking new lessons, while mistakes take away one of your hearts. Lose all of your hearts and you have to redo the level. As you learn more, you translate more-complex sentences, and, as your attempts are compared with those of others, useful, accurate translations are produced.

According to von Ahn, two great things about Duolingo are, “People really can learn a language with it, and they learn it about as well as the leading language-learning software,” and, “The translations that we get from people using the site, even though they’re just beginners . . . are as accurate as those of professional language translators.”

Oh, yeah, and did I mention it’s free? That’s possible because the sites that submit their text for translation are paying the tab—sites like Buzzfeed and CNN, which, von Ahn announced just a couple weeks ago, are the first to come on board.

Of course, even when there’s no monetary cost, not everyone wants to invest his time into the hours that are required for learning a language. If there could be a way for monolinguals to help out with just a few seconds—kind of like with the reCAPTCHAs—that might bring more people in.

Enter MonoTrans.

The Power of Widgets

MonoTrans (named MonoTrans2 in its newer version) is a process that combines machine translation with help from monolingual humans to produce accurate translations. A team from the University of Maryland’s Department of Computer Science, led by Chang Hu—a PhD candidate at UMD—proposed the process in 2010 to overcome the problem of not having enough bilingual translators to work on (a) texts in rare languages, and (b) huge amounts of text that would require enormous amounts of human effort.

MonoTrans starts with a computer translation of a passage, which is notorious for producing flawed (and often humorous) results. The output is then passed on to a person who speaks the target language. She then makes a guess as to the correct meaning and phrasing of the sentence, and her efforts are back-translated into the source language. Then a speaker of that language compares the results to the original passage, and the process between the two speakers is repeated until a satisfactory translation is produced. Along the way, the two monolinguals can help each other by including annotations, such as images and Web links, and multiple participants can vote on results.

While the process doesn’t necessarily take a large number of steps, it can be complicated and time consuming. MonoTrans2 addresses this problem by breaking the process into smaller, individual “microtasks,” so that many more people will take part in a translation, with each one handling only a small part of the whole process.

This new method was tested using children’s books at the International Children’s Digital Library. Visitors to the Website were presented with “widgets,” windows on a page that run a simple program. These widgets allowed users to edit or paraphrase a sentence, identify errors, or vote for the sentence they think is best.

The results of the trial show that using the MonoTrans Widgets in conjunction with Google Translate is a significant improvement over using Google Translate alone. And while this method also introduced some inherent problems, it indicates that the future of crowd-based computation by monolingual humans is very promising.

A Match Made in Cyberspace

Luis von Ahn coined the term human computation to describe using people to accomplish tasks that computers usually perform. Hu, in a blog post, sums up the relationship of human computation to translation in this way:

[H]uman computation presents a unique opportunity to significantly lower the threshold to do translation. At the same time, translation provides a set of interesting problems for human computation.

It sounds as if the relationship is something like a dance, with the dancers figuring out the steps as they go. Or maybe it’s more like a marriage, where both partners aid and challenge each other at the same time.

It’s a good union, and I’m glad there are people like von Ahn and Hu to serve as matchmakers.

(Luis von Ahn, “3,2,1 Takeoff! And We’re Translating the Web!“ Official Duolingo Blog, October 14, 2013; Chang Hu et al., “Translation by Iterative Collaboration between Monolingual Users,” University of Maryland Department of Computer Science, July 25, 2010; Chang Hu et al., “Deploying MonoTrans Widgets in the Wild,” University of Maryland, May 2012) 

[photo: “Crowd,” by James Cridland, used under a Creative Commons license]

Race, Culture, and Ethnicity in America: Checking Boxes and Switching Codes

In many ways race is about difference and how those differences are codified through language, categories, boxes, segmentation, and even the implicit sorting that goes on in our heads in terms of the way we label others and even ourselves.

—Michele Norris, The Race Card Project

Here’s Proof

4443977630_3b27ef0eb5
The results from the 2010 US Census include six single-race and 57 multiple-race groups.

This month’s issue of National Geographic marks the magazine’s 125th anniversary. That’s quite an accomplishment, and it represents thousands and thousands of pages of amazing photographs and stories. But National Geographic has more to share, and last month it opened up a new avenue: Proof, a blog “launched to engage ongoing conversations about photography, art, and journalism.”

Proof‘s first post is “Visualizing Race, Identity, and Change.” It features photographic portraits by Martin Schoeller (I wouldn’t have recognized that name before writing my last post) and discusses the dilemma faced by so many multi-racial Americans who find it difficult or impossible to check only one box on the census.

The post is a companion piece to a feature article in National Geographic, entitled “Changing Faces.” It’s written by Michele Norris, host and special correspondent for NPR, and curator of The Race Card Project, a site that collects views on the topic of race, all expressed in sentences of only six words. And you thought Tweets were short and to the point.

Of course, quite a few of the boxes were checked in the 2010 Census, and the results show a multi-colored collage of racial diversity across the American landscape. You can see that collage at The Racial Dot Map, created by Dustin Cable of the University of Virginia’s Weldon Cooper Center for Public Service. With one dot per person, the zoomed-out map shows a blending of colors representing the five categories of White, Black, Asian, Hispanic, and Other/Multi-Racial. But by zooming in, you can see the distinct contrasts at the neighborhood level, both intermixed and segregated.

Speaking in Code

NPR has started a new blog this year, too. It’s on race, ethnicity, and culture.

“Remember,” write the blog’s authors, “when folks used to talk about being ‘post-racial’?”

Well, we’re definitely not that. We’re a team of journalists fascinated by the overlapping themes of race, ethnicity and culture, how they play out in our lives and communities, and how all of this is shifting.

They call the blog Code Switching. In linguistics, code switching is when a multi-lingual speaker switches between languages within a conversation. More loosely defined, it can also include moving from one dialect, set of vocabulary, accent, or speaking style to another because of a number of factors, such as setting, relationship to the listener, and expectations.

Imagine that the doorbell rings. You answer the door and see that it’s your boss at the ad agency or your grandmother from Mexico or your childhood friend from the city or your ESL student or an acquaintance from the gym or a policeman. How might you talk differently to each of them? We all do it to one extent or another. But it’s an even bigger factor for those who move between races and cultures.

Gene Demby, host of the blog, calls this movement “hop-scotching between different cultural and linguistic spaces and different parts of our own identities.” Code Switching goes well beyond just methods of expression. It also covers news, info, and opinions on race relations and cultural interaction within America’s borders.

One of the things I like about Code Switching is its broad range of topics, from pop culture—”Why Black Heroes Make Zombie Stories More Interesting“—to historical—”The History of How a Shogun’s Boat Made Lincoln a ‘Tycoon’“—to current issues—”It Takes a Classroom to Learn the Family Language.” When it comes to race, culture, and ethnicity in America, they’re covering it all.

If only the staff here at Clearing Customs had the resources of NPR.

(Michele Norris, “Visualizing Race, Identity, and Change,” Proof, September 17, 2013)

[photo: “Question#9—Multiracial ID’s,” by Spot.us, used under a Creative Commons license]

Do You Hear What They Hear? Babies Are Listening Bilingually Even before They Can Speak

I remember having a conversation with an American raising his children in Taiwan. The father was fluent in Mandarin, and he’d started teaching that language to his son at a young age. He told me that it hadn’t worked for him and that he’d read that parents who speak more than one language to their small children only confuse them, as they aren’t able to tell one language from another.

It seemed like sound reasoning to me.

42052685_df923ad167So it surprised me to see new research showing that infants are better at becoming bilingual than I’d thought. As it turns out, by the age of seven months, babies can distinguish between languages by recognizing their different grammar structures.

The study, published in Nature Communications, focused on languages with opposite grammar patterns—such as English, which most often has the verb before the object, and Turkish, which follows the object-then-verb arrangement. Infants in bilingual environments pick up on these patterns and can distinguish between the languages, by listening to differences in pitch, duration, and word frequency.

Janet F. Werker, of the University of British Columbia, is co-author of the study, along with Judit Gervain, of the Université Paris Descartes. Werker reassures parents in bilingual households. “If you speak two languages at home, don’t be afraid, it’s not a zero-sum game,” she says. “Your baby is very equipped to keep these languages separate and they do so in remarkable ways.”

Mental Cartography

Werker and Gervain’s research is one more step forward in what we know about infants and language learning. In 2001, Patricia Kuhl, the director of the University of Washington’s Center for Mind, Brain, and Learning, told the Smithsonian magazine that six-to-eight-month olds can already distinguish between different vowel and consonant sounds in the languages they hear everyday and in languages “foreign” to them. But by their first birthday, they can no longer differentiate between sounds that are not part of a language that they’ve been exposed to. This is because they have developed a focus on familiar sounds, while “tuning out” unfamiliar ones. Then, later on in life, when the familiar competes against the unfamiliar, say, when learning a new language, the old sounds will usually win out. The result is a non-native accent.

To register what sounds infants can differentiate, Kuhl used a “head-turn” study (similar to that used by Werker and Gervain). In one example, two-thirds of both American and Japanese six-month olds could hear the difference between “la” and “ra.” But by the one-year mark, 80% of American children responded to the difference, while only 59% of the Japanese children did. Since the latter rate is only 9 percentage points above chance, this showed that the Japanese children had joined their parents in no longer being able to distinguish between the two sounds.

According to Kuhl,

The baby early begins to draw a kind of map of the sounds he hears. That map continues to develop and strengthen as the sounds are repeated. The sounds not heard, the synapses not used, are bypassed and pruned from the brain’s network. Eventually the sounds and accent of the language become automatic. You don’t think about it, like walking. [Familiar sounds] become more and more embedded into the map, until eventually they are almost ineradicable.

This accent map gets harder and harder to change as time goes by. On the other hand, if a child is exposed to multiple languages early enough—while the map is being drawn—the child can create more than one map at once.

Kuhl also has found (as shown in the TED Talk below) that if this exposure to languages is to have an effect on an infant, it must come from a live person. Listening to audio, even with an accompanying video of the speaker, does no good.

It’s Never Too Early to Learn

According to DNAinfo New York, some parents in the Big Apple are even learning a new language themselves in order to make sure that exposure to multiple languages happens for their children at an early age.

Take, for instance, Rhonda Ross, of Harlem, who went to a boarding school in a French-speaking area of Switzerland when she was a student. Later, when her son, Raif, turned one, she began speaking to him only in French. “I started with a French babysitter,” she said, “but a friend convinced me I would have to speak French to my son myself if I really wanted him to be fluent.”

Not being fluent herself, that means that Ross has to keep learning as she teaches her son. But she feels that the effort is worth it. In fact, she is so pleased with the outcome, that she’s introduced Raif to Mandarin and Spanish, as well.

Linguist Jennifer Wilkin, of Brooklyn, is another advocate of early bilingual education. In 2001, she founded Science, Language & Arts, where parents and children can learn French and Mandarin. “There is certainly a trend among New Yorkers to give a language to their children,” said Wiklin, who “knows several parents who are learning, and speaking, Spanish, Japanese, French and German to their children.”

While Wilkin’s school has students from preschool through fifth grade, Lyndsey St. John started Baby French in a Brooklyn ice-cream parlor and candy shop named The Candy Rush. The class caters to children who haven’t even learned to talk yet. “It’s really good to start those [language] pathways forming at a very early age,” said Wilkins. “Anywhere from 8 months to 3 years is when children are really sponges. They’re picking up everything.”

(Judit Gervain and Janet F. Werker, “Prosody Cues Word Order in 7-Month-Old Bilingual Infants,” Nature Communications, February 14, 2013; “Bilingual Babies Know Their Grammar by Seven Months,” The University of British Columbia Public Affairs, February 14, 2013; Edwin Kiester, Jr., “Accents Are Forever,” Smithsonian, January 2001; Julie Norwell, “New York Parents Learn Foreign Languages to Help Kids Become Fluent,” DNAinfo New York, March 6, 2013; “Even before They Utter First Words, Brooklyn Babies Take French Lessons,” DNAinfo New York, August 22, 2012)

[photo: “Mommy Tells a Story,” by Dan LaVange, used under a Creative Commons license]