In Love with SiSi

A wonderful bit of news has been hitting the headlines:

BBC News: Technique links words to signing: Technology that translates spoken or written words into British Sign Language (BSL) has been developed by researchers at IBM. The system, called SiSi (Say It Sign It) was created by a group of students in the UK. SiSi will enable deaf people to have simultaneous sign language interpretations of meetings and presentations. It uses speech recognition to animate a digital character or avatar. IBM says its technology will allow for interpretation in situations where a human interpreter is not available. It could also be used to provide automatic signing for television, radio and telephone calls.

Read the full story at IBM: IBM Research Demonstrates Innovative 'Speech to Sign Language' Translation System Demo or scripted scenario? Serendipity. Just this week a man called Thomas Stone inquired whether he could get access to the signing avatars of the eSign project. I passed him on to Inge Zwitserlood. She first passed him on to the eSign coordinator at Hamburg University, which was a dead end. Finally, he was pointed to the University of East Anglia, to John Glauert. And who is the man behind the sign synthesis in SiSi? From the press release from IBM:

John Glauert, Professor of Computing Sciences, UEA, said: "SiSi is an exciting application of UEA's avatar signing technology that promises to give deaf people access to sign language services in many new circumstances." This project is an example of IBM's collaboration with non-commercial organisations on worthy social and business projects. The signing avatars and the award-winning technology for animating sign language from a special gesture notation were developed by the University of East Anglia and the database of signs was developed by RNID (Royal National Institute for Deaf People).

Well done professor Glauert, thank you for keeping the dream alive. Now for some criticism: the technology is not very advanced yet. It is not at a level where I think it is wise to make promises about useful applications. The signing is not very natural and I think much still needs to be done to achieve of basic level of acceptability for users. But it is good to see that the RNID is on board, although they choose their words of praise carefully. It is amazing how a nice technology story gets so much media attention so quickly. Essentially these students have just linked a speech recognition module to a sign synthesis module. The inherent problems with machine translation (between any two languages) is not even discussed. And speech recognition only works under very limited conditions and produces limited results.

IBM says: "This type of solution has the potential in the future to enable a person giving a presentation in business or education to have a digital character projected behind them signing what they are saying. This would complement the existing provision, allowing for situations where a sign language interpreter is not available in person".

First, speech recognition is incredibly poor in a live event like a business presentation (just think of interruptions, sentences being rephrased, all the gesturing that is linked to the speech, etc.) and second, the idea that it will be (almost) as good as an interpreter is ludicrous for at least the next 50 years. The suggestion alone will probably be enough to put off some Deaf people. They might (rightly?) see it as a way for hearing people to try to avoid the costs of good interpreters. I think the media just fell in love at first sight with the signing avatar and the promises it makes. I also love SiSi, but as I would like to say to her and to all the avatars I've loved before: My love is not unconditional. If you hear what I say, will you show me a sign?

Doodling, Gesture, and Language Origins, the Movie

Here is a very entertaining video (nice music) that tells the tale of gesture and the origins of language in a nutshell. Much has been written about how the language capability may have evolved in humans with gesture as a stepping stone or how Man's first language may have been a signed language. Recent brain research findings (gesture+speech, mirror neurons, lateralization, sign language aphasia) have added more indirect 'evidence' for these theories. It is still hard to really prove anything about pre-historic events though... One thing that struck me is how the author talks about how people might be aided in their thinking when the gesture, or doodle and fidget. A reference to fidgeting! Hooray! Should I point out that I think gesture and fidgeting are quite different? No, I will just let it be.

Lead Guitar Body Language

Here is the Air Guitar World Champion 2007, Ochi "Dainoji" Yosuke (Japan) performing at Air Guitar World Championships 2007, Oulu, Finland: What a nice gesture performance: the pantomime, the gestures, the emotional expressions, the mimicry of the actual guitar play, and of course the dramatic gestures of a lead guitar player on stage. It makes me realize that a language may be found around in every hidden corner of human activity. In this case Dainoji shows a hilarious command of the body language of lead guitars. It also makes me wonder what exactly would remain of 'musical gestures', when all of a musicians 'body language' were hidden to the audience? I guess something would remain, and that would then be the real musical gesture.

Tags: guitar, body language, play, music

Baby Sign Mini Dictionary Flash

Here is a wonderful flash animation from Babystrology, featuring a signing baby: Lord knows, I am not the world's biggest fan of baby signing, but this is positively funny. I hope the creators keep treating baby sign with the same sense of humor. It is far too important a subject to ever talk seriously about. *Another nice example of using flash to present sign language online: Avon and Somerset Police. *Another nice example of animated kids signing: XV Congreso Mundial de la WFD.

Tags: baby sign, child development, dictionary, flash

Plains Indian Sign Language, Browning 1930

A wonderful collection of videos with Plains Indian Sign Language has been put on YouTube by Tommy Foley. A short 'teaser' with subtitles The videos were recorded in 1930, Browning Montana, when sign talkers from 14 different Plains nations gathered as participants in a conference organized by General Hugh L. Scott for the purpose of demonstrating their use of sign language. The first four videos (see this playlist) contain material from the participants at the conference themselves: Indians telling stories. Another six videos are a video version of a dictionary of the language (see this playlist).

Following the 1930 Plains Indian Sign Language Conference, General Scott intended to produce a cinematic dictionary of over thirteen hundred signs. Due to the Great Depression it would have been too difficult to get a second appropriation bill passed through congress to finish the cinematic dictionary. He did manage to get over three hundred signs filmed. (Note from Tommy Foley)

An important documenter of the Plains Indian Sign Language was Col. Garrick Mallery. He wrote 'Sign Language Among North American Indians Compared With That Among Other People And Deaf-Mutes' a report for the Smithsonian Insitute which was published in 1881, which is avaliable for free download as an e-book via Project Gutenberg.

Wiki Interactive Gestures

Dan Saffer, from Adaptive Path, and a member and contributor to the Interaction Design Association (IxDA), calls upon interaction designers to share their knowledge on gestural interaction in this new Interactive Gestures wiki. Read his 'Call to Arms' for more info and some nice links to web resources. Today's screen capture of the Interactive Gestures wiki I think it is a a good idea, which is why I am repeating the news here. Share and enjoy.

Signed Story Click, Clack, Moo

This wonderful multimedia video is a signed children's book. Ah, the revolution that is YouTube.

Gesturing Vervets and Sexual Harassment

Thanks to Alexis Heloir, a fellow PhD working on gestures, for sending me this story: Wild Vervet Monkeys Wreak Havoc in Kenya (or check the BBC which is the source). The most interesting part of this nice story (which tells of a group of vervets monkeys stealing food from a village and threatening specifically the women) is the following quote:

"The monkeys grab their breasts, and gesture at us while pointing at their private parts. We are afraid that they will sexually harass us," said Mrs Njeri.

Well, that is an interesting statement by Mrs Njeri. In the picture below you can see a vervet monkey next to a couple of children (from the Colobus Trust website, which also has more info on pest behavior by vervet monkeys) Vervet Monkeys (source) It must be said, these monkeys are not very big and the idea of 'sexual harassment' seems to me at first glance to be a tale of imagination gone wild. What are they going to do? Pinch a ladies bottom? Squeeze a boob? Certainly that is as far as they can go? Or is it? Perhaps I am thinking in the wrong direction. Perhaps sexual harassment is more like psychological warfare? Indeed, wikipedia states on sexual harassment that it can include many types of behavior and has a variety of purposes, most of which appear to be psychological rather than directly involving sexual intercourse. Dominance and humiliation can be important parts of it. From wikipedia we also learn that Vervets seem to "possess what has been called the "rudiments of language". Vervet Monkey alarm calls vary greatly depending on the different types of threats to the community. There are distinct calls to warn of invading leopards, snakes, and eagles." Now, there is an excellent web page on The Phallic Threat: Giant Penises and Similar Threat Devices. From it, I gather that the idea of a phallic threat is not unheard of, but instead common in both men and monkeys. Specifically on Primata, it is stated that Vervets have the folllowing use of the penile display.

penile display: This is when an adult male vervet monkey will present his erect penis and scrotum so that a neighboring group will see them (Estes, 1991). This display is used to demarcate territory (Estes, 1991). red-white-and-blue display: This display is used to communicate dominance by one male over another within a group (Estes, 1991). The male walks back and forth with his penis and scrotum in full view for the receiver to see; the sender will encircle the receiver (Estes, 1991). Occasionally the sender will stand on his hind legs and present his penis and scrotum to the receiver (Estes, 1991).

Moreover, the pigmentation of the Vervet Monkey's scrotum is a vivid blue that pales when the animal falls in social rank. In other words, Vervets may perhaps refer to their dominance over someone else by referring to the color of their genitals. So now we may have (1) an ability to communicate a variety of message, (2) a phallic threat with (3) a reference to dominance. Suddenly it is not so difficult to imagine that it is real. Or at least as real as sexual harassment gets. If the monkeys mean to express their dominance, mark their territory or humiliate the women and the women feel dominated or humiliated then that is a successful (if you will pardon the expression) case of sexual harassment. Unfortunately we cannot be sure of anything from such a distance. The whole story could just be exaggerated. It could even be an excuse for the villagers to start physically harassing the monkeys. Elsewhere: Atheism Central on this story A YouTube playlist on monkeys and their penal displays

US Police vs. The Finger

Giving the finger to the police, it remains an interesting case. Earlier I wrote that the Dutch police had booked and fined a man for it, which was overturned in court by a judge. Now, a similar thing has happened in Mobile, Alabama (USA). Philly.com brought the news: Mobile Appeals $3K Payment Over Gesture.

MOBILE, Ala. - The city of Mobile is appealing a judge's decision to award $3,000 to a motorist who was arrested for making an obscene hand gesture to a police officer. City attorney Ashton Hill said Wednesday the city is seeking to have Addison DeBoi's civil suit heard in circuit court. On July 31, District Court Judge Michael McMaken ruled in favor of DeBoi in his wrongful arrest suit and ordered the police department to pay $3,000. DeBoi, 56, was arrested by Officer Bristol Hines on Sept. 2, 2005, on a charge of disorderly conduct after he made a hand gesture while the two men were in their vehicles. He was acquitted last year and sued the city for $10,000 in damages, citing time lost from work, the threat of losing his engineering job , which requires a government security clearance , and the embarrassment of being put in jail. In awarding him $3,000, the judge said police officers must have "thicker skin" than the general public.

The rulings by the judges in the Netherlands and Alabama are along the same lines: the police should be less sensitive about being insulted. Unfortunately, being sensitive to insults is probably required if you do not want your authority challenged. Sensitivity lies at the heart of the perception of insults, which is a very subtle process. If the police are instructed to 'not be too sensitive' it probably greatly hinders their functioning. After all, we all rely on their judgment in all sorts of situations regarding aggression, violence, or misconduct, which all require a policeman to rely on his personal perception of other people's behavior. Why can't we rely on them to judge whether someone was 'insulting'? On the other hand, I was once taught how to referee a football game by none other than Mario van der Ende (a well known referee in Holland). He said that verbal abuse was always flying around on the pitch, most of it directed at him. He recommended to pretend not to hear it the first time (tempers can fly). A second time he would rebuff the perpetrator with a sneer or mocking insult of his own. Only if that wouldn't take care of it, a booking (yellow card) would be given. I think it made him one of the most respected referees in the competition. Maybe respect is gained as much from restraint in handing out punishment as it is from punishment itself. ps. I just finished this post when I saw that the UK police gave a man a 80 pound fine for giving the finger to a speed camera (and/or the cops operating it). I hope Simon Thompson (a respected citizen and school headmaster) will fight the decision, and let it go to court. I wonder whether the UK judge will follow the above examples from Alabama and the Netherlands. Elsewhere: The legal history of the finger (also showing many examples of verdicts and appeals where fines for fingers are overturned and arrests judged to be wrongful) - Jalopnik

Virginia is for Gangsters and Lovers

Some agency spent about $400.000 on a ad campaign featuring among others, this picture: My first impression was that it shows a very happy and very busy young lady. It's not often one sees someone simultaneously stamping grapes for wine with one foot, perform a balancing act and still have time to make a perfect heart gesture: bravo! What a nightmare photo shoot that must have been. Or a good bit of photoshopping. But the news (see Fox) is:

"The Virginia is for Lovers "Live Passionately" campaign will remove images of models making the hand gesture, one of several signs associated with the Gangster Disciples, Virginia Tourism Corp. officials said Friday. The gesture shows thumbs and index fingers formed into a heart."

I looked up the Gangster Disciples, and I checked Gang Signs, and more Gang Signs, and the most complete Gangster Hand Sign Index, where I found the following sign indeed shown as one of the Black Gangster Disciples: We have a match. Now the question remains: is it a wise decision to remove the image from the ad campaign? It is hard to judge from a distance. However, I do not think it was necessary, because it is nearly impossible that actual misunderstandings would arise from the use of the picture. Given the clarity of the image and the text on it, I cannot imagine anyone believing that this lady is flashing a gangster sign. Any gesture that is used as a conventional semiotic sign, or symbol, is what people choose it to be. If the producers of this campaign and the public that is adressed with it both agree that in this context it conveys something like 'love/heart/passion', then so be it. Other people in other contexts or other countries are free to attach their own meaning to the gesture. But, even though I can see no principled reason to scratch the image, it is understandable to do so anyway. Personally, I do not find it a very good picture, nor do I think that introducing a special gesture for a fairly complex message is going to be a succes (do they actually think people are going to adopt it in some way?). And why grant a bunch of gangsters a good laugh if you can avoid it (although they are probably already laughing their heads off from all the free publicity). In that sense, it is damage control to remove the picture. The best thing might have been to completely ignore the complaints and turn a blind eye to the gangster's sign. In that way, the gesture might have even been seen as ridiculing the gangster sign. Anyone flashing it could be mocked as a supporter of the campaign 'Virginia is for Lovers' instead of being a hardcore gang member. I admit, it probably wouldn't work that way, but it would have been better than granting the gangster disciples this PR victory. Other post on MS13 and gang gestures.