Did anyone order some natural interaction?

Here is an interesting company that seems to be in the business of selling natural interaction (am I the only one to get weird associations here?). Of course they had to do the obliged Minority Report clone They seem to be out there doing actual projects with gesture recognition, and touch-sensitive floors and such, without a care in the world (more at YouTube). Here I am thinking about what a long way this technology still has to go. Meanwhile in Chianti, Tuscany (Italy) these fine craftsman are working together with museums, studios, and banks to create really cool tools. I wonder if they are making a profit. That would be even more cool.

Tags: natural, gesture recognition, HCI, multimodal, multi-touch

Chomsky on Gesture and Language

Somewhat old, perhaps, but prof. Noam Chomsky made a few remarks in 1996 which I think illustrate the position held by many US linguists and other scientists regarding the difference between gestures and (sign) language. Language is discreet, gesture is continuous. But see some discussion. Personally, I think the characterizations of discreet and continuous have led to some remarkable claims. Some researchers looked at how people talked and found certain continuous systems of signification. For example the rising and falling of pitch to indicate height. And they dubbed it 'gesture' in language. Likewise, people studying sign language find continuous mechanisms and call it gesture in sign language. And vice versa, people who study gesture find discreet systems and point out that there are language-like properties in gesture. It seems more useful to me to think of speech and gesture as two of many forms of meaning-making. We use our hands, mouth, face, tools, media, canvas, even the web, to create meaning for eachother. And when we create meaning we use (sub)systems of form and meaning. Some of those systems are discreet and others are continuous. Ther e c a n e v e n b e continuity in the way I type this sentence. Were you ever forced to point the finger at someone? (source EurekAlert) Consider pointing. The question is not whether pointing is continuous or discreet as such. The question how we use different pointing mechanisms to communicate meaning. Some of those will be continuous, but even pointing can be as discreet as it gets.

Tags: gesture, sign language, pointing, Chomsky, discreet-continuous, speech, language, psychology

Practice Sign Language with CopyCat

Once upon a time (2005) in a far-away land (Rochester) there was a symposium. A band of researchers traveled there: Seungyon Lee, Valerie Henderson, Helene Brashear, and Thad Starner from the College of Computing, Georgia Institute of Technology, together with Harley Hamilton from the Atlanta Area School for the Deaf. They talked about User-Centered Development of a Gesture-Based American Sign Language Game. And then they presented it at a couple more conferences again (good output, guys :-) I would love to see some action, wouldn't you? (source Seungyon Lee Their educational game is called CopyCat. It was tested in a school with deaf kids using a Wizard-of-Oz setup. Our ELo-project will produce an educational application and not a game. But the distinction gets vague with very young kids. And our methods are quite similar. Let us see who will have the best recognition results, or better yet, the bigger benefit for signing children.

Tags: gaming, education, child development, sign language, gesture recognition, ELo

Sony Car Gear with Finger Gestures

The new Sony NaviGadgets, the NV-U81T and NV-U80, both have a touchscreen with Gesture Command. Hocus-pocus-take-me-home! (press release) Sony: "Frequently-used commands like “take me home” or “find the nearest fuel station” can be made with a quick, simple stroke of a finger on the large touch screen. This unique ‘Gesture Command’ improves driver safety and concentration by reducing the need for fiddly option selection or scrolling through on-screen menus. For safer calling while on the road, it can also dial a hands-free call to your home number on a Bluetooth equipped mobile phone with a single gesture."

Tags: gesture recognition, touchscreen, car, hands, HCI

Interpreters on Strike after VGT Recognition?

In Flanders, Belgium, sign language interpreters are on strike. Parliament recognized the Flemish Sign Language a while ago. But interpreters barely get paid for their services (in the classrooms). This appears to matter more than an offical statement. Or perhaps the interpreters felt they were backed by the new status of VGT and can now push for better policies? Are Flemish Deaf Finding their Pride? (source: SignWriting on the online VGT dictionary)

Tags: Deaf rights, sign language, official, policy, education, interpreting, Deaf culture

'Shy' Killer Plants are Spreading Fast

There are several forms of rapid plant movement which could be seen as intentional actions. (Let us leave aside their turning towards the sun and other tropism, taxis or kinesis, but temporarily grant them intentions). Why do these plants act? Sometimes to capture prey, like in the movie below. Sometimes to release seeds or pollen. The reasons for these movements are fairly obvious. To eat and reproduce. I think of these as practical actions, not gestures, since there is no intention to communicate. Other plants seem to shy away or curl up their leaves in response to touch or other stimuli. Dr. T. Ombrello - UCC Biology Department: "We can only guess as to why Mimosa pudica and a few other species have evolved to exhibit nyctinastic and seismonastic movements. It has been observed that folded and drooped leaves are not attractive to herbivores, and are often passed by in favor of more normal appearing leaves to eat. Also, folded and drooped leaves exchange less heat and water than fully expanded leaves, and this might have some survival value when the plants are under environmental stress." Although we may like to project our own shyness onto the plants behaviour, I do not think the plant is trying to appear modest. If it had legs it would probably flee, whatever impression that might leave behind. So, even if we grant a plant intentions it still doesn't intend to communicate with the movements treated here. No plant gestures here I am afraid. Maybe we need better examples, anyone?

Tags: gesture, plant, movement, behaviour, intention, anthropomorphism

Shakira: Ambassador of Gesture?

Shakira performed her "hips don't lie" at the MTV awards. Boy, is she a lady who loves to let her body do the talking? In this show she even mixes mudras into her cosmopolitan repertoire of dance moves.

Tags: mudras, shakira, seduction, body language, expressive, cross-cultural

Asimo Gestures

Asimo was in the news again (in Manilla), and I thought an update would be nice. "howwa YOU dowin?" (can Asimo look a girl up and down?) The little guy is currently equipped with some gesture recognition (he can wave back) and other computer vision capabilities. He has always had a bunch of synthesized gesture routines in his repertoire. When they made him speak they made him do co-speech gestures. I guess otherwise there would be nothing to watch, since he has no lips to move nor eyebrows to raise. Asimo is making a name for himself on stages everywhere. Rumour even has it he is getting pretentious and is plotting a coup against Mickey Mouse.

Tags: asimo, gesture synthesis, gesture recognition, humanoid, anthropomorphism

Mozilla Mouse Gestures

Could it be a real killer application of gesture recognition? It appears that mouse gestures are a viable option for browsing and games. Optimoz Mouse Gestures is a popular Mozilla Firefox extension which I have been trying for a while now. One could of course argue that this mouse gesturing is hardly new. We've had Palm Graffiti, handwriting recognition, pen gestures, Opera used it first, and we all drag-and-drop with the left mouse button. One could even say that every action used to control a computer other than talking is a gesture. All true. But none of that matters really. A killer app is not necessarily new. Just a really good combination of factors that sparks succes. My Favourite: Making any gesture ending with a straight Right-Up-Left movement opens all crossed links in tabs One of the great things about the current mouse gestures is the use of the right button. I always have that handy for mouse-down actions anyway. I can learn the gestures from a nice sidebar in Mozilla which I like. I did not have to pay anything. Installation was a breeze. No extra kit. And I can impress my colleagues. What more do you need?

Tags: gesture recognition, mouse, software, pen gestures, 2D

In Memoriam Bernard Tervoort

A founding father of sign language research has died on August 17: Professor Bernard T. M. Tervoort. His 1953 dissertation at the University of Amsterdam (UvA) was titled 'Structural analysis of visual language use in a group of deaf children'. It was groundbreaking, pioneering work in sign language research. He was the first to propose a linguistic status for sign languages. As such it predates the works of William Stokoe which usually receives that credit. Ben Tervoort, a dutch Umberto Eco? (source: crime.nl) His bibliography of works on sign language lists 102 works. He also wrote crime novels and childrens books. Some other memorials: Viataal, Doof.nl, Nederlands Gebarencentrum

Tags: sign language, literature, Tervoort, linguistic, memorial