A Nice Gesture by Jeroen Arendsen

A Nice Gesture by Jeroen Arendsen

A mix of posts on gesture, HCI, perception or robots and some personal information.

A Nice Gesture by Jeroen Arendsen RSS Feed
 
 
 
 

Avatar Kinect


Introduction to Avatar Kinect by Microsoft.

Avatar Kinect is a new social entertainment experience on Xbox LIVE bringing your avatar to life! Control your avatar’s movements and expressions with the power of Avatar Kinect. When you smile, frown, nod, and speak, your avatar will do the same.

Ah, new developments on the Kinect front, the premier platform for Vision based human action recognition if we were to judge by frequency of geeky news stories. For a while we have been seeing various gesture recognition ‘hacks’ (such as here). In a way, you could call all interaction people have with their Xbox games using a Kinect gesture recognition. After all, they communicate their intentions to the machine through their actions.

What is new about Avatar Kinect? Well, the technology appears to pay specific attention to facial movements, and possibly to specific facial gestures such as raising your eye brows, smiling, etc. The subsequent display of your facial movements on the face of your avatar is also a new kind of application for Kinect.
 

The Tech Behind Avatar Kinect

So, to what extent can smiles, frowns, nods and such expressions be recognized by a system like Kinect? Well, judging from the demo movies, the movements appear to have to be quite big, even exaggerated, to be handled correctly. The speakers all use exaggerated expressions, in my opinion. This limitation of the technology would certainly not be surprising because typical facial expressions consist of small (combinations of) movements. With the current state of the art in tracking and learning to recognize gestures making the right distinctions while ignoring unimportant variation is still a big challenge in any kind of gesture recognition. For facial gestures this is probably especially true given the subtlety of the movements.


A playlist with Avatar Kinect videos.

So, what is to be expected of Avatar Kinect. Well, first of all, a lot of exaggerating demonstrators, who make a point of gesturing big and smiling big. Second, the introduction of Second Life styled gesture routines for the avatar, just to spice up your avatars behaviour (compare here and here). That would be logical. I think there is already a few in the demo movies, like the guy waving the giant hand in a cheer and doing a little dance.

Will this be a winning new feature of the Kinect? I am inclined to think it will not be, but perhaps this stuff can be combined with social media features into some new hype. Who knows nowadays?

In any case it is nice to see the Kinect giving a new impulse to gesture and face recognition, simply by showcasing what can already be done and by doing it in a good way.

Some more Garfield gestures

A pleasurable pastime it is. Browsing through Garfield comics and smiling at the nice gestures Jim Davis draws to convey Garfield’s communication. I created a couple of lists ealier (here and here), and here is another list:

Polite palm out point & reproach with arms on hip
Here’s me! arms wide & reproach with arms on hip
Thumb pointing backward
Challenging grin
Jon’s angry glance and Garfield’s defensive point
Duh!

Brilliantly funny ‘body language’ instructional tape

Here is a must-see video for anyone who is interested in gestures and body language and has a sense of humour. Be warned, it may force you to rethink some of your ideas about the conventionality of body language and the extent to which interpreting it can be taught (should you be a communications trainer).

In any case, it’s good for a laugh :-)

Update:

Here is a collection of the sort of body language instruction that the above video is a parody of (with the exception of the fifth which again is a parody):

Book Review of: Imitation and Social Learning in Robots, Humans and Animals

In 2007 an interesting book was published that I believe is also relevant to gesture researchers:

Imitation and social learning in robots, humans and animals: behavioural, social and communicative dimensions.
Chrystopher L. Nehaniv, Kerstin Dautenhahn (Eds.). Cambridge University Press, 2007 - 479 pagina’s (available online in a limited way, here)

The book is an excellent volume with many interesting chapters, some with contributions by the editors themselves but also by many other authors. Personally, I found the following chapters most interesting (of 21 chapters):

  • 1. Imitation: thoughts about theories (Bird & Heyes)
  • 2. Nine billion correspondence problems (Nehaniv)
  • 7. The question of ‘what to imitate’: inferring goals and intentions from demonstrations (Carpenter & Call)
  • 8. Learning of gestures by imitation in an humanoid robot (Calinon & Billard)
  • 10. Copying strategies by people with autistic spectrum disorder: why only imitation leads to social cognitive development (Williams)
  • 11. A Bayesian model of imitation in infants and robots (Rao et al.)
  • 12. Solving the correspondence problem in robotic imitation across ambodiments: synchrony, perception and culture in artifacts (Alissandrakis et al.)
  • 15. Bullying behaviour, empathy and imitation: an attempted synthesis (Dautenhahn et al.)
  • 16. Multiple motivations for imitation in infancy (Nielsen & Slaughter)
  • 21. Mimicry as deceptive resemblance: beyond the one-trick ponies (Norman & Tregenza)

I’ll probably update this post with more in-depth review remarks later… But at least chapter 21 has connections to earlier posts here regarding animal gestures, such as here.

Gesture Recognition in the Operating Room

In the news recently: Future Surgeons May Use Robotic Nurse, ‘Gesture Recognition’.

ScienceDaily (Feb. 3, 2011) — Surgeons of the future might use a system that recognizes hand gestures as commands to control a robotic scrub nurse or tell a computer to display medical images of the patient during an operation.

Purdue industrial engineering graduate student Mithun Jacob uses a prototype robotic scrub nurse with graduate student Yu-Ting Li. Researchers are developing a system that recognizes hand gestures to control the robot or tell a computer to display medical images of the patient during an operation. (Credit: Purdue University photo/Mark Simons)

Purdue industrial engineering graduate student Mithun Jacob uses a prototype robotic scrub nurse with graduate student Yu-Ting Li. Researchers are developing a system that recognizes hand gestures to control the robot or tell a computer to display medical images of the patient during an operation. (Credit: Purdue University photo/Mark Simons)

I have noticed similar projects earlier, where surgeons in the OR were target users of gesture recognition. The basic idea behind this niche application area for gesture recognition is fairly simple: A surgeon wants to control an increasing battery of technological systems and he does not want to touch them, because that would increase the chance of infections. So, he can either gesture or talk to the machines (or let other people control them).

In this case the surgeon is supposed to control a robotic nurse with gestures (see more about the robotic nurse here). You can also view a nice video about this story here; it is a main story of the latest Communications of the ACM.

Well, I have to say I am in doubt if this is a viable niche for gesture recognition. So far, speech recognition has been used with some succes to dictate operating reports during the procedure. I don’t know if it has been used to control computers in the OR. Frankly, it sounds a bit scary and also a bit slow. Gesture and speech recognition are known for their lack of reliability and speed. Compared to pressing a button, for example, they give more errors and time delays. Anything that is mission-critical during the operation should therefore not depend on gesture or speech control would be my opinion.

However, the real question is what the alternatives for gesture or speech control are and how reliable and fast those alternatives are.  For example, if the surgeon has to tell another human what to do with the computer, for example displaying a certain image, then this can also be unreliable (because of misinterpretations) and slow.

The article mentions several challenges: “… providing computers with the ability to understand the context in which gestures are made and to discriminate between intended gestures versus unintended gestures”. That sounds like they also run into problems with fidgeting or something similar that surgeons do.

In sum, it will be interesting to see if surgeons will be using gesture recognition in the future, but I wouldn’t bet on it.

Funny Gesture Ad: Devil or God

Here is a picture from the Dutch site Jijdaar.nl. It asks kids to choose between listening to the Devil or God (and invites them to come to a lecture). They are worried about children listening to music that does not carry the word of God, praise the Lord, etc. In fact, all (pop) music about love (not aimed at God), sex, having a good time, is considered diabolical.

Invitation to lecture about the presence of the devil in pop music

Pretty cocky to expect kids will make the desired choice when forced. What would you choose?

A silent, gestural sponsor message: Li-Ning

Source: Graceclick.ca

Source: Graceclick.ca

What are these guys doing? Are they flashing a gang sign? Nope. It’s a gesture that mimics the logo of their sponsor, Li-Ning.

Now, in the news today, you can follow a nicely evolving story about this practice (here, here, and here for example).

The immediate cause of the story was in incident on a podium at the Asian Games involving gymnasts Lu Bo, Teng Haibin and Hisashi Mizutori…

… After the all-around ceremony, Teng was forced to explain a hand gesture he and Lu made on the podium, where they appeared to aim a gun at Mizutori. The two Asian powers have a long history of war and animosity.

“I would like to explain that the gesture which looks like I am using a gun is not hostile,” Teng said, explaining that the gesture is meant to symbolize the Li Ning logo.

Mizutori made the same gesture back, though he said later he was not sure what it meant.

“I thought maybe I should just go along with it, so I just did it,” Mizutori said. …

Amanda Turner – IG

So, are these gymnasts doing something improper? On the one hand, taking a ‘high’ semiotic perspective, a gesture such as the one they display is a typical symbolic sign. In gesture terms, it is an emblem. As such, it is much the same as the sponsor logos on athletes’ clothing (which are accepted).

On the other hand, the method of delivery, the carrier of the symbolic meaning may be very important here. A logo on a shirt delivers a message much more quietly than a gesture made on a public podium. Humans are hard- and/or softwired to pay attention to gestures, because there may be some important communication on that channel that may concern you. That is at least how I see it. It is nearly imposssible to see a man and ignore his gestures. That is not true for a logo. I can easily ignore a logo.

In other words, I would not be surprised if we look back in about five years time, and remember how Li-Ning pioneered a sales tactic that became booming business. I am very curious if the gymnasts will be sanctioned.

M. Obama and Sembiring: The Handshake that Shook Indonesia

In the news today, another wonderful gesture story. An Indonesian politician, Tifatul Sembiring, shook hands with Michelle Obama. Sembiring is a conservative Muslim who states that he should not touch women who aren’t related to him (he avoided doing so while meeting Tsunami victims for example). Sembiring quickly tried to spin the story via Twitter and said:

I tried to prevent [being touched] with my hands but Mrs. Michelle held her hands too far toward me [so] we touched.

Here are a couple of video’s for you to see for yourself…

At first glance, it looks Sembiring doesn’t have much of a point, but I am actually inclined to grant him his point anyway, though not entirely. Here’s why: Michelle Obama can be seen to initiate the handshake, as she raises her arm and hand towards him (although it is not very clearly visible). At that point, Sembiring is already ‘forced’, in a way, to respond, or otherwise, if he left her hand hanging there without taking it, he might cause an awkward situation and possibly a loss of face for her, for him or both.

There is however a counterargument: Sembiring appears to quickly glance down just before the handshake which Michelle Obama could have seen as a cue to initiate a handshake. He is also in a posture that is open to shaking hands, probably caused by just shaking her husband’s hand, with hands available and an inviting demeanor. She may be looking for such cues if she is sensitive to the issue of shaking or not shaking hands.

It would be interesting to know if she avoided shaking hands with some other conservative Muslems in the line, or that she simply shook all their hands? The man to the left of Sembiring appears to do a better job of avoiding the handshake, by keeping his hands to his side and by looking mostly at pres. Obama’s face. As far as I can tell, Michelle Obama does not (try to) shake hands with that man.

So Sembiring’s mistake, if you want to call it that, might not be that he took her hand and shook it, but that he invited the initiating of the handshake through his behaviour (his eye gaze, and his posture). And then again, if you just shook hands with the US president, would you be strong enough to control your body language to such a degree?

Thesis of Mats Andrén

For those among you who are interested in good gesture research it may be of interest to know that Mats Andrén, from Lund University, has published (online) his thesis called Children’s Gestures from 18 to 30 months. So far, I am enjoying reading it very much. Good job, Mats, and good luck with the defense :-)

What is Social Robotics?

Is it possible to give a good definition of social robotics? Is it a field of scientific study or is it only a catch phrase for exciting robot stuff? If it is a field of study, can we identify what belongs to it and what is outside of the field? Should we already set such boundaries or should we wait a while to give maximum growing room to the first seeds being planted by enthusiastic researchers and engineers around the world?

Instead of answering these questions here directly, I want to give you two answers of a different kind.

The first answer is that sometimes things can best be defined by identifying good examples (see explanation of Prototype Theory). If enough people can agree on good examples of social robotics then this defines the phrase ‘Social Robotics’ as a usable concept. This kind of definition plays an important role in the study of language and, given that the word ‘robot’ came from literature rather than science, it appears appropriate to try to define it in this way. Therefore, I collected the following videos that, in my opinion, each deal with one or more aspects of social robotics. They are all good examples of social robotics.

As a second answer, which may be more useful if you need more clarity fast, here is a reference to the call for participation of our recent workshop ‘Robots that Care‘, which contains a description of the field of social robotics.

Categories

RSS My Del.icio.us Links

Archives

RSS Books in My Library