Various enterprises and personal interests, such as Man-Machine Interaction (MMI), gesture studies, signs, language, social robotics, healthcare, innovation, music, publications, etc.
Hi Jeroen, Have you heard of Galton’s measurement of boredom with fidgets?
“Many mental processes admit of being roughly measured. For instance, the degree to which people are bored, by counting the number of their fidgets. I not infrequently tried this method at the meetings of the Royal Geographical Society, for even there dull memoirs are occasionally read. […] The use of a watch attracts attention, so I reckon time by the number of my breathings, of which there are 15 in a minute. They are not counted mentally, but are punctuated by pressing with 15 fingers successively. The counting is reserved for the fidgets. These observations should be confined to persons of middle age. Children are rarely still, while elderly philosophers will sometimes remain rigid for minutes altogether.”
The text apparently comes from his “memories of life” but many references to it can be found on the web and on Google Books. Kind regards, Gaël
Here is the fragment quoted in the book ‘Boredom: The Literary History of a State of Mind’ by Patricia Meyer Spacks.
Apparently, I was not the first to use the word ‘fidgets’ in a scientific context. How sad and how wonderful. I think I should read a bit more about Francis Galton. Hmm, first impressions: what a giant of a man… As far as interest in fidgeting goes, I seem to be in good company 🙂
Typically, people can see whether a movement is intended to communicate (a.k.a. a gesture) or whether the movement’s producer has some other intention, be it practical or just fidgeting. There are however plenty of examples where the movement is ambiguous: it could be a gesture but it could also be a meaningless incidental movement. Barack Obama produced such a movement during a speech. Watch and judge for yourself.
Did Obama just flip off Clinton or was he merely scratching his cheek?
Again, like in many other cases where the nature of a movement was debated, there is a potential insult to be considered. It is almost as if people are more sensitive to potentially insulting gestures then to other gestures. Some people, like Lehmann or Mr Wood even use this sensitivity to their advantage. They camouflage their insulting gesture and thus create ambiguity on purpose. Those who have a reason to feel offended are insulted by the ‘gesture’. Other people only see a cactus or someone scratching his head.
I would predict that if people must judge if a movement is intended to communicate they will do so more often when that would mean it is an insult than when that would mean it is some other gesture. (Question: Can you think of an experiment to test this prediction?)
BTW, there is a very interesting related paper on this topic from a psychiatric perspective:
Bucci, Sandra, Mike Startup, Paula Wynn, Amanda Baker, & Terry J. Lewin. (2008). Referential delusions of communication and interpretations of gestures. Psychiatry Research, 158(1), 27-34. (Scopus)
Gestures are an important aspect of non-verbal communication, but people with schizophrenia have poor comprehension of them. However, the tests of gesture comprehension that have been used present only scenes in which interpersonal meaning is communicated, though there is evidence that people with psychotic disorders tend to perceive communications where none were intended. Such mistakes about non-verbal behaviour are the hallmark of a subtype of delusions of reference identified as delusions of communication. Thus we hypothesised that patients with delusions of communication would tend to misinterpret incidental movements as gestures and, since delusions are often derogatory to the self, they would also tend to misinterpret gestures as insulting. Patients with acute psychotic symptoms (n = 64) were recruited according to a 2 × 2 design (presence vs. absence of delusions of communication by presence vs. absence of auditory hallucinations). They, and 57 healthy controls, were presented with 20 brief video clips in which an actor either made a well-known gesture or an incidental movement. After each clip, they selected one of four interpretations: a correct interpretation if a gesture had been presented; the interpretation of a different gesture; an insulting interpretation; no gesture intended (correct for incidental movements). The patients made significantly more errors of all kinds than the controls, perceived significantly more of the incidental movements as gestures, and selected significantly more insulting interpretations of the clips. These differences between patients and controls were almost wholly due to patients with delusions of communication. These results suggest that the difficulties that people with delusions of communication experience in understanding gestures can be explained, at least in part, by the misattribution of self-generated internal events to external sources.
Perhaps we all suffer from delusions of communication to some degree when we are in a situation where we expect to be insulted (rightly or wrongly). I know I always check for fingers when I feel I did something impolite in traffic. Don’t you?
Here is a very entertaining video (nice music) that tells the tale of gesture and the origins of language in a nutshell. Much has been written about how the language capability may have evolved in humans with gesture as a stepping stone or how Man’s first language may have been a signed language. Recent brain research findings (gesture+speech, mirror neurons, lateralization, sign language aphasia) have added more indirect ‘evidence’ for these theories. It is still hard to really prove anything about pre-historic events though…
One thing that struck me is how the author talks about how people might be aided in their thinking when the gesture, or doodle and fidget. A reference to fidgeting! Hooray! Should I point out that I think gesture and fidgeting are quite different? No, I will just let it be.
Do signers fidget during signing? This is a question I often pondered. How do signers handle fidgeting during signing. I have often witnessed that signers were fidgeting (rubbing their nose, or other self-adaptors) during signing. Even for me, a hearing non-signer, it seemed easy to spot such fidgeting movements as not being part of the signing. Now, luckily I came across a nice joke told in ASL at DeafJoke.tv by Kenton Hoxie, a Deaf vlogger (one of a fast growing group of ASL video bloggers cruising on YouTube). Obviously I cannot follow the joke, maybe you can.
But I will propose a game of my own to you: Spot the Fidgeting!: Try to spot the fidgeting in this video, and ask yourself what makes you see it.
Is sign perception hampered or disturbed in any way by fidgeting? I would find it hard to believe, as even I can filter it out, and ignore it while trying to understand the signing.
It is also interesting to check whether fidgeting is like saying ‘ehm’. Surprisingly, fidgeting occurs with one hand while the other is signing. But it does seem to occur at pauses in the signing (between phrases?). It might therefore be related to utterance planning (which I think saying ‘ehrm’ is as well). If anyone has suggestions about this, they are more than welcome.
Some people are actively interested in the stuff I am doing in my PhD studies, or at least they ask me questions about it. I usually tell them about my first experiment. That experiment was entirely about the difference between meaningless movements I call fidgeting and meaningful gestures, in this case sign language signs.
“Press the spacebar as soon as you see a sign”
It struck me then, and it still strikes me, that a bunch of people talking respond to each other so appropriately. Many, many times I saw people reacting to gestures of all sorts. Maybe just a little headnod or a palm-up gesture, or a raising of the eyebrows. And how often do you see anyone accidentally responding to a movement that was not intended to communicate after all?
Imagine the following chitchat:
You: “Nice weather huh?”
Her: “Yeah” (and makes some sort of movement)
You: “What do you mean, you think I am crazy?” (misinterpreting the movement)
Her: “I didn’t do anything, what are you talking about?” (now starts thinking you are crazy)
Rather unlikely? It just doesn’t happen. No matter how much we talk and interact, it hardly ever goes wrong. I will take the exceptional examples as exemplifying the rule.
So, I set out to see if I could test this in a lab. How fast can people make judgements about the status of a movement. I used sign language signs and fidgeting, and told people to press a button as soon as they saw a sign. And I found people could do that very well and very fast. Even non-signers could do it. (In case you want to read more: the journal Gesture recently accepted my publication of these results, hooray!).
If you want you can repeat the experiment in real life whenever you (and a friend) watch a conversation. Just put up your finger as soon as you see the talking people make a gesture. I bet you will both skip the fidgeting and spot the gestures.
Now, imagine a gesture recognizing computer trying to do the same trick and ignore fidgeting. Currently computers that are programmed to recognize gestures, simply assume any movement is a gesture candidate, and will try to classify it against their vocabulary.
In speech recognition one might see a similar problem. People say things like “ehm” or “ehr..” during an utterance. They may also cough, sneeze or scrape their throat. But is that really comparable to fidgeting? I am tempted to think that they are quite different. Coughing or sneezing is a bodily function, whereas fidgeting is usually just a ritualized watered-down version of some bodily function, if any. The reason behind it is quite different. Saying “ehm” is mostly a way to fill the gap, or keep the floor, in a poorly planned utterance. It is in a way as much a deliberate part of the communication as the words used. Nevertheless the computers task is more or less the same: it must withstand the disruptions and continue recognizing the words (or gestures) as if nothing happened. Both “ehm” and fidgeting should be ignored without damaging other processes. And that is quite a challenge as it is.
In speech recognition several techniques have been invented to cope with “ehm” and out-of-vocabulary (OOV) words. Most importantly ‘word spotting’ and ‘filler and garbage models’. Perhaps gesture recognition would do well to have a closer look at those techniques to start safely ignoring fidgeting?
Some people are actively interested in the stuff I am doing in my PhD studies, or at least ask me questions about it. I usually tell them about my first experiment. That experiment was entirely about the difference between meaningless movements I call fidgeting and meaningful gestures, in this case sign language signs.
“Press the spacebar as soon as you see a sign”
It struck me then and it still strikes me that a bunch of people talking respond to each other so appropriately. Many, many times did I see people reacting to gestures of all sorts. Maybe just a little headnod or a palm-up gesture, or a raising of the eyebrows. And how often do you see anyone accidentally responding to a movement that was not intended to communicate after all?
Imagine the following chitchat:
You: “Nice weather huh?”
Her: “Yeah” (and makes some sort of movement)
You: “What do you mean, you think I am crazy?” (misinterpreting the movement)
Her: “I didn’t do anything, what are you talking about?” (now starts thinking you are crazy)
Rather unlikely?
It just doesn’t happen.
No matter how much we talk and interact, it hardly ever goes wrong.
I will take the exceptional examples as exemplifying the rule.
So, I set out to see if I could test this in a lab. How fast can people make judgements about the status of a movement. I used sign language signs and fidgeting, and told people to press a button as soon as they saw a sign.
And I found people could do that very well and very fast. Even non-signers could do it. (In case you want to read more: the journal Gesture recently accepted my publication of these results, hooray!).
If you want you can repeat the experiment in real life whenever you (and a friend) watch a conversation. Just put up your finger as soon as you see the talking people make a gesture. I bet you will both skip the fidgeting and spot the gestures.
Now, imagine a gesture recognizing computer trying to do the same trick and ignore fidgeting. Currently computers that are programmed to recognize gestures, simply assume any movement is a gesture candidate, and will try to classify it against their vocabulary. In speech recognition one might see a similar problem. People say things like “ehm” or “ehr..” during an utterance. They may also cough, sneeze or scrape their throat. But is that really comparable to fidgeting?
I am tempted to think that they are quite different. Coughing or sneezing is a bodily function, whereas fidgeting is usually just a ritualized watered-down version of some bodily function, if any. The reason behind it is quite different. Saying “ehm” is mostly a way to fill the gap, or keep the floor, in a poorly planned utterance. It is in a way as much a deliberate part of the communication as the words used. Nevertheless the computers task is more or less the same: it must withstand the disruptions and continue recognizing the words (or gestures) as if nothing happened. Both “ehm” and fidgeting should be ignored without damaging other processes. And that is quite a challenge as it is. In speech recognition several techniques have been invented to cope with “ehm” and out-of-vocabulary (OOV) words. Most importantly ‘word spotting’ and ‘filler and garbage models’. Perhaps gesture recognition would do well to have a closer look at those techniques to start safely ignoring fidgeting?
Here is a nice Garfield comic (always a good source for gestures, see here and here). It illustrates how a bit of fidgeting (thoughtful lip-touching) can be turned into a pose. Suddenly Jon displays a gesture of thoughtfulness instead of actually being lost in thought.
Sometimes science is what happens while you are making research plans. One of my favorite topics for research plans is what makes a gesture a gesture. I ask myself if gestures are always clearly visible? And if so, is that universally true or culture-specific? What is the influence of context? Which circumstances increase the chance that a movement will be seen as a gesture? What role does the sensitivity of onlookers play in their own perception but also in the production of gesturers? To my surprise my 1 year old nephew Marco, visiting the Dutch family with his Italian mama and Dutch papa, answered my unspoken questions. What did he do? He twisted his finger in his ear during lunch with a big smile. Although you may find his action a bit uncivilized let me rush to his defence. At first I just saw his action, and barely noticed it. I had discarded it as just fidgeting. But then his mother remarked ‘ah, you like it, yes?’. Suddenly it dawned on me. It was a cheek screw! But little Marco did not make it exactly right. He screwed his ear instead of his cheek. His mom told me later this happens often.
I was forced to go over my own perception of this event. For me, the cheek screw is a gesture (meaning ‘Good’ or praise) that I only knew from reading. The Dutch do not use it. Perhaps I would have seen it immediately if I was Italian? In all likelyhood, Marco’s cheek screw is part of little rituals of communication with his mom and dad. He is learning, practising and his family is alert and responsive to his behaviour. My own involvement, as an uncle at a distance, is much smaller, though I try to interact with him as much as possible.
Let us return to the questions at the beginning with this case in mind to see the answers it may give. This cheek screw gesture was not clearly visible. It is a culture-specific gesture which may explain partly my inability to see it at first. The context of eating and Marco’s interaction with his mom made me see the gesture for what it was: an reasonably succesful attempt at a cheek screw to indicate he liked the food. The circumstances of his age and stage of development of his communication skills work to increase my sensitivity (I pay more attention) and tolerance (I try to understand what he intends to communicate). That is true for my interaction with my own children as well. Does this mean that seeing a gesture in a movement always requires the right context and the right cultural knowledge? No, it is just one case of a small kid growing up, a cheek screw, a mother and an uncle. Or in other words, for fans of human universals: It may well be that all little kids in all cultures learn to make gestures in ways that are so similar (universal) that when they are a bit older their distant uncles will always see their nephews’ intention to communicate, even if their exact meaning may escape those uncles. At the same time little kids gesturing can rely on several things. They operate within their parents’ culture. Mom and dad pay attention to their communicative attempts and are tolerant about mistakes. Furthermore they usually understand the needs of the situation.
From kids it is a small step to adults. We all may rely on exactly the same things. A gesture for a cultural insider who is paying attention can be made with just a subtle movement. If the context is clear the gesture will be perceived without problems. Conversely, if the context is ambiguous (not a shared understanding), two people from different cultures may well have more difficulties. In a way, strangers should act towards each other as a parent to a child: Use whatever clues you can to understand the needs of the situation, pay attention to communicative attempts and tolerate mistakes. Such goodwill gestures will surely be appreciated.
