This is a guest post by David Kerns, one of Greta's top student writers for Spring 2007.
As movie special effects technology improves, more and more live-action shots are being replaced with computer animation. Harry Potter flies across the Quidditch field; Spider-Man swings from web to web through the cityscape of New York City, and miniaturized Hobbits fight the overpowering Orcs of Middle-earth. All of these are examples of human movements that have been reconstructed with computer animation. But sometimes this type of animation fails to come across as real. When Harry falls from his broom, and his computer animated body contorts in a way that appears not humanly possible, we are reminded that it is just a computer generated figure. So what is it about physical movements that make them appear human or artificial?
Body language is a critical form of communication for human beings. We can pick up a lot of meaning from physical movements, even when we only see a very limited amount of information about that movement. For example, most special effects animation is created by putting sensors on several parts of the human body to determine how body parts interact when the body is in motion. A human figure made up of just ten dots located in the different major body regions is enough to convey a wide range of emotions and complex physical movements. How does this work? And what is it about this movement that allows us to distinguish natural movements from artificial ones? A research group lead by Jan Jastorff sought to answer these questions by testing if people could learn to distinguish between complex physical motions of artificial movements as well as they could distinguish between the complex physical motions of natural movements.
So what makes natural movements different from other types of movements? As Jastorff's team explains, biological movements have three distinct characteristics:
- Natural movements are pretty smooth and fluid.
- Natural movements are connected by an underlying structure. In the case of humans, this is a skeleton.
- Natural movements represent a physical form or familiar motion pattern that is familiar to us. That is to say, we can easily tell that the shape we see represents a human body in motion.
These three factors might be important in how we recognize natural movements and distinguish them from other types of movement. Jastorff's team tested how these factors play a role in how we contribute to our perception and understanding of physical movement.
The team conducted four experiments testing the roles of these factors. Participants in the study had to watch videos of natural or artificial movements. Jastorff's team then measured how easy it was for the people in the experiment to distinguish between different artificial movements versus how easy it was for them to distinguish between different natural movements. The movements could be anything from running or kicking to jumping or punching. All of the figures performing these movements were point-light displays -- nothing more than ten black dots displayed on a gray background. The natural moving forms looked like normal human figures, but the artificial moving forms either looked like contorted skeletons or completely random dot configurations.
One type of movement simulated the actions of an actor (QuickTime required).
A second type mimicked the movements of contorted artificial skeletons.
Why are these movies all wobbly? The researchers found that viewers focused too much on the individual points when they moved normally, so a bit of random motion was added to the display.
By comparing how fast people learned to distinguish the different actions within each movement group, the researchers could determine what factors are important in conveying information about a moving form. For example, is it easier to distinguish kicking from punching when the moving form is a natural human figure?
For this last question, the answer is not always yes. When comparing ability to learn about human-like figures and artificial ones, the results were clear-cut:
As you can see, after learning how to differentiate the motions in a pretest, viewers were equally accurate learning to distinguish different motions in both human-like and artifical figures.
But what about when the forms do not have an underlying structure - that is, when the dots aren't connected by a skeleton? Jastorff's team created a new sort of movie to test this. The motions of each dot were identical to the human-like movie above, but the position of each dot was scrambled, so that the motion couldn't easily be interpreted as being connected by an internal skeleton:
This graph compares how well viewers were able to differentiate between different motions, for human-like and scrambled figures.
By posttest 2, viewers were significantly better at distinguishing between human-like figures compared to scrambled figures.
Jastorff's team argues that a key factor in learning physical movements is having a connected unifying structure, but having a familiar shape or familiar motion pattern is not a key factor. This study also brings up further questions about what else is important in conveying physical information. For example, we have yet to see if changing the fluidity and smoothness of a given motion will make it harder to distinguish between actions. Only future studies will answer this question and help complete our understanding of how physical communication works. For movie-animation gurus, clearly having an internally connected structure is a key to generating realistic images. But if their goal is to create an otherworldly, completely imaginary figure, perhaps removing the internal skeleton could have a dramatic impact.
Jastorff, J., Kourtzi, Z., & Giese, M.A. (2006). Learning to discriminate complex movements: biological versus artificial trajectories. Journal of Vision, 6, 791-804.
- Log in to post comments
This all looks god but I think it is not sufficient. Even when things are well connected and move smooth, I and most of the people I know can still almost instantly pick up when a human movement is generated. (within 10 seconds or so).
What we can't figure out is why we can! The movements all individually seem ok, maybe the accelerations are wrong? or there isn't little quavers that people have? Anyway there still is something.
I think sometimes CGI movements are *too* fluid. Our natural movements are very stop-start. Animators sometimes seem so intent on the animating process that they forget to leave a few bits where their CG figure is just staying still.
It's even more noticeable with CG creatures - there's just excessive movement all over the place. A CG monster will leap in to frame, sway its head about a bit, maybe roar, maybe take a few steps. The most successful ones pause just like a real animal would. Compare, say, any of the creatures from Star Wars Episode I-III with the awesome T.rex of Jurassic Park.
Nice article. Minor nitpick: the y-axis of the figure you've used is labelled 'percent correct' when it only goes up to 2! I had a look at the paper and the authors use d' for that figure, which I assume is their figure 8.
(For those interested, d' is used in signal-detection theory. See here for examples.)
An interesting corollary question to this one would be "Why does our technology make generating natural-seeming movements difficult?" Human beings created this technology that is generating the artificial movements, so why hasn't our ability to detect these patterns translated into an ability to easily create them? Part of the non-technical discussion in Stephen Wolfram's book "A New Kind of Science" deals with this. As humanity developed, it decided that 'taming' complex systems to create the results desired was too difficult and opted instead to create systems to remove the complexity altogether. It has gotten us far, but it has it's own fundamental limitations. Complexity science is, to me at least, all about taking that other path we turned from as a species so long ago. And hey, once we've got a few hundred years in it maybe it will make generating artificial movement that is indistinguishable from natural movement a piece of cake!
I'm not sure if this is correct, but I once read (saw? Probably on Discovery Channel, so feel free to tell me I'm wrong) that if you like a person you will subconsciously mimic their actions to understand how they are feeling. Perhaps our ability to recognize non-human/naturual actions results from our brain realizing that we would not be capable of mimicking the actions we see on the screen. This ties into the "Uncanny valley" theory that was recently brought up in video games and robots.
Uncanny valley article http://www.slate.com/id/2102086
The uncanny valley is a sudden loss of "trust" of an image that is rendered too closely to a human face. I personally think that this has to do with the body language and micro movements of muscles in the face and the body. What if when we look, we copy what we see in our mind in order to interpret it? and with all the subtly in human expression, our minds must be aware of every single muscle in the face and all its possible movements, even if we can not consciously pick out and name each one. chances are, we see more than we think we see. One of the reasons that CG animation seems off is because animators (and people in general) are not consciously aware of everything that happens when a person moves. One thing I've always noticed with computer rendered characters is that the lighting is inconsistent. This is something I know is wrong and see as wrong, but other people might not know that it's the lighting that is throwing them off, and they won't understand why the cg character is unbelievable. This same lack of knowledge may lead animators to make figures that aren't quite right, and the animator won't know why.
Another reason its so difficult for our technology to mimic human movements perfectly is because computers have only been around for a very very short period of time. If you look at the original Doom game, and compare it to something like F.E.A.R, you will see that we are getting very close to perfectly mimicking our environments in video games, and it will happen. Like in painting, the western civilization wasn't aware of the principal of perspective until the renaissance period (prior to that, only sculpture was really human like).
I'm a little confused by otakucode because they say that humans (and I find it hard to believe that a species would collectively do this unless there was a substantial form of communication between all members of the species) decided to invent systems that weren't complex and that we are only now starting to look at complex systems. Didn't early scientists predict things because they thought the systems were simple, and the systems became progressively more complex as other scientists realized that those theories didn't work, and had to be modified or completely rethought? From what I've heard, this isn't new, its been around for a while.
Here's my theory, it's all about force. The force of gravity, and the exerted force of the cgi creation. Neither one exists and it's unnatural. The creation is unbelievable in its environment, since there is no discernable gravity or atmospheric pressure affecting its movement, and its movements are not credible because they don't come from within, they are just elaborate marionettes.
We can see the strings, we can tell that the creation is not moving on its own accord, because, well, it isn't.
As a postscript, I'd like to add that the spellchecker doesn't like the word 'discernable'. It also dislikes the word spellchecker. :)