Led by Associate Professor Yasue Mitsukura is a Keio University group that has developed something pretty awesome. They found a way to let an avatar copy the user’s facial expression just by using an ordinary PC and a normal USB camera. The system that they developed accomplishes this task with high speed and high precision, again just by using an ordinary PC and an ordinary USB camera.
As quoted from the one of the members of the group and also from the video: “We think this system could be used by CG animation hobbyists, in Web dialog systems that show a character instead of a person’s face, and for making characters move in real time at events. Because the system uses just one PC and one camera, it can be applied in many situations very easily.”
The system basically uses time-series signal processing in order to track and detect faces. At high speed with high precision, it is able to track human characteristics. In the video, you’ll notice that there are white dots. They are used to track the face while the red line (also in the video) will show the orientation of the user’s face. So, you can see how the system copies exactly the user’s facial movements.
As quoted: “We’re using an algorithm that gets updated in line with the motion of the face, so it can track the face very fast and with very high precision. That’s the basic technology for this avatar system.”
On top of that, this unique system is able to analyze the shape of the person’s expression, so that it can copy whether the user is laughing, surprised, angry or sad.
This avatar system can also be used for games. Games that are able to detect and respond to any changes in the player’s face.
“Systems using motion capture with markers on the face aren’t convenient for ordinary users. In that sense, until now, there hasn’t been a technology for moving a model freely in real-time.”
The group is currently working on developing a motion generation software for standard PCs that will enable them to apply and implement this avatar system.
Now here’s the video: