Sony Patent Sketches Mood-Tracking System for Games

The era of gesture-based controls is only just taking off, but before it's own wand-waving system sees production, Sony is already charging ahead with an emotion-tracking system.

In a patent spotted by Silicon Era, Sony describes a system that uses a camera and a microphone to capture metadata about a player's emotions. Laughter (and the inevitable shouts of frustration) are recorded by the mic, while gestures of triumph (like a high five) or more subtle smirks and grimaces are captured by the system. These signals of human disposition are then converted to metadata, which can be used locally on the console or passed over a network in a multiplayer environment.

While the proposed technology has an obvious application in grafting player's emotions onto their avatars — imagine how much more subtle and rich player interactions will become once body-language and facial expression are part of the equation — the system could be a huge boon to non-player characters as well.

One of the things that makes creating realistic NPCs so difficult is the clumsy way in which they must receive input. Dialog trees and even limited natural-language parsing leaves out much of the context required to hold believable conversations. With emotional tracking, characters could use a player's emotional state to resolve linguistic ambiguities. They could also use those cues to gauge, anticipate and respond to player reactions.

Another application we're really looking forward to is games that can tell whether you're frustrated or bored and tune themselves accordingly.  And if (when) this technology ever sees the inside of a living room, get ready for a whole new genre of "mood games." Could Sony Psych Ward be the next Wii Fit?