Maybe your computer just needs a hug

Posted by Anthony Francis

Maybe your computer just needs a hug
I'm Anthony Francis, an artificial intelligence researcher working in Google's Search Quality group.  One of the things I like about Google is that we give back to the world in ways that make sense for our business, both as a company and as individuals.  Google's search engine runs on electrical power, and so Google as a company invests in renewable energy and encourages Googlers as individuals to conserve energy.  Google's search engine runs on open source software, and so Google as a company open sources its software and encourages Googlers to participate in existing open source projects.

What may be less obvious is that Google's search engine runs on ideas.  Google as a company has a great Research department, but it also encourages Googlers to participate in the research community - both in visible, obvious ways like writing papers and attending conferences, but also in less visible but still necessary chores that keep the research community running, like reviewing papers and organizing conferences.  What I'd like to tell you about today is how Google helped me give back to the research community by letting me take time to write a paper on work I did before I came to Google.

The Personal Pet Project
Ashwin Ram's Cognitive Computing Lab at Georgia Tech has been working on emotional agents for over ten years, first with robots, and more recently in computer games and virtual environments.  I worked on one of our earliest efforts in this area, the PEPE (Personal Pet) project, a joint effort with Yamaha to develop an robotic pet.  Dr. Ram noticed that many consumer electronics require configuration steps that are frustrating, even baffling: everything from VCRs to toasters has blinking clocks and unused features.  Pets, on the other hand, don't have to be "configured": they understand us on emotional level, noticing what makes us happy or angry, remembering what brings reward and punishment, and learning to respond appropriately without us reading a manual or pushing a button.

PEPE emulated this intuitive understanding with an emotional long term memory module, which I developed.  This module had three parts: basic emotions, emotional memory, and emotional reminding.  Basic emotions gave the robot realistic behavior "out of the box": for example, the robot interpreted being petted on the head as being pleasant, which made it want to socialize; it interpreted a kick to the rear as unpleasant, which made it want to hide.  The emotional memory associated these primitive emotions with the people and objects the robot saw in its environment.  Emotional reminding closed the loop: when the robot saw a person or object again, it was reminded of the past emotion, which influenced but did not determine the robot's ultimate emotional expression.  So the robot naturally wanted to play with people who petted it and fled people who had kicked it in the past, but it would be possible to overcome that past learning by ignoring the robot (if you didn't want to play with it as much anymore) or by petting it (if you had kicked it but no longer wanted it to be scared of you).

I worked with a team of engineers at Yamaha in Japan to implement this emotional long term memory on a small robot they were building.  Our job was easier because PEPE's basic architecture enabled creating many different kinds of behaviors that could easily interact with each other, in parallel or in sequence.  On top of this architecture, we added basic emotions, emotional learning and emotional reminding in stages.  Our initial tests were positive, and upon my return to the United States, we began porting this software back to Georgia Tech's PEPE robot.  PEPE even had a brief moment of fame, appearing on a local TV station; ultimately, however, Georgia Tech and Yamaha decided to move on to new projects.

Characters with Personality
But the dream of emotional agents didn't die.  Manish Mehta, a graduate student at Georgia Tech researching interactive games, began working with Ram to try to develop more sophisticated models of emotional change.  Mehta realized that emotional events can act as a trigger for behavioral change: if we humans try something that works really poorly, or spectacularly well, the emotions generated by that experience can prompt us to make changes to our behavior in the future --- for example, remembering to bring that umbrella in the future so we no longer have to walk home wet in the rain.

Mehta used ABL, an agent language developed by Michael Mateas and Andrew Stern for the Facade project.  Like the fundamental architecture of PEPE, ABL enabled the author of an agent to design complicated behaviors that could interact in sequence, in parallel, or through complex triggering.  Mehta extended ABL to allow an agent to rewrite its own behaviors, and triggered that rewriting based on charged emotional events.  This worked: using two agents playing tag, Mehta was able to make them revise their behaviors based on how well they played the game. However, it had an unintended consequence: one of the agents decided it was less stressed out when it was "IT".  When it was "IT", it wasn't being chased and wasn't feeling stressed, so it just let itself get tagged, enabling it to "chase" the other player at a leisurely pace.

While this result was surprising, even amazing, it failed to achieved the original objective.   Mehta realized this had exposed one more property of human behavioral change: emotional events can prompt change, but we also have a self image - a model of what we think our personality should be like.  As our personalities change, we audit the changes to make sure they fit our self image and apply further corrections.  For a computer game character, that "personality" is really the designer's intent, so Mehta augmented his emotional learning system to check potential behavioral changes and make sure that they did not violate the original intended design.  With this change, both characters were able to learn from their good and bad experiences playing tag, and neither "quit playing the game" just because they didn't want the stress of being chased.

Putting It All Together
The paper that Ram, and Mehta and I wrote for the Handbook of Synthetic Emotions and Sociable Robotics reports this work and ties together all of these threads.  We were not alone in challenges we faced in developing intelligent agents: others faced them too, in robotics, computer games and academic AI.  Many people use flexible behavior systems like PEPE's architecture or like ABL, but building large systems out of these can be challenging and extremely labor intensive.  What Mehta and I found, however, is that the use of emotion models acted like a force multiplier: adding basic emotional responses to an existing behavior system radically increased the flexibility and apparent realism of an intelligent agent, whether it was a robotic puppy or a character in a computer game.

Moreover, the more components of the emotion stack you add to an agent, the easier it becomes to extend.  While adding emotion made PEPE extended the behaviors it could perform; modifying those emotions using memories changed the character of its behavior, making it seem more lifelike.  Mehta's work in extending ABL took this further, enabling creative behavior changes in response to agent frustration.  Adding personality models makes these changes stable, enabling the agent to adapt to new situations while staying consistent with their designer's intent.  Often, just adding more behaviors to an existing agent can make it more brittle; we found in contrast adding new layers inspired by research in human and animal emotion made the agent more sophisticated and robust.

Giving Something Back
Writing this paper did not take much time from Google.  Most of it was done in the evening on my own time, with the occasional email exchange with my coauthors and editors during the day while I was waiting on compiles.  And that is how it should be: it may be a long time before this work directly benefits Google, since we do not currently develop robots or computer games.  But Google still benefited because the advances that we need to improve our search engine often start in academia.

By encouraging our staff to follow up on their research work and to contribute to the research community, Google supports the growth of the next big idea.  Knowing that we can follow up on our past work makes researchers at Google feel more empowered to pursue new ideas and continually exposes us to sources of inspiration.  And that inspiration, which makes Google a better place to work, is exactly what I need to keep trying new ideas, which in the end will help make Google a better place to search.