"We are building a knowledge base that corresponds to all of the relevant background for our synthetic character--where he went to school, what his family is like, and so on," said Selmer Bringsjord, head of Rensselaer's Cognitive Science Department and leader of the research project. "We want to engineer, from the start, a full-blown intelligent character and converse with him in an interactive environment like the holodeck from Star Trek."

Currently, Bringsjord is stocking his synthetic character will all sorts of facts, figures, family trivia and personal beliefs gleaned from what he calls his "full-time guinea pig," a graduate student that has agreed to bare all for his synthetic doppelganger. The synthetic character will be able to converse with other human-controlled avatars about his educational and family history, his personal pastimes, and even his feelings and beliefs.

"This synthetic person based on our mathematical theory will carry on a conversation about himself, including his own mental states and the mental states of others," said Bringsjord. "Our artificial intelligence algorithm is now making this possible, but we need a supercomputer to get real-time performance."

The Computational Center for Nanotechnology Innovations (CCNI) is donating the supercomputer time this fall, when the Turing-test demonstration will open along with RPI's new Experimental Media and Performing Arts Center (EMPAC). There, the Turing test will be limited to controlling avatars in a virtual world--probably Second Life. Both the synthetic character and his human doppelganger will be operating different avatars. If the human-operators can't tell who the RPI synthetic character is, then it passes the Turing test

Mental states

The key to the realism of RPI's synthetic characters, according to Bringsjord, is that RPI is modeling the mental states of others--in particular, one's beliefs about others' mental states.

"Our synthetic characters have correlates of the mental states experienced by all humans," said Bringsjord. "That's how we plan to pass this limited version of the Turing test."

Mimicking the behavior of a human-controlled avatar in a virtual world like Second Life is possible, according to Bringsjord, if you craft the necessary algorithms carefully and run them on the world's fastest supercomputer. Bringsjord's synthetic-character software runs on the supercomputers at CCNI, which together provide more than 100 teraflops, including a massively parallel IBM Blue Gene supercomputer (the title-holder to world's fastest supercomputer), a Linux cluster-supercomputer, and an Advanced Micro Devices Opteron processor-based cluste supercomputer.

In the limited version of the Turing test that Bringsjord is devising, a synthetic character modeled by software running on the supercomputers will converse with the human-controlled avatars in a virtual world. If the supercomputer can interact convincingly with the humans controlling the other avatars--that is, without them guessing the fraud--then the test will be judged a success. EMPAC will provide the visualization, 3D audio and immersive environment, but the brains of the algorithm running on the supercomputers will come from software called Rascals, for Rensselaer Advanced Synthetic Architecture for Living Systems.

Rascals is based on a core theorem proving engine that deduces results (proves theorems) about the world after pattern-matching its current situation against its knowledge base. Each proven theorem then initiates a response by virtue of having a synthetic character speak and/or move in the virtual world.

"Upon analysis, anything that our synthetic character says or does, is the result of a theorem being proven by the system," said Bringsjord. "So far, theorem provers have only been used in toy-problems. We are scaling that up to enough knowledge for a synthetic character, which requires a very fast supercomputer."

Bringsjord's research group recently passed a milestone by programming a synthetic character to understand a "false belief." For instance, to create a false belief you could hide a stuffed bear in a cabinet in front of a child and an adult, and then when the adult leaves the room, move the bear to a closet while the child is still watching. Here, the child should know that the adult now has a false belief--that the bear is still in the cabinet.

Unfortunately, children less than four years old don't typically conceptualize the beliefs of others, but only their own, and, thus, expect the adult to know that the bear has been moved. Somewhere around five years of age, however, children begin to have second-order beliefs--that is, beliefs about the beliefs of others, enabling them to understand that other people can have beliefs different from their own. Now, Bringsjord's research group claims to have achieved second- and third-order beliefs in their synthetic characters.

"Children below about four years old don't have second-order beliefs--they just have first-order beliefs. But we have a mathematical account that passes the test," said Bringsjord. "We have a formal theory of second- and third-order beliefs, and a way to make them computationally real."

The next milestone for the group will be passing the avatar version of the Turing test in October 2008, when EMPAC officially opens. The Rascals-powered synthetic character will carry on a conversation with human-controlled avatars, perhaps without the humans noticing the difference, and thus passing the Turing test.

Bringsjord's research is supported by IBM and is being performed with the assistance of Rensselaer doctoral candidates Andrew Shilliday, Joshua Taylor, and Micah Clark, as well as undergraduate researchers Ed Charpentier and Alexander Bringsjord.