How do computer controlled characters perceive and interact with the world? In the past, they didn't - it was all smoke, mirrors and scripted sequences. But with greater processing power and more complex AI, non-player characters can be given the same perceptive powers as human players. They can 'see' the game world like us, and react to what happens in real-time.

Or at least that's the theory. Last year, Edge Online interviewed Halo lead AI programmer, Damián Isla, about the third iteration of the FPS series about the processing 'cost' of the latest techniques in the field. He said, "actual decision-making [for AI characters] is very quick and cheap – it's always spatial awareness that's expensive."

This is another area where masses of middleware solutions have popped up to meet the needs of busy developers. According to Nvidia, games that integrate its PhysX physics engine with NaturalMotion's character animation toolset, Euphoria, will be able to generate characters that are truly responsive to their environments. They'll be able to 'ray cast' to spot players and environmental threats. "When explosions cause walls and building to come crashing down, Euphoria characters would be informed via calls backs to trigger their AI to save themselves from getting hurt," says Nadeem Mohammad at Nvidia. "In addition, Euphoria characters can learn to avoid running into dangerous obstacles or situations..."

But what do the studios think are the key challenges with creating truly reactive characters? Here are a few thoughts...

The Panel
George Torres, Senior Software Engineer, Fight Night Round 4
Jasen Whiteside, Project Art Director, Red Faction: Guerrilla
Dave Gargan, Principal Engineer, Havok
Richard Hackett, Technical Director, Blitz
Mike Enoch: Lead coder, Ruffian Games
Chris Hecker, veteran programmer and game physics expert

Procedural animation techniques have been touted for a few years with limited success - in what ways is this area being advanced? What new techniques will be employed over the next couple of years to sharpen interaction with environments and other characters?

George Torres: With interaction between characters and environment, I think we need to see more balanced systems. The problem is not necessarily in having the most advanced path-finding technique with largescale awareness; we need to have more micro behaviours, with a proper physics awareness of the environment. I say kill the walking in to walls.

Generating variety to get believability is the right solution. Mixing layering behaviours is just the natural replication of our human behaviours; I think we have a long way in mimicking this complexity, but we should approach it by breaking it down to small sub-systems, and then mixing them all together.

Jasen Whiteside: I would like to see procedural techniques used to give characters in motion a realistic lean. For example, walking, running or falling up or down hills, stairs, etc. This has been achieved to some extent, but not to huge success. You still see characters trying to walk up inclines they clearly shouldn't attempt. There are so many subtleties in how a human moves that we have yet to successfully mimic in games.

Another area that has seen limited success is when characters run into one another. They typically play a staged animation and carry on with their business. Think of the fun you could have if you could toss something in an NPC's path, causing them to realistically trip over it; or have your friend crouch down behind someone while you push them from the front, causing them to topple over backward. Incorporating the right amount of ragdoll with animation or procedural reactions would make for some really believable characters in most situations.

Dave Gargan: We're starting to see more attempts at integrating animation and simulation either to control very simple situations like body lean during turning or even more adventurous situations like balance control. With traditional rigging tools and techniques it is difficult to see how we move much beyond these reactive effects like balance control, tackling, flinching and reacting to being hit.

We still have to see a very compelling example of procedural locomotion (i.e. a full system that control a characters ability to walk or run in any arbitrary direction) in full production. Again the key here is to ensure that we don't dilute the original artistic input in the characters motion - procedural techniques can very quickly dilute style and become robotic.

Richard Hackett: Fully procedural animation is technically possible but as we've seen, the results are often disappointing. This comes back to role of the creative guys - even using the correct AI and constraints it is rare that we are after a highly realistic response, more often it is something more stylised or 'hyper-real'.

It is interesting that techniques from AI are coming into play on the character behaviour side to define how they move and respond. Something we have been looking at in our avatar research is taking established human response research and applying that to procedural movement. For example we took medical research on eye movement and focus and translated that into a procedural system for eye movement. The results were amazing - it injects an element of realism into the scene that was previously missing.

Mike Enoch: I think we've just been waiting for the technology to mature. It needed to be easier to integrate, faster to process, and have well-understood tools for animators, so they can get a lot of style and character into the animation, even though they don't always have direct control over the end result. The middleware market has definitely grown in this area, I think we'll see more and more games start to use it.

It's all about making the characters more physical and in touch with the world around them. There are a few techniques that have been used for years, like using Inverse Kinematics to make sure the player's feet are placed on the ground properly while standing on sloped surfaces, but with new technologies becoming available it's possible to take this further and have characters react realistically in any situation. There will be small details making characters look more natural in their environment, like holding railings while going down stairs, or grabbing onto things when falling, but the most impressive use of this technology will be in making characters deal with complex dynamic environments.

Imagine being able to knock a whole city block down with a simulated earthquake, then having your character able to clamber up the debris, grabbing hand holds and placing their feet accurately and naturally all the way up and in various orientations. It would be a lot more fluid with procedural animation, requiring less binary decisions about what the player can and can't grab onto, because the motions would adapt more naturally to a wider range of situations.

Chris Hecker: We're still a long way away from having a fully procedural physically based character that looks anywhere close to as good as a hand-animated one, so it's going to be a while. Eventually, say 20 years from now, characters will be much more like living creatures, at least from a low level motor control standpoint. They'll be self-balancing, able to look ahead and run up some stairs while jumping over some debris, etc.

The bigger problem here is that once you start going down this path, it becomes an AI problem as well as a physics problem. How does the character sense the world around him, how does he make decisions that seem plausible and exhibit common sense?