Jeff Hannan studied computer science at Dundee University and subsequently did a PhD at the Department of Cybernetics at Reading University (Kevin Warwick's department!) on neural networks and electricity demand forecasting.
Jeff is now working for Codemasters, where he created the Artificial Intelligence for the excellent Colin McRae Rally 2.0 (see Generation5 review).
G5: Why did you choose to use neural networks in CMR2.0? Is there anything you can tell us about the networks themselves?
Controlling a rally caron mud and gravel is a lot more complex than controlling a car on a tarmac circuit. On tarmac its mainly a case of pointing the car in the right direction, and maintaining the right speed. On slidy surfaces, the car has to turn into corners early, and slide round. I tried to create a set of rules to control the car, but was unsuccessful. I thought that neural networks would help me find a quick solution, by modelling the way I drove the car. In the end, the neural networks made it very easy to create the AI.
I used a standard feedforward multilayer perceptron. There was nothing exceptional in the network structure, the key to its success lay in selecting the right input variables. I would recommend using the RPROP learning algorithm though, it is very fast compared to any other algorithm I know, including those fancy mathematical ones.
G5: What kind of flexibility did the neural networks give you in terms of AI design and playability? Did the networks control all aspects of the AI?
Obviously the biggest challenge was actually getting a car to successfully drive round the track in a quick time. Once that was achieved, I was then able to adjust racing lines almost at will, to add a bit of character to the drivers. The neural net was able to drive the new lines, without any new training.
The neural nets are constructed with the simple aim of keeping the car to the racing line. They are effectively performing that skill. I felt that higher level functions like overtaking or recovering from crashes should be separated from this core activity. In fact, I was able to work out fairly simple rules to perform these tasks.
G5: How did you feel about the resulting AI? What did you like/dislike about the results? Would you use a neural network again given a similar problem?
I was very pleased with the performance of the AI. I am very confident about it as well, because I understand what the neural net is doing. I haven't just created a big mysterious black box, I can map out the internal workings of it.
However, even my best models were doing laps a few seconds slower than what I can achieve. This is fine for the game, in which we have to provide opponents across a range of difficulties. But I'd still like to find out what the model is lacking that prevents it from matching the very best human players.
You'll notice that the arcade tracks are less hazardous than the rally stages. That was done for the benefit of the racing. There was no remit to have opponents on stages, since that is not what rallying is about, but it would have been tremendously difficult to produce AI for all the stages, because they are so complex and there is such a variety of conditions.
I'm wary about creating a massive neural net that takes dozens of inputs in the hope of capturing all the information about a problem. I'm sure that any problem can be modularized in some way. If it isn't simple enough to solve with a few rules, then a neural net can be a good solution.
G5: How do you see game AI evolving over the next couple of years? In what direction would you like to see game AI mature? Why?
An important lesson I learnt is that producing the cleverest AI is not the requirement for a game. Most players will never master expert level anyway, so the emphasis must be on producing a great gaming experience, rather than an invincible opponent.
I think that the nature of game AI is very dependent on the genre. In games such as racing, sports or strategy, the player and the AI are usually on a level playing field. Its you versus the computer, and it can't cheat. In some types of games, it is a difficult task to get AI to play as well as the average human player, for example, racing a tricky rally stage. Improvements in AI will enable more complex player vs computer type games. In many games though, it is straightforward to get AI to play to a fairly high standard. So the most important feature of AI generally is that it performs realistically. This requires advances in both animation and AI. It is a challenge that will increase as game environments get ever more complex.
In games such as RPGs and Action games, the emphasis is more on providing an enjoyable experience of the appropriate difficulty. In reality, one person is never going to take out a heavily defended base. So again, the emphasis is on realistic behaviour.
The ultimate in realistic human behaviour is AI that you can have a conversation with. I think that will require several advances yet in AI, not least in understanding human sentences. There is the further problem of giving each character in the game the necessary depth to hold the conversation.
G5: Which game genres do you see "mainstream AI" (neural networks, genetic algorithms etc) seeping into the most, now? In the future?
Neural networks and genetic algorithms are powerful techniques that can be applied in general to any suitable problem, not just AI. Therefore, any game genre could make use of them. It would be ridiculous not to consider them for a difficult problem. However, experimentation is generally required. They help you find a solution, rather than give it to you on a plate.
With my experience of using neural nets, I'd say that they are particularly good at skills. When a human performs a skill, it is an automatic movement that doesn't require high-level reasoning. The brain has learned a function that automatically produces the right behaviour in response to the situation. Sports games may be the most obvious candidate for this in the near future.
One area I'm not convinced about is AI that learns during the game, particularly for improving performance. Any published game is put through a lot of testing. The vast majority of players will not be as good as the best testers, so it would be something for the hardcore only. There is also the developer's fear that any adaptable AI may go wrong. Finally, most AI has to be dumbed down, because the odds are stacked so heavily in its favour. If you want the AI to gradually improve in response to the players improvement, then just replace it with an already prepared better AI opponent, or have some scaled degradation of performance.
G5: How did you get into the game industry, and more specifically into the AI department of it?
I didn't have a long term aim of getting into the games industry, just the computing industry and research. So I did a Computer Science degree and eventually a PhD in neural networks. After Windows programming for a small software firm, I felt that games would be a more exciting product to work on. I hadn't intended to work in AI research specifically, but with my AI background I was asked to look at the AI requirements for Rally2. Fortunately, Codemasters gave me plenty of time to do the research.
You don't need to have done a PhD to work on AI in games, but if you do intend to use some of the specialised techniques, its a good idea to read as much as you can on the subject, and experiment a lot. I've found that most books really only give you the basics.
G5: Any funny stories about the development of the CMR2.0 AI?
You mean when the AI didn't work properly? There was one time when all the cars just drove forward and back across the road, as if doing a waltz.
All content copyright © 1998-2007, Generation5 unless otherwise noted.