This article is featured in the first-ever DZone Guide to Game Development. Get your free copy for more insightful articles, industry statistics, and more!

Dr. Penny De Byl is a former Professor of Games and Media at Australia's Bond University, now CEO of online games education provider Holistic3d. Her books include Holistic Game Development and Programming Believable Characters for Computer Games. She also teaches several online courses through Udemy and holistic3d.com. I spoke with Penny about her research in AI, the player experience, and how character AI might evolve.

Could you give us a quick review of your resume?

I was an academic for about twenty-five years until last year, I completed a Ph.D. focused on artificial intelligence in game characters, and an honor's degree before that in computer graphics. Since then, I've been teaching programming in games and computer science stuff. Last year, I decided to branch out on my own, teaching online courses in game development, and it's going really well.

When games became popular around the late 90s, when they became quite visual, running on more devices, and getting traction, academics were starting to take them more seriously as a research domain. I found that teaching programming with these visuals was really appealing to people who didn't "get" programming and didn't want to deal with the math without seeing results, and games give you immediate results on the screen, which was really motivating for them.

I teach a lot of beginners, but we do touch on AI in getting characters to move around and act and behave in the environment. Not a lot of the hardcore stuff. I just finished writing a course (you can find it here) that looks at neural networks and machine learning in Unity. I'm trying to make it as visual as possible, not to "dumb it down," but to make it more accessible. We want to get rid of all those big scary equations in research papers and say: "this is what it means, this is what the code looks like, and it's not that scary after all."

What are some issues that come up in AI and game characters?

Well, there are two issues with emotions and AI: there's modeling how the user is feeling at the time and there's also making the agent itself behave emotionally. You've got recognition of emotions and you've got the generation of emotions. I did do a research paper on modeling emotions about two years ago, and that was a whole journal looking at the psychology around games and game characters. I guess it's a matter of figuring out what to do with emotions.

There are a lot of AIs out there commercially that aren't quite ready, so they get a bad rap when they behave unpredictably or incorrectly, which is a lot of the issue with AIs in public. Even with the new Unity ML agents, they're in this developmental and experimental phase, but I noticed in the footnotes in the documentation they make it clear that this is an experimental system and they can't guarantee what your AI is going to do if you release them. Then, you think of the old days where you have characters that are bumping into walls constantly, and they just look completely stupid. It's getting there, and there are areas where you can apply it and get away with it, and there are areas where if you push the boundaries too much they can come off looking like they haven't been programmed at all.

What spurred your interest in researching AI?

I happened to be at a university where there were a lot of academics studying computer graphics and AI, so that's where I got involved and interested in it. Then, when I went to do my Ph.D., I was looking at modeling or simulating different organizations but on the people level. Kind of like The Sims, but in a serious scenario where you're looking at how people are moving around an environment and behaving during the day and simulating that.

What drives human behavior? The answer is "emotion." We were looking at how people make decisions and found that emotion comes into it a lot. When people make decisions, they tend to make them quickly, and they're based on a lot of background information which has been stored as emotional reactions to things, whereas AIs have to make a decision based on an awful lot of parameters. We want to condense those parameters into an emotional reaction that they might've had. So, if we created a situation, the AI made a decision based on variables, and the outcome was "good," we'd say the agent was "happy" with that experience, so the next time a similar situation comes up, it knows it was "happy" and it doesn't need to calculate it again.

Emotion in AI also appealed to me from a female point of view, not to make it about the gender debate in computer science, but I found AI to be really dry when I was working with it. It was all focused on these equations and code, and I just thought it was really boring. As you may know, a lot of girls play The Sims, and it's been great at getting them into gaming. I guess that being able to visualize what's happening with little people walking around was really appealing to me. It helped me look at AI from an angle other than the dry, abstract stuff underneath it.

You mentioned earlier seeing issues with AI in game characters like walking into walls, so in your experience what's made AI feel natural to people playing games?

The classic problem is nonreactive AI. It's not even stupidity because stupidity is a level of intelligence. It's just programmed behavior like a machine instead of what you'd expect a human to do. You keep coming back to the Turing Test. Can an AI completely fool you into thinking it's another person in a multiplayer environment?

If the player becomes immersed in a world, and an AI does something it shouldn't do, it throws people out of that immersive state. Even programmed stupidity is a good thing because people do stupid things in life and MMOs and games. You include spelling and grammar mistakes when they chat because that's what real people do. Programming bots to run around an environment doesn't really require AI algorithms, just pathway scripting and putting in random tweaks to make them do different stuff. It's way more controllable than using adaptive algorithms, and a programmer has more control over what an AI is going to do. What machine learning can do in games is observe, live, how other people are playing and adapt to that on the fly. But, the challenge is monitoring how that adaptation happens.

In neural networks and genetic algorithms, there's always a chance that a mutation happens in order to explore the whole problem space, and sometimes that mutation can create really weird behaviors that you don't want the player to see, but that also restricts how a character learns and evolves. Having said that, if you come back to games like Spore, the player is interacting with a genetics-based game that operates in a similar way and it's quite exciting to see mutations happen.

Is there any sort of "uncanny valley" for AI behavior where characters can seem too real and make people uncomfortable? We're definitely starting to get that with computer graphics, particularly in some movies.

That's an interesting question, and it would be a good research area to see if there is. I guess the AI could become really creepy. The thing is, you'd have to know it was an AI, and then know it was acting inappropriately or that it was doing something you didn't feel it was capable of. If you knew a character was a machine or AI and it was having an emotional — or overly emotional — reaction to something, it would become quite creepy. Even those humanoid-type robots that you see coming out, they're kind of creepy because they can tell you how you feel. In a game, you'd have to be aware it was AI-controlled. With the uncanny valley in graphics, you can still tell it's not real but if you're in an MMO, for example, you may not know if it's real or not. Now I'm getting creeped out thinking about it!

There are characters that are scripted to have emotional responses to you that can make you grow attached to them, so if they die or something, you become sad. It would be weird for this thing to adapt to you and for you to become aware of it. That's interesting, I haven't thought about that before.

When you've worked with students, how important has programming AI for NPCs been for the overall experience of the game?

I'd say quite a lot because they want to make intelligent characters, or believable characters, and they feel to do that there need to be NPCs that act believably. Most students start off wanting to make this massively multiplayer first-person shooter that they have to do in 13 weeks and by the end, they only have one character or one AI. They want an AI or group of AIs to be a good opponent for players in the game. All games my students have ever created haven't been puzzles, but instead, need an NPC.

Part of their idea of a game is that it needs to be competitive, there needs to be a worthy opponent. Most of the AI techniques we start with are the "patrol/chase/shoot/run away" stuff. That's where I start teaching because they can see how to create a behavior, which mathematically and programmatically is simple but still compelling.

What advice would you give to developers who are looking to create their first game with respect to AI or just generally?

Prototyping is very important. If you've got a whole bunch of cubes running around on a map, that's all you need. It's about getting timing, behavior, virtual spatial aspects, and gameplay right. A student will want everything to look spectacular right away, but if the gameplay isn't there, then it's a waste of time. A lot of developers build prototypes with cubes and then toss them away if it doesn't work. If you spend all your time creating beautiful art assets, then you'll waste a lot of time.

With respect to AI, you should start with a simple algorithm and then build on it later if you need to. Often, so much in the background of computer games is faked. It's about what you can make the player believe what is happening, not what you've actually programmed. If you can make your character look intelligent by walking down some stairs with three lines of code, then do that. It's not about having these fancy algorithms if it has the same effect as far as the player's concerned. Having said that, the point of having clever AI is so the player can't tell whether it's just been scripted or there's something else going on there. Overall, it's about player experience, not how clever you are as a programmer.

The really important thing when getting into AI is to know your math. So many people see the math and get intimidated, and so much of this is driven by mathematics. If you get into games, you'll see what you learned in school every single day and you'll be thankful that you learned it.

Penny De Byl's online courses and books can be found at .

This article is featured in the first-ever DZone Guide to Game Development. Get your free copy for more insightful articles, industry statistics, and more!