The Artillery Blog
Building Games Through Playtesting
by Manoj Koushik
How do you build a successful game? How do you build a product that consumers will love? How do you keep players coming back to your game long after it’s been downloaded and installed? These are some of the questions we ask ourselves every day. In a free-to-play game (like ours will be), we need players to engage with the game on a long term and repeated basis. That time commitment from our players is something that needs to be earned by building a game they want and not something we think they want. Research has taught us that the way to get there is through playtesting. And lots of it!
What do we mean by lots? We playtest our game twice a week with a small group of external players. They play from their homes - while we do occasionally have onsite playtests, they are much more logistically difficult to coordinate. In addition to external playtests, we play our game a lot in the office while experimenting with gameplay features. Each playtest is intended to get feedback from the players on the latest changes to the game. Those changes can include everything from minor unit-balance tweaks to completetly new mechanics and systems.
The early Artillery team came from Facebook and Google where we learned the importance of building working prototypes quickly. So when we started to build a game we approached game development just like web development: get something working quickly, show people and ask them what they think, and then iterate. Strangely, this doesn’t seem to be the norm in the AAA games industry.
Professional game developers we’ve met have told us stories about how their game didn’t work until two months prior to shipping. This was surprising. How can you know that your customers will like what you’ve made if you don’t put it in front of them? How do you know that your game is fun?
Playtesting is all about testing hypotheses. We generally work on one aspect of the game at a time. Most recently we’ve been working on new unit abilities and the tech tree. We’ve spent a lot of time in the past focused on things like the user interface, map objectives, and battles. We usually start by finding a part of the game that we or our players think is not fun enough or mature enough and then brainstorm possible solutions. We implement the best ideas and show them to our playtesters. The results are often surprising.
The fundamental assumption underlying our playtesting methodology is that we do not have all the answers. The solution is to ask people who do have the answers: our players. Many things that we thought would be fun turned out to not be fun. For example, for a while we experimented with map objectives. We added seven special points on the map – every few minutes a special “artifact” would be spawned at a random point, and the players would need to find which point had the artifact, harvest it, and score a point. The game would end when a team had achieved a certain number of points. We were certain that by adding an unpredicatable element that players would have to react to on the fly, the game would become more dynamic and more fun.
But, we were wrong. The artifact game mode led to all of the battles happening in one or two places on the map. We wanted to use the artifacts as a way to keep players from focusing too much on specific areas of the map. But it turned out that they consumed all of the players’ attention, and the entire game took place at or near artifacts. Surprisingly, they soon began to ignore the objectives altogether, and simply battled near them, not for them! This made us realize that we should focus on making battles fun, not on artificially trying to make them happen in certain places.
Lots of real-time strategy games (RTSes) feature base-building as a core element of the game. In these games, structures define the upgrade path or tech tree. For example, you might need to build building A before you can build building B, and only once building B is complete will you be able to build unit X.
For a while, we were sure we’d have to create a structure-based tech tree in order for our game to have strategic depth. But before we got around to creating such a tech tree, we had only a single structure, the HQ. The HQ could build one unit at a time and queue five additional units to be built later. Each player started with one HQ and could build more of them using a worker. This allowed players to build more units closer to the enemy and get their armies to the front lines faster.
However, players began to use buildings to wall themselves in. These players clogged up choke points and “turtled,” and the result was that games became boring because players weren’t engaging. We tried to punish this behavior by making the HQ more expensive, but the real stroke of genius came later. What if factories built units instantly? Then there wouldn’t be any incentive to build more of them. And then, why let the players build any structures at all? So we tried giving players a single base that built units instantly, and it was a hit. Now players concentrate on their armies and battling instead of building up a fortress and turtling themselves in.
But if there aren’t any buildings, how is there a tech tree? Well, some of us have never liked the idea of needing to memorize a tech tree, so we decided to make it much simpler. We now have an “Upgrades” panel in the UI where players can upgrade their units by spending resources. There’s no ambiguity or special relationships between buildings, and our players like the simplicity a lot. We’re especially happy that we were able to solve this problem by side-stepping it, rather than by applying band-aids to a system that was causing us problems.
If you’re going to constantly iterate any product, the only way to do it is through a well-established feedback process from the consumers. For games, that means playtesting as early and as often as you can manage. Fun can’t be predicted, but you can search for it efficiently.
