Know Your Enemy, Especially if He or She Has a Sniper Rifle
The fellow on the right looks like a pretty nice guy, no? Well, take my advice: don’t go to a party where he is lurking outside.
Meet Paul and Ian, my elite SpyParty playtesters. Both have played around 10 hours of SpyParty across multiple sessions. They’ve played the game more than anybody else, including me if you don’t count my daily code testing1. Paul, on the left, is a designer on DarkSpore, and he specializes in playing the Spy. Ian, a designer on The Sims, is partial to playing Sniper.
When you’re trying to make a really deep player-skill game, you need to have people like Paul and Ian who will play your game a lot and get very good at it. I’m really lucky to have these two friends who are happy to play my game so often and for so long2. Given my depth-first, accessibility-later development philosophy, I am basically designing the game for these two guys. If anybody else has a good time playing, that’s a great bonus, but right now, Paul and Ian are the most important SpyParty players in the world. If I come to a decision where I have to choose between making the game better for newbies or better for Paul and Ian, I choose the latter. It’s an audience of two.
For most of the development, everything was rosy, and Paul and Ian were evenly matched. This was a huge relief, because SpyParty is a highly asymmetric game, so balancing the Spy versus the Sniper is a subtle and difficult challenge. Every time I’d add a significant feature, I’d test with Paul and Ian, take a bunch of notes, go back and tune the feature if it was unfair (sometimes while they waited), and then we’d test again until it was balanced. The fact that they’d played the same amount, and they’re both ultra hard-core gamers and very competitive, meant I could trust that if they were each winning about 50% of the time, I had a balanced mechanic.
The Good Old Days
Naturally, when I got the Action Test feature working, I called up Paul and Ian and the rest of our playtest gang and had them come over to try it out. We ordered pizza, and everybody had a good time casually testing the new feature and discussing its pros and cons. Finally, it was time for Paul and Ian to sit down and see what impact the Action Test had on the elite level game.
They hadn’t played each other for a few months because I’d been doing so much traveling, but last time we played they were at the same level, so I figured after a few warm up games they’d be back to form and the intensive balance testing could begin.
Except Ian kept shooting Paul almost instantaneously. Over and over again.
What the heck was going on? Had I screwed something up with a recent feature? Did the Action Test actually do the opposite of what it was supposed to do? Had Ian planted a mirror behind Paul?
Spy Bloodbath
I decided to sit down and play the Spy. I’m not really a fair person to play against, since I wrote the code, and I know exactly what percentage of the time people take books from the shelves, and how fast they sip their drinks. These are huge advantages when I’m playing Sniper, but even for Spy, I can usually beat anybody. However, even though I did a bit better than Paul, Ian managed to kill even me over and over again.
Okay, something was wrong.
Most people who hear about the game think the Sniper must have an advantage, but this isn’t actually true. The metrics are pretty clear on this: even for newbies, the Spy and Sniper are pretty evenly matched. For example, in the recent playtests at NYU, the Snipers won 48% of the time to the Spy’s 52% over 188 games, and at an even more recent playtest at Valve, the Spy won 55% of the 104 games played.
However, at PAX, an observant fan in line noticed a trend…the Sniper was winning more and more as the show went on. Day 1, the results were 47%/53% Spy/Sniper. Day 2, the score was 35%/65%. Finally, on the afternoon of day 3, the issue was noticed and debugged, just in time for the tournament, but overall the day was 36%/64%. What was going on was quite interesting and subtle: because there was a huge line with a long wait, everybody had plenty of time to read and digest the Four Page Instruction Manual and then watch a whole bunch of games before they got to play. This, it turns out, is Sniper practice. The Spy is a kinesthetic experience; you need to be holding the controller to get better at playing Spy. But the Sniper is mostly about perception, so simply watching 10 or 20 games while you waited in line, seeing the various tells from different angles, and watching what worked and what didn’t for each side, gave the Snipers a huge advantage. We fixed the problem by not giving verbal help to newbies playing Sniper, figuring they’d had enough practice while waiting in line, and only helping out the Spies. This was enough to restore balance for the people playing the game for the first time.
However, did you happen to notice anything about all the pictures from the PAX days linked above? Here, let me help you out with a couple examples:
Our pal Ian was one of two friends who helped out in the SpyParty booth the entire three days of PAX. Without Ian and Tom doing the bulk of the work, I’m sure I would have died right there on the show floor.
Well, this means Ian got something like 25 hours of additional Sniper practice over the course of PAX by tutoring people and observing games the whole time. Look at him in those pictures, sucking up information about how the NPCs move and how often they do and don’t do things.
This is the current best theory explaining why he’s winning all the time now. He’s an unstoppable machine because he’s played three times as many hours as the next best player.
This is good news and bad news. The good news is if this theory is true, the game is working exactly as I’ve designed it. He should be beating Paul (and even me) every game now, because he’s played so much more, even if his hand didn’t touch the controller more than a few times at PAX. I’m designing the game to have an intense player-skill ramp, and this is what should happen if I’m successfully achieving that design. The bad news is it’s hard to know if the theory is true without me or Paul practicing enough to get to the same level, which is a lot of practice!
So, I’m going to need to practice my own game, which seems a little cosmically unfair, but such is life when you’re striving to make a truly deep game, I suppose.
I’m also using this as an opportunity to fix as many of the subtle places where the NPCs and the Spy diverge due to lazy coding. Ian noticed some of these during PAX, and I need to deprive him of that advantage.
An example of this kind of divergence is the Ambassador would only have his or her personal space violated by the Spy character. Ian observed that the Ambassador would run away sometimes, and deduced this was a tell. Now the Ambassador will move away from any character that stands too near for too long. This is subtle stuff!
I’ve also changed the way the conversation groups work. Now NPCs look for open areas, and randomize within the entire open arc, rather than just aiming for a slightly random point near the center of the arc. Also, there are no longer any hard coded maximum number of characters in a given conversation group, the NPCs look for an opening big enough, and if there is one, they can go for it. Ian was asking me the other day about how many people would be in a conversation at once, so now there is no straight answer to the question. Take that, Sniper! There is a probability table that tells NPCs how likely they are to want to join a conversation given the number of people already there, but I’m not going to publish that, sorry Ian.
One thing I’m not going to fix is how NPCs won’t go out of their way to be near the Ambassador, while the Spy needs to sometimes. This is happening at the behavioral level where I want the game to live, so the onus is on the elite Spy to bide his or her time waiting for the Ambassador to be in a plausible position to walk near.
One other good thing to come out of this is an interesting modification of the “3 out of 4 missions” game mode. When newbies start playing, they play with the Spy required to do 4 out of 4 missions; it’s too overwhelming for newbie Snipers if the Spy doesn’t have to do everything. Once the Sniper starts winning a lot, usually around 10 or 20 games in, we switch to the Spy picking 3 out of 4 missions. With this “subsetting”, the Sniper can’t camp any individual mission, and there is interesting yomi in which missions the Spy picked. However, given Ian’s skill gap, I realized a good new mode would be “any 3 out of 4 missions”, where the Spy doesn’t have to pick the 3 up front, all 4 are available, and the Spy can be opportunistic during play. This should be slightly harder for the Sniper because now the Spy has more options, yet still only has to do a subset of the missions to win. I’ll need to playtest this to be sure, but I think it’ll work well at elite levels.
In the meantime, watch out for this guy:
- …which you probably shouldn’t since it’s not very good practice [↩]
- My pet theory about why Ian likes the game so much: during his day job, when a Sim misbehaves, it’s a bug and he has to go pester a programmer to fix it, and it’s a big pain; while testing SpyParty, he gets to shoot the misbehaving Sim in the face and win for doing so. [↩]
You mention changing up the Ambassador’s behavior so that he reacts to the movements of NPCs and the Spy. What if you extended movement rules like this to the behavior of all the NPCs? If the behavior of each NPC could be influenced by the movements of the Spy and the other NPCs, an experienced Spy might be able to use that influence to create false tells and trip up the Sniper. Each NPC could be given a unique set of player- or NPC-triggered movements (the guy in the plaid shirt leaves a conversation group if more than 4 people are present, the woman in the purple dress makes a right turn if someone walks directly towards her, etc.)
A Spy who had observed and kept note of these movement rules could manufacture suspicious behavior in the NPCs by forcing them to move erratically or towards suspicious areas. That said, a similarly experienced Sniper would probably be aware of these rules as well, and would be on the lookout for NPCs who didn’t abide by them. If the Spy failed to move as expected when her special movement rule was triggered by an NPC, the experienced Sniper would be able to single her out.
Such a system might also do a bit to alleviate the imbalance you mention between players who’ve watched a bunch of games, as the Spy would have the opportunity to learn about the NPC movement rules and improve her game even while observing.
While typing this out I realized that this idea was probably subconsciously inspired by the behavior rules for the ghosts in Pac-Man: http://gameinternals.com/post/2072558330/understanding-pac-man-ghost-behavior
I definitely want to have more feedback in the room, for sure. Like you say, it’s a delicate balance, but I’m hoping to have per-character personalities that are learnable, so the Generalissimo might follow the pretty girls around, and things like that.
Neat, “per-character personalities” is a much better way to describe what I was getting at. The example you mention with the Generalissimo is a bit worrying though, as it’s a personality that the Spy can only leverage to her benefit if she’s picked one of the pretty girls. My gut feeling is that the Spy should have equal influence over per-character behavior regardless of their own character choice. Otherwise you’ll inevitably end up with certain characters that are better-equipped to complete the Spy’s list of tasks than others, which would almost certainly lead to experienced Spies picking those characters far more frequently.
High-level Snipers would obviously pick up on these character tiers as well, and could adjust by weighting their attention more heavily towards high-tier characters from the outset of each match. I guess that could be an interesting dynamic in it’s own right (and it would introduce mind-games where the Spy could intentionally pick low-tier characters to try and throw the Sniper off the scent), but it seems like such a setup would detract a bit from the Sniper’s focus on pure observation.
Anyways, I’m sure you’ll figure everything out! Playing a bunch of the broken Assassin’s Creed multiplayer has only made SpyParty’s eventual release even more exciting.
Yeah, there’s lots to experiment with in this area. I think if some characters have some advantages for some missions, that’s okay, because if the missions are the Spy’s choice, then you get some yomi going there. For example, if the Spy can be the waiter, it makes poisoning the drink easy, but picking up the microfilm from the bookcase hard. Etc.
Nice post ^^
Still can’t wait to play sniper !
Good luck.
Thanks for the post!
I can’t wait to see it finished.
Very interesting observation. I’m curious as to how well Paul and Ian would fare if their roles were reversed, and whether the in-depth familiarity with one role would transfer over as well. I find in fighting games it’s not enough for the player to specialize with a single character and they typically need to branch out in order to improve their overall mastery.
They occasionally switch when playing less skilled players, and you’re totally right that playing both sides is important from the start through medium skill levels for learning, but they’ve both been playing so much that they’re highly specialized now. They get all nervous and complain when they have to play the other side against each other! :)
hey chris, I think your choice to focus your design based on these two players is very interesting and i’m very curious to see how it plays out. In my experience to play in the very high percentile of players in a skill based game usually requires several hundred hours of practice. Do you think the relatively low amount of practice your testers have gotten so far is affecting the design in any way? It seems this is the reason only a couple of days of extra practice can put one of these players so far ahead of the other.
Yeah, it remains to be seen. Certainly, somebody’s who’s played 35 hours of Counter-Strike will be owning people who have played 10 hours, so now Paul and I just need to catch up. If we can catch up, it means it’s still balanced. I’m just trying to figure out how we’ll practice without Ian getting practice at the same time. :)
This game is really starting to look interesting, to the point where I want to beta-test it. :/
Oh wells! It’ll be done in time. :D
Any concern that once Spy Party is released publicly, Sniper players will be able to watch YouTube videos and the like to improve, leading to a broader Spy/Sniper skill imbalance? Genuinely curious, not trying to presume.
It’s definitely a concern, but I hopefully it’ll be more fun to play other people, and so there will be Spies getting practice too!
The development process you are taking is fascinating! Thanks for sharing!
Disregard the following if it’s already implemented in the game or just sound like a bad idea…
Reading this i was thinking that as a sniper, having other characters that are acting suspicious might confuse me and so help to draw some of the attention away from the spy.
Such characters might be a guy having a secrete love affair with one the ladies in the room (and trying to hide it), or a messenger with a private message to the Ambassador… My point is that some like-spy behavior can be observed by none-spy characters, and used as a tool in the game mechanics.
Most chances thats something you already considered, and if so… no harm done :-)
It’s not implemented yet, but it’s definitely part of the plan! Those are great examples of the kinds of things I’d like to do, and I’m all ears for ideas anytime, so thanks!
I was wondering, do you have any plans for a practice mode, where you play either as sniper or spy against an AI opponent, with tweakable settings.
Yeah, there’s already a (crappy) practice mode that lets you get used to the controls on both sides, but eventually there will be a full single player mode, with tutorial, practice, and hopefully a campaign of sorts.
Well, the thing my mind jumped to instantly when you described the problem with the action test, I jumped to this conclusion: because the sniper already had time to adjust to the “normal” time frames on the animations, any difference was easily discernible. One possible solution could be have NPCs randomly succeed “fake” action tests. They would never fail an action test, but if they would randomly succeed, it would take away the the “punishment” for succeeding the action test.
After I thought about this, it really didn’t make as much sense as I first thought. That’s because the NPCs don’t really have tells when just performing normal actions.
So I went back and watched this clip again. http://www.youtube.com/watch?v=FwxEqFr5h2s . The thing I noticed once I watched it about 10 times in a row and started observing it without focus was that the action test version was actually the most visible, even though it was the quickest. It might benefit from actually making it the same speed as the “normal”, just without the rub down at the end.
The “good” result on that clip is actually the hardest to see in playtests for the Sniper, because the head doesn’t move (much), and that’s the main thing Snipers look for since the head is visible from multiple locations. It’s really hard to see the hand, if the Spy times it right with people around and is facing the right way (like at the corner bookshelf).
Good explanation. All comments I make a speculation due to never playing.
Interesting… Reminds me a bit of the Spy-skill relationship in TF2.
When TF2 first came out, nobody knew what a spy did, making spies dominate all the unaware players. Once players began learning how to search for spies, it became one of the hardest classes to play. If you jump into TF2 today and play a spy, you will die alot. It isn’t easy to play if you don’t know what you are doing.
With other classes like the Soldier or Heavy, you can just see what a class does by observing them. Its easy to pickup from observation. Which is why Medic is such a great starter class, so you can observe more skilled players.
Just my two cents