Beta Bug Fixing Meets Game Design
Cast Character Display
Update: If you ended up here looking for the open beta, I have no idea why wordpress is sending you here when it should be sending you here. Sorry, and I’m trying to figure it out!
So, I’m motoring along through my beta todo lists,1 and I start to investigate a bug Rachel found during a recent playtest: “suspected double agent on rollover but no triangles?”
Seems simple enough, and I’m able to reproduce it quickly. Allow me to explain the bug, and then the naive fix, and then the real fix that doesn’t mess up the game’s tuning…
Missions, Casting, and Game Modes
First, some missions have cast characters, meaning the Spy (or the code) gets to pick a character at the party to cast into a role, for example the Ambassador, the Seduction Target, or the Double Agent. These cast characters are displayed to the Spy and Sniper with a triangle over their heads, and the Sniper also gets a bit of rollover text when the target is on the cast character. You can see an example to the right.
The Sniper sometimes doesn’t get to see the correct cast character, so while both players get to know which character is the Ambassador, only the Spy gets to know who the Seduction Target is. For the Contact Double Agent mission, the Spy gets to see the real Double Agent, and the Sniper gets to see multiple Suspected Double Agents, and doesn’t know which one is really it.2
There are also multiple game modes on most maps, and some of the game modes are known as subset modes, where the Spy can select for play a subset of the total available missions for the map, and then optionally enable a subset of the selected missions. This is for more advanced play, and subsets prevent the Sniper from camping a given mission, or at least make it a very high risk strategy to do so because he or she can’t be sure the Spy is actually attempting that mission. The Sniper can see which missions the Spy has selected, but not which he or she has enabled, so in the shot below, you can see the Transfer Microfilm, Swap Statue, Inspect Statues, and Seduce Target missions are selected, but the Spy only has to do 3 out of 4 of them.3
The Spy’s Selected (but not necessarily Enabled) Missions
The rollover text but no triangle!
The Bug
The bug reproduces when I deselect the Contact Double Agent mission on the Ballroom map, so there is no real Double Agent. The mission code still randomly casts the Suspected Double Agents and sends them to the Sniper. The Sniper code does “the right thing” in the code that draws the triangle and checks if the mission is selected, but doesn’t check in the rollover text code, and so you get the latter but not the former. It looks like the shot to the right.
I obviously shouldn’t be displaying the rollover text to the Sniper for Suspected Double Agents when that mission isn’t selected as available. Or should I…
The Fix(es)
My first thought was the obvious: check whether the mission is enabled, and if it is, don’t display the text. But as I was looking into making that fix, I started thinking about the ramifications of that change. None of the cast characters can be the Spy (currently), so that eliminates a bunch of people at the party from the Sniper’s point-of-view, including the Waiter, the Ambassador, the Security Guard, and yes, the Suspected Double Agents. If I fixed the bug and removed the rollover text, it would actually increase the number of potential Spies at the party, and change the tuning to make things more difficult for the Sniper. The more partygoers the Sniper has to worry about, the more information he or she has to sift through to find the real Spy.
But, if I go the other direction, and just remove the number of Suspected Double Agents from the party if the mission is not selected, then things get easier for the Sniper, because even if the Sniper knows the suspects can’t be the Spy, having that density of people in the room increases the cognitive load.
After bouncing this issue off a friend, it seems like the right thing to do in the near term is to still send over the Suspected Double Agents, and display the triangle and the rollover text, even if the mission is not selected. This way the number of people at the party stays constant regardless of whether that mission is selected, and the game stays tuned correctly. It’s actually a kind of interesting solution, because it means the Sniper has just a tiny bit more cognitive load, because the suspects will be at the party, but the Sniper will have to either lowlight them or remember to ignore them if the Contact Double Agent mission is not selected.
Here’s the fix that actually removes the enabled check on the triangle draw code:
=== modified file 'project/spyparty/code/situations/double_agent/double_agent.cpp'
--- project/spyparty/code/situations/double_agent/double_agent.cpp 2012-02-24 03:13:37 +0000
+++ project/spyparty/code/situations/double_agent/double_agent.cpp 2012-03-14 21:20:46 +0000
@@ -414,8 +414,8 @@
}
- if(enabled && ( (GameMode == MODE_SNIPER) || (GameMode == MODE_DEBUG)) ) {
- // highlight the suspects
+ if4 {
+ // highlight the suspects, even if we're not enabled, else we're affecting the tuning of the map...sigh
for(uint32 i = 0; i < NumSuspectedAgents; ++i) {
Eventually, once I’ve got enough people in the beta to get valid statistics, I can experiment with other solutions, like removing the suspects. Another thing I’m not sure of is whether I need to display the Double Agent on the Spy view in this case…currently I don’t. It’s hard to see how that would matter since the Spy’s gameplay is so different, but I’m not quite sure.
This was not the blog post I was planning on making today, but I found it interesting that such a simple bug could have subtle tuning impact like this, and decided to share. I firmly believe basically all game programming is game design, and the little decisions made while writing code are where the game design rubber hits the road, but this one surprised me with the subtlety of the ramifications. I’m trying to imagine what would have happened if this was a big game with a team of designers, a team of producers, and a team of programmers, and this bug got entered by QA into the database, shunted to a programmer via an associate producer, and then “fixed” and checked off the list. How long would it have taken to figure out why the Sniper was winning or losing more often than before after the next official beta build rollout?
I’ll post GDC pics next time!
- Yes, sadly, there are multiple todo lists. [↩]
- The real Double Agent is always a suspect. The Spy doesn’t get to know who the other suspects are, just to make things even crazier! Asymmetry everywhere! [↩]
- There are actually two kinds of subset modes, one where the Spy has to pick which subset he or she will attempt before starting, and another where the Spy can opportunistically attempt the missions at play time. [↩]
- GameMode == MODE_SNIPER) || (GameMode == MODE_DEBUG [↩]
I like posts like this!
It’s a great way to kind of peak in on the actual development and game balancing that usually just all happens behind the curtains.
I’m very excited for the chance to take part in the beta, and can’t wait until spy party rolls out!
Thanks for this post!!
Steve B.
@Sb00ya
I fully agree, interesting stuff!
If only it was this easy http://i.imgur.com/LvW2Z.jpg
The bug database at Microsoft when I was there in the 90s was called RAID.
Interesting ruminations. The counter-intuitive fix is the one that preserves the balance the most. Definitely something to play around with in the beta. If you went for the fix that removes double-agent indicators it’d be a good example of a factor for choosing mission subsets that’s not immediately apparent, and emergent information like this tickles me just right. There’s another option: let the spy be a suspected double-agent!
Also regarding sub-note 2: the spy doesn’t get to know who is the fake-double agent. I didn’t know this, this is definitely a subtle game-changer! The more small things I learn, the more I think SpyParty has a huge scope for customizable gametypes.
The Spy will definitely be able to be some cast characters eventually, especially the Security Guard and the Waiter! Maybe the Ambassador, too.
I fully agree. I was surprised reading this that the spy is not allowed to be a suspected agent. =)
I agree with Hecker’s decision to not let the spy see who the suspected agents are, or even if he himself is selecting a suspected agent.
And it wouldn’t make sense for him to be the actual double agent… unless there’s some kind of reverse mission, “Be contacted by the AI spy. Pretend to not be the double agent, and listen carefully for any comments about delicious food substances uttered by nearby party goers.” =P
A rare glimpse at a code sample from SpyParty :P
That definitely sounds like the better solution – more consistency in gameplay (and in difficulty).
I miss using C++. Java is all the rage these days…
What are the chances of being able to buy “one for you and one for a friend” 2-packs during the beta? I couldn’t think of a more fitting game for this kind of model, as I imagine a lot of people (myself included) are going to want to start out playing against a specific person (who may not have heard of the game yet and isn’t signed up for the beta) so as to deliberately avoid having too much of a head start and keep things competitive.
On a different note, the captcha took me four tries. These things are getting worse and worse for humans, and algorithms only get better.
I hope to do a two-pack kind of thing during beta, like Frozen Synapse, but I haven’t implemented it yet. I don’t even have “invite a friend” yet, which I also need.
On the captcha, the best part is it doesn’t even stop the spammers anymore. I get 10 or 12 spam comments a day I have to moderate. I don’t know if they just have people doing them now, or if their algorithms are good enough, or what, but it’s basically lose-lose at this point.
Would it be possible to switch your comment engine to have accounts (w/ email confirm, maybe captcha too) so we can stop with the captchas?
Another option would be an anti-spambot question like “What is the spy’s enemy called?”
or at least, ignore captchas for people who have already completed one, via a cookie
Wait, you’re seeing a captcha every time you comment? It shouldn’t display them once I’ve approved a comment by you. Is your information being filled in correctly?
Huh, I just tested it and I am too…that’s not supposed to happen…I think something must have broken. No, wait, I think I must have misunderstood something. Hrm, I’ll figure this out.
Yeah, have to fill the captcha everytime. Gets old after awhile but ive managed to get pretty decent at reading the dang things.
I recently came across an article that talked about a “security flaw” with captchas and that is the audio portion. I wouldn’t be surprised now if they have bots just listening to the audio and giving it the best shot. Voice recognition has come a significant ways so far! It worked surprisingly well though. Now they are experimenting with how to stump the bots that listen to the audio. The best thing they found so far is to have other people talking in the background (sort of like at a sports pub or at your local tavern).
Yeah, 2-packs would be a great idea. I wonder what would be the best price for such packs! My guess would be something like 16 or 17 dollars, so you get the second copy super-cheap. This is not only because sales apparently generate lots of extra profit for video games (http://www.geekwire.com/2011/experiments-video-game-economics-valves-gabe-newell), but also because it means you would only need to convince one person to buy, while you get probably two enthousiastic people in the end. And more enthousiastic means more word of mouth. On the other hand, if this same couple would have to pay 30 dollars to play, there is a serious chance neither of them would buy it. I may be wrong of course.
Pricing strategy is interesting but terrifying. I will probably price it at a discount for two people, but not that steep of one at first. It’s a lot easier to come down in price than it is to raise prices if you mess up, but, more importantly, I’m okay with selling fewer copies right now as I figure things out. Later, when I open things up and have everything scalable and humming along smoothly (ha!), I’ll play with prices a bit, do sales, and all that stuff.
Yeah, $1-2 for the second copy is a bit ludicrously low.
Here’s one possibility: during the beta, a second copy is $5, but you need to pay the other $10 sometime after the beta ends to reactivate the key. Sort of like “pre-ordering with beta access,” which seems to be all the rage these days. (Honestly, I’m really glad you’re not saying ‘If you give me your money, I’ll let you in the beta.’ Betas have become more ‘trial’ periods than they are about bugs and balance.)
That way, the second copy is still potentially full price, as you can change second copy pricing later if you feel think it is necessary.
On another note, it didn’t ask me for a captcha, nor did it display an error. However, the URL did contain “incorrect-captcha-sol” so I loaded the resulting URL in another browser, and now it claims “That reCAPTCHA response was incorrect.”
@Checker: I agree, it’s probably the best (or at least the safest) not to start with risky experiments at the beginning; especially during the beta selling as much as possible is not the main goal. Still, I really believe that once you start experimenting, it’ll turn out that ‘ludicrously low’ priced two-packs are the most profitable. But we’ll see :-).
Yay! I’m not even in the beta and sort of filed a bug. As if you didn’t have enough work already :P
Okay, I think I fixed this. If you’ve had a comment approved previously, you shouldn’t have the terrible recaptcha now. Somebody let me know if it’s working.
I’ll see if I can fix the recaptcha-fail-hoses-threading thing now.
Captchas definitely not fixed. I placed my comment about 30 seconds ago with no trouble. Had to open an incognito window in order to post this one; same problem as before :(
Hrm, testing it again.
Edit: ah, I suck. I fixed it so it doesn’t display, but then it fails the captcha. Hold on, I’m an idiot, I thought it went through the same code.
Okay, trying again.
Yay, worked! Now it’s fixed for realz.
Teessttt tetstsrfwyhjeudg
Sweet, now I can spam this blog without a hassle! That being said: what’s your guesstimate on next BETA wave :P?
Within the month, easily. :P
You’re mightily optimistic there, seeing there are only 15 days left in this month.. but should be soon anyway, so I guess within this year is reasonable :)
Ill bet you my spot in line :) if I am right, you give me yours. If I am wrong, ill give you mine…..
Hmm, now things are getting interesting. Jordy is #6 in line, Keith is #1108.
(No, I will not look up what number everybody is, it takes a few manual mysql commands for each subscriber and is a pain in the butt! :)
You do realize you are betting your entire BETA-entry on something as feeble as Checkers estimation of release time?! You ought to know better :P!
Though my trust in Checkers ability to estimate release dates has been slightly tainted, there is still a sliver of faith I’m holding not to risk such a BIG thing..
But not to be a pushover, I’ll bet you a week of silence on the blog, facebook and twitter?
I may have bet my spot in line, but i’ll be in the beta sooner or later, win or lose.
I’ve thought about your counter-offer, and i REALLY want to wait till later to actually post a response, but time is limited for me as those invites are coming soon. So, i will post my response and sacrifice the suspense.
I like your offer but i want to raise it up a bit. The loser will not be able to check Spyparty’s facebook, blog, or twitter ( Aka, you can’t visit any of them for the entire week and must remain in the dark from everything spyparty. Any subscriptions will have to be unsubscribed to for the length of the week. )
In addition, if you happen to be in the next round of invites, you will also not be able to post anything in the beta forums EXCEPT bug reports. No discussions of strategy, no ideas, no nothing.
Let me recap, I am betting the next round of invites will happen within the month. The loser can’t check or visit Spyparty’s twitter, facebook, or blog. If the loser happens to get an invite, the only thing they are allowed to post on the beta forum are bug reports but nothing else.
If it happens within the month, the week for you will start as soon as the invites are sent/announced. If it doesn’t happen within the month, my week will start on April 1st.
Do we have an agreement?
P.S. There is an interesting sort of “power” that we are giving Checker.
We’re almost there :), but your are proposing an asymmetrical bet, which is, I admit, in the vein of the game, but not really fair.
So to balance things, I offer you my 1 week of not checking: blog, facebook and twitter AND not posting on forum except for bugs. In return, if you lose, you’ll have to stay away from the: blog, facebook and twitter for 1 week and not post comments for 2 weeks.
How about it?!
(I need that extra week to tease the hell out of you :)
And so, we’ve come to an accord. I accept your proposal.
I’ll allow you to make 1 last comment, after you found out that you lost, as a sort of “goodbye for a week” kind of thing. Mwhaha!
DEAL! And I’ll return the favor :). In a sense, this is win-win for me, either I get to play SpyParty and have no need to obsessively check the blog etc. or I get to win the bet.
It’s a nice hedge against Checkers unpredictability.
Yeah I was aware of the win win for you. In fact, I almost typed the exact words you used but its the assumption that you get in. This is not a certain. BUT its a win win for me too. I can use the time doing other things, which I’m completely okay with. Your extra week of teasing is worthless. You can not see my reactions or know if I actually read the comments.
There is only one person that could possibly be on the ‘losing’ side and its not me or you.
Well, I think the plan is to actually invite some people from the first sign-up list, so let’s just hope the cut-off ain’t at 5 :S!
And I’m sure you will be desperate to ask me questions when I’m hinting how great the game is and what I’m doing but not fully tell the story :P, but I’m curious, who could be on the losing side?
Sadly i don’t think you are guaranteed the next round of invites. Here is a quote from the blog for the time frame after GDC
“After GDC, or maybe just before GDC if I’m feeling really confident and things are going swimmingly, I’ll invite in a small group of the earliest and hardest-core fans from the invite list, the blog, Twitter, Facebook, GDC, and PAX, and a small hand-picked batch of press who’ve been covering the game closely since the beginning.”
Now after those invites, you will for sure 100% get in!
Checker is the only person that could possibly be on the losing side.
Get a room, you two =P
Yea, I interpreted that the invite-list was the BETA-sign-up list, I hope he invites first 10 subscribers and that my interpretation is actually right :S.
@Quirken, wanna join?
Yes, the “invite list” in that paragraph means the beta signup list.
I was under the impression that he’s going to invite people that he thinks are the ‘hardest core’ and earliest from all these different outlets vs inviting 10 from the list + press.
Don’t get me wrong though, I totally expect you to be in the invites coming up!
@quirken if its the bug I’m thinking of, that was a good bug find indeed. You deserved a high five for that one but we are out of range so it just ended up being a lonely air five.
I’ll make you an offer, to spare you the pain of missing out for 2 weeks JUST when BETA launched.. You can accept your loss and I’ll let you off the hook with 1 week, whadda ya say?
It’s been almost a full week; I am not worried. In fact, i’ll even up the ante. Your spot in line for my spot in line.
UP the ante you say? You say, Don’t know if I could stand losing that bet.. No I’m good, just saying though.. just 1 week left for you to win this bet, after that, I win!
And seeing how Checker was inspired by Blizzard once before, I’m suspecting he’ll release on a monday, just because it seems more practical as well, so we’ll see what coming Monday will bring us :), if not BETA, you better worry!
Thanks for the reminder but I’m still not worried :)
By the way, yo SpyMomma is so fat, when she stands at the statues you can’t tell which one she’s picking up.
YoMomma is so fat I ain’t even have to move my laser-sight.
I’m beginning to think you’re worried sir. Tell me, how are you feeling?
A bit anxious, Monday MIGHT be the day, then again, who knows right..? Certainly checker doesn’t :p.
You’ve run out of YoMomma jokes though? YoMomma is so ugly it takes her 10 minutes to seduce a target ;)
(@Checker, sorry for polluting your blog with this nonsense, if there was any other way, I would use it, like in-game chat.. but..)
There are plenty of other places you could take your banter >_< I can't speak for checker, but I think it'd be great if SpyParty comments were at least somewhat SpyParty related.
Yeah, it’s a delicate thing. Having fans excited by the game is totally awesome, and having some really hard core fans that follow the game is totallier awesomer, but, at the same time, we don’t want it to seem like some insider’s club where new people or less frequent commenters aren’t welcome. So, I guess maybe Jordy and keith could dial it back a notch here and on facebook, but I would be super sad if that request made them stop posting completely (chances seem low of that happening :). Obviously, as Jordy says, the solution is to get them invited into the beta as soon as possible so they have some place to spend their energy. Also, I’m maybe slightly regretting fixing it so approved commenters don’t need to do the captcha! ;)
Gotcha! I’ll tone it down a bit, I must say no captcha makes it quite a bit more tempting ;)
Yeah, see, it’s those second order consequences of a mechanics change that really interest me about game design! :)
That said, I think I’ll have to collect the Yo Spy/Sniper Momma jokes into the FAQ at some point. There were some good ones!
Commenting to show i acknowledge request. :)
I don’t know about anyone else but I already have a huge load of thoughts I want to talk about in the forum with regards to strategy, and that’s from only playing 4 games at gamecity. But yeah, instead of theorising on the blog I’m letting them stew inside my head until the mass beta invites.
First I thought I should keep tips and tactics to myself to give myself the edge, but in truth it’d be so much more rewarding to share anything and everything so the metagame can grow as fast as possible. I want to see gambits go in and out of fashion, mission subsets to shift in popularity, and I want fluke discoveries to completely break the game and make every round a sniper win until checker fixes it.
“I’m trying to imagine what would have happened if this was a big game with a team of designers, a team of producers, and a team of programmers”
Oh, please don’t jump on the “indies are strictly superior to AAA” bandwagon like other indies seem to do a lot. I’ve been enjoying SpyParty and your blog so far, don’t ruin it with offhand remarks like these :(
If I or any of my colleagues was handed this bug we’d think just as carefully about this as you did, probably more so because we’d generally talk to each other about it too. We’re not robots. And we have generally pretty decent analytics so we’d be able to measure the effects of our decisions. I don’t know how things worked at Maxis but that’s how they are for me.
Anyway, I’m looking forward to playing SpyParty and hearing about its development. Thanks for being so candid :)
Whoa there, nobody said “strictly superior”! There are plenty of things that the mainstream development teams do better, but I do feel pretty strongly that the feedback loop on things like this is WAY tighter when it’s inside a single person than when there’s a big team. Your experience might be different, and it probably varies a lot by company and team, but it’s hard to get the same level of iterative improvement when things have to cross out of one brain and into another, either via a bug database or even talking. This is admittedly a bit of a philosophical point on which reasonable people can disagree. :)
Ok, I know you didn’t actually say that, I just got worried you were going to turn into ‘one of those guys’ :) I do agree that iteration times will be faster for you than for bigger studios. I just don’t like that you implied a bigger studio would commit a bad fix and have to wait to find out it was bad – there’s just as much chance of you doing that as anyone else.
Right, that’s enough negativity – sorry about that. Keep up the good work; I’m loving it :)
(Oh man, the captcha beat me! These things are getting too hard)
(WHAT?! It beat me twice? This makes me sad.)
Also it appears failing a captcha makes it forget that you clicked ‘Reply to this comment’ :(
Yeah, that’ll get better when I fix wp-recaptcha to not require the captcha after I approve a comment by you. Also, fixed the threading for you, sorry about that!
Just to be clear, I think “bad fix” is too strong a phrase for what I was saying. The obvious fix wasn’t necessarily bad, it just would have changed the tuning (or, at least, I think it would have, no time to A/B test it, which a big team might have!). In any game like this, there are so many things contributing to the tuning, and especially this early when things are broken in a bunch of ways, and the tuning is not well understood by anybody (including me). My point was that if this fix was done the obvious way and rolled into a build that had 150 other checkins in it before it was pushed out to the players, then it’s totally possible it wouldn’t be noticed for a while.
If only the “many worlds” hypothesis was correct and we had access to the other ones so we could see how things play out! :)
I can’t tell you how excited I am for SpyParty. It’s the epitome of fantastic indie development.
I’m glad I’m not the only person who sighs in their code comments :)
“sigh” is a relatively benign comment for me…although maybe the saddest, because it implies a sense of defeat :)
I remember playing with the HL1/HL2 SDKs… there are all sorts of amazing comments like
// HACKHACK – Should we be doing this?
or
// HACKHACK – Why does this work?
The best I’ve seen is:
// If this is production code and this part is still here, we’re fucked
Needless to say, in production code.
Haha, where was that?
(also, if this goes through, captchas are fixed)
Looking awesome! Love posts like this!
I was wondering if the game will include some sort of LAN play or if all games have to go over the central spyparty server.
Right now you have to be connected to the internet. I’ll figure out LAN play at some point, but not for a while.
Cool. I can’t wait for LAN; it’s the most enjoyable form of multiplayer gaming in my experience. I can’t wait to see the faces of my friends after I shot them just before finishing the last mission ;-).
Oh wait, nevermind… not being able to play over LAN doesn’t mean that we won’t be able to play to someone in the same room, and that was what I was talking about. Forget what I said :-P.
Yep, and that’s the best way to play, too! Although, as the Spy, be careful of your mouse clicks, they’re huge tells.
Keep doing posts like these, I really like seeing the “Inner workings” of the game. :)
Don’t you need a double agent regardless of what missions are enabled, because you’re going to have mechanics like the spy ordering him to create a distraction? And therefore it makes sense for the sniper to always have suspected double agents given to him?
I’m not sure how that’ll play out, actually. Yes, when I make the Spy able to give the Double Agent submissions, then I’ll definitely have the Suspected Double Agents present, but the first draft of that will be where you have to contact him or her first, so they’d be present anyway. Then, there are maps where there’s a Double Agent but no Suspected Double Agents, like the balcony, because there are so few people there that being able to eliminate even one would be a huge advantage. Lots of things to explore!
I feel like I’m missing something here. A part of the “spy picks X out of X” and “spy finishes any X out of X” gameplay – in which the sniper does not know which missions the spy is trying to accomplish – is forcing the sniper to figure out which signs to look for. As such, the suspected double agent should always be present because otherwise, de-selecting the “contact double agent” mission would be a tell to the sniper. They would know that of these 4-6 missions the spy is going after, because there’s no suspected double agent (when there normally is one) then there is no banana bread to worry about.
This possible (but currently avoided) tell wasn’t mentioned at all anywhere, which makes me think I’m missing something.
Wow, this is me not looking at the dates of posts. Whoops.
The suspected double agents are always present on maps where they’re ever present so that the number of potential Spies stays the same, but the Sniper knows whether Contact is a potential mission so they know whether to pay attention to them.
Hah, no problem, I don’t close comments on old posts because there’s no reason to, I’m happy to answer questions about any of the topics. It always annoys me when blogs close comments on posts.
It means I have to moderate more spam attempts, but besides that, it seems like the right thing with no downsides!
SpyParty is great game! Make you feel badass, very high stress fun. You should sign up to pay money, good deal! Click this link: http://www.spyparty.com/beta-sign-up/