Beta Bug Fixing Meets Game Design

Cast Character Display

 

Update:  If you ended up here looking for the open beta, I have no idea why wordpress is sending you here when it should be sending you here. Sorry, and I’m trying to figure it out!

 

So, I’m motoring along through my beta todo lists,1 and I start to investigate a bug Rachel found during a recent playtest “suspected double agent on rollover but no triangles?”

Seems simple enough, and I’m able to reproduce it quickly.  Allow me to explain the bug, and then the naive fix, and then the real fix that doesn’t mess up the game’s tuning…

Missions, Casting, and Game Modes

First, some missions have cast characters, meaning the Spy (or the code) gets to pick a character at the party to cast into a role, for example the Ambassador, the Seduction Target, or the Double Agent.  These cast characters are displayed to the Spy and Sniper with a triangle over their heads, and the Sniper also gets a bit of rollover text when the target is on the cast character.  You can see an example to the right. 

The Sniper sometimes doesn’t get to see the correct cast character, so while both players get to know which character is the Ambassador, only the Spy gets to know who the Seduction Target is.  For the Contact Double Agent mission, the Spy gets to see the real Double Agent, and the Sniper gets to see multiple Suspected Double Agents, and doesn’t know which one is really it.2

There are also multiple game modes on most maps, and some of the game modes are known as subset modes, where the Spy can select for play a subset of the total available missions for the map, and then optionally enable a subset of the selected missions.  This is for more advanced play, and subsets prevent the Sniper from camping a given mission, or at least make it a very high risk strategy to do so because he or she can’t be sure the Spy is actually attempting that mission.   The Sniper can see which missions the Spy has selected, but not which he or she has enabled, so in the shot below, you can see the Transfer Microfilm, Swap Statue, Inspect Statues, and Seduce Target missions are selected, but the Spy only has to do 3 out of 4 of them.3

The Spy’s Selected (but not necessarily Enabled) Missions

 

The rollover text but no triangle!

The Bug

The bug reproduces when I deselect the Contact Double Agent mission on the Ballroom map, so there is no real Double Agent.  The mission code still randomly casts the Suspected Double Agents and sends them to the Sniper.  The Sniper code does “the right thing” in the code that draws the triangle and checks if the mission is selected, but doesn’t check in the rollover text code, and so you get the latter but not the former.  It looks like the shot to the right.

I obviously shouldn’t be displaying the rollover text to the Sniper for Suspected Double Agents when that mission isn’t selected as available.  Or should I…

The Fix(es)

My first thought was the obvious:  check whether the mission is enabled, and if it is, don’t display the text.  But as I was looking into making that fix, I started thinking about the ramifications of that change.  None of the cast characters can be the Spy (currently), so that eliminates a bunch of people at the party from the Sniper’s point-of-view, including the Waiter, the Ambassador, the Security Guard, and yes, the Suspected Double Agents.  If I fixed the bug and removed the rollover text, it would actually increase the number of potential Spies at the party, and change the tuning to make things more difficult for the Sniper.  The more partygoers the Sniper has to worry about, the more information he or she has to sift through to find the real Spy.

But, if I go the other direction, and just remove the number of Suspected Double Agents from the party if the mission is not selected, then things get easier for the Sniper, because even if the Sniper knows the suspects can’t be the Spy, having that density of people in the room increases the cognitive load.

After bouncing this issue off a friend, it seems like the right thing to do in the near term is to still send over the Suspected Double Agents, and display the triangle and the rollover text, even if the mission is not selected.  This way the number of people at the party stays constant regardless of whether that mission is selected, and the game stays tuned correctly.  It’s actually a kind of interesting solution, because it means the Sniper has just a tiny bit more cognitive load, because the suspects will be at the party, but the Sniper will have to either lowlight them or remember to ignore them if the Contact Double Agent mission is not selected.

Here’s the fix that actually removes the enabled check on the triangle draw code:

=== modified file 'project/spyparty/code/situations/double_agent/double_agent.cpp'
--- project/spyparty/code/situations/double_agent/double_agent.cpp      2012-02-24 03:13:37 +0000
+++ project/spyparty/code/situations/double_agent/double_agent.cpp      2012-03-14 21:20:46 +0000
@@ -414,8 +414,8 @@
     }
-    if(enabled && ( (GameMode == MODE_SNIPER) || (GameMode == MODE_DEBUG)) ) {
-        // highlight the suspects
+    if4 {
+        // highlight the suspects, even if we're not enabled, else we're affecting the tuning of the map...sigh
         for(uint32 i = 0; i < NumSuspectedAgents; ++i) {

Eventually, once I’ve got enough people in the beta to get valid statistics, I can experiment with other solutions, like removing the suspects. Another thing I’m not sure of is whether I need to display the Double Agent on the Spy view in this case…currently I don’t. It’s hard to see how that would matter since the Spy’s gameplay is so different, but I’m not quite sure.

This was not the blog post I was planning on making today, but I found it interesting that such a simple bug could have subtle tuning impact like this, and decided to share.  I firmly believe basically all game programming is game design, and the little decisions made while writing code are where the game design rubber hits the road, but this one surprised me with the subtlety of the ramifications.  I’m trying to imagine what would have happened if this was a big game with a team of designers, a team of producers, and a team of programmers, and this bug got entered by QA into the database, shunted to a programmer via an associate producer, and then “fixed” and checked off the list.  How long would it have taken to figure out why the Sniper was winning or losing more often than before after the next official beta build rollout?

I’ll post GDC pics next time!

  1. Yes, sadly, there are multiple todo lists. []
  2. The real Double Agent is always a suspect.  The Spy doesn’t get to know who the other suspects are, just to make things even crazier!  Asymmetry everywhere! []
  3. There are actually two kinds of subset modes, one where the Spy has to pick which subset he or she will attempt before starting, and another where the Spy can opportunistically attempt the missions at play time. []
  4. GameMode == MODE_SNIPER) || (GameMode == MODE_DEBUG []

I Should Not Be Posting About This Beta Playtest

I really should be working on my GDC1 talks. Instead, I am writing up yet another beta playtest, so I will make this quick!

On Friday my friends and awesome game artists Luis Antonio and John Cimino playtested.  They were only supposed to play 4 or 6 games, but kept playing for an hour, which is a good sign.  Of course, I took a bunch of notes, so here are the tasks from that:

  • don’t get that click on guy selects during cast characters
  • mouse overrides cursor keys in menus if over item…don’t override if key change but no mouse change
  • have box with ? on side for casting characters – progress status and neat
  • why is volume not working on last 30 seconds timer on sniper machine?
  • sniper review tells on waiting screens
    • chooser through animations playing on loop!
    • require sniper okay to continue
  • detailed stats stay on mouseover and off…good?
    • mode on chooser that says if mouse moves off, and not keyboard select, then help goes away?
  • pgup->PageUp
  • ambassador was doing tapdance again…is it always the ambassador? I think so…!2
  • microfilm get animation is same as put, so you get it out of your pocket, which makes no sense

I need to consolidate the recent punch lists so you folks can see what I’m working on.  I occasionally check items off the older lists in the older posts, which is kind of silly.  Sadly, the tasks on this list aren’t going to get checked off for a while due to the conference this week.

The best anecdote from the playtest was when John shot Luis and then said, “I lowlighted all of the men at the start because you’ve only ever picked women as the Spy.”   Luis realized his meta-mistake, and had to consciously start randomizing his character choices more. That is exactly the kind of know-your-opponent yomi gameplay I want to encourage!

If you notice the laptop to the left in the picture below, John is actually chatting with us using Google Talk.  Luis and John didn’t notice because they haven’t played that much yet and I didn’t want to point it out, but I could clearly hear the Spy’s mouse clicks through the connection, and they are huge tells, so I’m going to have to figure out what to do about that before I enable voice chat as a feature.

SpyParty causes bad posture.

All the auto-update, crash dump, and login stuff seems to be working really well, so I’m super happy about that.

Now, back to my talks!

  1. The interview with Notch is supposed to be live-streamed on Gamespot, no link as of yet. []
  2. I’ve been seeing this rare bug where one of the characters would slide across the floor, jittering.  I noticed it again during this playtest and realized I always see it on the Ambassador, which is a huge help in debugging it! []

More Playtests, More Tasks

Last Thursday we did a small playtest, the second so far in the Early-Access Beta!  This time it was Rachel, Lauren, Ian, and me, and we played and chatted (and restarted after crashes) for a little over 3 hours.

As it should, the playtest generated a ton of tasks, but this time I actually fixed a bunch of them before I got around to posting this, so here is the pre-struck-through punch list!  It seems much more satisfying this way, if only all my task lists were already mostly done before I posted them…

  • coming out of settings menu doesn’t eat the final up and it performs action
  • what is invert-y’s key? – was shift-y, made ctrl-y
  • notify new chats if pageupped
    • /ring bell when message comes in if in background?
  • display match or lobby [channel] on /say and /emote
  • escape should clear timestamps on all chats if displayed
  • floating briefcase
  • change white chat help to darker
  • stop echoing command on /say /emote /whisper
    • also do /reply and /retell local emulation
  • chat help on sniper machine adjust up like chat line for portraits
  • gameid sync is because server sends, then get request to change, which clears game
    • include gameid in request?
    • this is because of the clearing of the gameid in ClearGame
    • the server gameid comes in before the be_role packet, which toasts it
    • uh, undo previous CurrentGameID.Zero() change at least, still see sync issues?
  • if page up, then escape, new chats don’t show up, and new chats disappear
    • need to reset paging on escape?
  • suspected double agent on rollover but no triangles?
  • assert IsChatAllowed(), File: .\spyparty.cpp, Line: 5565
    • when I chatted when cancelling invite
  • go back on selection screens
  • enter focus for chat?
    • or just use it for chat and fuck it for menus
    • fine, make it /say
  • enter on empty chat should close
  • crash dump!
    • pigo says: i hit play again, and when i selected the sniper it crashed right away
    • i’m in results, he e2mm, hit sniper, i hit no, he hits sniper again
  • no confirm on end match in match menu
  • just do SpyWins & SniperWins for lobby stats
  • some whipsers getting eaten
  • tolower username in game
  • MOVEFILE_COPY_ALLOWED

2012-02-29 16:16:25 – I don’t think whispers were actually being eaten, I think it was /say in matches, which now describe which [channel] they’re talking to.  I think.

Of course, there are still some tasks left on the previous list, not to mention the list before that, but this is forward progress.

The funniest issue from this playtest came from chatting in the lobby.  I use Counter-Strike keys for chat, so ‘y’ is public chat, and ‘u’ is whisper.1  Lauren and Ian are big World of Warcraft players, so they would constantly hit ‘enter’ to chat, which when you’re in the lobby would start a match with somebody. About the third or fourth time one of them ended up playing a game in the middle of a chat conversation, I acquiesced and made ‘enter’ a synonym for /say just like ‘y’, and made ‘space’ the key for selecting menu items.2

Maxima

When you’re working on software, you get a feeling when you approach a local maximum, meaning a point where you’ve got the thing working pretty well given its current overall state.  This is always a long way from the global maximum3 but it means the game is pretty robust and does all the things it’s trying to do at this point in time reasonably well.  The current SpyParty builds need a bit more testing before they’re really at that local maximum, but I feel like they’re getting close.  The builds feel like this towards the end of PAX, after I’ve shaved off the rough edges found from 20 hours of playtesting.

This means I seem to be on schedule for the near-term beta rollout plan, which is nice.  I gave the “invite-some-fans-in-before-GDC” idea mentioned in that post a little more thought and I realized I was completely insane to even think about that…I have no time to fix or support anything during GDC,4 so everybody’s going to have to wait until the week of the 12th for more invites to go out.  Sorry!

Of course, with numerical optimization, moving off a local maximum means things get worse for a while until you find another maxima, and that’s true with software as well.  My plan is to simply bring you all along for that ride in the short term.  Once there are enough people in the beta who complain about me breaking stuff all the time, I will probably allow people to opt-in to the more bleeding edge builds, and keep them running on a separate server.  This is a pretty common thing to do to minimize pain during open betas, but it reduces your pain by increasing mine, so I hope I can put it off as long a possible.5

And no, I still haven’t gotten my first $14.26 from PayPal.

  1. really team-chat in C-S, but whisper is the closest I have to that. []
  2. mouse click works as well, obviously. []
  3. if such a thing even exists for works of art and entertainment! []
  4. or even sleep, sadly. []
  5. especially since my server deployment team is me. []

The First Beta Playtest

Late Sunday night, I was looking at the lobby server logs, and saw a login from “rachel” but no logout.  Uh oh, I thought, this must be a bug, since with only 9 people invited into the Early-Access Beta so far, the chance of actually finding another person in the lobby without making a pre-arranged date to play was basically zero.

I don’t have great tools for inspecting the running lobby yet, so I started up a local copy of SpyParty and logged on to poke around.  Hey look, it wasn’t a bug, it was actually my friend Rachel Reynolds, the real live beta tester!  She’d started up the game after registering, and was just checking things out.  She invited me to a game, and away we went!

According to the stats, we were in the match for 1 hour, 30 minutes, and 17 seconds, and we played 10 games for a total of 44 minutes and 31 seconds of playtime.  We spent a lot of time testing the new chat system and finding bugs and things that need cleaning up.  We also spent a lot of time re-learning how to play because neither of us had played since last summer!

Here’s the punch list we came up with from this official First Early-Access Beta Playtest:

  • chat
    • make local echo look like remote print, no /say
      • actually, echo command and print message
    • help on cursor line feels weird
    • put “hit ‘y’ to chat, page up/down to scroll history” at bottom
      • at least until person has done it a couple times?
      • nah, just in small text, want to encourage it
      • hmm, this is just going to be annoying if it’s always there…
    • mask keys from game when chatting!!!
      • sniper camera
      • spy movement!
    • don’t fade messages if app is background
    • do a “checker is typing” meta message?
      • can only do this out-of-game or it’s a tell
    • word wrap text to screen
    • uh, where do I cap the length of the chat message?!
      • no where…need to do it in HandleKey, and have help?
      • need to scroll the chat line too, including the help!
        • or word wrap it? yikes
  • statue in hand anim bug
    • repro: grabbing statue on veranda in overlap area?
  • fix statues to use the pad instead of radius
  • character/level chooser mouse tip for select versus next/prev
  • laser event still sending on sniper finished screen
  • add back 3 of 4 known missions on ballroom
    • have another menu mission state to pick available known missions
      • pick 4 of 6 that will be the available
      • then pick 3 of 4 to accomplish
      • make sure this journals
    • need to store some way of telling which map and game mode in db!
      • a single dword?
  • document action test is same button?
    • put small icon next to it?
  • remove briefcase verbs? or ignore them?

Not bad for an hour and half’s “work”!

However, for me personally, the biggest (re)discovery of the playtest was that I really like playing my game!  That probably sounds odd, but in the seemingly endless months of getting the beta ready, losing myself in server code that is necessary but feels very far away from making a work of art and entertainment for other humans to experience and enjoy, I’d actually forgotten how much I love just plain playing SpyParty, and it was great to be reminded of that.

Also, goddamn this game is stressful to play!

2012-03-02 13:59:07 – Just checked off the statue bugs.  Nobody is reading this old entry anymore, though, are they?

The Near-Term Early-Access Beta Rollout Plan

I still do not have my $14.26 from PayPal

But, I have fixed most of the bugs that Jonathan Blow—the first official SpyParty Early-Access Beta invitee—found, so I guess I’m not going to wait for PayPal1 and hope things are working now.

There have been a lot of questions here on the blog, on Twitter, and on Facebook about how the beta rollout is going to work, and a few misunderstandings based on ambiguous statements I’ve made, so I figured I’d try to clear all that up in this post.

First, and most importantly, everybody who signed up will get invited into the beta, it just might take a while.  Just getting to the point where I could send a single invitation email was way more work than I thought it would be, so it is going to take some time to get things to where I can invite in all 11,147 people who have currently signed up (as of this blog post).

I can, however, say how the very near future of the next few weeks is going to go…

Near-Term

Today, I need to finish writing up some basic documentation, including the Welcome Message in the private beta forums, the How to Report Bugs post in the forums, the Beta Test FAQ, and I need to update the game’s README.txt.  Hopefully I will grind these out in the next few hours.

Then, this evening, I’m going to send out a small number of invitations to my hardest-core and longest-term playtesters, folks like Paul and IanThis will be sent to only about 8 friends, all of whom are game developers and who have played a lot of SpyParty already, and who are very comfortable playing with very early software.  This group’s job is to play a lot, to find more subtle bugs—especially if I’ve broken any tuning since I haven’t playtested since PAX West 2011—and to make sure all the new score ranking and player database stuff works.  I’ll be fixing the bugs they report and implementing the features they request on a daily basis, in addition to playing a lot myself so I can stay even remotely competitive in my own game.

While these folks are testing, I’m going to finish the Invite-a-Friend feature, so they can test that as well, which will mix a few more people into the beta.

At this point it’s probably around March 1st, and I’m hoping the extremely-low-hanging bug/feature fruit will be mostly picked and everybody will be having a fun time playing and I’ll be ready to invite some more people in, including people I don’t know personally.  Unfortunately, this is also when GDC hits, which means I lose a week of productivity, so I’m going to hold off on new invites until after that’s over.  I’ll be doing a little SpyParty press at GDC, but I’m going to keep it pretty mellow.  I also might set up the laptops at some parties again, not sure.

After GDC, or maybe just before GDC if I’m feeling really confident and things are going swimmingly, I’ll invite in a small group of the earliest and hardest-core fans from the invite list, the blog, Twitter, Facebook, GDC, and PAX, and a small hand-picked batch of press who’ve been covering the game closely since the beginning.

Then the real bulk invites will happen shortly after that.  I’ll start with 20 people from the beta sign up list, mostly in order from the beginning, but also probably randomizing 25% of them from the whole list so at least a few people won’t have to wait quite as long to get invited.  Then, once the dust settles from those 20, I’ll invite 50, and then 100, and so on.

Less-Near-Term

At some point, my server will start melting from too many simultaneous players, but I don’t expect that to happen until I’ve got 1000 or more people invited in, so this first group should get in pretty quickly.  I will look at the bottlenecks and optimize the server, and that might get me to 2000 invited, or around 200 simultaneous players.2  By that point I want to have my server infrastructure scalable and running in the cloud, but that’s going to be a significant chunk of programming, so there might be a delay at this point in getting more invites out.

Finally, once the backend is scalable, I can start inviting larger groups of people in at the same time.  When I can send out 2000 invitations and everything runs smoothly as those people sign up, download the game, and start playing, it will be time to invite the rest of the beta list in, shut down the list, and let people join the beta directly without an invite. I think I’ll probably have a couple week period where I invite all the confirmed signed up people in, but don’t allow new signups, as a way of saying Thank You for signing up.

But, that’s all months and months away.  The Near-Term section above should be reasonably accurate, but this part is mostly me making stuff up right now.

  1. Give me my $14.26, PayPal!  And yes, I am going to provide other payment provider options soon. []
  2. That is, if 10% is a reasonable estimate for peak concurrent users relative to registered users, which it seems to be. []