Playtesting: A How-To?

[Nathan P.]

I see a gap between "Here's the game I'm playtesting" over at the Design forum and "Here's how the playtest session went" over here that I think would be a good one to fill*.

So how does everyone run their playtests?

Of course it depends on what stage of design you're at, and your particular goals for your game, and all that stuff. But I think it would be valuable for designers in particular to post how they playtest their games, what they look to get out of those playtests, and what kind of techniques they use to support those goals.

I GM my playtests, thus far. I tend to go into a given session with a broad "lets see how things work out" attitude. Sometimes I'll have a particular mechanic or something that I want to work on, in which case I explain that to everyone  before we get playing. I also explain any rules changes since the last time we played. I encourage players to stop play in order to ask questions that they have, and when these come up we'll pause for as long as we need to to clear up issues.

I always try to get player reactions after a session, which is sometimes valuable and sometimes not.

During a game I usually have a hard copy of the most recent draft for general perusal, and have the text pulled up on my laptop for my own reference. I make notes about the session itself on seperate paper, and notes about the text in the margins of the hard copy.

I'm sure there's lots of other methods out there, and not just about a specific session, but about the whole playtesting process. Whats yours?

* My search-fu didn't bring up anything about this, but if there's threads valuable to this discussion by all means link 'em up.

[Keith Senkowski]

Hey,

Regardless of the stage, we all take notes during the session about mechanics and what not.  We discuss everything afterwards.  However, if an issue arises during the game we usually have a brief discussion and mark it as an issue to go over when we are done.  It seems to work pretty well combined with running situations instead of full sessions.  That way we cover every aspect of the game and how they all interact with each other.

Keith

[Ron Edwards]

Nathan,

To keep this thread viable, please post some actual play experiences which illustrate your point about playtesting.

Best,
Ron

[Valamir]

I distinguish between mechanical tests and actual playtesting under real world conditions.  Mechanical tests are easy.  Often times you can do that solo just to see how the numbers work out.  The only real value in opening mechanical tests up to others is when searching for exploits or broken areas.

For me the real juice comes from actual play play testing.  In otherwords running the session not as anything special (so for me, no particular special notes or instructions to player to take notes) but as if it were a regular session of a long established game.  That's when I discover things don't work right.  These sessions may be under "demo" conditions rather than "campaign" conditionis but the key is to run it as you would any normal demo for an already published game.

One of the more effective playtests for me was a brief demo I did in the hotel room at Gen Con for some folks.  It demonstrated conclusively that the mechanics were still not streamlined enough.  Search and Handling time was still too high for furious swashbuckling adventure.  That might not have been obvious in more artificial conditions where perhaps play was being suspended to talk shop.  But since I ran it as close to a regular play session as I could, I could tell that it wasn't flowing as desired.

[Bill Cook]

I playtest ideas for mechanics as fixes to other games and then use 'em for my own. I ran a 1st ed. AD&D module called .. something about caverns and a place called Tsojcanth. Anyway, I ruled that flyers could only be targeted by sword attacks if they'd already attacked the sword fighter that round. Boy, did that piss everyone off.

Another thing was, the group had adopted this .. to me, weird way of determining initiative: roll a d10; call out 1 through 10; take your turn on your number; within the same window, monsters go last; resolve per action. So many things about this aggravated me.  I won't go into it.

So my mod was: write every character or group's name on uniform markers; turn them face down and mix; turn over, one by one, and call for action; after the last action, apply results. Combat became a mad blur. No one would take a chance on leaning back in their seat and looking up a spell because they might get called next!

*********

For straight up playtest of my own stuff, I look for two things: (1) player discomfort and (2) broken mechanics. I generally stop play for the ten seconds it takes to note the problem. Oh, another tell is unbidden enthusiasm.

Example: a project I've been working on breaks into two sections: (1) players saying what they want their character to do next and (2) killing stuff. I noticed that everyone was free with input as to how to kill things but they put a dentist chair grip on their dice at the notion of coming up with some lead-in to the shootout. Since then, I've been tooling with structures that support improvizing narrative.

[Green]

My playtesting process is pretty much like Ralph's.  The first time I playtested  Kathanaksaya, it was with a large and very diverse group of roleplayers: newbies, intermediates, and veteran roleplayers.  Here is the thread where I describe how a couple of those sessions went.

That said, after designing and playtesting my own game, I've grown to rely on actual play to decide what I need to do with it more than a casual reading and critique.  For myself too, I often abstain from critiquing or judging systems until I've played them.  Playing the game is the acid test, though I do use the initial readings of a game to make sure that there are no glaring contradictions or holes in the rules.

[Nathan P.]

The (eventual) point here is to gather some accounts of how people playtest, in order to head over to the Theory forum and talk about a methodology of playtesting. Personally, I've never had a conversation with another designer about how they playtest, so I have no idea how common or useful my techniques are.

However, here's one issue that may be of interest/concern.

In my last Timestream playtest session, a question came up about the limits of one temporal manipulation technique, and possible changes to those limits as currentely written. We took a good 10 minutes discussing it, which was great for me as a designer. As a GM, however, it broke the flow of the game and, in my memory, subsumes actual in-play events.

Now. It seems to me to make sense that there's a tension between goals of playtesting and goals of play, and this tension may easily skew the "results" of playtesting, and thus be problematical for the entire process. That is, the session wasn't as fun for me in an actual play sense as it was satisfying in a playtesting sense.  

Is this something that others see when playtesting? Are there ways to avoid or minimize this tension? Is this a valuable exercise in the sense of attempting to develop a methodology of playtesting?

[Bill Cook]

That's a good point. As a designer (or fiddler), I find it hard to remain at surface level, where play occurs. I think there's value in either approach: pressing forward, making mental notes, or stopping when your spider sense tingles and opening the hood.

Obviously, you want to consider your audience before you indulge. If you're demoing at a con or FLGS, I'd think you'd keep rolling; if it's your buds, they might be just as interested as you in stopping everything and hashing out a better way.

And you should decide, for your own purposes, what you're focused on: flow of play or mechanical integrity.

[Ron Edwards]

Hiya,

Geez, this puts me in mind of this ancient thread: Standards for playtesting. Unfortunately, at the time it didn't really take off as a topic, but I think a couple people made some useful points.

In my last Timestream playtest session, a question came up about the limits of one temporal manipulation technique, and possible changes to those limits as currentely written. We took a good 10 minutes discussing it, which was great for me as a designer. As a GM, however, it broke the flow of the game and, in my memory, subsumes actual in-play events.

I know exactly what you're talking about, especially from the period 1994-1996 when Sorcerer really got beaten into shape. I made a few decisions then which I've stuck with ever since, and it's helped me and others immeasurably. The decisions are -

1. We do "get into" the game and really try to enjoy our characters and the whole imaginative context of what the game is presenting, or making possible. In other words, yeah, we really do the whole SIS thing. This often means making a jump into a game-world etc that we might not ordinarily be interested in - but we know we gotta make that jump.

2. But it also means that we try to use the rules as presented and apply them in every way possible. We try to find out how they work together. I think the most important part of this point is the "We." It's not one guy who's playtesting and the other guys who're "just playing." We're all invested in the game as a text/explanation, and we want to help it get better.

3. When it's clear that a rule is acting as a roadblock, or that we don't understand it directly, or that its indirect effects on some other rule aren't working ... then we try not to get mad or to get disconnected from the activity. We know the game is in flux. We try to state what happened, to state why we didn't get it or why it doesn't work, and to figure out what might be the culprit.

So the point is that everyone has to invest, not only in playing, but also in the design process, as well as with the actual text. This is why I think con-style playtesting, where people "just play" and then fill out some form or something, is trash.

Best,
Ron

[jdagna]

My usual playtesting method is this:
1) development testing (i.e. me, sometimes alone, sometimes with others who have been with the game from the start) to hash out major issues and look for the big problems
2) in-house testing (me and a group of players; I usually GM) to get some wider input and see how things play out over longer sessions
3) third-party testing (a group of players who have no supervision at all).  This is best at catching things that just aren't explained well.  For example, someone discovered a missing link in the damage instructions.  It seemed intuitive enough in stages 1 and 2 that it just never got specifically stated.

One technique I've used that has been very successful at all levels of playtesting is to record the sessions on cassette tape.  I bought a cheap recorder that can compress time, so you fit 180 minutes of play on a 90 minute tape, which is enough that you only have to change it once during play, if at all.  (I'm still seriously looking at getting an iPod to do this all digitally, with essentially infinite capacity, but the tape method can be set up for $20 and most people have something already).

In games that I play in, I find that I hear things during play back that I didn't even notice at the time, and those things didn't show up in my notes at all.  In the third phase, I find it invaluable to really get into people's heads and see what's going on.

For a specific example there - one group never mentioned it as a problem, but I discovered that every single time someone got shot, they spent about 30 seconds looking stuff up in the book.  I took the hint and changed the layout on the pages so that all the damage stuff fits on two facing pages in the book and put other information they needed on the character sheets.  Now they use a bookmark and spend 5 seconds looking it up.  When you consider that this alone saves 15+ minutes during a combat-heavy game session, it's a huge improvement that only came about because of the tapes.

[lev_lafayette]

From a simulationist-gamist POV, I think it's possible to playtest without playing as it were.

In a nutshell, take a game system and (a) reality-check it's simulations and (b) try to break the system.

Of course, you can make this a cooperative venture among several hard-nosed gamers as well. So maybe that's "playing".

[Nathan P.]

Mmm...good stuff on that linked thread, thanks Ron.

Also good stuff about actual playtest techniques - the audio recording is something that I was intending to do at my next session anyway, and its good to hear that it's effective.

I'm gonna go ahead and shift over to Theory to talk about this some more, but if more people have more accounts of actual playtest, please continue posting.