Playtesting strategy

[Matt Wilson]

I'm currently playtesting a game, and I wondered if anyone has any thoughts on strategy based on experience or otherwise.

I've been taking the laboratory approach of focusing on one or two details per game session, and once those seem to be working fine, shifting focus a bit to study other details.

Does it make sense to doit that way? Are there other strategies? An alpha test approach and then beta, and things like that?

And one last question: is there any benefit to me describing the playtest sessions in Actual Play?

-Matt

[Mike Holmes]

The laboratory approach is only suitable if you have particular mechanics that are really contentious. If the whole game simply is in need of tweaking, you have to take a holistic approach to playtesting, and just canvas for opinions afterward. Otherwise, you risk missing something that's problematic on a systemwide level.

For example, you could look closely at your reward system, playing it different twice to see what effect occurs. But that might then miss the effects that the mechanic is having outside of the area of advancement.

Worse, to make this at all scientific, you have to do a number of tests at each level. This seems fairly prohibitive, especially in terms of independent playtests (which are indespensibly valueable in completely different ways than self tests are).

Playtesting for the most part has to remain an art, unfortunately. Obly the analysis can remain even remotely scientific.

Mike

[Marco]

I have only one concrete thought on this matter:

If your game is of the simulationist or gamist variety (and this goes 10x if your game is of the super hero genre) playtest the "mirror match"--a character against himself.

There's no right or wrong--but if it comes down to who fires first and the game isn't supposed to resolve combat that way ... you'll know.

-Marco (that woulda saved GURPS Supers a *world* of grief)

PS: I have another--it's Steve Jackson's thought on war game playtest but it applies to *some* RPG's: Playtest the Dumb Strategies. Try building a character in a way that seems at odds with common sense and determine if the effects are what you had thought (this would also have saved GURPS Supers a lotta grief ... man, what were they THINKING!?)

[Bruce Baugh]

The mirrored-character tactic is a really good one. (In Vampire, Potence and Fortitude are supposed to be symmetrical. They aren't. Pfeah.)

"What happens if I try to build myself, as I am or as I'd wish to be in this setting?" is another useful question.

It also helps to have at least one friend who is very good at mathematical analysis and sympathetic to your goals. Or at least it's good for those of us who can use some help with the math. Someone who can say "this is broken...and this would bring it into the line" is worth their weight in satisfactory compensation.

Incremental changes can, though they don't have to, set up a trap. After a bunch of them, take some time to make fresh characters and check the thing as a whole again. Cascading implications are terribly easy to lose track of.

[Bruce Baugh]

Oh, one more good sort of quick match-up:

A character with the maximum allowed value of Trait X against a character with the minimum allowed value of Responding Trait Y. If the extreme cases seem bad, many players will assume that problems lurk closer to the center even if they don't.

[Walt Freitag]

Going back to the opposite (holistic) extreme, I strongly recommend "blind" play-testing. That is, play-testing sessions that the game author observes but does not participate in. Not participating also means not helping with GM or player preparation and not making comments ("helpful" or otherwise) during the game sessions. Not a word, not a gesture, not even a revealing facial expression. The one exception being to intervene if the game play goes so wrong (e.g. due to a gross misinterpretation of a central rule) that further observation of play would be of no value.

This is torture for game designers. But the rewards are commensurate with the agony. After all, publishing your game means you won't be there to make helpful or corrective comments every time someone plays it, let alone be the GM. It's the only way to learn how (not whether) the game in the text differs from the game in your head.
 
- Walt

[Bruce Baugh]

Excellent advice. For that matter, it's good to have some playtesters you never deal with in person at all.

[Matt Wilson]

Good stuff. Thanks for the input.

[Sylus Thane]

Another thing to go along with the blind testing is have a single side printed copy of your book, this way a player or gm can write in notes about something exactly on the other side of the page the problem area is on. I have done this and it has helped alot. Also if you put your book in a binder they can easily add extra pages of notes to certain topics. You can also do this yourself as you observe and put them in with your playtesters.

Hope this extra bit helps a little,

Sylus

[Othyem]

Good point, Matt. I print my game one sided exactly for that reason, to have room for my chicken scratch. Our game is still in the first trimester of life, where we are looking for typeos, and still inventing the smaller print. After I feel I have used up enough blank space with notes, or have made enough changes, I'll reprint and replace in the binder.

By the way, I'm the new guy. This is my first post. Hi.

Othyem
http://www.geocities.com/othyem

[Bruce Baugh]

Another practicality of playtesting:

See how folks play with a manuscript presented in different ways. The effects of page breaks and the like can be significant in people's perceptions of the flow of text. Likewise, as soon as you have a prospective layout, see how folks respond to the actual text presented that way.

[Mike Holmes]

See how folks play with a manuscript presented in different ways. The effects of page breaks and the like can be significant in people's perceptions of the flow of text. Likewise, as soon as you have a prospective layout, see how folks respond to the actual text presented that way.

Hero Wars would have benefitted from doing this. Makes me wonder if they did, and there are just a small group of people who like that sort of presentation.

Mike

[Matt Wilson]

Another practicality of playtesting:

See how folks play with a manuscript presented in different ways. The effects of page breaks and the like can be significant in people's perceptions of the flow of text. Likewise, as soon as you have a prospective layout, see how folks respond to the actual text presented that way.

I'm pretty good at figuring out the text flow and layout, but what's driven me mad is the larger picture. In what order do I present information. That's been a toughie to figure out.

Note to Ralph and Mike: Universalis must have been tough to arrange. It's different enough that you probably couldn't reference existing games as easily.

[Mike Holmes]

Note to Ralph and Mike: Universalis must have been tough to arrange. It's different enough that you probably couldn't reference existing games as easily.

You have no idea. Ralph rewrote the game seven times, and each time we agonized over what should go where in relation to the other parts. There were times when the text seemed to need to refer to itself circularly. So what do you put first?

What was worse, nobody else seemed to have any idea either, even after playtesting the game. We got some comments that it was disorganized in it's ordering, but rarely any feedback on what the correct order should be.

Sometimes playtesting just doesn't work. Sometimes you have to wing it based on your own perceptions.

Mike

[Valamir]

heh, yeah.  The circular text was killing me.  I wound up having duplicated information in different chapters stuff all over the place...brr.

Alot of this was because of trying to fit the game into standard RPG layout.  This is the section on building a character, then this is the section on basic game mechanics, then this is the section on advanced game mechanics...etc.  Didn't work for Uni.

What I finally did, is that after having demoed the game enough times I realized that there was a distinct difference between how I presented the game when teaching it (order and priorities) and how it was written.

So the final rewrite I started from scratch and wrote the sections up in the order that I taught them.  The initial chapter on setting up tenets in the pre-game for example.  You don't really need to understand how Complications work, or the nature of Importance, when you're setting up the tenets of the game.  The next step was turn order, bidding for turn, interruptions and challenges.  You're halfway through the rules before you get to the section on how to build a character.

I'm not 100% satisfied with it, but it works ALOT better than the older versions.