The Forge Archives

One thing I've wondered about for years is seeing enormous lists of names under the "Playtesters" credit in - bluntly - some really lousy role-playing games.

How can this be? I have some notions, but most importantly, an overall point.

Notion  #1: Many publishers take lists of names from people who participate in con demonstrations. However, actual criticism from these folks is (a) probably not forthcoming, as they just play and go; and (b) probably not relevant to any fundamentals of the game so much as whether "combat sucks" or not.

Notion #2: The really important issues of RPG design are fixed prior to any kind of "playtesting" - especially for games that are built to a set of corporate specifications (huge metaplot, planned supplements, timed releases of accessories, companion card game, etc). In this case, there really IS no playtesting going on, just promotion.

Notion #3: What people comment about may not be what is actually wrong - a stated desire for "more setting" may not actually be satisfied by adding setting (e.g. they may be looking for more Premise without knowing the term/issue).

My overall point is this: in deciding to create, develop, and eventually sell an RPG, I think that the actual designer must have a set of questions to resolve. The big one, of course, is, "Does this damn thing work?" How is that broken down into functional questions? How are those answered, in whole or in part, by playtesting?

I am convinced that playtesting is crucial, but I am also convinced that one must have a philosophy or structure for dealing with it. For instance, in Sorcerer, tons of people told me, "You gotta have a game-world setting!" I decided that all of them were wrong and that little old Me was right. On the other hand, I originally made up a combat system that resolved two opponents' actions at once, and was very proud of it. However, players didn't seem to like it, and it didn't resolve stuff as well as I thought - so I changed it. Interestingly, no player actually said "This doesn't work," and I had to OBSERVE them as we played rather than ASK them.

In other words, I had REASONS for accepting or rejecting recommendations of one sort or another. I had an approach to playtesting in my head, such that it wasn't so simple as saying, "OK, what?" to my players and then transferring their responses directly into design.

I'm interested in other designers' playtesting approaches and experiences. When is it useful? How can our own attitudes get the most out of it? What responses can be identified as clinging to tradition ("No levels??"), without writing off valid points?

Best,
Ron

Hey Ron,

Something I've always assumed about games like Mage and Wraith that are extensions of a game line started with Vampire, is that a portion of the playtesters listed were carried over from the development cycle of the ancestor game. But now you've got me thinking I should compare a few lists of playtesters to see whether that assumption holds up to scrutiny.

Paul  

IMHO, the goal of playtesting is to discover the answer to this one question:  am I meeting my design goals?  Now, that may seem fairly obvious, but I wonder how many designers actually figure out their design goals with enough detail to know if they are reaching their goals.

I would imagine that a designer should have a list of goals that he wants to achieve with his game.  Some are standard to all games (e.g. scan time and handling time are appropriate to the game style), but others will be more specific.  These are the questions that he needs to ask his playtesters.

Blind playtesting, on the other hand, is more for the sake of testing the manuscript than the actual game.

I don't playtest m(any) of my games.  The reason is simple -- I don't care.  They do what they're supposed to do.  If they don't, I'm pretty much aware of it.

However, that doesn't mean it makes sense to someone Not Me.  That's when I do playtest stuff (like octaNe).  Just to make sure the rules are clear and to make sure it flows during play.

But really, I don't care much for feedback re: shoulda done thing/shouldn't of done this.  Just tell me -- did the stuff I give you work or not?

I'm first and foremost concerned that the system functions.  'How well' is another matter and is usually revealed through asking people to play-test it with no real set-up or explanation.

I think that works best because it (hopefully) will lead to situations and so-on that the designer didn't foresee.

I also encourage people to try to break the system (e.g., be a munchkin to your heart's content and find loop-holes that allow hacking & blasting followed by the always popular looting of corpses or generate the equivallent of Darth Vader meets Thor).

I'm also very interested in a player's perception of what the game's setting is supposed to be like; how it feels to them (e.g., funny, serious, dark, heroic, etc.)

To me, play testing is important for one very good reason (the only one for me, really): I didn't like getting ripped off by game companies, so I don't want to rip off other gamers.  You know, a buck's a buck. You worked for it and I'll do the same.
 
I'm not so interested in the proper terminology because, as Ron mentioned, the average gamer isn't going to bone up on the latest game design theory.  But if someone says 'Orbit combat sucks' I'll be feeling lucky to have the opportunity to ask 'why?'

God bless play testers and in-store demo teams!

Jeff Diamond
http://www.geocities.com/allianceprime">



[ This Message was edited by: JSDiamond on 2001-04-23 19:34 ]

I've got an interesting twist on this issue:

A game I did a lot of work on through the early to mid nineties had a combat system that, upon playtesting, was obviously broken.

However, the players all loved it, because they enjoyed thinking up new ways to break it ....

I suspect that the gamist-oriented player LIKES rules problems, because it gives him something to exploit.

Personally, I find that there is an absolute need for play testers.

If a game's unbalanced, people will stop playing it and generally NOT tell you.   As per a previous post mentioned, "They don't know how to say why 'it sucked' in a useful way".

A game needs to create an element of immersion and testers really help with that.

The "immersion" point aside (way too problematic for this discussion), let's focus on any of the following:

1) What should one inform playtesters of before play?

2) What should one ask of them afterwards?

3) In what context should the most important playtesting occur? With friends? At a con? What?

4) How does one establish playtesting using other GMs besides oneself? At what stage of the game's development is this helpful?

5) And finally, how does one separate signal from noise?

Best,
Ron

Quote
1) What should one inform playtesters of before play?

I think that's it's a very good idea to let playtesters know what information you're looking for.  For example, if you're looking for playability of the rules, playability of specific rules (e.g. combat, magic), manuscript quality, or presentation of key concepts in the manuscript.

Quote
2) What should one ask of them afterwards?

I would follow up with a few specifics, especially if you mentioned specific rules that you were interested in. I also think that interviewing them, and asking a lot of clear but open-ended questions is a good idea.  Specifically soliciting them for additional problems or suggestions is also important.

I tend to do this follow-up interview even when I'm running a game that I purchased. I use it to (hopefully) improve my GM style.

Quote
3) In what context should the most important playtesting occur? With friends? At a con? What?

I think the context depends on where you're at in the playtesting phase. If you're early in the game and testing core rules concepts, I'd start with friends. I wouldn't take it to a con environment until I had the system pretty well fleshed out, and here I would be testing general playability of the game, not just the game concepts. Finally, I'd have a sort of blind playtesting once I've moved from having a solid system to working on the actual product for release. At this stage, I would hand out my manuscript to people who had never played my game before, to see what they could make of it.

Quote
4) How does one establish playtesting using other GMs besides oneself? At what stage of the game's development is this helpful?

I haven't reached this point with any of my games yet, but the solution that I'm leaning towards is to first give the manuscript to people I know who have played the game, have them run it, and see how it goes. Then I'd try to find somebody I didn't know, sell them on trying out my game, and follow up to see what they thought about it.

My gut feeling is that this stage of playtesting isn't useful until the game passes from a collection of rough notes to a complete or nearly complete manuscript.  I'll probably be better able to make this judgement once I actually have a game to this point.

Quote
5) And finally, how does one separate signal from noise?

First, make sure you get a decent sized sample, and the noise will identify itself, because it won't fit the norm. I also find that it helps to know something about the gaming style of the playtesters. For instance, my game is very narrativist, inspired by The Window and Sorcerer. When I was testing with a couple of guys who really liked the MERP and Warhammer RPG systems, I knew that I could really brush off their complaint about a lack of hit location and weapons tables. I could take it as a good sign that even without these intricate combat mechanics, they had a blast and we had the single most intense role-playing experience it has ever been my pleasure to participate in.

I believe that Ron and Clay have just created what would make a very good standardized form for handing out to playtesters.

Would either of you care to compose such a form and please post it for all of us to use?

Jeff Diamond
http://www.geocities.com/allianceprime">6-0 Games

Jeff,

I'm not sure that what has been described here is most appropriate to a fill-in form to give to the playtesters. I'm of the opinion that one on one interviews will work the best. At minimum I would want to discuss it over the phone. Ron has emphasised that personal contact for purchasers of the game (which I agree with), and I think that you woul want an even closer bond with the playtesters.

What might be useful is a general format for a playtest report, to be completed by the game designer, and based on interview questions.

This thread is too good to let it languish.

Developing Children of the Sun has been interesting because system preceeded setting.  Actually, that's not an entire truth, but close enough.  The choice of setting was somewhat arbitrary, but I had a couple of very basic principles in mind for the system.  I worked very hard, especially early on, to not think too hard and let those principles guide me where they would go.  I hope that the end result is a system that "hangs together" reasonably well, and has a certain consistency to it.

Ron, it's interesting that you mention your original combat system dealt with resolving more than one action at the same time.  CotS does that, so it'll be interesting to see what you think of it.

All in all, I think the system is a little clunkier than I might like, but at the same time, I think it is relatively efficient, in the sense that there is a great deal of flexibility in what you can do in combat.  The game handles a lot of factors without beating you upside the head with hundreds of little rules.  I like to think of it like Diplomacy in a way.  The basic premises of combat aren't that complex, but when you give a great deal of freedom to the player and they start mixing relatively simple things around, the end result can get pretty complicated.

Another question related to this topic that I think is important: When playtesting, do you prefer taking the driver's seat as GM, or giving one of your playtesters the honors?

I can see valid points in GM-ing a session of your own game (the players get to play it as it was meant to be run) but that gives you a controlled environment and a somewhat one-sided approach to playtesting.

If you let one of the playtesters run the game, you can then see what works/doesn't work for the GM, but how the playtesters react to your game might be tied a little too closely to how good the GM is.

Thoughts? Comments?

Regards,
Corporate Dog

Hi there,

Necessarily, initial playtesting of any of my games means I'm the GM. Half the material at that point is still conceptual or in development, so writing it up to the point that someone else could use it has a "castles in the sand" quality, to me.

As a one-man-army, that also means that con demos and so on are usually just me as GM too. However, to my joy, I found that more and more people would show up at a demo having already played the game somewhere! Cool!

Still, getting the game from "my thing that I run for people" into "a game that lots of people play" is very, very difficult. Lots of people tend to want to play a game AFTER it has that well-playtested gloss. However, to get there, you need lots of people to play it, off your own leash, but able to give feedback. It's kind of a bitch, actually.

Best,
Ron

Alyria is currently in the very early stages of playtesting and I'm finding that Scarlet Jester and I have different styles of playtesting.  In my tests so far, I've been the GM working with a subset of my normal gaming group.  Soon I hope to expand to my entire group, which has a broader range of tastes and interests than the current subset.

Jester, on the other hand, sits down with his group, explains the concepts to them, and then lets them run an entire session without his participation.  He remains available for clarification but does not participate as either GM or player.  I guess that you could say that he participates as designer.

I can see advantages to either method.  Hopefully this is helpful.