One thing I've wondered about for years is seeing enormous lists of names under the "Playtesters" credit in - bluntly - some really lousy role-playing games.

How can this be? I have some notions, but most importantly, an overall point.

Notion #1: Many publishers take lists of names from people who participate in con demonstrations. However, actual criticism from these folks is (a) probably not forthcoming, as they just play and go; and (b) probably not relevant to any fundamentals of the game so much as whether "combat sucks" or not.

Notion #2: The really important issues of RPG design are fixed prior to any kind of "playtesting" - especially for games that are built to a set of corporate specifications (huge metaplot, planned supplements, timed releases of accessories, companion card game, etc). In this case, there really IS no playtesting going on, just promotion.

Notion #3: What people comment about may not be what is actually wrong - a stated desire for "more setting" may not actually be satisfied by adding setting (e.g. they may be looking for more Premise without knowing the term/issue).

My overall point is this: in deciding to create, develop, and eventually sell an RPG, I think that the actual designer must have a set of questions to resolve. The big one, of course, is, "Does this damn thing work?" How is that broken down into functional questions? How are those answered, in whole or in part, by playtesting?

I am convinced that playtesting is crucial, but I am also convinced that one must have a philosophy or structure for dealing with it. For instance, in Sorcerer, tons of people told me, "You gotta have a game-world setting!" I decided that all of them were wrong and that little old Me was right. On the other hand, I originally made up a combat system that resolved two opponents' actions at once, and was very proud of it. However, players didn't seem to like it, and it didn't resolve stuff as well as I thought - so I changed it. Interestingly, no player actually said "This doesn't work," and I had to OBSERVE them as we played rather than ASK them.

In other words, I had REASONS for accepting or rejecting recommendations of one sort or another. I had an approach to playtesting in my head, such that it wasn't so simple as saying, "OK, what?" to my players and then transferring their responses directly into design.

I'm interested in other designers' playtesting approaches and experiences. When is it useful? How can our own attitudes get the most out of it? What responses can be identified as clinging to tradition ("No levels??"), without writing off valid points?

Best,
Ron

Hey Ron,

Something I've always assumed about games like Mage and Wraith that are extensions of a game line started with Vampire, is that a portion of the playtesters listed were carried over from the development cycle of the ancestor game. But now you've got me thinking I should compare a few lists of playtesters to see whether that assumption holds up to scrutiny.

Paul

IMHO, the goal of playtesting is to discover the answer to this one question: am I meeting my design goals? Now, that may seem fairly obvious, but I wonder how many designers actually figure out their design goals with enough detail to know if they are reaching their goals.

I would imagine that a designer should have a list of goals that he wants to achieve with his game. Some are standard to all games (e.g. scan time and handling time are appropriate to the game style), but others will be more specific. These are the questions that he needs to ask his playtesters.

Blind playtesting, on the other hand, is more for the sake of testing the manuscript than the actual game.

I don't playtest m(any) of my games. The reason is simple -- I don't care. They do what they're supposed to do. If they don't, I'm pretty much aware of it.

However, that doesn't mean it makes sense to someone Not Me. That's when I do playtest stuff (like octaNe). Just to make sure the rules are clear and to make sure it flows during play.

But really, I don't care much for feedback re: shoulda done thing/shouldn't of done this. Just tell me -- did the stuff I give you work or not?

I'm first and foremost concerned that the system functions. 'How well' is another matter and is usually revealed through asking people to play-test it with no real set-up or explanation.

I think that works best because it (hopefully) will lead to situations and so-on that the designer didn't foresee.

I also encourage people to try to break the system (e.g., be a munchkin to your heart's content and find loop-holes that allow hacking & blasting followed by the always popular looting of corpses or generate the equivallent of Darth Vader meets Thor).

I'm also very interested in a player's perception of what the game's setting is supposed to be like; how it feels to them (e.g., funny, serious, dark, heroic, etc.)

To me, play testing is important for one very good reason (the only one for me, really): I didn't like getting ripped off by game companies, so I don't want to rip off other gamers. You know, a buck's a buck. You worked for it and I'll do the same.

I'm not so interested in the proper terminology because, as Ron mentioned, the average gamer isn't going to bone up on the latest game design theory. But if someone says 'Orbit combat sucks' I'll be feeling lucky to have the opportunity to ask 'why?'

God bless play testers and in-store demo teams!

Jeff Diamond




[ This Message was edited by: JSDiamond on 2001-04-23 19:34 ]

I've got an interesting twist on this issue:

A game I did a lot of work on through the early to mid nineties had a combat system that, upon playtesting, was obviously broken.

However, the players all loved it, because they enjoyed thinking up new ways to break it ....

I suspect that the gamist-oriented player LIKES rules problems, because it gives him something to exploit.

Personally, I find that there is an absolute need for play testers.

If a game's unbalanced, people will stop playing it and generally NOT tell you. As per a previous post mentioned, "They don't know how to say why 'it sucked' in a useful way".

A game needs to create an element of immersion and testers really help with that.

The "immersion" point aside (way too problematic for this discussion), let's focus on any of the following:

1) What should one inform playtesters of before play?

2) What should one ask of them afterwards?

3) In what context should the most important playtesting occur? With friends? At a con? What?

4) How does one establish playtesting using other GMs besides oneself? At what stage of the game's development is this helpful?

5) And finally, how does one separate signal from noise?

Best,
Ron

1) What should one inform playtesters of before play?

2) What should one ask of them afterwards?

3) In what context should the most important playtesting occur? With friends? At a con? What?

4) How does one establish playtesting using other GMs besides oneself? At what stage of the game's development is this helpful?

5) And finally, how does one separate signal from noise?

I believe that Ron and Clay have just created what would make a very good standardized form for handing out to playtesters.

Would either of you care to compose such a form and please post it for all of us to use?

Jeff Diamond
6-0 Games

Jeff,

I'm not sure that what has been described here is most appropriate to a fill-in form to give to the playtesters. I'm of the opinion that one on one interviews will work the best. At minimum I would want to discuss it over the phone. Ron has emphasised that personal contact for purchasers of the game (which I agree with), and I think that you woul want an even closer bond with the playtesters.

What might be useful is a general format for a playtest report, to be completed by the game designer, and based on interview questions.

This thread is too good to let it languish.

Developing Children of the Sun has been interesting because system preceeded setting. Actually, that's not an entire truth, but close enough. The choice of setting was somewhat arbitrary, but I had a couple of very basic principles in mind for the system. I worked very hard, especially early on, to not think too hard and let those principles guide me where they would go. I hope that the end result is a system that "hangs together" reasonably well, and has a certain consistency to it.

Ron, it's interesting that you mention your original combat system dealt with resolving more than one action at the same time. CotS does that, so it'll be interesting to see what you think of it.

All in all, I think the system is a little clunkier than I might like, but at the same time, I think it is relatively efficient, in the sense that there is a great deal of flexibility in what you can do in combat. The game handles a lot of factors without beating you upside the head with hundreds of little rules. I like to think of it like Diplomacy in a way. The basic premises of combat aren't that complex, but when you give a great deal of freedom to the player and they start mixing relatively simple things around, the end result can get pretty complicated.

Another question related to this topic that I think is important: When playtesting, do you prefer taking the driver's seat as GM, or giving one of your playtesters the honors?

I can see valid points in GM-ing a session of your own game (the players get to play it as it was meant to be run) but that gives you a controlled environment and a somewhat one-sided approach to playtesting.

If you let one of the playtesters run the game, you can then see what works/doesn't work for the GM, but how the playtesters react to your game might be tied a little too closely to how good the GM is.

Thoughts? Comments?

Regards,
Corporate Dog

Hi there,

Necessarily, initial playtesting of any of my games means I'm the GM. Half the material at that point is still conceptual or in development, so writing it up to the point that someone else could use it has a "castles in the sand" quality, to me.

As a one-man-army, that also means that con demos and so on are usually just me as GM too. However, to my joy, I found that more and more people would show up at a demo having already played the game somewhere! Cool!

Still, getting the game from "my thing that I run for people" into "a game that lots of people play" is very, very difficult. Lots of people tend to want to play a game AFTER it has that well-playtested gloss. However, to get there, you need lots of people to play it, off your own leash, but able to give feedback. It's kind of a bitch, actually.

Best,
Ron

Alyria is currently in the very early stages of playtesting and I'm finding that Scarlet Jester and I have different styles of playtesting. In my tests so far, I've been the GM working with a subset of my normal gaming group. Soon I hope to expand to my entire group, which has a broader range of tastes and interests than the current subset.

Jester, on the other hand, sits down with his group, explains the concepts to them, and then lets them run an entire session without his participation. He remains available for clarification but does not participate as either GM or player. I guess that you could say that he participates as designer.

I can see advantages to either method. Hopefully this is helpful.

Well, I know this thread is old, but I've recently had my first delving into playtesting (i.e.: sending my game out to be tested) and I can definately say it's invaluable.
My suggestions would be as follows:

First, get a working set of mechanics and a little bit of "setting" then send it out to as many other designers as you can get to look at it. For the most part, they don't need to run a full game.. they can tell you what they see as problem areas. This will save you the problem of sending it to the average gamer and them not realizing something is a problem. For example, I sent a version of Any Town, U.S.A. out (very recently) to my little brother's group. They live far away so they were on thier own to test it. They reported it was workable and that they'd like to see a full version. The only problems they saw were the obvious lack of content. I was happy. It wasn't untill a fellow designer informed of some major flaws that I became aware.

second: once you've gotten obvious flaws fixed, add a little meat (setting, theme, writing) to it and send it to some average gamers. The most you should get from them is "is it fun or not?" "were you able to immerse?" "Or were you just thinking 'it's not as good as *** game'" Which, at this point, is what you need to know.

finally: Once you've revamped it, make a copy of the game with no artwork and less filler then send it out to anyone you can. Check for final glitches, get some real reviews, and hunker down for the official release.

Hope this helps, it seems to be working for me.

T