Sending it off to playtesters

[Jeffrey Miller]

I'm curious as to what the general experience of people here has been regarding playtesters?  I've been involved in my fair share, and as a playtester I've had both good and bad experiences - one extreme offered a nicely structure, intelligent questionaire but no response, credit, or thanks, another just wanted free-form "fuzzy" feedback but did a good job responding to and creating dialogue about our criticisms, and a number in between.

Is it a good idea to offer a questionairre to help playtesters organize their thoughts?

Is it good form to engage playtesters in conversation?

How far afield did you go from your normal group?  I personally think this is an important piece, as my concepts and ideas are naturally influenced by the people I converse and game with on a regular basis - I'm already biased towards their likes, dislikes, and sensitivities.

Anything else I missed? ^_^

[Bailey]

I find playtesters at the local gameshop.  I've never gotten any response of value when I could not engage in face to face interaction with the playtesters.

[Mike Holmes]

What Ben said. Talk with them until you think you have gotten some valid feedback.

There are two potential downsides to conversation. The first is that the player giving feedback will often tend to slant their answers to what they think you want to hear. It's hard to tell a person the truth in person if its negative. Second, you have to be very careful to just start out with "How did it go?" or something similarly generic. If you start asking more specific questions right off, you'll tend to subconsciously lead the conversation. Most importantly, if you are going to talk to the player about the game, only do so after they play. Talking to them before hand will almost certainly bias their responses.

The advantages of the questionaire avoid all the above problems. So, perhaps the best solution is to do both. Give them a questionaire first, and then use that as something to spur discussion.

I've found that in some cases, I've had to void the "don't talk before" rule because the player was having problems getting started. But this is a good exeption, because if people are having a hard time getting off the ground, it's good to learn early on just why and make adjustments ASAP. Later you can then hand others a game that can at least get going without your help. And then you can get your unbiased after action review.

I say Unbiased, loosely. It's impossible to remove all bias from even a questionaire (much less a conversation). What you have to go for is Unbiased in the sense that you are actually getting some sort of insight into the process, rather than just confirming your own suspicions. Take it from someone who's made that mistake before.

Oh, and as far as how far afield you should go, the farther the better. The more independent and different the group, the better. The more international, the better. John Wick once told me, "You haven't really playtested the game until you've put it in front of a group of really bad players." Get as much variety as possible. This is the best way to get useful feedback.

Mike

[Jeffrey Miller]

Thanks for the good solid responses - man, I love this place ;)

The advantages of the questionaire avoid all the above problems. So, perhaps the best solution is to do both. Give them a questionaire first, and then use that as something to spur discussion.

That's my current plan.  I'd love to be able to tape a session to see what issue come up.. but that's my experience testing UIs and ad campaigns ;)  

Does anyone have a sample questionaire they'd like to share, or have ideas about questions to ask other than "Name 3 things you liked/didn't like about my game"?  I think this could e a useful resource for folks to get a feel for, as most of us are not going to have the luxury of instant feedback or face-to-face debriefs.

I've found that in some cases, I've had to void the "don't talk before" rule because the player was having problems getting started. But this is a good exeption, because if people are having a hard time getting off the ground, it's good to learn early on just why and make adjustments ASAP.

..one of the reasons the first round of playtesting will be with people we game with regularly or know socially from the local gaming scene, then passing it out willy-nilly to unknown people.  Get the big bugs out of it with friends or people you trust to respond in a positive, constructive manner, then dash your hopes, er, trust in providence with mystery guests.