I’m not sure this is the right place for this. Its not about actual play, but it is about playtesting. I want to kick off a thread about the formal evaluation of playtesting. I mean actually using an independent 3rd party, who doesn’t play and doesn’t GM, who observes the sessions and takes notes, collects qualitative data from the players and GM and comes up with some concrete independent feedback.

I’m involved in this kind of thing professionally, mostly for debate based public events, but also for planetarium shows, story telling workshops and buckets of other stuff.

You could use GNS and The Big Model to provide a solid benchmark, which would help formulate an evaluation strategy - particularly if a game is designed with GNS/TBM in mind. What I'm really thinking about here is creating a playtesting quality standard, with free evaluation materials for playtesters, that makes playtesting a more powerful tool.

Any formal evaluation would also benefit from including information about the players. There are free online tests to measure multiple intelligences or the ‘big five’ personality traits that would really help differentiate between player issues and system issues. I suspect you’d find a solid empirical basis for personal creative agendas - maybe even refine the model. I’ve already encountered hints of a non-theoretical basis for GNS using something called the Strength Deployment Inventory. I agree with Ron that debating GNS/TBM endlessly is pointless, but I think hard data is different to endless talk.

If this isn’t the right place tell me to go to hell, but forgive me - I'm a new fish. If it is the right place, encourage me!

Bert

“No good fish goes anywhere without a porpoise”
Lewiss Carroll

I would be interested in the results of something like this as well, and would gladly participate to help gather such data. As it is, I don't have a clue what metrics to use, what observations to note, or how to go about gathering data in other ways.

In other words, I'll be keeping an eye on this thread X-)

Dan