WotC Market Research & GNS

[greyorm]

Over at ENWorld, there was an announcement about the following survey data complied by WotC in 1999:  Breakdown of RPG Players.

Here's my questions: how accurate do you believe the categories are? That is, are they functionally dissimilar enough? Or are their distinctions far too fuzzy to reasonably be utilized.

That is, do you believe that an individual has enough coherent understanding of the differences between styles to actual detail their own play styles or choices without bias (especially given the observed phenomenon that gamers tend to proclaim they want or desire certain things during games, but upon examination by outside observers, it is discovered their desires do not match with their claims).

The above is highlighted in particular for me given the following:

We also found no additional segmentation based on what games people identified as their "favorite"; in other words, there are just as many Power Gamers as there are Storytellers who like Vampire, and just as many Thinkers as Character Acters who like D&D.

But of course, the data from WotC does not account for Drift in any of those games (and I dare wonder how similar a "Storyteller" D&Der game would be to a "Character Actor" D&Der game).

Which immediately calls into question whether or not the ending statement is even accurate, and thus whether the model as presented is even a useful tool

It reflects deep seated psychological aspects of the gamer mind

Does it? I can't see that it does, or at least not that it provably represents the deep-seated psychology of gamers -- except as what they tend to believe, rather than what actually is.

And finally, given the false dichotomy set-up "story" and "combat," and the particularly unclear difference between the the definition of each (ie: are not interactions between world/characters also conflicts that must be resolved?), how useful can the model presented actually be?

Ultimately, I question the rigor of the presented model as hinging on some particularly weak premises and definitions.

I also note some of the observations or hypotheses put forth directly conflict with our own observations. Frex

in fact, having different kinds of players tends to make the RPG experience work better over the long haul.

Whereas we have found precisely the opposite: that functional groups are more centered in play-goals, not diverse.

And then at the end, there seems to be a statement which is nothing more than a swipe at a number of existing efforts to define game design and gamer psychology, which in its context is completely out of place with this article, and makes me wonder about the actual intention of this model's presentation, or at least the writer's mindset:

Unlike some of the discussions which rage from time to time about the nature of game design paradigms, the above information was extracted from general market research data that had as much bias as possible removed.

Anyone else have any commentary?

[greyorm]

As an addition to my post above: does anyone see the categories created by WotC fitting into our model -- where and how?
What sort of data might the survey's results provide for GNS?

[xiombarg]

The irony here is the idea that marketing data is objective, when the structure of the survey itself could have biased results, and probably did. Admittedly, that survey is the closest thing we have to objective data about RPGs, but because of that it's over-analyzed, in many cases in an attempt to make some sort of "point" -- not unlike the quiet agenda you point out.

[Walt Freitag]

Despite the claims of these ideas coming out of the pure survey data, is the existence of these particular axes and the fact that players self-identify strongly along them news to anyone? I think it's likely that the two-axis model reveals gamer culture rather than meaningful differences in player preferences or behavior.

To be blunt, if the strategic-tactical and combat-story classifications do not correlate with game system preferences, play experience, or ability to coexist in groups, then what use are they? (Well, OK, identifying differences in GMs vs. non-GMs and designers vs. players might have some value.) And do those eight core values illuminate anything at all? (Gee, people with strong interest in role playing games want their role playing games to have "role playing" -- who'd'a thought?)

- Walt

[jdagna]

My first reaction to WotC's breakdown of players is that the results seem awfully neat for real data.  If this had been turned in to one of my Biochem professors, she'd give it a 0 and hand it back on the assumption that they hadn't really done the experiment and had simply faked data based on equations in the book.  I would really want to see the data.

As far as their basic questions and categories, I think the story/combat axes are reasonable enough for valid results.  Most gamers understand the difference between the two even if they might debate the precise meanings.

The analysis I can mostly agree with (provided the data actually supports it).  I think it proves what we've all known about Drift and System all along - the rule book is only a portion of System, and groups will selectively ignore or modify it until it fits what they want.  This is the only explanation for why D&D's player distribution looks the same as Vampire's.  Or, put another way, system does matter, but the rule book is a relatively small portion of system when it actually hits play.

What about their decision that mixed groups do best?  I actually agree with this.  For one thing, they specify that it's good over the long term.  To me, this means that people have had a chance to reconcile their differences and accomodate each other's styles.  Additionally, since most people enjoy a little of everything (even if their preferences are different), a diverse group encourages a little of everything to happen.  

Also, keep in mind that WotC's definitions are vague enough in GNS terms that you could have all five of WotC's categories in a group that all preferred Gamist (or whatever) styles.  We've met quite a few Gamists who would describe their competition in terms of story (be it the literary type or a social conflict).  Likewise, there are plenty of Sim and Nar players who might have ranked combat high (exploration of system or just combat-related themes).  Still, I think the best groups have people who like all of the GNS modes as well - such groups simply have to communicate well enough to cooperate when priorities might collide.

[Emily Care]

The questions were not designed to find these four quadrents; they correleate to all kinds of player interest and behavior. The original survey had several hundred questions, but only about two dozen have a bearing on the segmentation results. Once the study was complete, the data was plotted in several dimensions to look for clusters of responses; those clusters became the five player types. Once we know the segmentation was there, we reverse-engineered the axes, by comparing the responses of the people in each segment to find similarities.

The reverse engineering accounts for the neatness.  They should have published the questions that made up the clusters.  How things are asked is as important as what is asked.  And as Walt pointed out, it's hard to see how useful this analysis will be if they types do not align with rule set preference.  Anyone see how it may have affected WoTC's product marketing or design?

As far as gns, the obvious (and superficial) grouping would be to class the Thinker and Power Gamer  as gamist, Character Actor as sim-character, and Storyteller as nar.  The descriptions touch on the stances prefered by that segment (Power Gamer "this kind of person plays a character that is simply an extension of the player"-pawn, Character Actor 'This player will have a character that makes sub-optimal choices (from an external perspective) to ensure that the character's actions are "correct"'-actor, Storyteller "This kind of person finds enjoyment from the logical progression of the narrative of the scenario." - author, Thinker "This kind of person is likely to enjoy min-maxing a character, spending hours out of game to find every conceivable advantage available in the system to deliver maximum damage from behind maximum protection, even if the min-maxing produces results that are seemingly illogical/impossible." - metagame, avatar.)  

The story/combat axis doesn't seem quite right, though it's an intuitive division: what unites the "thinkers" and "power gamers" is a desire for challenge, and what divides them from the other two groups is a lack of care for in-game continuity.  

What do you think about the 8 core values?

Strong Characters and Exciting Story
Role Playing
Complexity Increases over Time
Requires Strategic Thinking
Competitive
Add on sets/New versions available
Uses imagination
Mentally challenging

Would anyone say they don't want these things in a game?

Thanks for posting it, Rev.

Regards,
Em

[jdagna]

Thanks for pointing out how they got such nice results, Emily.  I think I'll do the same as my Biochem profs - mark a 0 on it and hand it back.  If you massage data enough, you can make it say almost anything.

[contracycle]

1]Strong Characters and Exciting Story
2]Role Playing
3]Complexity Increases over Time
4]Requires Strategic Thinking
5]Competitive
6]Add on sets/New versions available
7]Uses imagination
8]Mentally challenging

All except 1, 2 and 6 (an odd category in itself) apply to chess.
Probably all but 1 applly to Magic the Gathering
Arguably, the L5R CCG incorporates all 8

Number 6 looks like backwards gamerthink to me.  Do I like a product BECAUSe it has extensions, or do I like the extension products BECAUSE I like the initial product?  Was the Matrix better BECAUSE there were going to be sequels?

Also, the use of "story" has so many meanings it really needs to be specifically qualified every single time its used, IMO, especially in this sort of exercise.  Edit: and looking at it again, I'm reminded that I find it inexplicbale that story and combat should be seen as opposed on a single axis.  This seems like a huge, 800lb gorilla style assumption that needs to be explained explicitly.

[beingfrank]

Thanks for pointing out how they got such nice results, Emily.  I think I'll do the same as my Biochem profs - mark a 0 on it and hand it back.  If you massage data enough, you can make it say almost anything.

Um.  Sounds like they used a factor analysis.  This is not 'massaging data' but a valid statistical analysis technique.

I'd also love to see the data myself and run my own analyses of it.  But I'd be wanting to run a factor analysis on it too.  Seems to me that if you're trying to uncover the underlying dimensions that influence how people answer these questions, then that's probably the best way of going about it.

You can aruge that it's not the appropriate technique to use, but I can't immediately think of a better one to get at the sort of information they wanted.

Does it? I can't see that it does, or at least not that it provably represents the deep-seated psychology of gamers -- except as what they tend to believe, rather than what actually is.

This is the problem with all reseach into people's attitudes, beliefs and even behaviours.  It's easy to directly measure how big a rock is, it's hard to directly measure someone's preference for a particular style of roleplay, or how racist they are.  Indirect measures are never perfect, but dismissing them all for not being perfect means that you can't say anything about the area because there's no way of knowing anything.

I'd love to run a study where I observed thousands of roleplayers in situations carefully controlled for different aspects of roleplaying (by GNS theory or whatever), surveyed them at regular points in the experience about their enjoyment, what they were feeling, and so on, and run a huge analysis of it all.  But it would cost stupendous amounts of time and money.  Until I become incredibly rich, or take up bank robbery, this is not going to happen.

The fact that this study exists is a huge leap forward, in my view.  It might not be perfect, and certainly has some flaws, but it puts us in the situation of knowing something (however imperfectly) which is always better than knowing nothing, in my view.