The Forge's Anti-Polling position

[Domhnall]

In the "Conceptualizing[Cian]" thread, Ron wrote:

...
My point is that, if either were the case, it would serve you absolutely no purpose whatsoever to tell you. None. Even if you polled 9000 people for such responses, the information would be worthless. And as you expended all that energy to run the poll, and subsequently infinitely more energy attempting to apply its results, ... well, that's a whole bucketload of energy you're not spending on game development.
...

I'm curious about this Forge anti-polling position.  Would a Forge vet (or Ron) elaborate on this?  
First, let's say someone did do a poll on a very simple concept (like if a system's title turns one off or on) and got even 300 answers.  How can you say that this information is useless?  If the overwhelming majority of respondents agreed that a title (or any other single facet) truly repulsed them, isn't that useful?  I agree that a full discussion of these facets is more thorough, but that takes much more time than a poll.  

Second, what effort?  If I could post a poll somewhere (like the Forge) and have faith that the respondents were giving honest answers, my effort consists of the 3 minutes to carefully word my poll, and then check on it when I'm curious.  While the overall value of polls is lower than full discourses, the energy to information ratio makes polling seem worthwhile.

[Jasper]

Hi,

I think the short answer is that, first, the Forge is about discussion and polls are not discussion -- they're just statements of opinion, and thus not very deep or constructive. (And, being that, would distract from what else goes on here.) Second, the sample of respondants you'd get here is going to be small and non-representative of the gaming community overall; certainly not even the online gaming community, much less all those people who have no interest in the net at all, yet game happily wherever they are.

Plus, overall, polling is a lame way to make a decision.  The philosophy here is that you should make a game that you want to play, above all else.  If you're not totally excited by something, don't do it.  Thus, decision by committee isn't a valid/productive approach.

That's my understanding of the justification (and one I agree with). Hope that helps.

[Eero Tuovinen]

I've always understood the "poll-ban" to be about quality of discourse. Opinion-fishing is a typical part of the social dance on many internet forums, and may even take the lead role. For ex. the currently primary Finnish rpg forum at majatalo.org is a constant orgasm of questionnaire threads, where a participant throws out a question, tells his own opinion, and there we go. Properly these threads are meant to open a dialogue about the matter at hand (however banal the topics might be), but the form of the opening prescripts the course of the thread: ask for "opinions", and you get just that, with no analysis or thought. Most threads in such environment seem to be about "The best game you played", "Your favourite die size", "Opinions about Harn" and so on. It's rule by the commons, where the dialogue is constantly pushed to a level where everybody can have a voice.

Really, if you're Finnish, go to majatalo and see. It's a sight to behold. Not unlike the Forge birthday forum, but going on all year long.

The interesting thing is that I'm not seeing a blanket ban of questionnaires at the Forge. I imagine that if I wanted to have some empirical data, put up a simple questionnaire form in the web and asked people here to fill it, I wouldn't be going against the rules. The rule is against opinion threads, not polling in general. It's more about the Forge being the wrong place to do your questioning, not about the questioning per se.

I myself tend to hold the opinion that by far the most common reason for people to do polls is either
- social affirmation, like at majatalo, above
- misguided decisionmaking process
, which neither is a very good reason to start polling. So perhaps part of why we don't see polls here is that for a great majority of cases they're unnecessary. Protecting you from your own folly, perhaps?

[Ron Edwards]

Hiya,

Several things.

1. Disciplinary bias on my part: I consider that people trying to please the pollster or outright lying contribute far more to the spread of data in a poll than any "response" the poll is intended for.

In other words, you say, "How can 300 responses be useless?" and I say, "Because such responses are useless in any quantity." The large number of jobs and business decisions made on the basis of such "data" only means that a whole lotta jobs and decisions out there are a big scam.

And even if all those attempts to please and so on were eliminated ... so what? You are dealing with reactions, not about actually being interested, but about some vague-ass hypothetical "would be interested" reaction.

Case in point: you can ask a ton of people whether they would do X or Y when dealing with a person of A or B ethnicity. You can get tons of responses. And then if you carry out observational studies, you will find that the actual behaviors will differ radically, in both directions. (Yes, the verbally-racist person is not necessarily the behaviorally-racist one.)

2. Lack of discourse, which in and of itself means "waste of space" for this site's purposes. One might try to correct for this by saying, "Hey, give your reasons." But if a given respondent is going to explain his or her response, you might as well start with that as the discourse.

Furthermore, there is no explaining a reaction. For real discourse about the utility or point of a title (for instance), you'd need some kind of basis for what makes it interesting or not interesting, and some agreed-upon basis for how we tell it's interesting or not interesting.

Satisfying as it may be for people to react-and-type, and satisfying as it may be for the pollster to read all these posts (in terms of receiving attention), it's all a big anti-intellectual wad of slop.

Finally, I need to make it clear that my moderator/content positions might as well be considered totally arbitrary. I'm not interested in justifying them whenever called upon - it's sufficient that I'm putting in the effort and that you guys judge the effort by its results in the long term. Every time I answer a post like this, ten emails, posts, and PMs are getting shorted.

Best,
Ron

P.S. Cross-posted with Eero. He's 100% correct, and his points about meaningful questionnaires are well taken.

[KingstonC]

It is an article of faith here (the only article of faith here, come to think of it) that game design should be a labor of love. Game design should be about sharing the designer's personal vision with others.
And, when you start asking other people "Do you think this is cool? What do you think about a race of cat people in my game?" or whatever, the game moves, even just a little bit, from a product of personal vision to a product of marketing, designed to please others. And there is quite enough of that, things made to please sombodies idea of "the market" around already, in games, in books, and in movies.
We know already what games "the market" wants. Those games already exist, and there is no need for another. We don't know what game you wanted, and saw didn't exist, and had to make with your own two hands. Show us that game.

[Clinton R. Nixon]

Very nice: Eero posted well, and Ron as the moderator confirmed his reasoning. I, as the other administrator, will confirm Kingston's reasoning.

The most important post you'll ever read at the Forge is right here, and it describes perfectly why I do not support polling as a useful method of design.

[anthony kilburn]

I understand and agree with some of the posts above, but I pose a question: Why is the information gleaned on a topic such as the effectiveness of a title useless?  If all you're looking for is whether or not the title of a product projects interest in those unfamiliar with it (a "yes" or "no" answer), why would such a poll's results be considered useless without discussion?

Beyond that, your poll could be married with discussion, such as an explanation for a certain opinion.  The poll simply allows for the "yes" or "no" to be easily referenced, and for certain "yes" or "no" answers (or those "none of the above" or "both" response), the author of the poll could simply scroll down to look up why a certain person answered the way they did.  While not ideal for design ideas, questions of effectiveness (again, those "yes" or "no" questions) can be easily responded to in a poll.

Besides that, it's much easier for someone to answer a multiple choice poll than jump in on lengthy discussions of terminology and principle.

After all, as in the original topic posted, the question was the effectiveness of a title.  Everyone has an opinion on whether or not they'd look into a game based on a title, even those that don't consciously think about it.  Based on that title, they either look at the product or don't.  No need for everyone to discuss what they don't consciously think about, just quick gut answers.  Polling, it seems, would be perfect for this.

[Shreyas Sampat]

Besides the non-discussion problem, there is also the point that the Forge is not only a very small but extremely skewed sample.

It's just not a good place to take polls.

[Eero Tuovinen]

I understand and agree with some of the posts above, but I pose a question: Why is the information gleaned on a topic such as the effectiveness of a title useless?  If all you're looking for is whether or not the title of a product projects interest in those unfamiliar with it (a "yes" or "no" answer), why would such a poll's results be considered useless without discussion?

As I understand it, this is basicly because Ron & Clinton want to keep the discussion going at the Forge, without diluting it in any way. One may disagree with the numerous prohibitions (like, say, the lack of a pure socialization forum), but in the end it's their forum. So the final answer (as Ron already said) is that things are like they are because he wants them that way. This doesn't of course mean that there is no explanation.

Think of it this way: an opinion thread or a poll has no significance for anybody but the originator of the thread; it only serves him, and is of limited interest to anybody else. Therefore it's hardly the kind of content one would like to have on his forums. Furthermore, a poll is movement of information (more properly, data) in one direction only, making the thread almost inevitably lesser in it's nature.

So it's not that a poll would necessarily offer useless information (although it's almost certain here, given the highly skewed nature of the Forge's participants), but that the moderators do not want to encourage polling in lieu of communication. Ron/Clinton are really big on genuine communication, and I can only agree; the Forge is a meeting of the minds, and encouraging opinion polling can only take attention from more important and valuable forms of communication. In a poll you're just shooting pellets of gut instinct into the abyss, instead of reading and replying; the mode of communication is totally different.

Furthermore, consider the question you're asking: if the poll you'd like to have is truly such that it doesn't matter who the question is asked of, a genuine gut instinct question, then why ask it at the Forge? Why not put it up on your own server and direct folks there, for example? Why should the Forge, for example, host such a questionnaire? The connection seems tenuous at best.

After all, as in the original topic posted, the question was the effectiveness of a title.  Everyone has an opinion on whether or not they'd look into a game based on a title, even those that don't consciously think about it.  Based on that title, they either look at the product or don't.  No need for everyone to discuss what they don't consciously think about, just quick gut answers.  Polling, it seems, would be perfect for this.

You aren't understanding. The advice about this particular issue was that you shouldn't be needing a poll about a game name. That's not a good way to name a game. You should give it a name you like, instead. Good art is brought to being through caring, not by calculation.

I understand that this is an ideological thing to a degree. The board owners are not motivated to support game design/publishing methodologies they deem ineffective or wrongminded. They aren't interested in supporting work-for-hire because they don't like it's cultural implications, and likewise they don't support opinion polling as a game design tool. All about the kind of activity they want on their forum.

[anthony kilburn]

As I understand it, this is basicly because Ron & Clinton want to keep the discussion going at the Forge, without diluting it in any way.

I'm not arguing the discussion focus or on the actual prohibition of polling on the boards.  I just wonder why polling is thought of as useless or as counter-productive to discussion.

Think of it this way: an opinion thread or a poll has no significance for anybody but the originator of the thread; it only serves him, and is of limited interest to anybody else. Therefore it's hardly the kind of content one would like to have on his forums. Furthermore, a poll is movement of information (more properly, data) in one direction only, making the thread almost inevitably lesser in it's nature.

This is no different than a situation I've already encountered on the boards.  I've tried to ask system-oriented or theme-oriented questions thus far (does this stat set work in general?  does this idea/write-up suffice as a concept for a game?).  In this, I've tried not to alienate other posters, since no one really cares about the answers as they relate to my project.  Rather, in asking such universal questions, I figured I was contributing to the community.

Instead, as I've discovered, I've only been asked for all the details on my specific game before I can get an answer to the questions.  How can such specific discussion help anyone else?  I fail to see it.  But if this is permitted, why not simply ask whether a title works for a project or not?  It's just as valuable as "does my system work?" and actually requires that the author of the post create something to be judged rather than ask the community to contribute to a game that the author himself will benefit from.  It's the difference between "does my system work?" and "what system should i have in my game?"  It avoids having the lazy simply "harvest" ideas from discussions and forces a real creation to be brought to the table in the first place.

So it's not that a poll would necessarily offer useless information (although it's almost certain here, given the highly skewed nature of the Forge's participants),

I disagree.  We play games (or at least have opinions on them!).  I personally wouldn't care what someone who plays Final Fantasy thought about my project, but I certainly would care what someone who played a variety of paper & pen games said.

In a poll you're just shooting pellets of gut instinct into the abyss, instead of reading and replying; the mode of communication is totally different.

What if gut instinct is what you're looking for?  Sometimes, a thoughtful response is not the most truthful.  We can convince ourselves that something makes sense out of sympathy or connection.  For instance, in critiquing a title of a game, instinctively, you may hate the title, but upon discussion, you get attached to another title suggested.  This is counter-productive.  The author wonders "what if" this was the title.  Yes, the discussion is valuable, but perhaps the author is really wanting to use the current title.  The discussion agreed that another was better, but then again, if the title was effective instinctually, what's the point of a new title?  Again, just an example.

Furthermore, consider the question you're asking: if the poll you'd like to have is truly such that it doesn't matter who the question is asked of, a genuine gut instinct question, then why ask it at the Forge? Why not put it up on your own server and direct folks there, for example? Why should the Forge, for example, host such a questionnaire? The connection seems tenuous at best.

I'm not challenging the anti-poll sentiment at the Forge, as I've had no real desire to poll anything.  However, I don't understand why the Forge shouldn't be used.  I myself rarely participate in multiple message forums, so I imagine there are many others who can only be polled here.  And people in the gaming world like us who play/create games have valuable opinions that can't be ignored.  Why should someone have to avoid polling here and do it somewhere else?  If I were to actively challenge the policy, perhaps set up a section specifically designated for polling.  Then it stays in one area, those who need polls to help them can use them, and those who are short on time can make a trip to this section to help many people quickly with a few multiple choice questions—then, if they desire, they can add a comment on why they answered the way they did.  Simple and neat.

You aren't understanding. The advice about this particular issue was that you shouldn't be needing a poll about a game name. That's not a good way to name a game. You should give it a name you like, instead. Good art is brought to being through caring, not by calculation.

When I originally asked this question about my game's title, I simply wanted feedback on whether or not people felt a game should be titled descriptively (like Dungeons & Dragons) or creatively (like Cian, the name of the continent in my game).  It helps to know whether or not Cian is too vague or not(as in, an unrecognizable word would inspire someone to check it out).  The name I had was created by care, the calculation just determines whether or not it works.  Much like asking the community if your system works or not.

I understand that this is an ideological thing to a degree. The board owners are not motivated to support game design/publishing methodologies they deem ineffective or wrongminded. They aren't interested in supporting work-for-hire because they don't like it's cultural implications, and likewise they don't support opinion polling as a game design tool. All about the kind of activity they want on their forum.

I don't wish to challenge, but rather discuss how it can be valuable and effective.  A quick, easy to answer question can be more valuable than a lengthy, vague discussion, IMHO.  It's more direct and avoids the lazy designer from just asking questions and taking ideas.

And besides that, it takes up less space than people asking a bunch of indirect questions and having the masses respond with "please clarify" and "what's your game about?" posts.  It forces a clear question—and a clear answer.

Feel free to disagree, I'm just presenting my thoughts. ;-)

[Ron Edwards]

Somebody else talk to this guy, please. Here, PM, whatever.

Thanks,
Ron

[anthony kilburn]

LOL.  I never intended to poll my question, and quite frankly, I don't even know how to create one!  Someone presented a question that I was interested in hearing about, and I just disagree with you all on the validity and effectiveness of polling.  I simply think it supports the design process.  Sorry, I had no intention of offending you.  My sincere apologies, Ron.

[timfire]

Instead, as I've discovered, I've only been asked for all the details on my specific game before I can get an answer to the questions.  How can such specific discussion help anyone else?.

I actually think it does help others, but more indirectly. When people ask you questions about your game, they're doing a couple things. First, they're demonstrating the types of questions you, the designer, need to be asking yourself. I know I've had those moments -- I'll be reading someone discussion about their game and I suddenly realize, "I never considered that issue!"

What people are also doing when they're talking about your game is showing various angles that an issue can be approached from. In the future, someone will come along searching for inspiration on a similar issue, find your thread, and possibly find ways to use those answers in their games.

Another thing these discussions do is generally demonstrate the process a designer should go through to develop their game. Even if people don't find the specific ideas helpful, you'll still see how a game is designed. And that's a very helpful thing, too.

[Ron Edwards]

Offending me is not possible in this venue. Others will be able to help you better than I can - so read what they have to say, carefully. I suggest waiting for a number of posts or PMs.

Best,
Ron

[greyorm]

This is not a bearpit. Stop behaving like the bear.
If you don't understand what I mean, use the search feature.
If you don't believe you are behaving like the bear, take Ron's advice: carefully read the responses that were and have been written. Additionally, if you wonder why I accuse you of being the bear, this is the only example I should need to bring to bear:

I just wonder why polling is thought of as useless or as counter-productive to discussion.

From this, it is obvious to me that you are not reading, digesting, and examining the posts being written in response to you, or you would not have had to ask this question...again. The answer has been detailed in a number of ways above (hence, no doubt, Ron's desire not to deal with the question again).

Hence, the advice: read, digest, examine. Sit and wait for a day, thinking about the responses -- not "what you gotta say about the responses" but about the responses, before forming/posting a response.

Regarding polls: Proper polling is not something you can do via a website or on a forum. Proper polling requires precisely sampled known diversity, well-crafted questions written to careful specifications, and a large base of participants per the above requirement from which to draw answers.

Polling -- the real, useful kind -- is a science, not a 14 year old kid or a 46 year old adult asking around what everyone thinks he should do or how they feel on their website. That's all there is to it. YOU (and ME, for that matter) cannot conduct a valuable, useful, meaningful poll because any data you gather from the exercise will not be valuable...even though you think it should be. What most people "think should" and what "actually is" are two very divergent beasts.

I further submit that if polling is not an issue for you, if you do not care about polling, then the question about polling is a red herring and the reason for this thread is at an end.