Falsifying GNS

[Ian Charvill]

In a couple of threads recently people have raised the idea that GNS is not falsifiable so therefore is questionable as a theory.  It occurs to me that GNS is falsifiable by doing one or more of the following:

1. Establish that there is no creative agenda by either
(a) proving that people have no reason to get together to roleplay; or
(b) proving that if people do have a reason to get together to roleplay, but that everyone has the same reason

2. Assuming that there is such a thing as a creative agenda you would need to prove
(a) creative agenda cannot be divided into several categories on a practical level (I suppose by showing that everyone games for a reason that is entirely their own, and that there are never 'reasons in common')
(b) creative agenda can be divided but that such divisions are unknowable from how people act (i.e. that a gamist and narrativist, frex, would behave when roleplaying in the same way at all times)

3. Assuming creative agenda can be divided into categories then you would need to prove
(a) the categories are contradictory (e.g. something about narrativism proves there can be no gamism or vice versa);
(b) the categories are insufficient (i.e. that there is a fourth (or more) mode that isn't accounted for by GNS); or
(c) that one of the modes doesn't exist (there is no gamism, simulationism or narrativism)

[Valamir]

Excellent call Ian.  I think I even agree with the order you put them in (whether you intended that order or not).  #3 would clearly be the weakest form of falsification, because all that would prove is that the theory hasn't finished developing...which we all know is the case any way.

I actually object to the tactic some Forgers have taken of suggesting or "acknowledgeing" that the theory isn't falsifiable.  I think that weakens the theory and does an injustice to how powerful it actually is and how demonstratable its tenets are.

I do feel, however, that attempting to search for falsification before one truly grasps the theory, is an inherently inefficient pursuit.  If one doesn't understand what the theory actually says, then the best one can do is build a case against what one assumes it says.

In this matter I think any claims of "X years experience as a roleplayer" is actually a liability, not an asset.  Those years of experience as a roleplayer invariably bring with them alot of excess baggage about how RPGs are played, what they are supposed to look like, etc.

Much of GNS flies right in the face of such traditional assumptions and seeks to ask "what's really going on here.  Lets peer beneath the hood of the Impossible Thing, and inside the black box of Immersion and try to identify what's actually going on at the table.

Putting the cart before the horse and trying to disprove when one only barely understands what one is trying to disprove is a road we've been down many times since the Forge started.  Invariably it involves ALOT of running to stay and place.  A tremendous amount of effort is exerted and the end result is that we're basically at the exact same place we were when we started...hopefully with at least one new person deciding theres something worth sticking around for.

Hopefully the new "whole model" writeup, polished and packaged with the 3 GNS essays will serve as an effective aid to us in this process.  But I do wish that the enormous amount of time and effort, and proven results of the theory's application and the pile of evidence of the same to be found in Actual Play and elsewhere in our 90,000 posts on the subject, would be given a bit more benefit of the doubt.  This is not some idea that was just thunk up in the shower yesterday afterall.

[Ron Edwards]

Hello,

Thanks, Ian. That's a good breakdown, and I agree with your purposes in presenting it. However, I do think it's premature.

In my experience, people who use the words "theory" and "falsifiable" without actually dealing with such things professionally tend to fall into a couple of errors. The errors are traceable to missing one key concept.

All hypothesis testing is based on constructing an understandable null hypothesis. It's not a hypothesis test until people who disagree about the "way things work" do agree on the validity of the null being examined.

The real question is whether "Creative Agenda works like Ron says" is a null or an alternative hypothesis.

If it's the null, then the experiment should set up conditions under which the principle might not apply.

If the null, on other hand, is "Creative Agenda does not work like Ron says," then the experiment should set up conditions under which the principle might indeed apply.

The real stumbling block for students, in my experience, is that they think you pick a null based on how you think things "ought to turn out," which is to say, make that statement the alternate hypothesis. (Or even more primitively, they might think a null means "no change, nothing happens" as so many science textbooks wrongly state. But that mistake seems less likely in this case.) But you don't pick a null based on any such thing. What constitutes the null is a matter of agreement among people who disagree about the likelihood of the various possible outcomes generally, not just in the case of one experiment.

What's all this about "falsifying," then? It's about the null. Either the data (clearly or muddily) is consistent with the null, or it's not. If it's not, then we say it "falsifies" the null, which is just a fancy way of saying "inconsistent with." In which case, at least for purposes of discussing the single experiment, we are stuck with the alternate statement.

(Scientists are sloppy about saying "accepting the alternate" sometimes. Don't let that fool you. There is no acceptance, just rejecting or not rejecting the null.)

I also caution everyone against using the term "prove" in the sense of a courtroom drama. Real critical analysis (hypothesis testing, e.g.) is like real law, not like dramatic/TV law. You don't prove anything. Nothing gets proved, ever. We even have fancy names for that, called Type 1 Error and Type 2 Error.

All you can do is make better sense with your explanation, given what we observe with various comparisons, then with any other explanation. Or if you can't reject the null, then you're showing that your (or "this") explanation isn't doing a great job and we should back up and look at the question again.

However! And this is what the textbooks always forget to explain; people don't learn it until they're knee-deep in mud in the field or twenty levels deep in the analytical stages. You don't construct hypotheses based on what you "feel" or even "know." You construct them based on what other principles and corroborated explanations (theories) suggest.

People who disagree vehemently on whether the world is X or Y can at least agree that, since ABC applies, then X can be held as the "just ABC in action" outcome and Y can be held as the "must be a D in there too" outcome.

So I'm kind of impatient, or more accurately, wearily tolerant of people hurling about "falsify!" as some kind of crtiticism of my and others' notions here. Why? Because I don't consider the ideas mature enough, and integrated with other ideas well enough, even to provide the right foundation for hypotheses.

There ain't no ABC yet from which "X or Y" questions can be properly constructed - which is to say, whether X or Y is best considered the null hypothesis.

Best,
Ron

[pete_darby]

Well, since it seems my position that's a bit of a bugbear to some people, I guess I'll set it out again to show where I stand: the model proposed here is (to me, at present) more akin to a theory of acting or writing than a scientific model. Attempts to treat it as a scientific theory, with everything (as Ron knows better than I certainly do) that entails are as useful as making scientific tests for the validity of the theories of Stanislavsky or Keith Johnstone.

What irks me is the assumption that such theories are worthless because they are not scientific but artistic, or at least creative. But I picked those two examples because they both arose from a passionate desire to improve the quality and experience of performance in a practical setting (something often lost on students of Stanislavsky's method), not a desire to classify, or restrict, performances into narrow bands defined by The Method. Toolboxes, not prisons.

The model could certainly form the basis of a scientific theory of the behaviour of people engaged in a co-operative creative endeavour, as could the two dramatic theories I've mentioned, but that doesn't mean that, as it stands, it isn't an excellent critical tool for the analysis and betterment of play.

*sigh* Someone's going to misread critical as criticizing now, aren't they....

Well, Ron, that's how I'm abusing your work. Wacky drama & philosophy graduate that I am.

[jdagna]

Well, since it seems my position that's a bit of a bugbear to some people, I guess I'll set it out again to show where I stand: the model proposed here is (to me, at present) more akin to a theory of acting or writing than a scientific model. Attempts to treat it as a scientific theory, with everything (as Ron knows better than I certainly do) that entails are as useful as making scientific tests for the validity of the theories of Stanislavsky or Keith Johnstone.

But I think ron has set himself up for scrutiny of GNS as scientific theory   I can certainly think of it as literary-style critical theory until I hit the point where it specifies observable behavior as the definition of the modes.  By specifying observable behavior, Ron makes it something that can be scientifically tested, and to my mind, he's issuing an implied challenge for just that.  I think quite a few people are picking up on the same implied challenge.

Now, whether this is a good thing or not I certainly don't know.  Ralph make a very important point, however, which is that few (if any) of the people demanding a scientific proof actually understand the theory.  We could show them a published, peer-reviewed article from a journal and they'd still reject it because it still wouldn't match up to their misconceptions.

[Christopher Kubasik]

All right, I'm throwing my hands up in the air.

Ron, is it a scientific theory?

And if so, could you please define for me what that means?  

I'm having an Alice in Wonderland moment here, and I'm looking for the right mushroom to bring me back to equilibrium.  For the record, I think I've got a pretty damned good idea of what makes solid dramatic narrative -- but I wouldn't make pretend that the knowledge reuired is on the same playing field as cancer research.  Or is it?

What's going on?

Christopher

[Ron Edwards]

Hi there,

Perhaps it will help if I state that I don't make any distinctions between scientific and non-scientific "theory" or "thinking" or "critique" or whatever you want to call it.

There's just rigorous vs. non-rigorous thinking, that's all. I hope that the ideas presented here and, more importantly, as being discussed, are rigorous. I've done my best so far and so have a lot of other people.

I addressed the issue of hypothesis testing in the above post because that's the methodology people (specifically Marcus, at this point) are referencing when they use the term "falsify." I hope to have articulated that the state of the ideas is not yet ready for applying that methodology, and certainly not at the level that a first-time reader can do on an internet forum.

Not surprisingly, all the most respected ideas of our time also went through such a stage. This stage is grossly "unscientific-looking" to those who think of science like a bunch of clean little yes/no tests which add to an edifice of certainty. During a stage like this, the intellectual framework and dialogue about the ideas is slowly constructed back-and-forth among "naked" ideas, existing observations, and novel experiments that were not even conceivable outside of this new framework. The stage usually lasts a long time, for decades. It's messy.

The most important point is that crackpot, unthinking assumptions, bullshit, and well-constructed but unfortunately-wrong ideas go through exactly the same process; in fact, they are all mixed up with the "good" ideas at the same time. You can't tell which is which, at the outset.

What most students never learn is that this process constitutes the bulk of science and is highly enjoyable and fruitful - the cleanly-constructed hypothesis test is an exceptional moment, not the routine. But I digress. Back to the present topic.

So do I think my notions are equivalent to the result of a full-on paradigm shift, which requires the intensive and critical efforts of investigators over many decades, from many directions? Of course not. And they'd have to be, if they were to mature enough to yield "falsifiable hypotheses."

But do I think my notions are rigorous enough to contribute positively to the current stage of paradigm-shift? Yes, I do. Thus I do not think they are equivalent to the the individualized "well I think so theories" of a dis-unified discipline (e.g. cultural anthropology, film theory), to be lined up in an array for people to pick from based on gut reactions.

More generally, I really think that a "scientific" vs. "artistic" distinction is so highly personalized, in terms of what it means at all, that it's not a very useful general point. Pete, I'm perfectly OK with the observation that saying this helps you fit how you want to apply the ideas into your head, but it doesn't have the same power in others' heads necessarily.

Best,
Ron

[marcus]

I agree with Ron.

I originally thought that GNS was more fully formulated and more definite in its statements about role-playing behaviour than is apparently the case. If the theory is not in a state whereby its proponents can agree on what predictions the theory makes about potentially observable phenomena, then the theory is certainly in no fit state to be tested empirically.

As Ron points out, the above does not mean that the theory is without value. Even if one sees value only in a theory that is testable (and many would not be of this view), the fact that the theory cannot make definite predictions now doesn't mean that this problem (if problem it be) will not be solved when the theory is refined and more precisely formulated. And GNS is certainly excellent food for thought whether testable or untestable, right or wrong.

Marcus

[pete_darby]

Ron, my points of disagreement about your last post are so minimal as for me to be struggling to find a coherent expression for them (Dogbert style handwaving and going "pah!" aside).

Is GNS presently in a state more rigorous than most artistic theories? Oh hell yes. For instance, your not starting with a dogma then applying it to the work... ah, a rant for another time.

[Ian Charvill]

Well, unless there's anything pressing anybody feels a need to add, I think we can call this one closed.

[Christopher Kubasik]

Hi Ron,

Thanks for the reply.

I am now pretty certain I'm out of my league on this one.  (I can't shake the feeling that matters of "taste" will haunt all this to the borders of science, but I'm probably missing something.)

I leave it to more capable hands.  But, if needed, I'll refer to this thread when others need a reference.

Christopher

[beingfrank]

I like the view of scientific discovery that follows the following stages:

1)  Is there something real there?
2)  What is it?
3)  What things influence it?
4)  How does it work?
5)  How can we use it to make real world predictions and other applications?

Most of science occurs at stages 3-5.  But the really exciting, important and groundbreaking stuff is in stages 1 and 2.  However, because these stages don't form a big part of people's consciousness, they tend to leap to stages 3+ when wanting to test things and do research.

Proving stage 1 for GNS is the first stage to proving the later stages.  But the very fact that it's so basic makes it tricky.  I haven't got any brilliant ideas yet, unfortunatley.

But maybe this view will help other people in their approach to the problem.

[MachMoth]

I'm not going to press my opinions into this discussion.  I have them, but they hold no validity in relation to it.  

The main point is that an experiment isn't classified as "non-scientific" because it contains elements of human thought and ideas.  My best comparison would by Psychology, a field which I believe has significant ties with RPG Theory.  Psychology deals entirely in the why and how of human thinking, yet it is an entirely scientific process.  In fact, the major goal/barrier to psychology is the research of human thoughts in a scientific approach.  It's a fairly difficult process, that has been handled with much success.

Whether Ron's theory follows the lines of scientific process is not my place to say.  I'm only saying that it is a very possible and defined process to apply human thought to hard science.