Please rip our questionnaire apart.

[J. Tuomas Harviainen]

We have done field research on larp experiences by using this questionnaire, with the intent of gathering data on how people experience the in-game stare. It also contains questions that provide us with potential comparison points and at the same time give the game's organizers direct feedback on their work. We've so far been kindly permitted to use it on two Nordic larps ("Moira" by Ericson & Tidbeck and "Four Colors" by Pohjola; academic standards prevent using it in our own games), and have a reasonably large set of results now. (In case there are U.S. based organizers reading this: We'd also love to get one run of this survey from an American Larp with 50-100 adult players, to give a much broader perspective. So if you're interested, PM me. I'll cover the costs of making copies of the questionnaire and mailing them to us, and will of course give you credit for the assistance rendered in the eventual article.)

The questionnaire has received its share of criticism, but we'd still like to get more. That way we can state within the result publication (upcoming in 2006) the criteria on which the results can be taken as reliable. So pleaser rip it apart. Really. In this case honest feedback to the point of being brutal is extremely welcome. Feel free to question our perspective, motives, etc. The idea is to get outsiders' criticism, as we see that we are still bound by the predominant larp concepts of our native gaming culture.

The questions try to include the key points of all dominant theories presented in the field so far. Section 4 - the main part of the text for us - is based on the Allport-Ross questionnaire, which is a classic of research on religious experiences.

(No matter how much I edit it, the dots do not seem to stay in line. Try see them like that anyway, please)
---
Larp Experience Research Questionnaire, version 3
(copyright) Harviainen & Heliö, 2005

1. General information:

Game:___________________________   Date:___/___/_____     Location:_______________ 
Duration of the game itself (excluding preparation and debriefing):_____________

Age:___
Gender: __________
Nationality:_________________
Place of permanent residence: ________________
Years of larp experience:______      Years of tabletop role-playing experience:_____


2. Analysis of the current game:
                                                          Very Negative  Negative  Neutral  Positive  Very Positive
How was your game experience in general?            O               O           O             O           O
How did the following elements affect your
game experience:
- Preparation of the game environment                         O               O           O             O           O
- Quality of the game material provided                          O               O           O             O           O
   (i.e. all written material you received for the game, such as character description and background information)
- In-game interaction between players                          O               O           O             O           O
- Duration and pacing of the game                      O               O           O             O           O
- Pre-game information provided by the organizers             O               O           O             O           O
   (i.e. advance information about how and where the game would be run, themes, logistics, etc.)
- The general plot structure of the game                  O               O           O             O           O
- Off-game discourse/workshops with other participants      O               O           O             O           O
- The use of new game techniques or mechanics               O               O           O             O           O

Were there any elements during the game that you found disruptive to your game experience? If yes, please elaborate._____________________________________________________________________
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

Were there one or more specific elements that particularly assisted in your enjoyment of the game? If yes, what were they: __________________________________________________________________
_________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________

3. How well do you think the following elements had been prepared in the game you just participated in?                                                                                               Very Badly  Badly  Decently  Well  Very Well
- The game in general                                              O         O           O          O       O
- Preparation and staging of the game environment               O         O           O          O       O
- Character creation                                                O         O           O          O       O
- Support for sufficient character interaction                           O         O           O          O       O
   (i.e. was the quantity of plot hooks, game elements, etc. sufficient to provide enough discourse?)
- Anticipation of optimal game duration and pacing                  O         O           O          O       O
   (i.e. was the game the right length and did events unfold at a good pace?)
- Provision of information about the game beforehand               O         O           O          O       O
- The overall plot structure of the game                           O         O           O          O       O
- Production of new mechanics and techniques for the game     O         O           O          O       O
- The participation-fee-to-game-quality ratio of the game              O         O           O          O       O
- Support for off-game interaction (workshops, mailing lists, etc)O         O           O          O       O

Please elaborate what kind of changes you would have preferred to see in the areas you considered badly done:
(FOLLOWED BY 7 EMPTY LINES)

4. The in-game state:

During the game, did you think of your character as: ___ "Me" or as ___ "My character"? What do you think were the primary reasons for this: (FOLLOWED BY 4 EMPTY LINES)

Which of the following statements do you consider accurate descriptions of the way you experienced the game? Please mark all that seem applicable.
___ My character was the only "self" I experienced during the game. My real-world personality did not affect my game experience during the game.
___ I was the same person that I am outside the game. My character was only a social mask that I wore, just like I act differently at work and at home.
___ My character and I existed as an interlaced persona, in which the choices of my character arose from the character's personality, but were influenced by my real-world personality.
___ The choices I made during the game were influenced by things I like outside the game.
___ I was on a vacation from the burdens of the mundane world.
___ My game experience was disrupted because the pre-game information (including character) that I received contained too much information that I had to remember/re-read during the game.
___ I was in a theatrical play without an audience.
___ I enjoyed the new forms of activity the game's reality made possible.
___ Occasionally I reacted without thinking solely on the basis of my character's world-view.
___ I was not the same person during the game that I normally am.
___ Ethical choices I made during the game were influenced by the game's genre and/or rules system.
___ Other participants identified with the characters more strongly than I did.
___ I felt fear during the game.
___ My character felt fear during the game.
___ The choices I made during the game were influenced by my assumptions on what would suit the plot structure of the game.
___ My character's actions that were observed by other players were more important to my game experience than were the ones I did without an audience.
___ I felt sexually aroused during the game.
___ My character felt sexually aroused during the game.
___ I enjoyed the sense of being someone else during the game.
___ The choices I made during the game were influenced by my assumptions on what would make the game better for all.
___ I could not identify with my character.
___ It was hard to concentrate on the game, because the game environment did not look like what it was supposed to look like in the game's reality.
___ My game experience was disrupted because I had to invent material that was not included in the pre-game information (including character) that I received.
___ I enjoyed participating in the game


If you felt a loss of the sense of self and/or a strong identification with your character, how long (approximately) did it take you to reach that state?_____________

Do you believe it is possible to identify so strongly with one's character that it becomes one's primary identity during a game (i.e. does, in your opinion, "character immersion" exist)?  ______
If yes, have you experienced it yourself? ______

5. Game results:

What was the most enjoyable element in the game:__________________________________________
___________________________________________________________________________________
What was the most unpleasant or disruptive element in the game: ______________________________
___________________________________________________________________________________
Did you experience the intended themes of the game? If yes, how did they manifest: _______________
___________________________________________________________________________________

Did you learn something new from the larp, such as new values, playing methods or points of view? If yes, please elaborate: (FOLLOWED BY 4 EMPTY LINES)

Any other feedback you want to give on the game:
(FOLLOWED BY 16 EMPTY LINES)

Any feedback you want to give on this research questionnaire:
(FOLLOWED BY 8 EMPTY LINES)


This research questionnaire is intended for direct feedback on larps for the organizers of those larps, as well as for non-commercial academic use. The results will be handled in an anonymous manner, and will only be utilized in furthering knowledge and design of role-playing games. All processed data will be made public.

This research questionnaire may be freely distributed and utilized within the abovementioned parameters, as long as no changes to it are made and the authors of this questionnaire are informed of its use.
---
On behalf of the research team,
-Jiituomas

[Eero Tuovinen]

I'd love to comment on the questionnaire, but I don't have anything to say. That usually indicates to me that there's nothing to say, yet. Could you tell us more about the exact features of the in-game state you're gathering data on? The questionnaire seems to be solely about the phenomenon of immersion. Is that intentional? If it is, then I find nothing wrong with the questionnaire as it is.

You're probably aware that this questionnaire already assumes the standard logistical and artistic organization of Nordic larps. There's writers/GMs who prepare themes, character backgrounds, plot and such with players doing mainly just immersion exercises, and much of the questionnaire operates on those assumptions. This isn't probably any kind of problem, as that particular set of techniques is pretty all-pervasive hereabouts, to the degree that our mutual acquaintance Juhana Pettersson writes in his book about it being the generic method, applicable to everything ;) It's nowhere near universal, though, when looking at larping or roleplaying as a general phenomenon.

Actually, I think I find the above assumption somewhat problematic, now that I think about it. If your intention is to research the birth and conditions of immersion, then surely you should concentrate some more on the absolutely crucial issues of rules system and social context? Your questionnaire seems to assume that all games operate on fundamentally the same system, and that system doesn't apparently have anything to do with whether immersion is reached. This is the only conclusion to make when you note that there's not one question about the rules systems, or what the player actually did in the larp, or what kind of decisions he had the opportunity to make, isn't it? Simply put: I couldn't take this kind of information as indicative of anything, because I would expect that the rules, actual in-game activity and social context are the most important factors in whether immersion is reached. So a questionnaire that doesn't cross-reference this information with information on the player's immersive state won't then tell me anything at all about the conditions under which immersion is reached.

Really, now my curiousity is aroused. Why didn't you ask anything at all about the activity or style of the game, or the rules systems or anything like that? The only practical matter you ask about is the length of the game. This has to be intentional.

Of course, the above is not a problem if you don't plan on addressing the conditions of immersion, and wish to just explore player experiences of the phenomenon. The questionnaire will probably suffice just fine for figuring out what kind of psychological phenomenons people associate with the idea of immersion.

[J. Tuomas Harviainen]

Actually, we're not at all intersted in immersion. All references to it are there for control purposes. "Do people who say they immerse favor certain character relationships". (Answer: Surprisingly enough, they probably do not.) :)

Instead, the main point is in understanding the general tendencies of the character-player relationship and the factors affecting it. That's why the questions you notice as being absent aren't there. What "Player X" did in the larp isn't as important as why she thinks she did it. For example, at test runs of the now-outdated version 2 of the questionnaire, some people claimed that they were both /only the character/ (the first option of section 4) during the game, but also marked that their actions were affected by plot structure assumptions and/or the wish to heighten the gaming experience or other players. I think that's extremely interesting.

If we'd ask what people did per se, it would leave to analyzing the actions on grounds that we do not truly have access to. Such value judgments, IMO, are not at all useful at this point. The same goes for the impact of rules. (But your critique is nevertheless now noted as "extremely relevant to explaining study parameters". So thank you, Eero.)

[Eero Tuovinen]

Thanks, that cleared up a lot. Let's see if others have anything to say, now.

[xenopulse]

Well, I used to design questionnaires for a living (and in soc sci grad school), so I have a few overall suggestions.

Mainly, get rid of the easy way out, i.e., the middle answer (Neutral, Decently...). You will get a lot of useless information that way from people who have a leaning but are too shy/undecided/unmotivated to write it down, and people will read the question more carefully. If people are really, thoroughly undecided, they will leave the field blank or write it in. So for your evaluation, you keep "undecided" as a possible answer, but don't give people an actual field for it.

You'll probably want the same four categories (I usually use "Strongly Agree, Somewhat Agree, Somewhat Disagree, Strongly Disagree" with the question being "Would you agree or disagree with the following statements regarding your LARP experience") for question 4.

"Years of experience" is a vague term given that some people play much more frequently than others. You might have to think of something else that's a little more specific, or add an "average days of LARPing each of those years" to it, which you then multiply with the years for your final value.

Since you have sexual questions in there, make sure to point out that the questionnaire is anonymous and that the person filling it out will not be identifiable.

That's it for now. :)

[J. Tuomas Harviainen]

A big thanks, Xenopulse. Some comments in return:

1. The easy way out needs to be there. It's a cultural thingy, coming both from the habit of people to think positively about larps they just were in (a phenomenon which has been noticed before) and from the fact that in some countries the acceptable level of praise varies, some being more enthusisastic and other considering "decent" a complement. Thus the local baseline varies, meaning we need more scale. If we leave it out, the default becomes "positive/well", something even less uselful. So far we have not had any trouble with people taking the easy way out, but it is indeed a serious concern. It was given a lot of thought beforehand, and apparently may still need more.

2. We intentionally left the categories out of question four. The data from there will be built from the interplay of the statements. The key is in what things seem to exist at the same time, and which ones preclude others.

3. Years of experience are the only way to get even some rough answer. To use any another method of counting (we tried) meant exhausting the players at the start of the questionnaire. The questions are there mainly because they provide two functions: a. knowledge about whether or not a larper has played tabletop (which has been suggested by theorists as a major factor in how perfectly staged people want their game environments) and b. some rough measure of how much time the player's views have had time to develop. To ask for days per year would lead to needing a follow up on what kind of larps they were in, which would lead to further value judgments from us, something we definitely want to avoid.

4. The sexual questions normally come up only in games that deal strongly with such elements, and on those occasions the anonymity issue is discussed as a part of the game process itself. Nevertheless, an anonymity statement is at the legalese section in the end, and will of course also be emphasized when the questionnaires are dealt out. The reason the sex questions are there is that sexual game material has been posited by several larp theorists (such as myself, Ericsson and  Wieslander) as being something that does not match the rules on how the player-character relationship normally works.

For most of the players in our primary area of potential games we can use this questionnaire on, players find anonymity for giving game feedback more important than for sex-related issues anyway. They have a harder time saying they did not like a friend's game than saying that a love scene felt erotic. In other areas, it may be just the opposite.We'll see how it goes.

[xenopulse]

Well, looks like you've everything figured out just fine. Let me know when the results are in, it's a fascinating topic.

[beingfrank]

How are you going to analyse the results?

I know, I know, I ask a huge question.  But I can't really comment on the questionnaire until I know more about how it fits into your overall methodology.

[J. Tuomas Harviainen]

How are you going to analyse the results?

Some of the material will be analyzed just as they are, but most of the stuff will be done through multivariable correlation systems. (I'll eventually pour everything into SPSS and look at what happens. :)

The primary questions were made so that they can be easily accessed, as the example case above shows. For the rest, what we are hoping for are pointers for what to look at in the future. For as far as I know, no one has attempted stuff like this on such scale before, so we do not exactly know where it all leads. We do have some initial knowledge on how certain things will probably correlate (from the testing stage of the project), and are thus able to form functional factor estimates on which we can base the rest of the process, and then polish it later to more precision.

We'd love to do a follow-up with interviews, but due to this being a self-financed hobby project despite our drive to stick to academic standards, it is not at this time fiscally possible to be as thorough as we'd like. Same goes for eventual publication: we do not yet know if we can get it out in an academic work or if we'll go for the next Nordic yearbook - both of which are options that have up- and downsides.

Do you want something more specific?

-Jiituomas

[Graham W]

Jiituomas,

Since you were asking for brutal feedback...

My immediate reaction is that I wouldn't fill the questionnaire in, because it's too long. If you emailed it to me, I'd probably forget to send it back; if you gave it to me after the game, I'd go to the bar instead.

I think some of the questions - especially the "I felt sexually aroused" one - would put me off filling in the questionnaire too.

Apologies for being so negative and good luck with the study. I hope it gives you some good results.

Graham

[J. Tuomas Harviainen]

My immediate reaction is that I wouldn't fill the questionnaire in, because it's too long. If you emailed it to me, I'd probably forget to send it back; if you gave it to me after the game, I'd go to the bar instead.

Graham, you're 100% right. We've long ago identified this as the biggest flaw of the questionnaire. Problem is that we have not found a way to make it shorter /and/ able to get us the data that we need. Plus we need it to contain a part that also gives the organizers feedback. The first draft was much shorter, but those people who did answer asked for more questions, which we then added (with good results as far as content was concerned).

Previous experience shows, though, that larping is so energy-consuming mentally that a significant portion of players would refuse to fill /any/ questionnaire that would be still useful. As it stands, we now suggest that people simply fill out the dot-parts of page one, and then question 4. The rest are left to those who want to elaborate.

In Moira, we got about 50% of the participants to fill out the paper - both due to cultural habits and the fact that there was a 7-hour bus ride back from the game. So at this point we luckily enough do not have to worry about the length.

-Jiituomas