Watching your readers

[Jasper]

Spawned from the thread on web-based presentation of game texts.

In web-page design (and elsewhere) there's always the desire to watch someone while they're using a page to see how the user interface works.  Just getting reports back is all well and good, but people are often unaware of their own use habits, so it's key for the designer to actually look over their shoulder and see how they use the page: what things confuse them, when do they take a long time looking for something, etc.

Some people around here have certainly emphasized asking for critical feedback on rules texts, but has anyone purposefully watched someone else read through their game, perhaps semi-serruptitiously? It might be hard to do, or to watch closely enough, but I imagine you could get some interesting data. If, for instance, a reader constantly flipped back to an earlier section, that might tell you that something wasn't explained well enough, or needs to be repeated.  

Of course the real test of rules-comprehension will be during a game itself. When the rule book gets consulted would be very telling.  If your players have to reach for the same page over and over, that could at least tell the designer that a quick-ref card is needed.  I've often tried to get others to GM my games, but I'm usually a player at the same time -- in part to clear up any rule confusions there might be.  But now I think a lot could be gained by just silently observing. Has anyone done this?  Just watched people playing, with a mind towards how they use the text?  (Of course, if they don't reference the text, but get the rules wrong, that tells you something too...)

[Matt Machell]

I think observing how a game goes in actual play, based on the reading, is probably far more interesting.

Like in usability testing, not being present (use a video or web cam), so that your reactions don't guide play might be a useful tweak.

I'd love to do this kind of test with a copy of D&D and a group of non-roleplayers, just to see reactions.

-Matt

[jdagna]

For a web page, something like VNC might be ideal, though you'd have to coordinate when the person was going to read the book with when you watched.  It should help reduce their feeling of being watched.

Of course, a lot of people will want to print the thing out, whatever format you provide it in.

[Luke]

Some people around here have certainly emphasized asking for critical feedback on rules texts, but has anyone purposefully watched someone else read through their game, perhaps semi-serruptitiously? It might be hard to do, or to watch closely enough, but I imagine you could get some interesting data. If, for instance, a reader constantly flipped back to an earlier section, that might tell you that something wasn't explained well enough, or needs to be repeated.  

In short, YES.

This is a crucial point in my playtesting. Handing someone the rules and watching them flip and read. Where do the eyes fall? What's the first thing they flip to? When do they start asking questions?

It drives me friends  crazy when I do it, "I need you to sit and look through this {while I stare at you)." But it's incredibly important to the design process.

Secondarily, I LOVE surreptiously watching people pick up my material in a store. First, it's a major visceral thrill -- he's looking at MY book! But watching the reactions, how they handle it, what they read first, etc is just so darn interesting. And damn informative, too.

I definitely use that info, too. If I section is being flipped past that I need to be more prominent, I reconsider it's placement, art, titles, charts or whatever.

-L

[Ed Cha]

Don't most people just look at the first few pages, then flip through the rest of the book? That's why I think a good introduction and some great visuals in the beginning are so vital. Of course, there's nothing more important than a friendly and persistent targeted sales approach.

[Jasper]

Ed, I'm specifically talking about *after* you've made a sale: so comprehension rather than...attraction I guess.


For those who have observed people reading (and playing with a mind towards the use of the text), have you noticed any trends in terms of what kinds of rules are problematic more often?  What about any of the following kinds of things:

[*]Introductory text.

[*]Mechanics involving dice and concrete manipulations

[*]Mechanics involving less cocnrete things like narration rights

[*]Play examples

[*]Anything else I've forgotten[/list:u]


I imagine this will depend a lot on your own writing talents.  But for any of the above: do people  actually read it?  Do they peruse or really focus?  Do they return to it later for reference or do they internalize it all?  Do they go back just to get a broad sense of how to play (i.e. how everything comes together)?