The Forge Archives

Archive => Indie Game Design => Topic started by: Paganini on January 12, 2002, 01:09:35 PM

Title: Accuracy vs. Precision
Post by: Paganini on January 12, 2002, 01:09:35 PM
Fang, and everyone, I thought maybe I was unclear in my last post, so I wanted to clarify.

Here's what I meant to say:

Detail in a system does not suggest accuracy, because the information encoded into the system may be incorrect.

Similarly, abstraction in a system does not suggest inaccuracy, because accurate results may still be arived at, even though not all factors were taken into account.

In the conversation in RPG-Create, detail and precision were used interchangeably, but Fang has given me a new idea, so I'm going to suggest a new nomenclature:

If a system is detailed, it has much information encoded into the mechanics. We assume that the information encoded into the mechanics is accurate, as this is part of the design process, and we assume good design in discussion of general design principles.

If a system is accurate, it produces outcomes that are consistant with the premise of the game. (In a superhero game, bullets bounce off Superman, for example. This means that you can have simulationist games of any premise.)

If a system is precise, it is both detailed (much correct information encoded) and accurate (produces correct outcomes).

A precise system is very hard to achieve, due to things like rounding errors that I mentioned previously.

Does that make sense?
Title: Accuracy vs. Precision
Post by: Ron Edwards on January 12, 2002, 08:53:17 PM
Hey,

I think this is a big issue that may well enter the Forge lexicon and engender much discussion.

Most of us here are probably familiar with the accuracy vs. precision concept, as applied to all sorts of things. What are they in terms of role-playing design? I think Paganini's suggestion works pretty well.

Comments? Examples, especially? Role-playing concerns?

Best,
Ron
Title: Accuracy vs. Precision
Post by: Joe Murphy (Broin) on January 12, 2002, 09:56:16 PM
Yeah, a few more examples would be lovely. I've just reread parts of the GNS essay (could not remember for the life of me what 'Karma' systems were) and Ron's examples enliven his theory.

Joe.
Title: Accuracy vs. Precision
Post by: Logan on January 12, 2002, 11:20:38 PM
It makes sense to me. We'll have to wait for someone to disagree before the debate begins in earnest.
Title: Accuracy vs. Precision
Post by: hardcoremoose on January 12, 2002, 11:32:20 PM
QuoteIf a system is detailed, it has much information encoded into the mechanics. We assume that the information encoded into the mechanics is accurate, as this is part of the design process, and we assume good design in discussion of general design principles.

I'm specifically wondering what constitutes "much" information?  If it's encoded into the game mechanics, are we talking rules-heavy, rules-lite, or something independent of such distinctions?  I suspect the latter, but some concrete examples of precise and imprecise systems of all varieties would certainly be welcome.

- Moose
Title: Accuracy vs. Precision
Post by: efindel on January 13, 2002, 01:00:10 AM
Quote from: Paganini
If a system is detailed, it has much information encoded into the mechanics. We assume that the information encoded into the mechanics is accurate, as this is part of the design process, and we assume good design in discussion of general design principles.

Why do we assume that the information encoded is accurate?  I've seen quite a few detailed games (using the colloquial meaning, not yours) that weren't accurate.

For example, consider combat turn order in Runequest.  Determining when someone can attack depends on their dexterity, their reach, the reach of their weapon, how far they have to move before they can attack, whether they have surprise, and whether or not their weapon is ready.  There are rules about things like when a character can both move and draw a weapon at the same time.

I'd call this a detailed system, and probably most other people would as well.  But how accurate is it?  There's one major thing that Runequest leaves out in this determination -- skill.  And most people with experience in hand-to-hand combat will agree that skill is a major factor in who goes first.

If we assume that information encoded into a game system is always accurate, then we'd also have to assume that a more detailed system is always a more accurate system.  Neither of these is true, though, and I think any reasonable set of definitions for these terms is going to have to take that into account.

There's also a certain vagueness in talking about "the amount of information encoded in the system" (leaving aside the question of accuracy for the moment).  Let's say I spend six months researching the results of 18th-century pistol duels, analyze that data to produce a set of percentages for how likely it is that a participant in a duel is killed, mortally wounded, wounded, or unharmed.  I then use that data to make a dueling table for my game, which simply has percentile dice rolled for each participant and gives one of those results for him/her.

This table has a lot of information encoded in it (heck, it even has a lot of accurate information encoded).  It probably has more information encoded in it than most actual RPG combat systems.  But is this a detailed system?  I'd say no.

So... what would I call a detailed system?  Well, I'm going to hedge the question a bit, and say that there's more than one type of detail.

Title: Accuracy vs. Precision
Post by: Paganini on January 13, 2002, 09:54:28 AM
Quote from: hardcoremoose
I'm specifically wondering what constitutes "much" information?  If it's encoded into the game mechanics, are we talking rules-heavy, rules-lite, or something independent of such distinctions?  I suspect the latter, but some concrete examples of precise and imprecise systems of all varieties would certainly be welcome.

Let me expand on my definitions. :)

We're pretty much talking "rules-heavy," although that distinction may be a bit of a misnomer. You can have a "mechanics-light" system that is still "information-heavy," if you follow me. As an extreme example, consider a system that only has one mechanic... rolling d% on a chart. This is a mechanics-light system. It only uses one simple mechanic. But at the same time, the entire book could be a huge mass of charts to roll on. You use different charts for different situations, they're crosslinked (like, "go to chart X, reroll and add 30") and so on. You *use* all these charts in the same way, with a single simple mechanic, but there is a lot of information encoded into the game.

Similarly, a mechanics-heavy will often have the information encoded with mechanics. In the previous example, falling damage would be determined on a d% chart like everything else in the game. In a mechanics-heavy game, falling damage might be determined by rolling a d6 for every yard fallen, while fire damage might be determined by rolling a d6 and mutiplying by the temperature. Whatever... these are just arbitrary examples. IME, the most common way to encode information into a system is with the mechanics. So, "rules-heavy" and "detailed" are not quite synonyms, but they're close.
Title: Precision vs. Accuracy
Post by: Marco on January 13, 2002, 11:10:56 AM
Precision (in slightly general terminology) is how *much* information is included in your measurement (i.e. number of significant digits). Accuracy is how correct it is. A computer can give you a highly precise low-accuracy answer if you feed in the wrong data.

(I know the chemistry/physics definitions of precision vs. accuracy against a known-true result: I think that level of definition isn't as useful).

These are the definitons I'd use for Precision and Accuracy with RPG's:

A Precise system is one that gives you a lot of detail about what happened (I hit him on the arm, causing a % chance of weapon droping, a % chance of shock, and these prevailing combat modifiers).

An Accurate system is one that "correctly" models the gener (or Premise, here, I guess).

So if you get lots of data about what major vein was hit when the bullets rip through Superman's skin you have a precise but not accurate system. If you know the attack failed (and always will fail with a hand gun) but the system leaves it to the player or GM to tell why it failed you have a precise but not accurate system.

-Marco
NOTE: Most S/G games (in my experience) have vastly different levels of precision for combat and non-combat but about the same accuracy.
Title: Precision vs. Accuracy
Post by: Paganini on January 13, 2002, 01:27:51 PM
Quote from: Marco
These are the definitons I'd use for Precision and Accuracy with RPG's:

A Precise system is one that gives you a lot of detail about what happened (I hit him on the arm, causing a % chance of weapon droping, a % chance of shock, and these prevailing combat modifiers).

This is only useful so far though, because there are two kinds of accuracy WRT RPG systems. One kind of accuracy is accuracy of results. If the results the system produces  are accurate, then the system can be said to be accurate. OTOH, a system can *also* be said to be accurate if the information encoded in it is correct. Such a system can still achieve off the wall results, in spite of the accuracy of encoded information.
Title: Accuracy vs. Precision
Post by: Paganini on January 13, 2002, 01:44:02 PM
Quote from: efindel
Why do we assume that the information encoded is accurate?  I've seen quite a few detailed games (using the colloquial meaning, not yours) that weren't accurate.

Well, here's may answer. This is all IMO, though, so you may not agree. :)

To me, it's fruitless to discuss general design principles without the preassumption that the design is implimented without mistakes. If I make a comment about a general design principle, it's not very useful for someone else to say something like "that's not true, because system X uses that principle, and it does blah blah blah that doesn't work."

This sort of thing is usually given in the spirit of a counterexample, but IMO it's not very worthwhile, because it's more of a straw man than a counterexample. Citing a specific instance of bad application doesn't refute, or even alter, a statement regarding a general principle. It's sort of boils down to:

Me: "Design principle X works well for Y."
Them: "No it doesn't, because game Z is poorly designed!"

Quote
For example, consider combat turn order in Runequest.  Determining when someone can attack depends on their dexterity, their reach, the reach of their weapon, how far they have to move before they can attack, whether they have surprise, and whether or not their weapon is ready.  There are rules about things like when a character can both move and draw a weapon at the same time.

I'd call this a detailed system, and probably most other people would as well.  But how accurate is it?  There's one major thing that Runequest leaves out in this determination -- skill.  And most people with experience in hand-to-hand combat will agree that skill is a major factor in who goes first.

Using the train of thought above, I would probably say that Runequest attempts to be a detailed system but falls short of the mark. Thinking about this, I see where you're coming from. This could be confusing in long term use, since people might tend to think "detail" simply as the amount of information encoded, independant of the accuracy of that information.

Quote
If we assume that information encoded into a game system is always accurate, then we'd also have to assume that a more detailed system is always a more accurate system.  

Exactly! That's the point I was trying to make on RPG-Create that Bradd didn't get ahold of for a while.

Now, having made these definitions, and identified a triend (more detail = more accuracy), then we can identify a continuum. Absolute detail and accuracy is probably impossible.  

But now we have some terminology that we can use, for example to describe Runequest. It tries to be a precise system, but errors in the detail stop it short of that goal. (I haven't read Runequest, so I'm just basing this on your description... the statement might not be true, I'm just using it as an example of how the terminology could be applied.)

(much snippage)

I think I covered the "what is encoded information" issue in another post.

Quote
Relating this back to detail -- with these terms in hand, we can now say things like "this system has a detailed representation system for X, but leaves interpretation largely up to the GM."  IMHO, this is more informative than simply trying to say that a system is or isn't detailed.

(Kung Fu mode)

Your terminology is pretty good...

(/Kung Fu mode)

:)

The three stages of detail idea goes deeper than my definitions do, while staying on the same line of thought. I think it could probably be combined with mine for maximum effect.
Title: Accuracy vs. Precision
Post by: efindel on January 13, 2002, 08:56:27 PM
Quote from: Paganini
Quote from: efindel
Why do we assume that the information encoded is accurate?  I've seen quite a few detailed games (using the colloquial meaning, not yours) that weren't accurate.

Well, here's may answer. This is all IMO, though, so you may not agree. :)

To me, it's fruitless to discuss general design principles without the preassumption that the design is implimented without mistakes. If I make a comment about a general design principle, it's not very useful for someone else to say something like "that's not true, because system X uses that principle, and it does blah blah blah that doesn't work."

I agree and I disagree.  On the mud-dev mailing list, I've talked quite a few times about "separating ideas from implementations".  The fact that an idea can be handled poorly doesn't mean that it's a bad idea.  On the other hand, though, the fact that an idea would work well if it were done perfectly doesn't mean that it's a good idea.

At some point in design, you have to move from theory into practice.  If you started out assuming that you're going to be able to implement everything perfectly, then you may run into some rude surprises.  Things don't work perfectly in the real world, and trying to design a system -- any system, not just a game system -- as if things always will work perfectly just isn't a good idea, IMHO.

Now, obviously you can go overboard and be too pessimistic, saying nothing's going to work.  There's a middle ground that has to be found between blind optimism and "it'll all just get screwed up" pessimism.

A second point is that I'd like for a terminology and theory about game design to be useful not only for designing new games, but also for talking about existing games.  In order to do that, we have to recognize that sometimes mistakes do get made.  (Of course, in doing so, we also have to be able to look at other perspectives -- sometimes, what's a mistake from one perspective is a very good feature from another.)

Quote from: Paganini
I think I covered the "what is encoded information" issue in another post.

Yes, you did, but my question still remains.  Let me try explaining it in a different way:

When you encode information to make it easier to apply, you often lose some of the information in the process.  To take my example from before, the dueling table encodes information about hundreds or thousands of actual duels.  However, there's no way that you could reconstruct all of that information just from the table -- thus, information has been lost in the encoding.  To put it what I hope is a better way, then:  is detail, in your definition, the amount of information that was used to create a system, or the amount of information that can be gotten back out of the system?

If it's the latter, then the definition could work for me.  If it's the former, then it doesn't work for me.  I think you probably mean the latter, but your original text and your explanations so far haven't been clear to me on which one you intend.
Quote from: Paginini
Quote
Relating this back to detail -- with these terms in hand, we can now say things like "this system has a detailed representation system for X, but leaves interpretation largely up to the GM."  IMHO, this is more informative than simply trying to say that a system is or isn't detailed.

(Kung Fu mode)

Your terminology is pretty good...

(/Kung Fu mode)

:)

The three stages of detail idea goes deeper than my definitions do, while staying on the same line of thought. I think it could probably be combined with mine for maximum effect.

Umm... small correction:  It's not three stages of detail, it's three stages of resolution.
--Travis
Title: Accuracy vs. Precision
Post by: Paganini on January 14, 2002, 12:35:27 AM
Quote from: efindel
When you encode information to make it easier to apply, you often lose some of the information in the process.  To take my example from before, the dueling table encodes information about hundreds or thousands of actual duels.  However, there's no way that you could reconstruct all of that information just from the table -- thus, information has been lost in the encoding.  To put it what I hope is a better way, then:  is detail, in your definition, the amount of information that was used to create a system, or the amount of information that can be gotten back out of the system?

Wow. I hadn't even thought of it like that before. Now that you make this distinction I see what you're saying. I guess I was thinking in terms of information used to create the system, not information that can be extracted from the system. When you lay it out, though, it makes more sense to talk about the information that can be gotten back out of the system, since that's the information that will probably be the most recogniseable. Whether or not it would be the most useful, I'm not sure.
Title: Accuracy vs. Precision
Post by: Mike Holmes on January 14, 2002, 03:22:48 PM
Yep, I thnk that we have several things going on here.

First I think that most of this talk of detail can be broken down into where the detail resides. We've already seen design detail (encoded information in th system), resolution input detail (how much information is required from the player as to his characters actions), resolution output detail (how much information the system produces). We can also look at things like stats. Sorcerer has greater detail for stats than many games, as, in addition to the number describing the stat, the player must select a descriptor for that level of ability (two data,instead of the usual one). Some systems have lots of statistics which makes them detailed. Some smight say that some systems are overly detailed. I think that Jared Sorensen works hard at achieving the right amount of detail.

There are lots of other places we can use the detail term I think.

I wouldn't speak of precision, as that has two distinct meanings (one implies accuracy, the other distinctly does not). If you want to say that a system is detailed and accurate, comment on those points separately. Accuracy has a meaning that I think we can all agree on.

This all also relates back as one poster pointed out to granularity, which has been well discussed previoulsy. Granularity does not entail more detail, but finer detail. A system that has three levels of health (say concious, unconcious, and dead) is at a different granularity than one that has five. You get the same amount of detail (one datum), but the first is more coarse than the second. BTW, as a reminder, when refering to granularity use Fine and Coarse. Everybody gets what you mean then. More and Less, for example, are confusing when refering to granularity.

Just my two cents,
Mike