Topic: Accuracy vs. Precision
Started by: Paganini
Started on: 1/12/2002
Board: Indie Game Design
On 1/12/2002 at 6:09pm, Paganini wrote:
Accuracy vs. Precision
Fang, and everyone, I thought maybe I was unclear in my last post, so I wanted to clarify.
Here's what I meant to say:
Detail in a system does not suggest accuracy, because the information encoded into the system may be incorrect.
Similarly, abstraction in a system does not suggest inaccuracy, because accurate results may still be arived at, even though not all factors were taken into account.
In the conversation in RPG-Create, detail and precision were used interchangeably, but Fang has given me a new idea, so I'm going to suggest a new nomenclature:
If a system is detailed, it has much information encoded into the mechanics. We assume that the information encoded into the mechanics is accurate, as this is part of the design process, and we assume good design in discussion of general design principles.
If a system is accurate, it produces outcomes that are consistant with the premise of the game. (In a superhero game, bullets bounce off Superman, for example. This means that you can have simulationist games of any premise.)
If a system is precise, it is both detailed (much correct information encoded) and accurate (produces correct outcomes).
A precise system is very hard to achieve, due to things like rounding errors that I mentioned previously.
Does that make sense?
On 1/13/2002 at 1:53am, Ron Edwards wrote:
RE: Accuracy vs. Precision
Hey,
I think this is a big issue that may well enter the Forge lexicon and engender much discussion.
Most of us here are probably familiar with the accuracy vs. precision concept, as applied to all sorts of things. What are they in terms of role-playing design? I think Paganini's suggestion works pretty well.
Comments? Examples, especially? Role-playing concerns?
Best,
Ron
On 1/13/2002 at 2:56am, Joe Murphy (Broin) wrote:
RE: Accuracy vs. Precision
Yeah, a few more examples would be lovely. I've just reread parts of the GNS essay (could not remember for the life of me what 'Karma' systems were) and Ron's examples enliven his theory.
Joe.
On 1/13/2002 at 4:20am, Logan wrote:
RE: Accuracy vs. Precision
It makes sense to me. We'll have to wait for someone to disagree before the debate begins in earnest.
On 1/13/2002 at 4:32am, hardcoremoose wrote:
RE: Accuracy vs. Precision
If a system is detailed, it has much information encoded into the mechanics. We assume that the information encoded into the mechanics is accurate, as this is part of the design process, and we assume good design in discussion of general design principles.
I'm specifically wondering what constitutes "much" information? If it's encoded into the game mechanics, are we talking rules-heavy, rules-lite, or something independent of such distinctions? I suspect the latter, but some concrete examples of precise and imprecise systems of all varieties would certainly be welcome.
- Moose
On 1/13/2002 at 6:00am, efindel wrote:
RE: Accuracy vs. Precision
Paganini wrote:
If a system is detailed, it has much information encoded into the mechanics. We assume that the information encoded into the mechanics is accurate, as this is part of the design process, and we assume good design in discussion of general design principles.
Why do we assume that the information encoded is accurate? I've seen quite a few detailed games (using the colloquial meaning, not yours) that weren't accurate.
For example, consider combat turn order in Runequest. Determining when someone can attack depends on their dexterity, their reach, the reach of their weapon, how far they have to move before they can attack, whether they have surprise, and whether or not their weapon is ready. There are rules about things like when a character can both move and draw a weapon at the same time.
I'd call this a detailed system, and probably most other people would as well. But how accurate is it? There's one major thing that Runequest leaves out in this determination -- skill. And most people with experience in hand-to-hand combat will agree that skill is a major factor in who goes first.
If we assume that information encoded into a game system is always accurate, then we'd also have to assume that a more detailed system is always a more accurate system. Neither of these is true, though, and I think any reasonable set of definitions for these terms is going to have to take that into account.
There's also a certain vagueness in talking about "the amount of information encoded in the system" (leaving aside the question of accuracy for the moment). Let's say I spend six months researching the results of 18th-century pistol duels, analyze that data to produce a set of percentages for how likely it is that a participant in a duel is killed, mortally wounded, wounded, or unharmed. I then use that data to make a dueling table for my game, which simply has percentile dice rolled for each participant and gives one of those results for him/her.
This table has a lot of information encoded in it (heck, it even has a lot of accurate information encoded). It probably has more information encoded in it than most actual RPG combat systems. But is this a detailed system? I'd say no.
So... what would I call a detailed system? Well, I'm going to hedge the question a bit, and say that there's more than one type of detail.
• One type of detail is explicitly taking into account more factors. For example, the Runequest combat turn order system explicitly takes into account several factors. A less detailed system might leave out some of those factors -- e.g., the character's reach.
• Another type of detail is having a fine-grained system. Using Runequest as an example again, it assigns weapons a number from 0 to 3 representing the weapon's reach. A system built on the same principles, but having weapon reach ranging from 0 to 9, is more detailed.
There's also a question of where the detail is. Some systems have a large number of inputs to resolution, but the only output is whether the attempt succeeded or failed. Other systems may have fewer inputs, but may have the output be a degree of success or failure. Still other systems may have multiple outputs -- e.g., attack resolution in combat might produce information not only about whether an attack hit, but also about the damage done, the area of the body it was done to, and secondary effects on the character (stunning, penalties to further actions, etc.)
This is probably a good place to introduce an idea I've talked about in other forums before: the three phases of a resolution method.
• Representation. In this phase, entities in the game world are assigned values to be used in the next two phases. Note that these values need not be numeric -- for example, a weapon's "type" could be a value. Nor need they be exact -- in a "systemless" game, a GM's feeling that "this factor is important, and is in favor of the characters" is its value.
This step may be more or less formalized. Some games only have assigned values for a few things, and leave valuation of any other factors to the GM; others have long lists of modifiers with valuations already assigned. Note that an implicit part of this is deciding what entities in the game world have significance to the thing being resolved.
As a further note, my usage of "entities" above is very loose. A character's strength or agility could be considered entities for this purpose, for example. This points out that often some entities in the game world will be "pre-valued"; with those, this step largely consists of deciding their significance.
• Resolution. In this step, the values that were determined as part of Representation are worked on in some way to determine an output. Again, this process may be more or less formalized.
In many (heck, probably most) RPG systems, a random factor is brought in at this point. However, it should be noted that this is not the only phase at which a random factor can be brought in. For example, an AD&D GM faced with the question of resolving a wrestling match between a PC and an orc might roll up a Strength for the orc on the spot; that would be a random roll in the Representation phase.
• Interpretation. This is the reverse of Representation; here, the output from Resolution is assigned a meaning in the game world. Interpretation might involve tables, more random rolling, or other such things.
It should be noted that these phases can be applied recursively or iteratively. In an iterative application, output from one event's resolution is applied to another event. An example would be the practice in Sorcerer of using successes from one roll as extra dice for another.
In a recursive application, resolving one event might require resolving another. For example, if the PCs decide to send a good-looking female character to the palace gate to try to distract the guard there by flirting, the GM might decide that the first important question is what sex the guard on duty is. The GM might then resolve this, with all three steps -- Representation (deciding what factors are relevant to the split of sexes guarding the gate), Resolution (either making a decision based on that, or perhaps rolling dice, or maybe having the PC make a luck check), and Interpretation (which in this case will probably be fairly simple).
Of course, separating out whether a use of multiple steps is iterative or recursive may not always be simple -- and, in truth, it probably isn't very important. Any good computer programmer will tell you that you can simulate iteration with recursion, and vice-versa -- so while the distinction may be academically interesting in RPG resolution systems, it's not likely to be of much practical use.
Relating this back to detail -- with these terms in hand, we can now say things like "this system has a detailed representation system for X, but leaves interpretation largely up to the GM." IMHO, this is more informative than simply trying to say that a system is or isn't detailed.
--Travis
On 1/13/2002 at 2:54pm, Paganini wrote:
RE: Accuracy vs. Precision
hardcoremoose wrote:
I'm specifically wondering what constitutes "much" information? If it's encoded into the game mechanics, are we talking rules-heavy, rules-lite, or something independent of such distinctions? I suspect the latter, but some concrete examples of precise and imprecise systems of all varieties would certainly be welcome.
Let me expand on my definitions. :)
We're pretty much talking "rules-heavy," although that distinction may be a bit of a misnomer. You can have a "mechanics-light" system that is still "information-heavy," if you follow me. As an extreme example, consider a system that only has one mechanic... rolling d% on a chart. This is a mechanics-light system. It only uses one simple mechanic. But at the same time, the entire book could be a huge mass of charts to roll on. You use different charts for different situations, they're crosslinked (like, "go to chart X, reroll and add 30") and so on. You *use* all these charts in the same way, with a single simple mechanic, but there is a lot of information encoded into the game.
Similarly, a mechanics-heavy will often have the information encoded with mechanics. In the previous example, falling damage would be determined on a d% chart like everything else in the game. In a mechanics-heavy game, falling damage might be determined by rolling a d6 for every yard fallen, while fire damage might be determined by rolling a d6 and mutiplying by the temperature. Whatever... these are just arbitrary examples. IME, the most common way to encode information into a system is with the mechanics. So, "rules-heavy" and "detailed" are not quite synonyms, but they're close.
On 1/13/2002 at 4:10pm, Marco wrote:
Precision vs. Accuracy
Precision (in slightly general terminology) is how *much* information is included in your measurement (i.e. number of significant digits). Accuracy is how correct it is. A computer can give you a highly precise low-accuracy answer if you feed in the wrong data.
(I know the chemistry/physics definitions of precision vs. accuracy against a known-true result: I think that level of definition isn't as useful).
These are the definitons I'd use for Precision and Accuracy with RPG's:
A Precise system is one that gives you a lot of detail about what happened (I hit him on the arm, causing a % chance of weapon droping, a % chance of shock, and these prevailing combat modifiers).
An Accurate system is one that "correctly" models the gener (or Premise, here, I guess).
So if you get lots of data about what major vein was hit when the bullets rip through Superman's skin you have a precise but not accurate system. If you know the attack failed (and always will fail with a hand gun) but the system leaves it to the player or GM to tell why it failed you have a precise but not accurate system.
-Marco
NOTE: Most S/G games (in my experience) have vastly different levels of precision for combat and non-combat but about the same accuracy.
On 1/13/2002 at 6:27pm, Paganini wrote:
RE: Precision vs. Accuracy
Marco wrote:
These are the definitons I'd use for Precision and Accuracy with RPG's:
A Precise system is one that gives you a lot of detail about what happened (I hit him on the arm, causing a % chance of weapon droping, a % chance of shock, and these prevailing combat modifiers).
This is only useful so far though, because there are two kinds of accuracy WRT RPG systems. One kind of accuracy is accuracy of results. If the results the system produces are accurate, then the system can be said to be accurate. OTOH, a system can *also* be said to be accurate if the information encoded in it is correct. Such a system can still achieve off the wall results, in spite of the accuracy of encoded information.
On 1/13/2002 at 6:44pm, Paganini wrote:
RE: Accuracy vs. Precision
efindel wrote:
Why do we assume that the information encoded is accurate? I've seen quite a few detailed games (using the colloquial meaning, not yours) that weren't accurate.
Well, here's may answer. This is all IMO, though, so you may not agree. :)
To me, it's fruitless to discuss general design principles without the preassumption that the design is implimented without mistakes. If I make a comment about a general design principle, it's not very useful for someone else to say something like "that's not true, because system X uses that principle, and it does blah blah blah that doesn't work."
This sort of thing is usually given in the spirit of a counterexample, but IMO it's not very worthwhile, because it's more of a straw man than a counterexample. Citing a specific instance of bad application doesn't refute, or even alter, a statement regarding a general principle. It's sort of boils down to:
Me: "Design principle X works well for Y."
Them: "No it doesn't, because game Z is poorly designed!"
For example, consider combat turn order in Runequest. Determining when someone can attack depends on their dexterity, their reach, the reach of their weapon, how far they have to move before they can attack, whether they have surprise, and whether or not their weapon is ready. There are rules about things like when a character can both move and draw a weapon at the same time.
I'd call this a detailed system, and probably most other people would as well. But how accurate is it? There's one major thing that Runequest leaves out in this determination -- skill. And most people with experience in hand-to-hand combat will agree that skill is a major factor in who goes first.
Using the train of thought above, I would probably say that Runequest attempts to be a detailed system but falls short of the mark. Thinking about this, I see where you're coming from. This could be confusing in long term use, since people might tend to think "detail" simply as the amount of information encoded, independant of the accuracy of that information.
If we assume that information encoded into a game system is always accurate, then we'd also have to assume that a more detailed system is always a more accurate system.
Exactly! That's the point I was trying to make on RPG-Create that Bradd didn't get ahold of for a while.
Now, having made these definitions, and identified a triend (more detail = more accuracy), then we can identify a continuum. Absolute detail and accuracy is probably impossible.
But now we have some terminology that we can use, for example to describe Runequest. It tries to be a precise system, but errors in the detail stop it short of that goal. (I haven't read Runequest, so I'm just basing this on your description... the statement might not be true, I'm just using it as an example of how the terminology could be applied.)
(much snippage)
I think I covered the "what is encoded information" issue in another post.
Relating this back to detail -- with these terms in hand, we can now say things like "this system has a detailed representation system for X, but leaves interpretation largely up to the GM." IMHO, this is more informative than simply trying to say that a system is or isn't detailed.
(Kung Fu mode)
Your terminology is pretty good...
(/Kung Fu mode)
:)
The three stages of detail idea goes deeper than my definitions do, while staying on the same line of thought. I think it could probably be combined with mine for maximum effect.
On 1/14/2002 at 1:56am, efindel wrote:
RE: Accuracy vs. Precision
Paganini wrote:efindel wrote:
Why do we assume that the information encoded is accurate? I've seen quite a few detailed games (using the colloquial meaning, not yours) that weren't accurate.
Well, here's may answer. This is all IMO, though, so you may not agree. :)
To me, it's fruitless to discuss general design principles without the preassumption that the design is implimented without mistakes. If I make a comment about a general design principle, it's not very useful for someone else to say something like "that's not true, because system X uses that principle, and it does blah blah blah that doesn't work."
I agree and I disagree. On the mud-dev mailing list, I've talked quite a few times about "separating ideas from implementations". The fact that an idea can be handled poorly doesn't mean that it's a bad idea. On the other hand, though, the fact that an idea would work well if it were done perfectly doesn't mean that it's a good idea.
At some point in design, you have to move from theory into practice. If you started out assuming that you're going to be able to implement everything perfectly, then you may run into some rude surprises. Things don't work perfectly in the real world, and trying to design a system -- any system, not just a game system -- as if things always will work perfectly just isn't a good idea, IMHO.
Now, obviously you can go overboard and be too pessimistic, saying nothing's going to work. There's a middle ground that has to be found between blind optimism and "it'll all just get screwed up" pessimism.
A second point is that I'd like for a terminology and theory about game design to be useful not only for designing new games, but also for talking about existing games. In order to do that, we have to recognize that sometimes mistakes do get made. (Of course, in doing so, we also have to be able to look at other perspectives -- sometimes, what's a mistake from one perspective is a very good feature from another.)
Paganini wrote:
I think I covered the "what is encoded information" issue in another post.
Yes, you did, but my question still remains. Let me try explaining it in a different way:
When you encode information to make it easier to apply, you often lose some of the information in the process. To take my example from before, the dueling table encodes information about hundreds or thousands of actual duels. However, there's no way that you could reconstruct all of that information just from the table -- thus, information has been lost in the encoding. To put it what I hope is a better way, then: is detail, in your definition, the amount of information that was used to create a system, or the amount of information that can be gotten back out of the system?
If it's the latter, then the definition could work for me. If it's the former, then it doesn't work for me. I think you probably mean the latter, but your original text and your explanations so far haven't been clear to me on which one you intend.
Paginini wrote:
Relating this back to detail -- with these terms in hand, we can now say things like "this system has a detailed representation system for X, but leaves interpretation largely up to the GM." IMHO, this is more informative than simply trying to say that a system is or isn't detailed.
(Kung Fu mode)
Your terminology is pretty good...
(/Kung Fu mode)
:)
The three stages of detail idea goes deeper than my definitions do, while staying on the same line of thought. I think it could probably be combined with mine for maximum effect.
Umm... small correction: It's not three stages of detail, it's three stages of resolution.
--Travis
On 1/14/2002 at 5:35am, Paganini wrote:
RE: Accuracy vs. Precision
efindel wrote:
When you encode information to make it easier to apply, you often lose some of the information in the process. To take my example from before, the dueling table encodes information about hundreds or thousands of actual duels. However, there's no way that you could reconstruct all of that information just from the table -- thus, information has been lost in the encoding. To put it what I hope is a better way, then: is detail, in your definition, the amount of information that was used to create a system, or the amount of information that can be gotten back out of the system?
Wow. I hadn't even thought of it like that before. Now that you make this distinction I see what you're saying. I guess I was thinking in terms of information used to create the system, not information that can be extracted from the system. When you lay it out, though, it makes more sense to talk about the information that can be gotten back out of the system, since that's the information that will probably be the most recogniseable. Whether or not it would be the most useful, I'm not sure.
On 1/14/2002 at 8:22pm, Mike Holmes wrote:
RE: Accuracy vs. Precision
Yep, I thnk that we have several things going on here.
First I think that most of this talk of detail can be broken down into where the detail resides. We've already seen design detail (encoded information in th system), resolution input detail (how much information is required from the player as to his characters actions), resolution output detail (how much information the system produces). We can also look at things like stats. Sorcerer has greater detail for stats than many games, as, in addition to the number describing the stat, the player must select a descriptor for that level of ability (two data,instead of the usual one). Some systems have lots of statistics which makes them detailed. Some smight say that some systems are overly detailed. I think that Jared Sorensen works hard at achieving the right amount of detail.
There are lots of other places we can use the detail term I think.
I wouldn't speak of precision, as that has two distinct meanings (one implies accuracy, the other distinctly does not). If you want to say that a system is detailed and accurate, comment on those points separately. Accuracy has a meaning that I think we can all agree on.
This all also relates back as one poster pointed out to granularity, which has been well discussed previoulsy. Granularity does not entail more detail, but finer detail. A system that has three levels of health (say concious, unconcious, and dead) is at a different granularity than one that has five. You get the same amount of detail (one datum), but the first is more coarse than the second. BTW, as a reminder, when refering to granularity use Fine and Coarse. Everybody gets what you mean then. More and Less, for example, are confusing when refering to granularity.
Just my two cents,
Mike