Getting accurate estimations from players

[M. J. Young]

Just to clarify the question--

[*]You aren't asking players to rate how good their characters are at something;[*]You are asking how important something their character is or does will be in play, in terms of how advantageous or disadvantageous it will be.[/list:u]
Here's a thought.

What I see you trying to get from the player is his own prediction concerning how useful/valuable or how detrimental each characteristic of his character is going to prove to be in play that has not yet happened. You're trying to get his honest assessment. You specifically want to avoid having the referee make that judgment at the outset, even though there is a mechanical means of making that judgment once one session has been played, based entirely on how useful or detrimental each factor in fact was in that previous session.

Why not make it a corporate decision? Once all the characteristics are on paper, make a list of them, give a copy of the list to each player, and have each player rate each of those characteristics in terms of how valuable or detrimental he thinks they will be. Then use those results. The referee might or might not be included as one of the voters. Depending on how many players you have, you could use any of these values as the actual value:

[*]The arithmetic mean, that is, the average--add everyone's estimate together and divide by the total number of players. This lets everyone impact the assessed value of the ability.[*]The average of all answers except the extremes--that is, toss out the highest and the lowest, and average the rest. This eliminates the player who decides to skunk everyone else, and the player who decides to advantage himself, either of whom might provide a value way out of range in order to shift the average up or down as desired.[*]The mode, that is, the number that occurs most frequently. I have a lot of trouble with the mode in small data pools though--the odds for example of two out of five people giving the same number on a twenty-one point scale (-10 to 10) are pretty slim, and to hope that this represents a common understanding of the value is really pushing things. If you had a large group and a small range, that would work, but that doesn't sound like it's going to happen here.[*]The median, the answer that appears exactly in the middle when all the answers are placed in sequence. This may be the best choice, as it inherently eliminates all the extreme answers. If three out of five players think the ability is a ten, it's a ten, even if the other two gave it a neg ten, because the middle answer is ten. Of course, you'd have to decide up front in the case of even numbers of participants whether you take the one above the middle or the one below the middle, or the average of the two (which may also require a rule for rounding). It also would not work so well if there are only three players, and not at all for two.[/list:u]There may be other ways to determine the answer from this data, but these strike me as the simplest.

I know that this doesn't really give you the player's honest assessment; however, it may give you the players' honest assessment, which may be sufficient for your purposes.

I hope that helps.

--M. J. Young

[Morgan]

A thought: Once a player has evaluated their attributes, the GM "buys" one attribute of the GM's choice, at the player's estimated value. The character no longer has that attribute, and the player gets that attribute's "value" in some metagame currency.

This wouldn't work in a vacuum, of course -- players would tend to overvalue everything. But since your concern is that your system will push them to undervalue everything, maybe it could balance that.

It's worth noting, though, that players will be evaluating their character's attributes' value to them, but not necessarily their value in-game. If you have a player who has a firm concept of who and what they want their character to be, but little interest in "winning," they might rate apparently trivial attributes higher than those that provide in-game advantages.

Also, it doesn't work on negatively-valued attributes. Even if they had a starting fund of the necessary currency, a player isn't going to want to pay the GM for the privilege of losing an attribute they chose -- even if it's technically a disadvantage.

So it's not perfect... but maybe you can do something with it.

[John Kim]

My first thought is that you imply that the usefulness of an ability is something that the player secretly knows but might not be honest about.  I don't think that's the case.  How useful an ability turns out to be depends a lot on everyone else in the group (i.e. what the GM does and what the other players do).

I think he's right. Most people are a little uncomfortable revealing their true preferences, as a defensive measure arising from the fact that almost all people are political animals prone to taking advantage.

I think we're misunderstanding here.  Even if he is inclined to cover up his preferences, I meant that the player generally doesn't know how useful a given ability will be. Given that the ability definition is open, the player has relatively limited secret knowledge.  For example, let's say a player chooses a "Silver Tongue" trait.  How the ability is defined and adjudicated is open to everyone.  Now move into play.  The other four PCs all turn out to be extremely violent types who make enemies of everyone.  So the game becomes combat dominated.  

One concept from several games (notably Theatrix and The Babylon Project) is to evaluate advantages/disadvantages during play.  That is, when an advantage proves useful to you, you have to pay out points for that use.  Conversely, if a disadvantage proves to be a real hindrance to you, then you gain points at that time.  That's the principle, at least.  There are a number of pitfalls.  (1) There is the possibility of players encountering "Well, I logically should be able to do that, but I'm out of points."  (2) Trying to evaluate usefulness in play adds more decisions and bookkeeping to action, thus slowing down play.  (3) This system in a sense penalizes creatively finding uses for your abilities -- or at least there is no advantage to doing so.

Nobilis has a system like this for disadvantages, but the pitfalls you suggest aren't the ones that actually arose in practice. (1) and (2) don't apply in a point-bidding based system like Nobilis -- the characters have a base level of competence they can spend points to increase, and the point-spending *is* the mechanic so there is no extra time lost.  And (3) doesn't happen because you can't solve a problem using an ability without a justification for it, so you still need to be creative in finding a way to make it applicable to the problem.

Instead, what most often happened is that the players played to their restrictions and then simply forgot to remind the HG to give them the points for it. I've seen this happen again and again as both a player and a game moderator.

I'm not familiar with Nobilis, but this sounds related to my #2, actually.  That is, the players are not putting in the time and attention needed for the system to actually work.  Here time isn't just the time to perform the physical motions, but the time/attention to mentally consider and register the system.

[Hudson Shock]

M. J.:  The methods you give are all subject to player collusion - everybody agreeing to minimize values to get the most out of it.  (Again, something I think that happens automatically, not because of people being deliberately dishonest.)  But something about them has a little back area of my brain percolating, which usually means that my subconscious has seen something not yet obvious to my conscious.  By the way, I wasn't really intending to use a 21-step scale to measure things, I just threw that in as part of the "what if".

I suspect the bigger stumbling block would be time - that's a lot of data for a group to go through before being able to start the game.  But maybe it could be simplified down by rating the character as a whole, instead of by each individual attribute.

Morgan:  there's definately something to your method, balancing competing drives to maximize and minimize at the same time.  Thanks; I'll play with that some.

[Sydney Freedberg]

I've been struggling with this same issue in My Eventual Game, and in sheer desperation wondered about turning the whole thing on its head:

Don't try to evaluate how often a given ability will be useful and price it from there. Instead, let the player say how often s/he wants an ability to come into play and let him/her price it according to that. If the player pays 90 character points per level of "knowledgeable about rutabagas," then, darn it, you as GM are now obligated to intrude rutabagas in some meaningful way into 90 percent of that character's scenes. If the player pays just 10 character points per level of "master swordsman," then you as GM only involve him in swordplay 10% of the time*.

* Which would be appropriate to depict, say, The Man in the Iron Mask , where all the main characters happen to have terrifying sword skills -- they used to be the Three Musketeers, after all -- but that's really incidental to the political intrigue that is 90% of the story.

[simon_hibbs]

Suppose abilities are rated according to how good the character is at them. Players are likely to put more points into the abilities they think are most useful. Many games do this. If I create a GURPS Traveller character and buy a skill level of 18 points into Handgun and 15 into Medic, it means I think being able to shoot a handgun well will be more useful than Medic skill.

If you don't want to have ability ratings directly related to percieved utility in this way, you could make the points the players allocate relate to the advancement system instead of initial ability ratings. The more points the player invests in a skill, the mroe easily it will be advanced in-game. That way you can use a different mechanism to determine initial ability ratings, but still get a fair assessment of how useful the players think the abilities will be in the long term. Also, it'[s only fair the players get some reward for doing all this work for you.


Simon Hibbs

[Hudson Shock]

Thanks for the ideas, everybody.  You've given me plenty to think about.

[Doctor Xero]

Thanks for the comments and your time.  But - and I don't want to seem ungrateful - comments that boil down to "there's no reason to do this" aren't really helpful.

There's a problem with your approach, however, and that is the lack of context.

How much I value a particular attribute or ability will vary according to the campaign genre and the game master's habits of running a game.

I might consider Style far more important an attribute than Self-Discipline in a swashbucklers-and-intrigue game, but I am more likely to value Self-Discipline than Style in a Cthulhuan game with temptations everywhere my player-character turns.

I remember one character-building session at which we were building characters for two different campaigns and neither of the game masters were present.  That was fine, since we were all a trustworthy lot, but it also gave us a chance to gossip about both game masters' styles as we constructed our characters.  We knew that one game master always defaulted to combat even in the alleged intrigue-not-battle campaign he was running, so we all knew enough to allocate considerable amounts of character creation points to defensive abilities and other combat-oriented abilities.  The other game master was an otherwise brilliantly creative woman who had a weakness for including supernatural seductresses who required astonishing saving throws, but never supernatural seductors, so everyone created female player-characters, male eunuchs, or gay male characters as the only safe creatures.

Would we be dishonest, as you imply, to tell her that we didn't value combat attributes that highly, even though we poured creation points into them for the guy's campaign?  No, because for her campaign, we honestly didn't see combat attributes as valuable or relevant.  On the other hand, if someone wanted to play a swaggering male romantic, she would play him only in the guy's campaign as playing him in the woman's campaign was only asking for trouble.

Without context, valuation is impossible.  Starving and trapped in a vault filled with gold, a sandwich matters more to me than a solid gold crown : context matters.

Doctor Xero

[neelk]

How much I value a particular attribute or ability will vary according to the campaign genre and the game master's habits of running a game.

Yes, and I think that's why auctions are interesting as game mechanics. One of the fundamental ideas of economics is called revealed preference: basically, it says that what people really want is what they choose when they get the chance, rather than what they tell people they want in surveys. Talk is literally cheap.

A good auction mechanism can be a knowledge-discovery tool to learn how much the players value different things in a given game. This is valuable because learning the players' true expectations is also a useful in changing the norms in the group for new games: we can see what areas we have to consciously focus on to develop the game in the right way.

[Hudson Shock]

How much I value a particular attribute or ability will vary according to the campaign genre and the game master's habits of running a game.

You are exactly correct, but I already addressed this:

(I also recognize that this does not reflect an "objective" value. A person may rate being a catburglar as very useful, but obviously if the adventure is all about court romance, it will be less so.)

The whole point behind my developing system is that it self-corrects for mismatches between GMing styles and player choices.  I'm simply seeking an acceptable solution for the first session, when no data on how the game actually goes is available yet.

The reason I'm hoping to be able to use player evaluation is mainly because, IMO, it removes some of the sense of adversarial-ness with the GM.  If the players guessed wrong, and things that they thought would be useful turn out not to be, well, they have no one to blame but themselves.  Besides which, it will correct itself quickly in later sessions.

(You may disagree whether this actually does remove the adversary feeling.  But it's the theory I'm going with.  Most of the game design so far stresses player choices in the things that directly affect them.)

I might consider Style far more important an attribute than Self-Discipline in a swashbucklers-and-intrigue game, but I am more likely to value Self-Discipline than Style in a Cthulhuan game with temptations everywhere my player-character turns.

Well, realistically, you'd have some notion of basics like genre and such.  You might even have a very detailed description from the GM as to what kind of game he intends to run.  The more information, the more accurate your estimation, of course.  But the point is still to make it your estimation, not the GMs.

Would we be dishonest, as you imply, to tell her that we didn't value combat attributes that highly, even though we poured creation points into them for the guy's campaign?

Just the opposite.  You'd be dishonest if you didn't tell her you didn't value combat.  And you'd be a little dumb, because you'd be passing up a point to get the Fortune Points that balance out useless skills.

More to the point, you'd be dishonest if you said you didn't value "male eunuch" as an attribute, so that you could get more Fortune Points.  I'm looking for a system that gets people to admit that while, normally, "eunuch" might be a disadvantage, in this situation they expect to it to be a plus.  Get it?

[Doctor Xero]

The whole point behind my developing system is that it self-corrects for mismatches between GMing styles and player choices.  I'm simply seeking an acceptable solution for the first session, when no data on how the game actually goes is available yet.

This only works if I have never played with that game master before nor with those particular players.  Otherwise, my knowledge of previous experiences with the game master(s) and players will alter context for me.

Ex: if I know Bill is going to be in the game and Bill always plays elves who hate gnomes, I will either avoid playing a gnome if the game master said the campaign needs characters who do not hate each other -- or discuss it with Bill and see if he can play a character who doesn't hate gnomes.

You'd be dishonest if you didn't tell her you didn't value combat.  And you'd be a little dumb, because you'd be passing up a point to get the Fortune Points that balance out useless skills.
More to the point, you'd be dishonest if you said you didn't value "male eunuch" as an attribute, so that you could get more Fortune Points.  I'm looking for a system that gets people to admit that while, normally, "eunuch" might be a disadvantage, in this situation they expect to it to be a plus.  Get it?

The above is the crux of why I disagree with your way of containing/shaping your inquiry.

I would be dishonest if I told her I didn't value combat, because in truth, when I am playing in the guy's game, I do value combat.  It is in her game alone that I don't value combat.  Similarly, I would be dishonest if I told her that I value "male eunuch" as an attribute, because that is not a default valuation.  Only in her functioning-heterosexuals-get-zapped campaigns do I value it.

In other words, the big flaw I see in your argument is that you conflate contextual valuations with default valuations.  For many of us who have been gaming for a decade or longer, we have few if any default valuations.  Don't expect me to build a character without a firm sense of context, and don't expect my character preferences in one campaign to be the same as my character preferences in another campaign.  (We had this problem once when a game master asked me to build any sort of superhero character I wanted but refused to give me any context: within an hour, I had fifteen fully detailed character conceptions, each of which would have been great fun in one context and misery to play in another.  Without context, I have no basis for valuation.)

Default valuations and contextual valuations are not identical, and experienced gamers tend to have few default valuations after having played so many types in so many genres under a diversity of game masters and in a diversity of gaming groups.

Doctor Xero

[Hudson Shock]

"In other words, the big flaw I see in your argument is that you conflate contextual valuations with default valuations.

I also recognize that this does not reflect an "objective" value. A person may rate being a catburglar as very useful, but obviously if the adventure is all about court romance, it will be less so.

and

Well, realistically, you'd have some notion of basics like genre and such. You might even have a very detailed description from the GM as to what kind of game he intends to run. The more information, the more accurate your estimation, of course.

and

I'm looking for a system that gets people to admit that while, normally, "eunuch" might be a disadvantage, in this situation they expect to it to be a plus.

If these three quotes don't make it clear that I'm not confusing context with default/objective values, I don't know what will.    I'm agreeing with everything you're saying, and yet you still think I'm missing something obvious.

I suspect we're actually on the same page, but on the chance that you are seeing something I don't, explain it to me one more time:

Given the third quote again, the one that emphasizes "this" situation being valued differently than another, and the second quote that clearly states that value judgements  will be more accurate with more information - why do you think I don't understand the value of context in making a decision?

[Doctor Xero]

If no one can't think of a way to pull this sort of truth out of people, that's okay.  Neither have I, so far.

You'd be dishonest if you didn't tell her you didn't value combat.

why do you think I don't understand the value of context in making a decision?

Because, as the earlier quotes I've given shown, you keep conflating the player response with some sort of universal honesty, and this conflation undermines your acknowledgements of context.

As soon as you tell players you are trying to keep them from being dishonest about their preferences, you imply that you demand from them a stance from which variation is not permitted.  Without variation, context no longer becomes an influence.  It's a self-defeating way of expressing oneself.  I apologize if I'm too blunt in saying so.

Now, simply critiquing what you've written hardly deals with your topic and can come across as rude, so I won't waste Forge space doing only that.  Here's a suggestion how to handle your concern:

Have players write down on their character creation sheets not only which attributes they choose but why[/b].  Make it clear that they are to write either why they want a specific attribute for this specific campaign or why they generally want a specific attribute.

For example, one person might write that she wants a high Agility because she imagines her player-character as leaping from bannister to chandalier and thence to surfing the tavern bar, while another person might write that he wants a high Agility because it makes it easier to duck and dodge in a campaign run by a game master far too fond of combat, and a third person may be a clumsy individual who wants a high Agility just to feel graceful as displaced through the player-character.

It asks them to explain themselves rather than asking them to verify their honesty about their choices.

I hope this helps.

Doctor Xero

[M. J. Young]

Here's a crazy idea that's probably not workable on one hand and doesn't give you enough information on the other, but take it as brain food.

I gather that there is no fixed number of "attributes" to be listed. Try this. Tell the players to choose their attributes, affix values to them, and then to list them from the most important to the least important in defining their character for this game. Indicate/imply that there's going to be a die roll which will determine how many of the listed attributes they actually get to keep.

There doesn't actually have to be a die roll, as long as they don't know there isn't going to be one.

This doesn't get you specific valuations, but it does get you relative valuations as a ranking. If from that you can derive the values you want, you're there.

--M. J. Young

[Hudson Shock]

Brain food indeed.  Interesting.  Hmm...

(Sorry if I haven't given much feedback on the ideas you have all given me - I haven't had much time to brainstorm on my system over the past couple of weeks.  I'm just glad these posts are here for me to come back to when I can.)