My normal playtest group in Blacksburg is mostly board game players, but I got eight of them to indulge me on 1/22 in doing self-assessments to see how well the system created plausible characters. In this case, plausible would be a self-assessment that generated a character that everyone agreed was similar to the person doing it. To add a little suspense I had the players do the ten questions in the rules for themselves to generate Aspects, and then everyone passed characters to the left and that person figured out the Traits and Beliefs.

The most accurate characters were ones with strong, tiered Aspects, like a 4, 3 and 2. These generated Traits and Beliefs that were generally agreed by the player to be representative. Only a few were dead on, but a number of them were still on target. The strict Roman Catholic player did not have a high "religious" score, but they did end up as "militant" and “traditionalist”. The self-described Libertarian did not have a high "libertarian” score, but did turn out as a "center-right capitalist”. So, I was pretty happy with that.

On the other hand, we had some characters who had a number of similar Aspects, including someone who had four Aspects at +1 and one at +2. Since the player choose the order of ties when doing Traits, and Beliefs are based on the number of check marks from Aspects, the results were all over the place, with only a random correspondence to the actual personality and belief of the player.

On the whole, I'd say it was a success, but I need to adjust the character generation in two ways: First, set it up so that the highest three Aspect ratings are unique, like a 4,3,2. This will require some tweaks to the basic and advanced generation rules. Second, as part of this, I need to add a "how important is this to you?" qualifier on the advanced questions. Some people said that some questions they had strong opinions on, but that they did not feel the subject of the question itself made much difference in how they thought and acted. That is, it was in the "I believe in X, but it wouldn't change the candidate I voted for" sort of importance.

So, adding an “importance" factor should help differentiate the personality results.

After the character generation, I went into the concept of the game and while we did not run a crisis, I presented one and we had a bull session on it, which involved the personalities of the players. Seeing the players go back and forth and adding in secondary complications based on their conversations was interesting, and it almost went into the negotiation-based mid-game for crisis resolution.

There were also a few mechanical issues about clarity on the character sheet and table layouts not being as clear as they could be, but those are minor issues. Making sure that characters have a few uniquely strong Aspects is going to generate some new quirks in the rules, I think, but I also think it is necessary to make sure that characters are believable. So, back to editing and I'll do a followup here once I have a revised set of rules.

Greg

Just sharing some feedback (which I've already substantially emailed to Greg) in a rephrase.

Tried it as a "Before the regular game game".

The current advanced aspect generation questionnaire gave us good, but not congruent with self-description, ratings; ratings which we all felt were only slightly surprising, and far more extreme than our expectation (Risk 4 for me? Wow... I figured risk 1, maybe 2!)

The checks/belief system is effective, but cumbersome. Automating it really helps; without automation, it's too slow for use, and needs dedicated CG worksheets.

The basics of the task system work well for group tasks. No one was upset about it.

The traits table works well if one gets the lookups in the right order. My wife, we didn't first try, and it resulted in very wrong traits; swapping the equally numerically rated 1st and 2nd traits, and it hit dead on. Similar for me.

Of the listed beliefs, when I ran the checks system, one's I'd accept were the 3rd and 4th highest, rather than the two highest, but I might be a statistical anomaly.

Love the concept and the setting, the execution is (by what I've come to expect of Greg's initial playtest releases) a bit on the rough side.

Character sheet is gorgeous, but worthless off a B&W printer.

Looking forward to next draft.

Hi Greg,

I think you're running into a few conceptual limitations.

1. I don't know why any attention is going toward the notion that a person will come full circle through this analysis and see his own preferred label looking at him. If that were the case, it'd be a useless exercise.

2. Regarding the game, and building the fictional organizations, I think that it's inaccurate to think that one's chosen group to work with is a mirror of one's most heartfelt convictions, or else one is a hypocrite.

Now, that's not a criticism of using the personality devices to make up fictional groups; that's wholesale fun. My concern is that there's some language in the rules about hypocrisy which, as I see it, doesn't need to be there. 

3. My next point is a little tricky and subject to a lot of debate. In the real world (geez, only here would we have to specify that), I think our political discourse runs into speedbumps concerning "freedom/rights" as a rhetorical tool. It's especially hard in discussions among somewhat-radical right libertarians and somewhat-radical leftist reformers. Both are, effectively, civil liberties advocates who tend to agree on many details, and yet each side will clutch its favorite source text (Atlas Shrugged and The Monkey Wrench Gang, respectively) as the sine qua non of the respective position, perceiving the others are obviously deluded, if not outright monsters.

I'm seeing those speedbumps in your methodology. At the moment, it seems as though scoring high on certain radical-liberties metrics moves one toward the libertarian labels, whereas one is moved toward the socialist and evironmentalist labels mainly via group-oriented, help-others-oriented metrics.

I'm going out on a limb and providing the full breakdown of doing all this for (on?) myself.

Aspects: Solo +2, Intellectual + 4, Chaos +1, Empathy + 4, Risk +2 (incidentally, I didn't realize what a pain in the ass I'd be making for myself with all the ties, but at least it shows I wasn't peeking ahead at the system when I chose the values)

Intellectual & Risk: inventive; then Risk & Chaos: informal

Fear: betrayal
Motivation: justice
Beliefs: Family, Beliefs, Self, Friends, Humanity, Nation, Race

Cultist 0
Religious 4
Spiritual 4 + 1 = 5
Agnostic 3 + 4 = 7
Atheist 3 + 2 + 3 = 8
Reactionary 0
Conservative 1
Center-Right 1
Libertarian 5 + 2 + 3 = 10
Centrist 3 + 4 + 3 = 10
Center-Left 4 + 1 = 5
Progressive 1
Liberal 4 + 1 = 5
Socialist 5 + 1 = 6
Communist 0
Anarchist 2 + 1 = 3
Green 5 + 4 + 1 = 10
Rational 5 + 2 = 7
Cynical 2 + 3 = 5
Capitalist 2 + 3 = 5
Reformist 5 + 4 + 1 = 10
Traditionalist 1
Conformist 5
Evangelical 1
Insular 5
Militant 2 + 1 = 3
Pacifist 5

(Note: I seem to have garnered rather higher values than others are reporting. I'm doing this right, right?)

OK, all this turns out to be consistent with no less than FIVE pairs: Libertarian Green, Libertarian Reformist, Centrist Green, Centrist Reformist, Green Reformist.

Now, when I did this, I was thinking of a fictional character, so I chose Green Reformist due to the "inventive and informal" tag for what seemed to me to be the most fun to play.

But let's look at it in the way I was suggesting we not do and see whether it "circled back" on me and my preferred personal labeling, which sadly, is not especially intense. I see the "reformist" tag - I'm a pretty intense activist on a few fronts - but that's about it in terms of simple labels. I can see a couple of things that seem like they might fit, such as the fact that I have some righty-friends that I respect more than some of the people involved in causes I support.

It makes a lot more sense when I draw the distinction group vs. person, and find that I think these exact things are in disjunct between group ideal vs. genuine activity. Hence I don't participate in libertarian politics because I think the groups involved, and all but a few individuals, are bigoted closet-conservatives; and I don't participate in much green activism because I think most groups involved, and all but a few individuals, are empty-headed science-phobes; and I don't participate in socialist activism (at least not the chapter-and-verse form) because with very few exceptions the groups involved, and all but a few individuals, are self-centered, bourgeois fuckazoids.

And on a related note that I forgot to mention, I don't think any of that makes me a centrist. I hate centrism like poison but I've been bumped there in this analysis via the methodology.

OK, going on with the process. Group size = -4, about 6%. University professor 2, Fighting 1, Game design 1.

1. How likely is your character to start a conversation with someone outside your immediate circle?
very likely: -2
2. How likely is your character to make sure others are comfortable and happy?
somewhere in between: +0
3. How likely is your character to use specialized jargon or esoteric words in conversation?
very likely: -2
4. How likely is your character to prepare things in advance?
somewhere in between: +0
5. How likely is your character to feel blue or depressed?
somewhat unlikely: +1
6. How likely is your character to plan social or other events?
somewhere in between: +0
7. How likely is your character to say things about people that they would consider insults?
somewhat likely: +1
8. How likely is your character to think about philosophical questions or social issues?
very likely: -2
9. How likely is your character to let things get into a mess or disorderly state?
somewhat likely: +1
10. How likely is your character to feel stressed or worried?
somewhere in between: +0

Solo 2, Intellectual 1, Apathy 1, Risk 4

Traits: Unexcitable, Unemotional, or Versatile for #1; Seclusive, Sedate, or Independent for #2
Again, if we're talking merely about making up a character, then any of the three for #1 and any of the three for #2 - probably not Unemotional and Sedate together because that's kind of a boring character, but otherwise, any combination would probably be fun.

But in terms of personal profiling, I don't know if this batch of three-and-three is particularly strong. If the idea is that one term out of the three (each) is all that matters, then versatile and independent might do. I'm a bit more sedate than I used to be, sure. Otherwise, the traits are pretty far off. Again, though, since the purpose here is to make a character and not to see yourself, or even worse, your self-image looking back at you, I'm not criticizing this as game design.

I am, however, suggesting that the whole "and does it look like me" at the end, which is understandly irresistible, doesn't have to be a design consideration.

Best, Ron

I think the issue is one of the data collection method being flawed

People are, generally, lousy self evaluators.

Greg's data collection (referenced in Crowdsourced RPG research) relied upon people simply asserting which elements were strongest associated with the labels, in an ordered list.

In looking at the result, I think it would be more accurate to have people take the inventory (possibly using a poll on a website), and correlate those various labels to how they rated from the tool, rather than by self evaluated.

My players decided not to play themselves, tho. And it's just fine in that mode. (aside from desperately needing automation of the checks system.

The point of the self-evaluation sessions was not to generate people playing themselves, but to see if the generation process could or would generate PC's that felt "real" in terms of the personality and beliefs and motivations. Getting a group to self-generate gave me a good sample for both whether the PC represented the character from the player's point of view, and also from the external point of view of the other players. It also gave me a bunch of new data points to add to my table for seeing what Aspects correlate best with what beliefs.

From a mechanical standpoint it was good testing of how the mechanical aspects of character generation worked. That brought up things like the muddy character sheet background, confusing tables, etc., but also the very excellent point on the questionnaire that it is not just about how you answer the question, it is also how important that question is to you. We had a number of people whose Traits were off when using the 1 and 2, then 2 and 3 Aspects, but which were dead on with say 1 and 3, then 3 and 2. Assigning priorities to the question responses and making sure that is done in a way with no ties to muddy the issue should improve the versimilitude and the speed of making a character.

By the time I get the next iteration done, it should all be automated. Get the Aspects or answer the questions and the rest can get calculated in an eyeblink, letting you get straight into the game.

There's always going to be error, I have to accept that. The real-world correlations are merely statistically significant rather than reliable predictors. That something this coarse works at all I consider to be a minor miracle. I suspect that getting it close about half the time is as much as I will ever be able to hope for. But as long as it makes believable characters who are interesting to play, interact and deal with the foibles of, then it is all good.

Ron, as far as your comments on the Beliefs, I see where you're coming from, but I'm trying to make the Beliefs a core of who and what your character is, to the extent of where that Belief fits into your Loyalties. If your Belief is X, but you have four other Loyalties that would take precedence when push comes to shove, then you're not going to be as much of a "true believer" as someone who has Belief as their #1 loyalty. It's the different between someone who buys "green" paper towels and someone who chains themselves to a bulldozer at a logging camp.

Do you think I should push the Loyalties more in terms of how and what motivates you to do something?

As an aside, how badly off was your self analysis on terms of friendly and unfriendly beliefs? You already said you don't like centrists, but your check marks generally show:

Friendly (two of which are probably you):
Libertarian
Centrist
Atheist
Green
Reformist

Unfriendly:
Cultist
Reactionary
Communist

Out of those eight, how many hits do we have? Again, this is a good part of the design process. Having a system that gives plausible results for who you dislike is as important as figuring our who you do like.

And for self-confessions, I came out as an unambiguous Atheist Anarchist with libertarian and rational sympathies. Go figure.

Greg

Hi Greg,

You? Really?

In case I haven't made this clear, your statement

But as long as it makes believable characters who are interesting to play, interact and deal with the foibles of, then it is all good.

As an aside, how badly off was your self analysis on terms of friendly and unfriendly beliefs? You already said you don't like centrists, but your check marks generally show:

Friendly (two of which are probably you): Libertarian, Centrist, Atheist, Green, Reformist

Unfriendly: Cultist, Reactionary, Communist

Ron,
To clarify the game terminology:

Aspects: the five paired items (order/chaos, etc.)
Traits: The two items you get from pairing the top Aspects on the table
Beliefs: The two items that you get after all the check marks are tallied up (green reformer, etc.)
Loyalties: The "self" through "humanity" scale of tie-breakers

I'm thinking loyalties needs to be upgraded in importance, as being loyal to self or belief is going to generate widely different responses to a situation, as well as influence how strongly you are tied to your belief group. Right now it just tells which way you jump if you have to make a choice, but does not actually drive your actions in other situations. I'll have to think about that one for a while.

Greg

Have you noticed that there is difference in definitions here, for your playtest, "plausible would be a self-assessment that generated a character that everyone agreed was similar to the person doing it". which is in fact quite distinct from the goals of the self-assement, which was "not to generate people playing themselves, but to see if the generation process could or would generate PC's that felt "real" in terms of the personality and beliefs and motivations"

Feeling real <> same as the person doing it.  Whether or not a a character felt real to you, when you did not know the player and could not check the self assement against what you thought of the player, is something entirelydifferent to whether such assesment is an accurate description of the person.  It is entirely possible that someone could produce a set of values that "feel real" without that being perceived as an accurate perception of who they are.  As such, the self reported survey has no bearing at all on whether or not self-assesement accurately reproduces the views of a specifiic person known to you or others.  Furthermore, I suggest the whole survey element is essentially irrelevant to the game as such; if you can't crerate a character that is other than yourself, what would be the point?

It is in fact possible that the self-assesment is most useful when it produces results that are surprising.  It is also possible that theb self-assesment is in some way broken.  It is perfectly possible, and I would say factually true, that the self assessment is totally insignificant when it comes to actually playing the game.  Thats is, the game only needs to produce characatyers that "feel real"; the goal or accurately reproducing player ideologies is at best a seperate, redundant issue.

Second, and let me try to not put this too harshly, as Americans you are all FUCKED UP.  You've been through half a century plus of Cold War propaganda; systematically reproducing a certain specific set of ideological concepts. 
Of course you don't see  yourselves this way, because that was one of the tropes you were systematically encouraged to believe.  None the less it is undeniable that the US specifically prosecuted Thought Crime in the McCarthyite witch-hunts, and that the political culture of today exhibits precisely such pruning of the range of permissable and discussable concepts.  Indeed, the very association of "socialism" with "group oriented" betrays this ideological conditioning; the reality of Communist thought as the apogee of self-determination is an unthinkable thing, barely imaginable, much less discussable.  And thus, because you are not allowed to discuss certain ideas, it is unsurprising that such self-assesment as you do results in certain appparent anomalies.  You don't have a permissable language with which to discuss a certain range of concepts, and the range for which you do have acceptable languange is a miniscule subset of all thought on these topics.  It is therefore not at all surprising that you tend to struggle with finding appropriate classifications for your positions, as you ceasely struggle to find an appropriate hole into which to hammer your impermissably square pegs.

That's going to be difficult to reply to.

1. Gareth, your point about making the game rather than ID-profiling agrees with what both of us said. So at least that part could be framed in a "OK this works," "OK you're making sense Greg" way.

2. In the essentials, I agree with you regarding American perspectives, although I don't think it's as simple as Americans vs. enlightened-everyone-else. So I want to stress that I'm not moderating that content; it's valuable input as far as I'm concerned when political RPG design is involved. And there's more of them every year.

3. Taking it to the personal level of (effectively) accusation toward Greg and me isn't OK here. Especially since we're dealing with the notoriously intellectually-flattening effect of internet prose and none of us are too familiar with one another's personal/political histories. I also stress that nailing a particular idea or rule would be fine. I appreciate that the line is not always clear and will say as moderator, this was perhaps on it or in it, not over it.

4. Any help or comment regarding the game itself is appreciated.

5. Further political discussion as such should be taken off-site.

I think that's fair. Moderating genuine content is way harder than moderating discussions of how to roll-to-hit.

Best, Ron

I would hope that, by answering the questions as myself, the resulting traits and belief should be me, or close to me. It's that simulation-oriented streak in me. That said, what my players chose to play, character-wise, wasn't them, but characters actually rather removed from themselves.

The checklist is effective (tho I don't adhere to the two that came up top by the instrument)...

With all 4 of my group, the questionnaire and traits (at least when tied traits were taken in the proper order), it did in fact hit dead on. The belief-component labels, however, didn't. And that's true for my wife and I for certain, and I think also for the guys we game with. Which means it's likely the broken part.

Noting that Ron also got a disconnect there, in that same step... implies the underlying data is bad for those labels.

And, having automated and then played around with altering the aspects by a point or two, it's surprisingly sensitive to single and 2 point changes on a single axis. Drop the risk from 4 to 2, and suddenly, it's got a fitting, but not dead on, belief descriptor pair resembling me.

--on to another issue--

Group and PC matching aspects at start: the group matching makes sense to uswhen one presumes the character to be high-power leader in the group (influence on the group being a large chunk of the group), and theoretically in power because of being a tight match to the group's collective average....

And also noting that they diverge from each other, due to end of crisis resolution and political and personal capital, that steady state of matching probably will not last.

Nothing I said should have been read as directed toward anyone personally.

I am attempting to run a forum based playtest, and hope to have more to add later.  But while doing prep some thoughts did occur, notably around the lack of colony structure.  That is, there is no systematic expression of the health or stability of the colony or anything along those lines.  This leaves the players with little information work with or play off, and relegates everything that has to do with the colony status into positioning statements of one sort or another, and I suspect predominantly into the GM's hands.

Possibly this could stand to be fleshed out a bit; we'll see how it works out.  But seeing that the characters are not necessarily the colony authorities, not necessarily in power positions in regards the group they represent, and not necessarily in direct contact with one another, I'm left with a worryingly nebulous feeling about how to bring on character action in a tangible way.

Also, there is nothing about Martian conditions, or colonial tech, or the like.  Perhaps this doesn't need to be specified but perhaps a discussion of possible sceanrios would be worthwhile.  I'm generally familiar with the topic and have other RPG resources to borrow from, which I'm doing, but again there is a certain absence of solidity. I'm not sure the players are going to be able to develop actions by surveying their status or the of the colony, or the technical scope for action, etc.

Oh and coincidentally:

"A wave of programmed conformity has swept over the Washington community on this question," he told The Huffington Post. "The substance of this issue has been placed on an index of forbidden thought. And anybody who expresses those thoughts is excommunicated."