Submitted for Your Consideration: Changes to ORIGINS Awards

[Jared A. Sorensen]

Anything that affects the integrity of the RPG industry, like an award, I think has a very big affect on the Industry as a whole.  From the big guns down to Indie Games, both free and for pay.

The chance to be able to affect something that may very well reflect well on the industry as a whole should not be ignored.  So I think it is very relevant to us here.

I keep hearing this word industry. What industry? Do we care? I mean, I guess you do...and I guess I don't.

- J, the Industry!

[ADGBoss]

I keep hearing this word industry. What industry? Do we care? I mean, I guess you do...and I guess I don't.

- J, the Industry!

Well thats a perfectly viable opinion to have and more power to you, but I find it incongruous to belong to a Forum dedicated to the improvement of Indie games and RPGs in general, but not care about "The Industry". I would say you me and us are part of the "Industry" whether we like or not.

Again though thats just my opinion so I guess we do disagree :)

Sean

[GMSkarka]

I saw that this was cross-posted on RPG.net

It was cross-posted on several public websites, and posted to one industry email list as well.

It was NOT, however, posted to the Academy mailing list, the forum for such discussions.  

Participating in these threads on the various places it has been posted does nothing more than further one individual's power-politics BS to "drive change from without".

Take that as you will.

GMS

[Ron Edwards]

Hello,

Couple things to go over.

THING ONE
Sean (ADGboss), Jared is making a point that you're not seeing, and I think it'd be valuable for you to consider it - that "the industry" is not necessarily important or relevant to RPG publishers and players. Jared's work has demonstrated this beyond any shadow of a doubt, and my own success at the industry level is based on my earlier success without it.

In other words, participating in "the industry" (retailers, distributors) is an option for, not the definition of, role-playing publishing. This is one of the founding principles of the Forge.

Therefore doing stuff to benefit "the industry" is not the obvious no-brainer that it might seem. To many of us, doing so would be a disastrous waste of effort and money. To others, including me, doing such stuff is necessary but limited - best handled from a strong "what's in it for me" perspective.

However - and this is important - some independent RPGs do rely largely on the "industry" functioning. Therefore this topic is relevant to the Forge (which is my answer to Jared), even though it's not overwhelmingly crucial to everyone here.

THING TWO
It strikes me as perfectly viable for someone, GAMA most likely best suited, to offer awards based strictly on monetary gain or volume of sales of some sort. In other words, Ryan, if you're looking for an indicator of RPGs' physical and commercial presence in our society, as a means of advertisement, why not jettison the whole "awards based on content" in the first place? Supposing that the numbers regarding profits and sales-volume could be gathered, just list the getters of the Mostest in order, as the winners of the awards. All done.

If these companies are themselves the recipients of other, content-based awards as well, regardless of commercial presence, then it's a bonus - both "big" and "good" would be rewarded independently.

Letting the one be the gateway for eligibility to assess the other, though, seems quite odd to me.

Best,
Ron

[ethan_greer]

In general, revising the Origins Awards is long overdue and badly needed. It's one piece of reviewing Origins per se, as a convention, and GAMA as an organization - also badly needed.

Ron, can you elaborate on the reasons underlying this opinion?  It's not that I disagree - just seeking better understanding of where you (and Ryan, and others) are coming from.

[ADGBoss]

Hello,

Couple things to go over.

THING ONE
Sean (ADGboss), Jared is making a point that you're not seeing, and I think it'd be valuable for you to consider it - that "the industry" is not necessarily important or relevant to RPG publishers and players. Jared's work has demonstrated this beyond any shadow of a doubt, and my own success at the industry level is based on my earlier success without it.

In other words, participating in "the industry" (retailers, distributors) is an option for, not the definition of, role-playing publishing. This is one of the founding principles of the Forge.


Therefore doing stuff to benefit "the industry" is not the obvious no-brainer that it might seem. To many of us, doing so would be a disastrous waste of effort and money. To others, including me, doing such stuff is necessary but limited - best handled from a strong "what's in it for me" perspective.

However - and this is important - some independent RPGs do rely largely on the "industry" functioning. Therefore this topic is relevant to the Forge (which is my answer to Jared), even though it's not overwhelmingly crucial to everyone here.

Best,
Ron

I do see what JAred is saying and I respect his opinion, otherwise I never would have taken the time to comment on it.  However, I think he is wrong but perhaps thats because I consider "industry" to be beyond the retail and distribution.  I consider industry to include all Players and Designers and Artists and Publishers as well as Retailers and Distributors. So that anything that could affect "the industry's" image or integrity either as a positive or negative, I think is extremely relevant to players and especially designers.  Thats where I was coming from.

Sean

[Pramas]

Ron, can you elaborate on the reasons underlying this opinion?  It's not that I disagree - just seeking better understanding of where you (and Ryan, and others) are coming from.

One of the biggest problems with the OAs is that they a hybrid peer award/fan award and they serve neither group particularly well. The Academy determines the final slate of nominees and the votes of its members are weighted in the final public vote. Right now, the final vote is 50% Academy and 50% public. This system has all sorts of problems.

First, the Academy is a small group, currently less than 200 strong. It costs $30 to join for a year, and even though you usually get back far more than that in comps (many companies send out samples of their prodcuts during the process) many professionals never bother to join. This allows a large and motivated company like Wizkids to buy a lot of Academy memberships for their staffers--and they have a lot of staffers. Any company that can bring to bear 20-30 votes is likely to get whatever it wants on the final ballot and to get a nice chunk of the Academy weighted votes in the final ballot as well.

The public vote end of things has its own problems. Most gamers aren't particularly well informed about the nominees, so they tend to vote for companies, games, and brands they've heard of. Witness the Lord of the Rings RPG winning this year. I doubt most of the folks who voted for the game ever read it, never mind played it, but they like Lord of the Rings so they voted for it. People also tend to vote for "funny" products, which has led to comic books dominating the Best Periodical category for the last 5+ years. Witness Hackmaster as Game of the Year last year as well.

The upshot of all this is that while the OAs are supposed to recognize excellence in design, you can look through the winners in any given year and find many products that make you say, "WTF." Which creates a credibility problem, which makes people less interested in the awards, which sort of defeats the whole purpose of having them in the first place.

Nicole Lindroos, the new chairperson, has many plans to try address the issues, but as she's been chair for all of a week, she hasn't gotten the ball rolling yet. Hopefully, by the time she's done, people will be give the awards another chance.

[HinterWelt]

Being an academy member let me point out that the main problem which has been sited will remain a problem and there is little this restructuring will do to solve it.

The main problem people seem to be sighting is one of a "Popularity Contest". The initial list of games we received out lined so many I had never heard of and thus had no opinion on. Guess what, I also had much more important things to do than research the many games mentioned. In the end I voted for the games I had a knowledge of (I happen to own the LOTR game and was greatly disappointed with it). Whether you have the vote split into three (of which I qualify for all three) or have it based on an elite "Academy" it will still come down to what you know. I received a bunch of Hackmaster and Kenzer co stuff as a preview of their material. Up until then I had not owned a single Kenser co item. I was seriously offended that Hackmaster won last year(I was not a member of the academy then) after reviewing their products. To be honest, I will probably not renew my membership as I see no benefit in it.

Allow me to also say I do not see much benefit being a part of GAMA as an organization. I find the GPA to be a massively more beneficial organization and cheaper. I am a voting member there for $60 while it costs $300 to be a voting member in GAMA. This all comes back to the Origins Awards and what does it say about the org. Does it garner more sales for winners? I honestly do not know.   Does it garner respect and is it something sought by the best of the best? All the companies I have ever talked to at the 3 GTS that I have attended fall into two groups. The small group of winners which claim it is a great honor and the larger group of individuals who really did not care.

In the end, I am not sure what Mr. Dancey's motives are but the overall change would seem to be to the benefit of the big companies that are most popular. Sum zero. No effectual change.

My suggestion would be to either make it a popular vote award where every vote is weighted equally or make it an elite rotating staff made up of respected individuals from all parts of the gaming community but very finite. This could be along the lines of 2 retailers, 2 distributors, two manufacturers (including indie), 2 gamers. Make it no bigger than 12-15 and allow submissions by all nominees to these individuals for their review. Establish a criteria by which each category would be judged. If you want sales to be a driver make a category for it. Take the top 5 selling games and have them judged against each other. Other than that category ignore sales and judge games on its own merits. Some of these ideas were already expressed in this thread but I wanted to express my agreement.

All the above is of course my opinion and may be ignored at will.

Bill

[ryand]

Hello,


In other words, Ryan, if you're looking for an indicator of RPGs' physical and commercial presence in our society, as a means of advertisement, why not jettison the whole "awards based on content" in the first place? Supposing that the numbers regarding profits and sales-volume could be gathered, just list the getters of the Mostest in order, as the winners of the awards. All done.

I'm convinced that an award process that is designed to select the "best" products by category is possible.  And I think that doing so, because we lack the ability to make quantitative judgements about what is "best" due to the subjective nature of the question and the products involved, requires the use of a system based on multiple inputs.

The sales volume metric does a couple of interesting things.  First, it essentially gives "the consumers" a vote at the table - their purchases matter in the selection process.  Individually, I don't think that consumers always make the best purchasing choices.  But in aggregate, I think that new hobby gaming products become bestsellers because they are of high quality first, and most other considerations second.

That market-based analysis is insufficient as a single factor though.  Limits on distribution and awareness are obviously barriers to widespread success for some kinds of products.  That's where the "peer review" component comes into play - especially in my "Special Master" revision.  The "peer review" component would be insufficient by itself as well, because oftentimes peers become very focused on the selective parts of a product that relate to their own jobs, and it can be hard for peers to see the forest for the trees.

I believe that a combination of the two inputs offers a reasonably high chance that the nominations list will contain a mix of products that both peers and consumers will accept as "valid" by any reasonable test.  A list constucted using just one of the two methods would fail that test.

If the list is considered "reasonable" by a large number of stakeholders, the resulting award will gain relevance as well.  Making the winner more relevant is a key to making the whole Award system more useful and meaningful.

[Ron Edwards]

Hi Ryan,

Well, real-world data can go two ways.

1. Just as you describe, including multiple variables can result in identifying a "vector" among their distributions that is not expressed by any single one, but which means something on its own and can be named.

This is how biologists measure "size," for instance. No single variable is itself "size," but the aggregate portion of variance among different variables that all varies together can be called size.

2. Conversely, the vector can be mathematically derivable, but conceptually not worth the paper it's printed on. IQ is quite likely such a pseudo-variable ("reification"); the data used to get it make more sense as a set of separate variables or categories than their shared vector does as a single number (i.e., there's no "it").

So which is it for RPGs? Damn good question. I favor the latter. I'd be happier with two separate awards, one with its winners and runners-up based on commerce, and the other with its winners and runners-up based on content alone (and play thereof). Again, if a particular game scores really well, well then hey, it wins both awards.

You apparently favor the former, which is OK too - because frankly, distinguishing between the two phenomena in the real world is often very difficult, and relies very heavily on the nature of the original measured variables.

Best,
Ron

[Bruce Baugh]

I wanted to point out a waiting trap in thinking about awards based on sales, with an example.

WotC sold Everway in mixed palettes with Magic: The Gathering cards for a while. That is, distributors who wanted their M:TG also had to buy Everway. So a lot of copies of Everway left WotC. But those copies sitting in distributors' warehouses didn't all move on to retailers, and certainly retailers did not sell Everway in earthshaking quantities. What WotC sold and what gamers bought differed quite significantly.

That kind of thing happens all the time, too, though usually less extremely. Sometimes a company sends out a lot of copies...and then gets a lot of them back, in chain bookstore returns, stock abandoned by distributor collapse, and like that. Sometimes things fly to retailers, and then don't move, because end customers prove less interested for whatever reason.

What we'd need to have awards based on sales is genuinely reliable sales data - the equivalent of Soundscan and Bookscan, with verifiable data and methodology. That doesn't exist in gaming at all. There are some stores (physical and online) that have reliable point-of-sale data, and every single one I know of can discuss ways in which they're not representative. None of us actually knows what's sitting around not moving in J. Random Gamer's semi-hobby store. Or flying off the shelves there, for that matter.

Okay, I suppose we could have awards for companies most successful in selling to distributors, but it wouldn't mean anything even to most of us in the business, let alone to the public at large. The data that matter, I think, are the ones about customers at the far end of the chain are doing, and they are terra incognita to a very large degree.

Now if WotC and its regular associates were to back a big push for point-of-sale data collection and processing in the gaming business, that would be an enormous boon, and I'd sign on with much enthusiasm as long as the results remained open to inspection and verification.

[Bruce Baugh]

So which is it for RPGs? Damn good question. I favor the latter. I'd be happier with two separate awards, one with its winners and runners-up based on commerce, and the other with its winners and runners-up based on content alone (and play thereof). Again, if a particular game scores really well, well then hey, it wins both awards.

This seems like a good way to go, to me.

Though I find myself pondering (and wrote about this on rpg.net, so I'm recycling a comment) whether there are meaningful comparisons across the whole of gaming.

Andy Kitkowski's awards deal with a population that I think does have a certain identity - the definition of indie rpg sets some boundaries, and the pool of games worth considering is of a manageable size. ENWorld's ENnie awards also encompass something that has an identity - the fact of being d20 (or OGL) draws some boundaries, and while the pool of possible contenders is very large, it's relatively easy to extract the subset that actually deserves serious consideration.

But then you get beyond communities like that and...I dunno. We're doing such different things in such different ways with such different assets and priorities. In theory, taxonomy like the Forge's could provide a conceptual frame of reference for looking at the whole field's ideas, but that would still leave the fact of varying format, both for individual works and their position in ongoing publishers' efforts.

I'm not crying doom here, mind. I honestly don't know.

[kwill]

> What we'd need to have awards based on sales
> is genuinely reliable sales data - the
> equivalent of Soundscan and Bookscan, with
> verifiable data and methodology.

but hey, aren't most gaming products just books? (or am I just biased? :)

Nielsen Bookdata (I think the new name for Bookscan) seem to cover almost everything with an ISBN, I think what they'll miss are:

- books without ISBNs

- books not distributed through gigantic distributors (Ingram, Gardners et al)

...in other words, indie games

GAMA could use this data if their members distribute through the right channels

all in all, though, I'm in favour of seperate bestseller tracking and awards

also, perhaps take another look at the Oscar model? that said I can't remember exactly how it goes, something like...

1) nominations by relevant academy members (directors nominate for Best Director, actors nominate for Best Actor) and award voted by The Committee

or 2) nominations by all academy members, award voted by relevant academy members

(where academy is replaced by GAMA)

again, box office success is tracked seperately in this industry

here's hoping for the day I see a special issue of The Bookseller on roleplaying

[Bruce Baugh]

A lot of specialty shops aren't covered by Bookscan; this isn't unique to gaming, by any means.