Review Criteria for RPGs

[Gold Rush Games]

Good suggestions.

 Which do you folks think would be better? A 1-5 score or a binary (Pass/Fail) score?

 I can see advantages and disadvantages to both, but I'm curious what you think, too.

[Kester Pelagius]

Greetings Mr. Arsenault,

 Part of the requirement for a submission to be considered for the program is that it has to be "ready to print," though. So we'd be evaluating the whole package (writing, style, design, "look," etc.) at once.

Otherwise, to play paranoid devil's advocate, I'd have to ask what is in it for the author?

 That's not paranoid. It's good sense to ask such a thing.

 In summary, what's in it for the author is:

[*] POD publication of their book (with their imprint/logo)
[*] Client maintains copyrights (we just license publishing rights)
[*] ISBN assignment & barcode
[*] Listing in R.R. Bowker's Books in Print database
[*] Listing in the Ingram Book Co. database
[*] Listing on Amazon.com, BN.com and others.
[*] Review copies sent out to those on our review list
[*] Association with an established publisher (est. 1995)
[*] Distribution sales via our sales office (Tundra Sales Org.)
[*] A percentage of all sales of the book
[*] Representation (and potential sales) at cons we attend
[*] Listing in the GRG catalog, on the GRG web site, etc.
[*] Marketing (not exhaustive, but comparable to what give our own products)
[*] A much cheaper alternative to traditional printing
[/list:u]

So, though it is not explicitly stated, we can assume that this is a program strictly for books.  No "boxed sets" or RPGs with "board game" or other gimmicky elements (like cards) need apply, yes?

That's very important to know, IMO, since some indie designers may make use of non-standard resolution mechanics and methods.  But if the program is strictly for books, that means anything that can't be printed strictly as a book need not apply, or can it?

Also, as an aside, what manner of binding are we talking about here?

Will authors have a number of choices to choose from ranging from, say, perfect bound to saddle stitched?  And what kind of paper will be available within the book?  For the art?  Covers?

Incidentally, will there be galleys or proofs made available for review by potential customers?

For instance what would turn YOU off to a submission?

 Poor quality art, writing, editing, a cliche product for an overused genre, and so on. We're looking for quality, innovative game products from indie designers. The good stuff.

Ah, road block right there in the "cliche" or "overused genre" bit.

The main thing here will be a response of "Who is to say what is a cliche or overused genre?"

Every author will (hope and want to believe) their MSS has that extra *something* that sets their endeavor apart, but if that work happens to fall into a genre category you aren't even going to accept submissions for that month. . .  Loss of potential customer.

Or, from a writer's point of view, if you say your magazine is "open to all submissions" but then have a form rejection letter that simply states that you did not like the work, personally, as a matter of your own reading taste what you've just stated is that, as an editor, I, the writer, need to entertain you.

Problem is that should NOT be my job, unless you're running a fanzine.

Sure, editors have to sometimes think that way.  It's a judgement call, but which is the more appropriate question to ask: "Does this story entertain me?" -or- "Will this story entertain my readership?"

Or, to put it a different way, ask yourself when the last time the market came knocking on an editors door to ask them what their taste in the next big trend was.  Never has happened, unless you're with the Illuminati or the Bilderbergers maybe.  In which case I've said too much already!!!  ;)

If a MSS submitted already had copyright filed (and the author made a point to let you know) what would be your response?

 Registering a copyright offers no more protection to a work than its mere creation and recording in tangible form. All works (in the U.S.) are protected the moment they come into being.

 With that said, a creator having a copyright registered already tells me that they are serious, know at least something about copyrights, and otherwise has little bearing on our consideration of whether to publish their book.

Interesting.  Let me rephrase. . .

Say I have a game, X, that I have both registered copyright and trademark for.  Would this make it easier for you or more difficult from the published end of things to work with?

Would you prefer that authors who approach you have their MSS ready to submit and print with all the bases covered, or at least as many as possible?

You mentioned artwork, does this mean you will be checking all illustrations used to be sure that artists copyrighted and intellectual material is not being stolen?

 All art will be reviewed, but not to ensure protection of IP. The publishing agreement has language in which the Client guarantees that their product does not violate any third party's IP rights.

Ok, but what if a third party sees a book published by your company that has artwork in it that they feel is derivative (or a blatant rip off) of something of theirs and decide to file suit against your company.

Are you sure that such a disclaimer will protect you?

If a author wanted to negotiate the boilerplate in your contract BEFORE submitting anything to you, would this impact how you viewed the authors submission?

 Not how we viewed the submission, but it could have an affect on our decision whether to accept it into the program or not. It would depend on what the specific item was.

So, to be accepted into the program, you're saying that the author would have to first sign the contract??

For that matter will you be reviewing the MSS to ensure that no copyrighted material- such as mention or use of established worlds and derivative works of fiction that may infringe upon intellectual properties- are used?

 Client declarations, guarantees and warranties are written into the contract. It would be a bad idea for a Client to submit material that violated another party's IP rights.

 We do reserve the right to reject any submission, howeve, including for reasons involving legal liability.

Granted, it would not be a good idea to submist material that obviously violated established rights of another property.  But what about something done in parody or spoof?

Are you familiar with the movie Zardoz?  If not it basically is a post-apocalyptic flick in which heavy use is made of imagery/ideas derivative of the Wizard of Oz.  But it's taken almost to spoof/paradoy.  Then again it could be argued such protrayals fall perfectly within fair use, being a cultural icon and all.  (Iffy.)

I assume that such borderline works will be considered on a case by case basis, but what if someone plagerizes a obscure work that no one reading your slush that week is familiar with?

There's probably a dozen other questions I could ask, but I think that suffices to illustrate the potential problems.

 I see no problems thus far. Feel free to keep the questions coming. All I ask is that you review the "Draft Agreement" and the "POD Costs & Royalties" documents before posting more questions, in case they are answered in the documents.

As you asked so nicely. . .  :)

I see that you are asking for a number of rights.  I asked earlier about modification to the boilerplate. . .  But now what I want to know is how one mediates disputes between the author and the publishing house if a problem arises.  What will be the recourse available to the undersigned if they have a dispute with contractual obligations?

As for the rest of if. . .  Kiddies, if you're reading this, and haven't read the PDF, or have and don't know what to make of it.  Speak up now while the man is here.  It's six pages and says a lot, but what does it really all mean?  Do you know?

Curious about why it refers to NET and not GROSS?

Have questions about liability, fees, charges, or other costs that you may incur as a result of. . . ???

Want to ask the man how this program will differ from going to the local Kinko's or vanity press?


Kind Regards,

Kester Pelagius



P.S.  As always grammatical and syntax errors are purely the fault of sleep deprived gremlins!  ;)

[busling]

Mark,

In my experience if you grade them with a specific number at the end it will help to make your final decision. Especially when there are several products that are similiar.

[Ron Edwards]

Hi Mark,

My first impression, based on your first post, was to recommend removing the numerical scale or collapsing it to 1-sucks, 2-OK, 3-good. But then I decided that 4 would be better - so that you really have to bite the bullet between 2 and 3. A mid-value on numerical scale tends to create an artificial cluster as the rater grows lazy. A poorish product with a great feature or two, and a pretty good product with a fatal flaw or two tend both to get rated "3" on a 1-5 scale.

Also, and this is a horribly unscientific anecdotal sort of thing, I recommend not going above a scale of 4. Once past 6 increments, issues such as the difference between "very average good" and "average average good" start cropping up.

Best,
Ron

[Walt Freitag]

Hi Mark,

The list of criteria look fine on the surface. But you're implying that you then intend to average the scores and compare the average to a minimum threshold.

The idea is that products will have to receive a minimum score (of, say, 3.5) in order to be accepted into the program...

* Marketability (is the genre a fresh twist or cliche?)
* Cover illustration
* Interior art
* Organization
* Writing (incl. style, voice, etc.)
* Interior design (readability, graphic design, attractiveness, etc.)
* Editing
* Playability (fun factor)

What does it really mean to do that? Averaging says that an excellent cover illustration can make up for poor playability, and that the cover plus good interior art and graphic design can make up for abysmally awful playability.

It also means that weakness in art or organization or writing style could keep a product with marvellous playability out -- so that, for example, the 1e AD&D books would not have been publishable in 1979 by these standards.

Neither of these possibilities is a good thing, at least not from my point of view as a potential customer.

I suggest a pass/fail threshold for each individual element, in addition to an average of open-ended individual scores that can be increased above the normal range when a feature truly merits it. So, truly excellent art or playability alone could get a product in, provided it was at least adequate in all the other areas. The pass threshold could be lower for some items than for others, and it could include specific cases and expectations -- such as, poor spelling in the editing is a failure condition, not because it's so harmful but because it's so easy to fix; or minimal cover art consisting of tasteful text and abstract graphics might pass while a poor or inappropriate illustration would not.

- Walt

[Jack Spencer Jr]

So you feel that it is a waste of my time to solicit feedback on review criteria from the people here in this forum? Gads. I've been criticized before for not caring what my customers think. Never before have I been criticized for caring what my potential customers think. ;)

Not sure what you're saying here.

I wouldn't say waste of time, but I personally wouldn't do it. This may explain all of the money I've made in the publishing industry, but having never been might be a better reason. I think I'm the sort who believes more in instincts. Devleoping an objective criteria takes more work and, from what little I've seen and read about business, works about as well as instincts (worse in some cases, I guess. Better in others). That seems to be where I'm coming from.

[Gold Rush Games]

So, though it is not explicitly stated, we can assume that this is a program strictly for books.  No "boxed sets" or RPGs with "board game" or other gimmicky elements (like cards) need apply, yes?

 Correct. Our printer is not able to produce and distribute boxes, cards, etc., via their POD print system.

Also, as an aside, what manner of binding are we talking about here?

 Perfect (square) glue binding.

Will authors have a number of choices to choose from ranging from, say, perfect bound to saddle stitched?

 No.

And what kind of paper will be available within the book?  For the art?  Covers?

 Interior pages will be printed on 60-lb. offset, opaque 480 PPI, creme white acid-free paper (for books 6x9" and smaller -- if we allow that option; it's undecided yet) or 50-lb. offset, opaque, 500 PPI, white acid-free paper (for 8.25x11" books).

 Covers are printed in full color, one side, on 80-lb. offset, enamel paper and laminated.

Incidentally, will there be galleys or proofs made available for review by potential customers?

 I wasn't planning on sending proofs off to the clients, but it's certainly possible.

 

The main thing here will be a response of "Who is to say what is a cliche or overused genre?"

 The publisher.

Every author will (hope and want to believe) their MSS has that extra *something* that sets their endeavor apart,

 And that would be part of the evaluation.

but if that work happens to fall into a genre category you aren't even going to accept submissions for that month. . .  Loss of potential customer.

 That would not happen. We would not ignore certain genres in certain months. That's silly. What I mean by a cliche or overused genre is, say... "A fantasy game. Just like D&D but better."

Or, from a writer's point of view, if you say your magazine is "open to all submissions" but then have a form rejection letter that simply states that you did not like the work, personally, as a matter of your own reading taste what you've just stated is that, as an editor, I, the writer, need to entertain you.

 No, as a publisher you need to sell you product to me and I, as the publisher, need to belileve that the product would appeal to our customers. There's a big difference between that and "entertaining me."

 Eureka! would have no obligation to print every submission that comes its way and I have no intention of doing so. There has to be some sort of evaluation criteria to weed out those products that simply aren't up to par in terms of quality.

Say I have a game, X, that I have both registered copyright and trademark for.  Would this make it easier for you or more difficult from the published end of things to work with?

 Neither.

Would you prefer that authors who approach you have their MSS ready to submit and print with all the bases covered, or at least as many as possible?

 If you're referring to copyright and trademark registration, it doesn't matter, really. If you're referring to something else, perhaps you could restate your question more clearly.

Ok, but what if a third party sees a book published by your company that has artwork in it that they feel is derivative (or a blatant rip off) of something of theirs and decide to file suit against your company.

 See the draft publishing agreement. In it the client agrees to certain conditions; essentially that the client is responsible for ensuring their work does not infringe on others' IP rights (including the d20 and OGL or other licenses), and client also agrees to defende and hold GRG harmless in such cases.

Are you sure that such a disclaimer will protect you?

 I don't think it's appropriate to dscuss legal theory here in this thread. I am not a lawyer. I do know the clauses are pretty standard fare, however.

So, to be accepted into the program, you're saying that the author would have to first sign the contract??

 To be accepted? Yes.

But what about something done in parody or spoof?

 We would evaluate parody product on a case-by-case basis. I cannot give you a blanket policy statement.

...but what if someone plagerizes a obscure work that no one reading your slush that week is familiar with?

 That has no bearing on the legality of a product being accepted as parody or satire, IMO. But that's what we have lawyers for.

...what I want to know is how one mediates disputes between the author and the publishing house if a problem arises.

 I believe section XIII.L. covers that. It describes the requirements for formal mediation in cases in which there is no resolution reached between the parties.

As for the rest of if. . .  Kiddies, if you're reading this, and haven't read the PDF, or have and don't know what to make of it.  Speak up now while the man is here.  It's six pages and says a lot, but what does it really all mean?  Do you know?

 We encourage clients to consult an attorney with any questions regarding the agreement (bearing in mind that the current version is a non-official draft). The agreement itself restates this in section XIII.C. Just FYI.

Curious about why it refers to NET and not GROSS?

 Because that's the way we structured it. Due t the POD pricing structure and other fees and costs involved with our bringing a POD book to market, we felt it best to structure it that way.

Want to ask the man how this program will differ from going to the local Kinko's or vanity press?

 That's easy. Take a product produced at a local print shop and compare it to one of our POD products (we list them on the Eureka! page), in terms of production quality and other features (barcode, distribution, etc.).

[Gold Rush Games]

Hi Mark,

 Hi there, Walt.

Averaging says that an excellent cover illustration can make up for poor playability, and that the cover plus good interior art and graphic design can make up for abysmally awful playability.

 True. We could require a product receive a minimum overall score of X and also state that no catagory can receive a score of, say, 1, and still pass.

It also means that weakness in art or organization or writing style could keep a product with marvellous playability out (**unnecessary personal criticism deleted**)

 Correct.

So, truly excellent art or playability alone could get a product in, provided it was at least adequate in all the other areas.

 Which is precisely what would happen using the system I suggested above.

[Kester Pelagius]

Greetings Mr. Arsenault,

Hope all is well in your part of the world this fine day.

Per your orignal question. . . Mr. Edwards has a point.  If you were offering a editing service a scale of 1-5 (or whatever) would be very helpful to the author, for sure.  But if this is purely a classification for you, as publisher, to decided "yeah or nay" about a submission then maybe that is what you should go with.   Keep it simple.  Unless you would like to provide some feedback to authors so they can get a better idea of why their submission wasn't accepted, in which case such a scale would likely be very helpful, if taken in the spirit intended.

Thus the questions that need answering are probably 1) which would work best for you, and; 2) what do you intend to do with the information after you review a submission?

If it's going to stay "in house" then keep it simple.  Just my opinion.


Now, on to. . .

but if that work happens to fall into a genre category you aren't even going to accept submissions for that month. . .  Loss of potential customer.

 That would not happen. We would not ignore certain genres in certain months. That's silly. What I mean by a cliche or overused genre is, say... "A fantasy game. Just like D&D but better."

Ah, IC.  That's sounds reasonable.  (Must have misunderstood your previous remarks, apologies.)  You're saying, and correct me if I'm wrong, that over used themes will have a lower chance of acceptance than, say, a game that happens to fall in a pseudo post-apocalyptic genre with flavorings of cyberpunk with the spin that the characters are, say,  Tibetan monks instead of the typical cyborgs, humans, or enforcers.  So long as the game is compelling and the standard genre tropes aren't abused or simply reused without anything to set them apart from half a dozen other games similar to the one being submitted?

Wow, that was a bit long winded.  *rolls eyes at self*

Or, from a writer's point of view, if you say your magazine is "open to all submissions" but then have a form rejection letter that simply states that you did not like the work, personally, as a matter of your own reading taste what you've just stated is that, as an editor, I, the writer, need to entertain you.

 No, as a publisher you need to sell you product to me and I, as the publisher, need to belileve that the product would appeal to our customers. There's a big difference between that and "entertaining me."

 Eureka! would have no obligation to print every submission that comes its way and I have no intention of doing so. There has to be some sort of evaluation criteria to weed out those products that simply aren't up to par in terms of quality.

Good point.  Come to think of it, and there's no reason for you to unless you think it's a good idea, how about including a paragraph stating this on the Eureka! page?

It might discourage some but, then again, it might make those who want to submit think twice about whether their work is really ready to be submitted for printing.  Just a thought.

Would you prefer that authors who approach you have their MSS ready to submit and print with all the bases covered, or at least as many as possible?

 If you're referring to copyright and trademark registration, it doesn't matter, really. If you're referring to something else, perhaps you could restate your question more clearly.

Apologies.  What I mean is, if I have a MSS set and ready to go, with grahpics and title (and the sort of things that can have a SM or TM applied to them) would you prefer to see such MSS as, say, opposed to just a MSS that hasn't had artwork and titles worked up for it yet?

Thanks for taking the time to reponsed and clarify.

Best of luck!




Kind Regards,

Kester Pelagius


edited for clarity

[Gold Rush Games]

Unless you would like to provide some feedback to authors so they can get a better idea of why their submission wasn't accepted, in which case such a scale would likely be very helpful, if taken in the spirit intended.

 That's our intent, yes.

 products can be resubmitted, but we thought it would be helpfull for potential clients to know what we felt needed to be improved before we would reconsider a submission.

Thus the questions that need answering are probably 1) which would work best for you, and; 2) what do you intend to do with the information after you review a submission?

 Yes, I understand that. I'm not asking for those reading this thread to decide our policy for us. I'm looking for feedback to help me determine how best to present the evaluation of product submissions.

 Ultimately it will be my decision and I certainly am under no obligation to consider any comments from those outside of my company. But I invited said feedback in the hopes of considering POVs that I may not have considered.

You're saying, and correct me if I'm wrong, that over used themes will have a lower chance of acceptance...

 Correct.

...if I have a MSS set and ready to go, with grahpics and title (and the sort of things that can have a SM or TM applied to them) would you prefer to see such MSS as, say, opposed to just a MSS that hasn't had artwork and titles worked up for it yet?

 All products submitted for consideration must be complete and "ready to print." We can be contracted to design the cover, of course. But not having at the very least a cover illustration knocks a product out of consideration.

Thanks for taking the time to reponsed and clarify.

 My pleasure.

[Walt Freitag]

Hi Mark,

My apologies for any unnecessary personal criticism. If you had any role in the writing, art, production, or publication of the books I mentioned, I was unaware of it and I certainly did not select that example based on any perceived connection to you personally.

The point I was trying to get at is that your list emphasizes style over substance.

When you average a bunch of ratings with equal weighting, those overall aspects of quality that are represented by the most specific and numerous criteria are emphasized. To show what I mean, I could shift the emphasis the other way by grouping Cover Illustration, Interior Art, and Interior Design into a single list item (e.g. "Visual Presentation") and expand Playability into three list items, such as: Accessibility, Usability (game design quality if it's a game system, suitability for the system if it's supplemental material), and Depth (sustained play value over time).

But instead of going that far, how about just evening the playing field:

Content
* Game design functionality OR Suitability of material for stated purpose
* Writing (inc. style, voice, etc.)
* Accessibility (organization, clarity, completeness)
* Sustained play value (relative to price)
* Originality
Presentation
* Cover illustration
* Interior art
* Interior design
* Editing
* Integration of elements (art is appropriate to content, etc.)

As the criteria list currently stands, you can take away one item and you'd have suitable rating system for a coffee table book, a home-and-garden magazine, a clothing catalog, or a corporate annual report. One item out of eight reflects that we're talking about games and materials to be used in games.

It makes me sad to think that when I buy a cookbook, the publisher might have chosen it for publication based on criteria just like that, except with "How well the recipes actually work" substituted for "Playability." It makes me even sadder to think of textbook publishers who might be stuffing "Subject Matter Accuracy and Completeness" into that lonely eighth slot.

I suppose this just reflects the reality of the modern marketplace which you can't escape. But I still find it unfortunate. That's my opinion, which you did ask for. Nothing personal.

- Walt

[Gold Rush Games]

My apologies for any unnecessary personal criticism.

 No need for an apology. I didn't perceive any personal criticism.

The point I was trying to get at is that your list emphasizes style over substance.

 How do?

When you average a bunch of ratings with equal weighting, those overall aspects of quality that are represented by the most specific and numerous criteria are emphasized.

 Ah. I think I follow what you're saying.

But instead of going that far, how about just evening the playing field:

Content
* Game design functionality OR Suitability of material for stated purpose
* Writing (inc. style, voice, etc.)
* Accessibility (organization, clarity, completeness)
* Sustained play value (relative to price)
* Originality
Presentation
* Cover illustration
* Interior art
* Interior design
* Editing
* Integration of elements (art is appropriate to content, etc.)

 That's something to consider. Thanks.

As the criteria list currently stands, you can take away one item and you'd have suitable rating system for a coffee table book, a home-and-garden magazine, a clothing catalog, or a corporate annual report.

 I fail to see a fault to this from the perspective of a company seeking to publish other peoples' games.

I suppose this just reflects the reality of the modern marketplace which you can't escape. But I still find it unfortunate. That's my opinion, which you did ask for. Nothing personal.

 And I appreciate you sharing it with me. :)