"But the 'consolation' of fairy-tales has another aspect than the imaginative satisfaction of ancient desires. Far more important is the Consolation of the Happy Ending. Almost I would venture to assert that all complete fairy-stories must have it. At least I would say that Tragedy is the true form of Drama, its highest function; but the opposite is true of Fairy-story. Since we do not appear to possess a word that expresses this opposite — I will call it Eucatastrophe. The eucatastrophic tale is the true form of fairy-tale, and its highest function.

The consolation of fairy-stories, the joy of the happy ending: or more correctly of the good catastrophe, the sudden joyous “turn” (for there is no true end to any fairy-tale): this joy, which is one of the things which fairy-stories can produce supremely well, is not essentially 'escapist', nor 'fugitive'. In its fairy-tale—or otherworld—setting, it is a sudden and miraculous grace: never to be counted on to recur. It does not deny the existence of dyscatastrophe, of sorrow and failure: the possibility of these is necessary to the joy of deliverance; it denies (in the face of much evidence, if you will) universal final defeat and in so far is evangelium, giving a fleeting glimpse of Joy, Joy beyond the walls of the world, poignant as grief.

It is the mark of a good fairy-story, of the higher or more complete kind, that however wild its events, however fantastic or terrible the adventures, it can give to child or man that hears it, when the “turn” comes, a catch of the breath, a beat and lifting of the heart, near to (or indeed accompanied by) tears, as keen as that given by any form of literary art, and having a peculiar quality."

That's J.R.R. Tolkien in On Fairy Stories... and ever since I've read it I wanted to incorporate it into an RPG... but how do you ensure a happy ending without devaluing the choices made by the players?

The illusion of choice of course.

No, but its not an illusion to the characters in the source material, and you'll note that Tolkien stresses the might or might not, but if it does not happen its not a fairy story, its either drama or comedy... maybe its an impossible dream.

I'm quite certain that Tolkien intended for the protaganists to prevail at the beginning of writing.  I'm also quite certain that readers of such fantasy - and most fiction for that matter - fully expect the protaganists to prevail as well, yet that doesn't devalue the enjoyment I or many others have had in reading his work.

In roleplaying games, however, the GM cannot guarantee his players to prevail unless if he manipulates the story in some way.  Dice are fickle; players make errors of judgment.  A railroaded game is not necessarily criteria for a bad play experience.  On the contrary, the skill of the wizard behind the curtain determines that. 

If I were to design an RPG that favors railroading, I would provide the GM the tools he needs to guide a session without breaking immersion: no deus ex machinas or other such contrivances.  For example, instead of forcing the players to go down one path, you may introduce elements into the story that make the intended path more seductive, so that players think they are making the decision to go down that path on their own.  Furthermore, dice can be fudged sparingly (or fake modifiers could be applied, "you can smell the alcohol on the orc from here" the GM says to the inexperienced hobbit), and enemies can make poor tactical choices. 

Do you know the card game Once Upon a Time? It's a rules-light storytelling game in a fairy-tale setting - there are kings, talking horses, dark forests, the whole shebang. Each player has a hand of cards representing these various elements and get to play a card when they use the corresponding element in the story (or interrupt someone else when they use that element). Each player also has a "Happy Ever After" card which is the ending they're trying to steer the story towards, but which can only be played once you've played all your other cards.

I think these ending cards could work as a model for the type of game you're talking about: have a finite number of possible (all happy) endings, and make the players differently invested in each of them.

Thanks for the suggestion guys but I think your missing the point. What I want to make is a mechanical framework for the narrative that recreates these kinds of results organically. The best existing example I can think of is the game Arkham Horror. Its a board game by Fantasy Flight where everyone cooperates to defeat a monster. In that game the mechanics and the whole set up make you feel like your going to fail but you don't you succeed, by the skin of your teeth. Once that illusions of "oh fuck we're going to die" is gone, then the game loses a lot of its playability, IMO. I think RPG mechanics can help you feel like Frodo, they can make you go "Oh fuck, no way." and then fairly reliable give you a win at the end. I just don't know how to do that over a long enough scale.

I don't think an algorithm (a set of mechanics that operate independently of the GM's judgment) can successfully do that without breaking immersion at some point.  If a player reads the rulebook, he can figure out just when and how to expect the man in the sky to save the day.  However, if the rules dictate the modus operandi of the GM to tactfully manipulate events to his discretion and provide him with the tools to do so, then in my opinion that is a more organic experience because the players will never know how events managed to turn in their favor.

Despite my penchant for creating mercilessly lethal systems, I would think such a concept successfully implemented would be extremely fun to play.  The illusion of danger and tension feels exactly the same as the real thing, with the added benefit of the players finishing the story. 

Then I understand better what you're after.

However, regarding the comparison to Arkham Horror, that works because there is a very real risk of failure. You don't always succeed - or at least my group doesn't. The exhileration of success comes because of that risk of failure, but if you knew that you would win there would be no tension. But that's because all the players know the mechanics (in a wide sense, not just the rules of how actions are resolved but how one part of the game leads to the next), there's no veil as in a GM-led RPG where one person knows things or makes decisions based on secret criteria.

However, if the rules of the game were to dictate that the fairy-tale story were to have a happy ending, then the veil would be destroyed if any of the non-GM players read them as was mentioned above, so the only way to preserve the certainty of a happy ending and have the perception of the risk of failure leads back to the illusion of choice: when the players do something that should prevent the happy ending the GM will have to rearrange things backstage so that success is still possible.

This is all dependant on the critical element being that sensation of joy when the seemingly assured disaster is averted, which seems to be what you're interested in. Either there has to be a real risk of failure or the fact that victory is assured has to be hidden.

Your right you can fail in Arkham, and I think that can is important, but I feel the game is specifically designed to make this not very likely. I don't want predictability, I want reliability, and

sorry, I'll continue

... and, thats not something any RPG I know is good at. Most of them do the opposite really.

As far as mechanics not creating the mood or feel of the game, I know they have a limited capacity to do so, but how the game as a text does this is what I'm concerned about here. How can you most reliable create these kinds of feelings within the mechanics.