Dice probabilities (split from New Review)

[Mike Holmes]

Those charts ought to convince anyone (assuming they can read a chart) that the differences are unimportant.

BTW, Ralph, in Donjon, the differences are way more pronounced because of the effect of ties, so maybe that's what you're thinking of. That's a completely different game going from d20 down to d10 (or, heaven forfend, d6).

Mike

[Valamir]

Ok, I hate to belabor this issue, but the graphs aren't really helping.  They're just graphing numbers without any indication of the accuracy of the numbers generated.  There are certain things about how the graphs are displayed that red flag them in my mind (note: red flag =/ they're wrong.  red flag = more information please).

First the graph of 5:5 and 6:6 are exactly what I'd expect.  All lines line up perfectly.  The tie effect comes from changing the ratio of the pool size, so with equal pool size, there is no effect.  However 3:3 and 2:2 ARE showing an effect.  Since I can't think of a good mathematical reason for why the graph of 3:3 would look different from the graph of 5:5 both of which show the expected 50%, Red Flag is raised.  Something is going on there I don't understand.

Second, I don't know how the number generation accounted for ties.  I see a note where graphs were altered to make the data continuous, but this is erroneous.  The graph cannot and should not be continuous.  You have either 1 or more victories for the large pool (positive numbers) or 1 or more victories for the small pool (negative numbers).  There should be a complete gap at 0 because ties do not stand.  I'm not sure why this adjustment was made, but it confuses the heck out of me and raises the question as to whether the tied results are being accounted for correctly in whatever system is rolling the dice.


I went and prepared a grunt work spreadsheet where I simply listed every single possible combination of a 3 die d4 pool in the columns and ever single possible combination of a 2 die d4 pool in the rows, and then a second sheet using the same pools with d6s.  I chose the small number of dice simply to keep the number of cells manageable.  I suppose one could do a 8d10 vs 4d10 this way but it would be enormous.

The results were exactly what I expected them to be.  

When the die size was dropped from d6 to d4 the number of times ties occured increased from 24.77% to 36.72%.

When ties occured on a d6, the smaller pool managed to win with subsequent dice 24.14% of the time.

When ties occured on a d4, the smaller pool managed to win with  subsequent dice only 20.75% of the time

More ties occuring with a higher ratio of those ties going to the bigger pool is exactly what I've been saying.


As for how pronounced the effect is overall at these pool sizes it made a difference of 3.78%.  Thats just going from d6 to d4.

The chance of rolling a double on 2d6 is 16.67%
The chance of rolling a double on 2d4 is 25%
That's 1.5x more likely dropping from d6 to d4

The chance of rolling a double on 2d10 is 10%
That's 1.67x more likely than rolling a double on 2d6
So I'd expect the difference to be even more pronounced going from d10 to d6s


As to whether a difference of 3-5% is significant enough to worry about.  Well, that I guess resides in the eye of the beholder.  Some people take their couple of % on D% very seriously, some do not.  

I'f one were playing d20 and one found a sword that was +1 anytime the roll was even and +0 anytime the roll was odd (essentially +1/2) they'd prefer it over a straight sword...even though the statistical difference is only 2.5%

[Mike Holmes]

I'f one were playing d20 and one found a sword that was +1 anytime the roll was even and +0 anytime the roll was odd (essentially +1/2) they'd prefer it over a straight sword...even though the statistical difference is only 2.5%

Sure, because they can percieve that. The GM will tell them so.

But a player has to roll 20 times to even get a sample that will have the anomaly in it for a 5% difference. And then he has to go through that 20 roll cycle multiple times before there's a chance that they can see the trend that the odds are different. And this all assumes someone looking for it, or who has a reason to pay attention, and someone who has played with the other method. Sorcerer doesn't really give you a reason to pay attention to the dice. D&D is quite Gamist, so, of course, every little advantage makes a difference. In Sorcerer, you don't care what the results of the roll are particularly; they're only a "spingboard for cretivity" (hence the reason that the "unrealistic" underdog effect is cool).

So what does it matter if it's slightly different, one way or another?

Mike

[rafial]

They're just graphing numbers without any indication of the accuracy of the numbers generated.

Fair cop.  Since the method being used is random sampling, the accurary is dependent on getting the number of samples high enough.  We can get some idea of the amount of "noise" by looking at expected versus actual results in the simple cases.  If you looke at 1:1, you see that the with 50:50 expected, we get 49.8:50.2 in one case and 49.9:50.1 in the others.  So it looks like we've got around 0.2% error in the results.

First the graph of 5:5 and 6:6 are exactly what I'd expect.  All lines line up perfectly.  The tie effect comes from changing the ratio of the pool size, so with equal pool size, there is no effect.  However 3:3 and 2:2 ARE showing an effect.

The effect being shown is that with smaller die sizes, there is a slightly lower probability of extreme differences.  This trend actually carries into the higher numbers, it just becomes harder and harder to see as the tails get longer.  Look at the raw data sets.

Second, I don't know how the number generation accounted for ties.

Do you mean tied dice, or tied pools?  The function that scores a roll is doing exactly what you'd do by hand.  tied high dice are set aside, and then the  pool is re-examined.  I keep a count of the number of tied dice to add back in for the donjon odds, but in the run that generated those graphs, ties were simply discarded per sorcerer.

If the one pool runs out of dice before a victory is scored, an additional die roll is added to each pool, and the pools are rescored.  Correct me if this is not the proper sorcerer way to break ties.

I see a note where graphs were altered to make the data continuous, but this is erroneous.  The graph cannot and should not be continuous.  You have either 1 or more victories for the large pool (positive numbers) or 1 or more victories for the small pool (negative numbers).  There should be a complete gap at 0 because ties do not stand.

There is a gap at 0.  ties do not stand.  "shifting up" the failure results was done simply as a workaround for gnuplot.  The labels on the axis of the graph are then "shifted down" to cancel out the original shift.  This has nothing to do with the original results generated, it is simply done for convenience of plotting.  The only reason I mentioned this was if you look at the raw data, the value of "-1" on the graph will be found as 0 in the data set, and "-2" on the graph will be found as -1 in the data set and so on.

More ties occuring with a higher ratio of those ties going to the bigger pool is exactly what I've been saying.

I believe that is born out.  My original arguments were based on my own faulty analysis of my data.

[Ron Edwards]

For reference's sake, all of the above posts were split from the New Sorcerer review thread.

Best,
Ron

[djarb]

I'm a new poster here, so howdy all :)

I've written and run a script similar to the ones described earlier in this thread, but the results are quite different. I'm seeing a marked advantage for the lower score when rolled with larger dice: for example, a pool of 3 versus a pool of 6 wins 18.8% of the time with d4, but 31.5% of the time with d20

I checked some of the simpler dice combinations (e.g 1 vs 2 for d2 and d20) against the theoretical probability, with results within a couple of percentage points of what the experiment turned up. I have reasonable confidence in these numbers.

You can find the results of the script here:
http://www.highenergymagic.org/sorcdice.txt

and the script itself here:
http://www.highenergymagic.org/sorcdice.py

If the effect of die face count is as strong as this indicates, perhaps some information about how they affect play should  go into the FAQ and/or later editions of the game.

[Lxndr]

The book DOES say that the # of sides on the die makes a difference, and that the larger dice favor the underdog.  I'm not sure if any more statistics are really needed (though they're nice to see).

Djarb:  Did you just roll a # of times, randomly?  And was it true-random (like with hexbits) or was it pseudo-random?  Or did you take every single possible roll in every case, and compare them that way?  To do it "right" (so to speak) I think you'd have to do the latter.

Or, I'm sure someone with more probability math knowledge than myself can figure out some sort of equation.

[djarb]

The book DOES say that the # of sides on the die makes a difference, and that the larger dice favor the underdog.  I'm not sure if any more statistics are really needed (though they're nice to see).

Ah, my bad.

Djarb:  Did you just roll a # of times, randomly?  And was it true-random (like with hexbits) or was it pseudo-random?  Or did you take every single possible roll in every case, and compare them that way?  To do it "right" (so to speak) I think you'd have to do the latter.

Or, I'm sure someone with more probability math knowledge than myself can figure out some sort of equation.

It was pseudo-random, seeded from the time, with a pseudo-random period of 2**19937-1. In case you don't know, that might as well be true-random for this purpose. If you're not convinced, I could use numbers from random.org

For each combination of die faces with dice pool sizes between 1 vs 1 and 9 vs 9 inclusive I rolled 10,000 times, recording who won and how many victories the winner got. I handled ties according to the rules and the errata, with ties that exhaust the smaller pool counted as 0-victory wins for the larger pool.

As to running an exhaustive search of the results: there are 167,960  ways to combine nine d20s, so for example 9 vs 9 d20 would produce 28,210,561,600 different rolls, and storing the results of that one combination would take up more than seven times the addressable memory on a 32bit computer, even if you only stored who won. Running the calculations that way would be a severe pain in the ass :)

A large random sample should be good enough, all things considered.