I've been trying to finish my next post involving a case study of the misuse and abuse of statistics to claim to prove global warming skeptics possess certain negative traits (my last post regarding this can be found here). Unfortunately, a number of things are getting in the way. Of special note is it's difficult to talk about statistics as I've largely lost faith in the laws of probability.
I've talked about this before. To make a long story short, I once combated boredom in high school by writing a program for my graphing calculator to let me play Rock, Paper, Scissors. I didn't expect anything to come of it, but it seemed like a decent diversion. I became more interested in the program over time as I consistently won more games than I should have.
A couple months ago, I decided to test those results by writing a new program in a different programming language. My thought was the previous results might have been caused by the calculator having a weak random number generator that led to its "random" choices being predictable.
To test that possibility, I wrote a new version of the program in the Python language. After 3,351 matches, I had 1,220 wins, 1,079 losses, 1,052 ties. That's a win rate of 36.41%, well above the 1 in 3 chance one would expect. Using the binomial theorem, we can determine those results are statistically significant at the 99.99% level. To give you an idea of the odds we're talking about, Here is a visualization of my results after 2,236 matches (798 wins, or a 35.7% win rate):
Here is the same visualization for after I completed all 3,351 matches:
The black curve indicates the probability density function for each occasion. The further any point on the line is from 0, the more likely it is for that outcome to happen. The area under the curve indicates the full range of outcomes and their likelihoods. For the sake of our purposes, we want to know how likely it is to get at least the specified number of wins. That is represented by the area of the curve left of the red line. The area to the right of the line displays the probability of getting more wins than the specified number.
Technical details aside, there is simply no way I should have won as many times as I did. There are several possibilities. One possibility is the one I considered with the graphing calculator - maybe the RNG of my game was flawed in a way that could allow a person to win more often than they would otherwise be expected to. To test this, I switched the algorithm I was using to generate random numbers for a stronger one, one suited for cryptographic purposes. In theory, there is no way it should produce predictable results, but There may be some flaw in the way this algorithm is being used that affects things.
Rather than go on with words, I'll just show you the results for each batch of matches I performed:
Wins: 117 Losses: 120 Ties: 99 Wins: 97 Losses: 115 Ties: 110 Wins: 175 Losses: 177 Ties: 174 Wins: 121 Losses: 111 Ties: 80 Wins: 115 Losses: 107 Ties: 88 Wins: 157 Losses: 122 Ties: 147 Wins: 90 Losses: 65 Ties: 81 Wins: 133 Losses: 94 Ties: 101 Wins: 68 Losses: 74 Ties: 63 Wins: 67 Losses: 72 Ties: 92 Wins: 112 Losses: 93 Ties: 78 Wins: 96 Losses: 94 Ties: 98 Wins: 129 Losses: 126 Ties: 131 Wins: 130 Losses: 135 Ties: 139 Wins: 255 Losses: 199 Ties: 226 Wins: 72 Losses: 68 Ties: 50 Wins: 67 Losses: 61 Ties: 66 Wins: 68 Losses: 62 Ties: 63 Wins: 74 Losses: 65 Ties: 68 Wins: 122 Losses: 111 Ties: 100 Wins: 100 Losses: 101 Ties: 112 Wins: 79 Losses: 71 Ties: 60 Wins: 71 Losses: 65 Ties: 84 Wins: 85 Losses: 77 Ties: 67 Wins: 50 Losses: 32 Ties: 40 Wins: 51 Losses: 40 Ties: 37 Wins: 61 Losses: 64 Ties: 53 Wins: 59 Losses: 54 Ties: 50 Wins: 104 Losses: 84 Ties: 78 Wins: 77 Losses: 66 Ties: 65 Wins: 93 Losses: 75 Ties: 84 Wins: 82 Losses: 80 Ties: 70 Wins: 138 Losses: 128 Ties: 113 Wins: 96 Losses: 103 Ties: 96 Wins: 147 Losses: 123 Ties: 148
I apologize for how long that list is, but I felt it was important to show some of the detail which has gone into my testing. These tests were performed by playing Rock, Paper, Scissors an unfixed number of times, with the total results and individual outcomes recorded in a timestamped data file.
Because subconscious biases might influence when I would choose to stop any particular run, the individual sets of results aren't too important. Some sets have fairly even results, some lean toward losses or ties, but overall, my tests leaned significantly toward wins. Here are the totals:
Wins: 3558 35.55% Losses: 3234 32.31% Ties: 3216 32.13% Total: 10008
To show how unlikely these results, here is the same visualization as before:
The chances of achieving these results by chance are nearly one in a million. It is difficult to explain these results, and no explanation I can come up with is comforting. Here are the explanations I can think of:
1) It's just chance.
2) The laws of probability don't work as expected.
3) I have psychic powers.
4) I am able to beat the RNG of my game.
The first explanation seems silly. It's practically impossible to prove something like this isn't just "luck." At the same time, it's difficult to believe things are just due to "luck" when a person predicts something with a one in a million chance of success and is right. If anyone wishes to pursue this explanation, I can continue playing the game to see if my results hold up.
The second explanation would be world-shattering. If we could prove the universe isn't "random" like it is believed to be, it would have philosophical, theological and metaphysical consequences beyond anything we've ever seen. Every religion and philosophy in the world would have to take these results into account.
The third explanation is perhaps the most worrying for me as I don't believe in psychic powers. Not only would being a psychic change a great deal about how I view myself, it would pose serious problems as I would be (as far as I know) the only psychic in the world who could prove their abilities are real.
The fourth explanation is the one I hope is true. I would like to test it by rewriting this program in other languages with other algorithms used for the "randomness" of the game. Preferably, I would also try it out on different machines with different operating systems. I'd probably want to try other things as well. Perhaps I'd find these results are limited to certain setups.
That wouldn't stop these results from being worrying though. Let's suppose both algorithms I've used in Python and the algorithm I used on my graphing calculator are flawed in a way that makes their results somewhat predictable. Let's suppose I am just subconsciously picking up on the flaws in these algorithms (or their implementations). Think about what that would mean. It would mean I was able to "beat" RNG algorithms after only a few hundred or few thousand attempts.
That shouldn't be possible. Remember, the RNG is selecting only one of three outcomes (rock, paper or scissors) but it does so by producing a large, "random" string/number. This game compresses that long string/number into one of three values. That results in a huge loss in information. If a human subconscious could pick up on patterns after even a few thousand guesses with such a great loss of information, computer analysis should be able to prove the existence of patterns in these RNGs with relatively little trouble.
They can't. We know they can't because this is an active field of study. People try to find weaknesses and flaws in RNG systems all the time. If we could find easily discernible patterns like these results suggest, the RNG systems would lose a ton of value. This would have huge implications on all sorts of things, including any number of security systems.
None of these explanations are comforting. Every single one of them raises serious concerns. I should be freaking out a bit. Maybe more than a bit. The only way these results aren't cause for serious concern is if I've somehow screwed up my program, but no test I've come up with indicates such. People who've tried it for themselves don't get the same results as me, I can't find any patterns in the outcomes,
If anyone wants to check for themselves, I posted the data (and code) of the earlier batch of tests online a while back. You can find a link to them and a description of what files are included here. You'll find results from a couple other tests I performed as well, tests which show the same ability for me to (seemingly?) predict outcomes.
I don't know what more to say. I try to be calm while discussing this, but it is freaking me out. I don't know what I'll do if these results hold up with a different program using a different approach to generating "randomness." I just want things to make sense.
As a final note, I am happy to share any details or information people might like about these tests, including all code and data files. I am also happy to take any suggestions on how to improve any aspect of my approach or to test my conclusions. And finally, if anyone has a better explanation than the ones I've mentioned in this post, I would love to hear it. I feel like I'm losing my mind right now.