How many hands do I need do know my winrate?
Am I a winning player?
These are some basic/usual/recurrent questions for someone thinking of turning pro, or some beginner worrying about his performance...
I have been thinking on this subject lately, and since I can't reach my favourite poker room because of an earthquake near Taiwan, I am laying down my thoughts. Hoping it might interest someone although it is certainly not very well exposed.
You don't need to understand the statistical concept behind to understand the conclusions... if you don't bother about the details, go to the conclusions directly 
Also figures are coming from my knowledge of usual winrate/standard deviation that I observed on some players in FR NL.
First: why is it so hard to know our very own winrate? Because poker winnings are subjected to huge variance. On a 100 000 hands sample, if your winrate is somewhere close to 5BB/100 hands, the standard deviation is more close to 80BB... so if we use classical confidence interval, on a given sample size n, and assuming that the winnings are normally distributed (more on this later), our "true" winrate (in BB/100 hands) lies - with 95% confidence - somewhere in between:
100*(0.05 - 1.96*8/sqrt(n)) < winrate < 100*(0.05 + 1.96*8/sqrt(n))
Replacing n, we get the following interval:
0.05 BB/100 mains < winrate < 10 BB/100 mains
That is to say that the only 95% confidence info we get from a 100 000 hand sample (with a 5BB/100 hands observed on this sample), is that we are a winning (or even a breakeven) player.
For some player, with a lower variance style, the situation is better, but still nothig spectacular. Say we divide the variance by two for a very solid FR player, we still have a wide interval:
2.5 < winrate < 7.5
Second: About the normality assumption.
It is absurdely false, at least in the few samples I have observed. However, the beauty of the statistics is that if we sum 100 consecutive hands for example, then it tends to become normal (the famous CLM).
I have made this transformation on my samples. Confidence intervals are similar to those explained above.
Third: How many hands do we need to detect a small difference in our winrate with a given frequency?
Basically, if we apply a statistical test, we want to guarantee we can detect a 1BB difference on our winrate with a given frequency (say 80%). This question is answered by the power of the test.
Since we use approximately normal variables now (the sum of 100 consecutive hands), we want to use for example the t test (to test if our winrate is greater than a given 5BB/100 hands for example). It yields the following conclusions.
Conclusions:
Basically computing the power of the t test, we obtain the following info:
- to detect a 1BB difference on our winrate (still with the 80BB standard deviation), we need 100 000 cluster of 100 variables, i.e. 10 million hands...
- to detect a 2BB difference: 2 500 000 hands
- ...
Similarly, we can get the following info: for a given sample size, what difference can we detect
- 100 000 hands => 10BB!!!
- 1 000 000 hands => 3BB
- 2 000 000 hands => 2.2BB
Hope this can give you a "rigorous" answer to some of your unanswered questions And if you spotted some errors, please correct!
