Is ROI predictable?

    • akrammon
      akrammon
      Bronze
      Joined: 17.05.2009 Posts: 3,142
      Hey guys!
      I was wondering the following thing: Everyone on this forum says that you can talk about your ROI after a sample of 1000 SnGs, and with a good reason: after just a few played games you can get very nasty ROI which doesn't cover the truth at all.
      However... I think you can make a prediction, and give a possible interval for your "true ROI" based on your current, low-sample ROI.
      Has anyone given a thought on that? I think it could be pretty useful when moving up a limit.

      Because if your ROI is like 80% after 200 games then you are probably on a big heater, but it is also pretty unlikely that your first 1000 SnGs will show you a negative ROI. So maybe there could be a way (yo-ho, mathematics) to predict it. Maybe it has something in common with the connection between ROI and variance (swings, the probability of going broke, etc)?
      So if one has a ROI of 30% after a sample of 100 SnGs (with solid play), how much do you think his ROI could be after 1000 SnGs? My guess is 5-40%, but it's merely a wild guess.
      Any thoughts?

      regards
      akrammon
  • 5 replies
    • pzhon
      pzhon
      Bronze
      Joined: 17.06.2010 Posts: 1,151
      One approach is to use confidence intervals. A rough 95% confidence interval after n tournaments is about your observed ROI +- 310%/squareroot(n). So, even after 1000 tournaments, you don't know your ROI very precisely, since it would not be a big shock to be up to 10% off. If you play 1000 tournaments and have an ROI of 10%, you only have strong statistical evidence that you are a winning player, and that your ROI is no more than 20%, not that your ROI is really 10%. Similarly, if you play 100 tournaments with an ROI of 30%, you have strong evidence that your true ROI is not -10%, but you still don't know how much you are winning.

      You should be careful that confidence intervals are less reliable if you did not choose the amount of play ahead of time. It is much more likely to be 2.5 standard deviations above the mean at some point than at a preselected point. If you do not choose the amount of play ahead of time, then it may be better to widen the confidence interval by 50% to 3 standard deviations, 455%/sqrt(n), instead of the 2 standard deviations represented by 310%/sqrt(n).

      Also, you should consider a prior distribution. Some ROIs are less plausible than others, and you need more evidence to accept a highly implausible ROI. If no one in your games is sustaining an ROI over 10%, even those playing at that level professionally over years, then you should not casually assess your own ROI as greater than that.
    • akrammon
      akrammon
      Bronze
      Joined: 17.05.2009 Posts: 3,142
      Whoah, thanks it's a great help!
      But I still have one question: If I get it right, I can calculate an interval for my ROI based on my observed ROI of a certain amount of SnGs.
      However, I somehow feel that if thats interval is somewhat like -10% and +50% (for example), then the edges of the intervals have less chance to be true than the middle. Is this right?
      (sorry for the crappy phrasing, so, to make it clear: in the given example, Hero has more chances of having the middle of the interval (+20%) as ROI, than either the -10% or the +50%. So is there any easy way to caclulate the possibilites of certain ROIs?
      I'm sorry if this was somehow implemented in your previous post, I'm not an expert of the subject:) )
      And thanks again :)
      akrammon
    • gavinonymous
      gavinonymous
      Bronze
      Joined: 18.12.2008 Posts: 1,146
      there is a link to a simulator from another forum.

      http://www.zshare.net/download/70886672f9227868/

      maybe it's good? I'm downloading now.
    • pzhon
      pzhon
      Bronze
      Joined: 17.06.2010 Posts: 1,151
      It is impossible to conclude correctly, "There is a 1/3 chance your ROI is exactly 10%." That's not the type of answer math provides.

      You have strong (but not bulletproof) evidence against the ROIs outside the confidence interval. You have weak evidence against the ROIs toward the edges of the confidence interval in favor of the ROIs toward the center of the confidence interval. However, again, this evidence needs to be added to your prior assumptions. If you get mild evidence that the sun will not rise tomorrow, this should not overturn your previous conviction that the sun will rise tomorrow.

      If you want to use a different threshold than a 95% confidence interval, you can look at an 80% confidence interval or a 99.9% confidence interval instead.
    • akrammon
      akrammon
      Bronze
      Joined: 17.05.2009 Posts: 3,142
      Thanks, I think it's more or less clear now :)
      Thanks for the help, I appreciate it.