Equilab 1.2.6.0 seems to have a bug with Monte Carlo sims

    • Dex77
      Dex77
      Basic
      Joined: 08.09.2012 Posts: 3
      The results of the Monte Carlo sims in Equilab 1.2.6.0 seem to be skewed.

      Look at these results:

      http://www.pokerstrategy.com
      Equity Win Tie
      UTG 11.26% 10.06% 1.20% { Tc9s }
      UTG+1 9.77% 8.75% 1.02% { random }
      UTG+2 9.87% 8.85% 1.02% { random }
      MP1 9.93% 8.91% 1.02% { random }
      MP2 9.89% 8.84% 1.05% { random }
      MP3 9.69% 8.68% 1.01% { random }
      CO 9.97% 8.92% 1.05% { random }
      BU 9.89% 8.87% 1.02% { random }
      SB 9.87% 8.86% 1.01% { random }
      BB 9.86% 8.84% 1.02% { random }

      That sim was run over 1.35 billion hands, which is the most Equilab will allow (after 1.35 billion hands, Equilab stops the sim).

      Look at the equities of the nine random hands. After 1.35 billion hands, shouldn't the equities of the random hands have converged a lot closer to each other?

      For comparison, here are the results of an identical Monte Carlo sim over 1.35 billion hands using Pokerstove 1.24:

      Text results appended to pokerstove.txt
      1,350,213,005 games 4798.984 secs 281,353 games/sec
      Board:
      Dead:
      equity win tie pots won pots tied
      Hand 0: 11.330% 10.10% 01.23% 136326752 16655719.33 { Tc9s }
      Hand 1: 09.851% 08.83% 01.02% 119172531 13831314.08 { random }
      Hand 2: 09.853% 08.83% 01.02% 119197117 13834783.33 { random }
      Hand 3: 09.853% 08.83% 01.02% 119201869 13836695.42 { random }
      Hand 4: 09.853% 08.83% 01.02% 119196288 13833835.42 { random }
      Hand 5: 09.852% 08.83% 01.02% 119191727 13830624.17 { random }
      Hand 6: 09.853% 08.83% 01.02% 119208795 13831819.25 { random }
      Hand 7: 09.852% 08.83% 01.02% 119196209 13831289.58 { random }
      Hand 8: 09.851% 08.83% 01.02% 119183087 13831185.00 { random }
      Hand 9: 09.852% 08.83% 01.02% 119186346 13835146.75 { random }

      Notice how close the equities of the nine random hands are there? That's more in line with what would be expected, precision-wise, from the results of a 1.35 billion hand sim. Also notice the difference in equity of the Tc9s hand: 11.33% with Pokerstove vs. 11.26% with Equilab.

      So, just to see what would happen, I tried running another 1.35 billion hand sim in Equilab with 10 random hands against each other:

      http://www.pokerstrategy.com
      Equity Win Tie
      UTG 9.97% 8.90% 1.07% { random }
      UTG+1 9.97% 8.90% 1.06% { random }
      UTG+2 10.00% 8.95% 1.04% { random }
      MP1 10.05% 8.99% 1.05% { random }
      MP2 10.08% 9.00% 1.09% { random }
      MP3 9.96% 8.90% 1.06% { random }
      CO 10.03% 8.96% 1.06% { random }
      BU 9.98% 8.91% 1.07% { random }
      SB 9.88% 8.82% 1.05% { random }
      BB 10.10% 9.03% 1.07% { random }

      After 1.35 billion hands, each of the 10 positions should have converged very close to 10.00% equity, but they haven't. The equities are all over the place. So, I ran a second identical 1.35 billion hand sim, to see if that would happen again:

      http://www.pokerstrategy.com
      Equity Win Tie
      UTG 9.97% 8.90% 1.07% { random }
      UTG+1 9.96% 8.90% 1.06% { random }
      UTG+2 9.99% 8.95% 1.04% { random }
      MP1 10.05% 8.99% 1.05% { random }
      MP2 10.09% 9.00% 1.09% { random }
      MP3 9.96% 8.90% 1.07% { random }
      CO 10.03% 8.96% 1.06% { random }
      BU 9.98% 8.91% 1.07% { random }
      SB 9.87% 8.82% 1.05% { random }
      BB 10.10% 9.03% 1.07% { random }

      Now, that is interesting. The results are slightly different this time, but the results seem to be skewed the same way from position to position in each of the two sims! That suggests some kind of bug which is skewing the supposedly random results.

      For comparison, here are the results of a 1.35 billion hand sim with 10 random hands against each other using Pokerstove 1.24:

      Text results appended to pokerstove.txt
      1,350,118,150 games 10337.321 secs 130,606 games/sec
      Board:
      Dead:
      equity win tie pots won pots tied
      Hand 0: 10.001% 08.95% 01.05% 120811366 14208790.68 { random }
      Hand 1: 10.000% 08.95% 01.05% 120800791 14206270.68 { random }
      Hand 2: 10.000% 08.95% 01.05% 120809133 14201947.85 { random }
      Hand 3: 09.999% 08.95% 01.05% 120791819 14206285.85 { random }
      Hand 4: 10.002% 08.95% 01.05% 120824316 14208410.60 { random }
      Hand 5: 10.000% 08.95% 01.05% 120800241 14210100.77 { random }
      Hand 6: 10.000% 08.95% 01.05% 120811678 14203494.93 { random }
      Hand 7: 09.999% 08.95% 01.05% 120788313 14207451.27 { random }
      Hand 8: 10.001% 08.95% 01.05% 120814938 14208363.87 { random }
      Hand 9: 09.999% 08.95% 01.05% 120796999 14207509.20 { random }

      Again, the Pokerstove results look closer, precision-wise, to what would be expected from a 1.35 billion hand sim.

      What do you think?
  • 5 replies
    • VorpalF2F
      VorpalF2F
      Super Moderator
      Super Moderator
      Joined: 02.09.2010 Posts: 8,904
      It seems to me you're asking quite a lot.

      Since there are 1326 possible hole card combos,
      there are 1326 x 1326 x 1326 combos of random hands to deal with even if you have only 3 rand hands.

      That already is over 1.34 x 10^9, however the results would converge to 2 decimal places sooner than 1.35 x 10^9 hands.
      (it does -- at about 6x10^ hands)

      By the time you are dealing w/ 5 random hands, though there are 4 x 10^15th combinations to deal with, and I would not necessarily expect them to converge even at 1 decimal place.
      (and I'm wrong -- it converges to 2 dec places at about 2.5x10^8 hands)

      Perhaps pokerstove smooths the results between identical ranges?

      I'm interested to see what the software people have to say about this.
      I suspect that the answer lies in how the sampling is done, since there is no way that all hand combos are tested
    • YohanN7
      YohanN7
      Bronze
      Joined: 15.06.2009 Posts: 4,084
      There is certainly a bug in Eqiuilab. I have seen similar things, not in huge samples, but systematic "skewing" by position at the table when running "Monte Carlo". [Please don't ask me what I did exactly :rolleyes: ]

      It can be as simple as a need to "reseed" the RNG or as bad as a subltle error in the algorithm, but yes, there are bugs.

      EDIT: Hmm, guess I'm not being very helpful here. Well, I do remember that I simulated full ring. I was trying to "reproduce" the starting hand charts. Anyway, what I saw back then was "Yes, there IS without doubt a bug" and at the same time "Not worth reporting". Still not helpful, I know. I'll try to remember what I did and get back with that. It was very blatant.

      /Johan = :f_confused:
    • Dex77
      Dex77
      Basic
      Joined: 08.09.2012 Posts: 3
      Originally posted by VorpalF2F
      I'm interested to see what the software people have to say about this.
      Me too. Have they had a chance to look at this yet?
    • Dex77
      Dex77
      Basic
      Joined: 08.09.2012 Posts: 3
      This bug still exists in Equilab 1.2.8.0 and is consistently reproducible.

      http://www.pokerstrategy.com
      Equity Win Tie
      UTG 9.96% 8.89% 1.07% { random }
      UTG+1 9.97% 8.90% 1.07% { random }
      UTG+2 10.00% 8.96% 1.04% { random }
      MP1 10.04% 8.99% 1.05% { random }
      MP2 10.08% 8.99% 1.09% { random }
      MP3 9.95% 8.89% 1.06% { random }
      CO 10.02% 8.96% 1.06% { random }
      BU 9.99% 8.92% 1.07% { random }
      SB 9.88% 8.83% 1.05% { random }
      BB 10.10% 9.03% 1.07% { random }
    • UPAY4DINNER
      UPAY4DINNER
      Bronze
      Joined: 27.09.2009 Posts: 21,921
      Hey Dex,

      Sorry this thread has gone un-noticed from me. I have moved it to a more appropriate board and hopefully we can get someone to look into this.

      Happy new year!

      Gary