Alternatives to ICM?

    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      Nibbana and I have been talking recently about whether ICM really captures all of the advantages of having a big stack, particularly on the bubble, the point being that with a bigger stack you can often bully the table by profitably pushing a wider range of hands than with a small stack and (this is the key point) you are therefore more likely to pick up a profitably pushable hand.

      As an example, consider the bubble in a 6max SnG. There are three players left, and the payout structure is 0.65:0.35:0. My experience is that, if there are three roughly equal stacks, this is a very unstable equilibrium, and the higher the blinds, the more unstable it is. It only takes one player to successfully steal the blinds a couple of times and he can start to push even wider than before. It seems that a small chip advantage is more advantageous than ICM would indicate.

      Now, assuming equal skill levels, ICM is a reasonable way of calculating the equity of the remaining players. But it is just a model (the clue's in the name!) for the distribution of equity that, for chip stacks s1, s2 and s3, assigns equities E1, E2 and E3 to each of the players as functions of s1, s2 and s3, not the actual equity. We don't know what the actual equity is. Now, apart from the constraint that if the stacks are equal, all players have equal equity, and that the equities must sum to 1, we could try to use different functions E1, E2 and E3 that give greater weight to larger stacks.
      (i) What could we use instead?
      (ii) How would we decide whether this new model (ICM+) was better than ICM?

      Here's a suggestion for (i). Using E1 as an example, and normalizing the stacks so that s1+s2+s3=1, if E1(s1,s2) is the equity according to ICM, let e1(s1,s2) = min(1, (s1+2/3)^k*(4/3-s2)^k*E1(s1,s2)). In other words, we multiply the ICM equity by a function that is fairly flat and equal to 1 at s1=s2=1/3, where the stacks are equal, but which does not exceed the total equity available.

      Here's E1


      And here's e1 with k = 0.2, which is just a little bit different


      We can do something similar with e2 and e3, and may have to monkey around a bit to keep things sensible, but hopefully this illustrates the idea.

      Actually, it doesn't matter if the sum of the functions exceed 1. It's just a model for assigning value to a stack of chips.

      Now, question (ii). Once you have the functions for ICM+, you can work out the Nash equilibrium solution for any given stack sizes. This will be different to the ICM Nash equilibrium. Which is better? Well, it just needs some simulation. We simulate lots of three handed bubbles with some players using ICM and some using ICM+ and see who has the best ROI. Simple!

      Now I could go away and write some code to do this, but it seems like a real pain in the arse (is there a flush, is there a straight, is there a full house, ach! tedious) when I'm sure some of you clever computer scientists must have the appropriate software just lying around. I also know that there are other alternatives to ICM. Anyone want to work on this with me?
  • 159 replies
    • nibbana
      nibbana
      Bronze
      Joined: 04.12.2009 Posts: 1,186
      Good start jb. Should be an interesting and fulfilling project.
    • pzhon
      pzhon
      Bronze
      Joined: 17.06.2010 Posts: 1,151
      Ideally, if you really know the theoretical equity function, then your average equity after the hand should equal your equity before the hand. That the Nash ICM calculator shows that players gain or lose equity indicates that the ICM is not perfect.

      The largest differences come from the blinds and your position. You tend to lose equity when you are in the blinds, and you tend to gain equity when you are outside the blinds.

      If you average over all of the positions, you see that there are some stack size effects, mainly that the big stacks may tend to gain equity on average. This indicates that the ICM may undervalue big stacks. There are also some relative position effects. It may hurt you if you have a worse relative position against a big stack so that you get few good stealing opportunities.

      So, the true equity should depend not only on the stack sizes, but also on the blinds, your position relative to the blinds, and your position relative to each other stack.

      Anyway, I'm not averse to looking for something better than the ICM, but there are a lot of advantages of using the ICM with minor adjustments. The ICM is simple, easy for computers to compute, and it has some nice local properties such as continuity and positive risk aversion heads-up. These are not advantages to give up lightly, even if another equity model seems slightly more logical or fits data slightly better.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      Originally posted by pzhon
      Anyway, I'm not averse to looking for something better than the ICM, but there are a lot of advantages of using the ICM with minor adjustments. The ICM is simple, easy for computers to compute, and it has some nice local properties such as continuity and positive risk aversion heads-up. These are not advantages to give up lightly, even if another equity model seems slightly more logical or fits data slightly better.
      Thanks for reading my post. Looking for something that's ICM with minor adjustments is exactly what I'm thinking of. Heads up, I can't see that there's a sensible alternative to ICM, but I want to focus on a three-handed bubble. And also, my idea is not to find an equity model that fits the data, but one that performs better than ICM as tested by simulation. For example, if two players are using the ICM Nash ranges and one the ICM+ ranges, can the parameters for ICM+ be tuned to give a positive ROI?

      I realize after some thought that my choice of modified function is a bit silly. An equity function e(s), with e = (e1,e2,e3) and s = (s1,s2,s3), stacks normalized so that s1+s2+s3=1 and prize structure P:1 should satisfy

      (i) ei>0 for i = 1,2,3
      (ii) e1+e2+e3 = 1+P for all s
      (iii) ei an increasing function of si for i = 1,2,3
      (iv) ei = 0 when si = 0 for i = 1,2,3
      (v) e1=e2=e3=(1+P)/3 when s1=s2=s3=1/3

      ICM satisfies all of these criteria with a nice simple formula so I can see the attraction. The challenge is to modify ICM so that these criteria are still satisfied and there are some extra parameters (maybe functions of blind level and position?) which can be optimized to produce a positive ROI against ICM.

      So the plan would be

      (i) Suggest an ICM+ equity function e(s; k), where k is a vector of parameters, possibly functions of position and blind level.
      (ii) Write a simulator that plays off an ICM+ Nash player against two ICM Nash players (or indeed vice versa, as the outcome might be different) over thousands of games to produce a function ROI(k).
      (iii) Wrap an optimization algorithm around this to determine the optimal value of k. Some sort of genetic algorithm would probably be the way to go here.

      I can think about (i), and I have a good idea of how to do (iii). Do you have code that could do (ii)?

      I use MATLAB for all my simulations (I specialize in pde's, so this is new to me!). Not sure what you like to use.
    • pzhon
      pzhon
      Bronze
      Joined: 17.06.2010 Posts: 1,151
      I do not yet have code which would do step ii, pitting someone using the equity function ICM+ against two ICM players.

      It is not clear to me whether you want the ICM+ player to assume that his opponents are playing the Nash ICM+ ranges, or to play to exploit players (according to ICM+) who are using the Nash ICM ranges.

      Because it is 3-handed, it is conceivable that the other two players are essentially colluding against someone with a better equity model. Perhaps the player with a better equity model would still come out ahead, but not necessarily by an amount which represents how much better the equity model is. So, the ICM+ model which performs the best against ICM players might not be best in other senses.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      Well that's research right? All interesting questions.

      It would probably do me good to write some code myself. I also have some ideas for interesting hold'em-like games that would be easier (for me) to code. I'll see what I can come up with. What I need is a PhD student! Or at least some funding for one. Maybe I can scam some at work. :f_cool:
    • gedwashere91
      gedwashere91
      Bronze
      Joined: 20.07.2009 Posts: 2,387
      I've been thinking this stuff for a while but have never had the mathematical know-how to put it into any sort of real model.
      It's times like these I wish I wasn't doing a Biology major. :P
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      Sat in my conference today and worked out how to calculate the Nash equilibrium for the HU version of the Hold'em-like game I mentioned above. Got back to my room to code it up and produce some graphs and realized that, as I have mrsjb's computer, I don't have MATLAB with me, so I can't do it! Am pissed off, as it will now have to wait until the weekend. Poo!

      Just to put some details down anyway, my idea is that each player has a hand, h, drawn from the uniform distribution on [0,1], and there is a hand strength function H(h), which decreases with h. If two players have hands h1 and h2, the probabilities that either wins are H(h1)/(H(h1)+H(h2)) and H(h2)/(H(h1)+H(h2)) respectively, with the obvious extension to more than two players all in. I think that the function H(h) = 2-h+2*exp(-50*h) would be a good one, although I can't plot it right now. The best 2% or so of hands would be strong favourites over other hands, but the rest of the hands would be similar in strength. Even the worst hand (h=1) would be 20% to win over the best hand (h=0). Of course, this doesn't capture all the subtleties of Hold'em hand strengths, but it lets me play with Nash calculations, game simulations and ideas about optimization of ICM v ICM+, which will take me a while to get my head around. I have the equations for the HU Nash equilibrium on a bit of paper in front of me. Aaaargh! So frustrating.
    • matel17
      matel17
      Bronze
      Joined: 11.07.2010 Posts: 1,278
      Is there any hope for mathematical **tards to get any of this?? :f_confused:

      I love this kind of theory but I never studied any maths beyond what is legally required (lol).

      :f_cry:
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      Well, you can learn the idea of the ICM and use ICM trainer to get an idea of the right ranges to push without knowing any maths. You don't need to know any maths to play around with SnGWiz. It's the ideas that are important. All I'm suggesting is that a slight tweak to ICM might give more profit. If that's true, you could use ICM+ trainer and SnGWiz+ just as easily.
    • Waiboy
      Waiboy
      Bronze
      Joined: 18.09.2008 Posts: 4,877
      Originally posted by jbpatzer
      I realize after some thought that my choice of modified function is a bit silly.
      Yes.

      My work is done here.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      Originally posted by Waiboy
      Originally posted by jbpatzer
      I realize after some thought that my choice of modified function is a bit silly.
      Yes.

      My work is done here.
      Lost for words. And that doesn't happen often.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      So I managed to remote desktop to my computer at work and use that to find the Nash ranges for my little model. I changed the hand strength function to H(h) = 1.2-h+2.8*exp(-50*h), as this gives ranges more similar to the Hold'em HU Nash ranges. Here's H


      and here are the pushing and calling ranges, hp and hc. s is stack size in bb. With stack sizes less than about 2bb, BB should always call. Note also that with stacks of 10bb, the pushing and calling ranges are about 60% and 40%, similar to hold'em.


      I'm now going to write some code to repeatedly play off two players and see whether someone playing the wrong ranges loses the amount that they ought to to someone playing the Nash ranges, just to check my analysis.
    • Waiboy
      Waiboy
      Bronze
      Joined: 18.09.2008 Posts: 4,877
      Originally posted by jbpatzer
      Originally posted by Waiboy
      Originally posted by jbpatzer
      I realize after some thought that my choice of modified function is a bit silly.
      Yes.

      My work is done here.
      Lost for words. And that doesn't happen often.
      It's all I had. My brain hurt after the second sentence. And I could agree that you had, in fact, realized something after some thought.

      Going now (again) to stick my head in a freezer.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      And here's the EV for pusher and caller. It's better to be the pusher at smaller stack sizes, and the caller at large enough stack sizes. Can't remember whether this is the way it works in Hold'em too.

      EDIT: Checked it out. Hold'em is like this too! :f_cool: But the crossover point is at about 7.5bb. :f_mad:


      Wrote a little Monte Carlo simulator, and it agrees with these EV calculations. Also checked by evaluating the EV for ranges close to the calculated Nash ranges that I got the Nash ranges right. It all checks out. Now, a three handed bubble! I'm going to do the calculations based on Chip EV first and make sure that all works. This is going to confuse me, so may take a little while to get my head around.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      As predicted, a three handed bubble (of course for cEV, it's not a bubble) confused me a lot. Particularly when everyone is all in and you have to divvy up some of the chips to one person and some to another. Here are the pushing and calling ranges with equal stacks.

      The BB overcall ranges are unexpectedly wide at smaller stack sizes, but 'computer says yes', so what can you do!

      And here are the cEVs for each player, again checked using Monte Carlo simulation.

      These are qualitatively the same for Hold'em, so it's all going OK so far.

      Now that I can work out cEVs, the next step is to wrap the ICM functions around them, work out the Nash ranges based on the EV predicted by that, and see how they looks.

      This is how I feel.
    • muebarek
      muebarek
      Bronze
      Joined: 31.07.2008 Posts: 532
      interesting topic!

      about a year ago i also tried to develop some ideas on improved tournament equity models but i was going in a slightly different direction since i'm MTT player. my main problems were that
      - ICM overvalues the short stacks (relevant for final tables)
      - every model i'm aware of completely ignores edges (which can be huge on a MTT final table compared to a SingleTable SNG since there are always huge fish and people who are on scared money)
      if someone is interested, i post something about my attempts on that.

      but i'm interested in this future game related stuff as well. good job so far, jbpatzer! i'm looking forward to see your nash results in this model using ICM functions!
      i like your approach of using a "holdem-like" hand strength function to get a qualitative understanding (somehow this idea never occured to me :f_frown: . i equilated a lot back then and tried to fit the equities in a function which didn't work too well ).
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      Glad someone else is interested. However, I am finding that there are some technical issues with using my simple model. For example, when three players are all in, to work out the EV you need to evaluate a triple integral. It's a real bottleneck. Presumably for the same situation in Hold'em, there's a triple sum that needs to be done, and similarly for more players simultaneously all in. I must be missing a trick though because online ICM Nash calculators do this in a fraction of a second! Do you or pzhon know the technical details?
    • muebarek
      muebarek
      Bronze
      Joined: 31.07.2008 Posts: 532
      Unfortunately, I can't help you out with technical details there. Tbh, runtime issues got me stuck as well (i'm just capable of abc-programming).

      But afaik, the online Nash calculators are also restricted to 3 players simultaneously all-in.
      Furthermore, according to their FAQs, they don't do an exhaustive calculation but already start with somewhat "well chosen" (i don't know how) ranges and then try to improve the EV for one player by adjusting his ranges assuming the other players' strategies as constant ("Fictitious Play"). This is done for all players and then repeated several times (often something around 300 iterations i think).
      [Source hereand here]
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,944
      Hmmmm. Interesting. OK. Here's my new idea. With continuous distributions of hands (i.e. a 'hand' is a number drawn randomly from [0,1]), you always end up with multiple integrals to evaluate. How do you evaluate multiple integrals? You split them up into little bits and add them all up, and if it's not accurate enough, you split them up into even more little bits until you get an accurate answer. So you're actually adding up lots of discrete bits, but the number of bits is determined by the accuracy rather than any practical considerations.

      So.... instead of a hand being drawn from [0,1] and then approximating [0,1] with lots of little bits, I should just say there are N possible hands, with 1 the best and N the worst. I can still use a hand strength function, so that f(h) becomes f(i/N) with i = 1,2,....N., and then all the multiple integrals become multiple sums, but there's no approximation involved. You just do the additions. In hold'em, N=169. Of course, it's not as simple as that with card removal effects and AK>76s>22>AK>..., etc. But if I take N=100 and redo all the above analysis (sigh!)...

      I'll leave this post here to see if anyone will cast any pearls of wisdom before me before I try this again. Term starts tomorrow. Bollocks!