Alternatives to ICM?

Originally posted by jbpatzer
Driving back tonight I realized that there's a fundamental problem with the way we're testing these models. The defining feature of the Nash equilibrium is that it's the unexploitable strategy. Consider the rock/paper/scissors game. The Nash equilibrium is to randomly choose each object 1/3 of the time. If I now come along and say 'I have a better strategy. I'm always going to choose the rock, because they're hard and tough and must be best.', how can you test whether I'm right? Well, what you don't do is play me off against the Nash equilibrium strategy. This just breaks even. The way to show that you have a better understanding of the game than me is to play 'paper' all the time and exploit my poor model to win.

I think that property of the rock-paper-scissors game does not generalize to model poker games with hidden information. In model poker games, you can make unilateral mistakes, like folding the nuts, or betting with a hand which will fold out worse hands while getting called by better. If you look at the heads-up Nash equilibrium, the GTO strategies passively exploit non-GTO strategies. They don't tie with everything. In fact, it is very hard for a non-GTO strategy to do as well as the GTO strategy against a GTO opponent. If your opponent pushes AA only or ATC for 10 bb HU, while you call with a 38% range, you do significantly better than against a Nash pusher.

One property which does seem to generalize is that the Nash equilibrium can be unstable. If someone plays Rock 33%, Paper 33%, Scissors 34%, this is pretty close to the Nash equilibrium. However, the exploitatively optimal (EO) response is Rock 100%, far from the Nash equilibrium. Choosing an EO response is not a nice procedure, and it introduces discontinuities and instabilities. A way to fix this when you are searching for the Nash equilibrium in poker is to take a small step toward the EO response to your opponent's strategy. This might still diverge for rock-paper-scissors, but with the right parameters it converges for poker.

Originally posted by pzhon
A way to fix this when you are searching for the Nash equilibrium in poker is to take a small step toward the EO response to your opponent's strategy. This might still diverge for rock-paper-scissors, but with the right parameters it converges for poker.

Well, the way I'm iterating is to take a step of no more than one hand on each pass, so maybe that will do the trick. Which parameters do you mean in the above?

Anyway, I'll see what I can find. Presumably if it is an unstable equilibrium, that's more interesting? I wouldn't have thought that there's going to be an equivalent of 33% rock -> 100% rock, as I think any model that's close to ICM isn't going to be hugely exploitable by ICM, or vice versa.

Btw good point about the HU GTO strategy.

Wow luckily I don't play SNG's :f_biggrin:.

Looks like a sick sick thread, keep it up ♠_cool:

One type of step that you can take is to say

strategy[n+1] = p*EO response to oppt[n] + (1-p)*strategy[n]

You can vary p. In push/fold situations in poker, my guess is that a small value of p leads to convergence, while p=1 diverges. However, it could be that p would have to be a function of n which decreases to 0 (while the sum of p[n] diverges) to ensure convergence to the Nash equilibrium.

Even if you need to use a small value of p later once you get very close to the Nash equilibrium, it might be that using p = 0.3 would get you close enough for all practical purposes.

Originally posted by pzhon
One type of step that you can take is to say

strategy[n+1] = p*EO response to oppt[n] + (1-p)*strategy[n]

You can vary p. In push/fold situations in poker, my guess is that a small value of p leads to convergence, while p=1 diverges. However, it could be that p would have to be a function of n which decreases to 0 (while the sum of p[n] diverges) to ensure convergence to the Nash equilibrium.

Even if you need to use a small value of p later once you get very close to the Nash equilibrium, it might be that using p = 0.3 would get you close enough for all practical purposes.

I'm struggling with this at the moment. My current algorithm is

i) Start with an initial guess of the six pushing and calling ranges.
ii) For each of the six ranges in turn, determine whether the TEQ of the corresponding player can be increased by changing the range by one hand, according to the TEQ model that player is using.
iii) Repeat (ii) to convergence.

Not sure how I can have p in that, unless I'm effectively already choosing p to be as small as possible.

The problem is that I get convergence, but that the result depends on the initial guess, and there seem to be multiple equilibria. Not sure I know enough of the theory of these sorts of games to know quite where to go next.

One possibility is to replace the pure strategy S[k] = "push hand k and anything stronger" to allow strategies of the form S[k+9/10] = "push hand k 10%, and any hand k+1 or stronger." You can adjust the k+9/10 in smaller increments than you can adjust an integer k.

Do you get a unique (or at least canonical) Nash equilibrium strategy for each equity model? I think it would be of interest if you determined the Nash equilibrium ranges for ICM and ICM+, and computed the transition probabilities for 0-3 ICM players versus the complementary numbers of ICM+ players, either for the simplified model with randomized positions or keeping track of positions. This would let you compute the tournament equities of each state, and you can compare the performance of the first player in a (15-10-5) state who has two ICM opponents if he follows the ICM equilibrium strategy versus the ICM+ equilibrium strategy. What you would hope to see is that ICM+ extracts more equity out of each situation than ICM regardless of how many opponents are playing by the ICM.

Originally posted by pzhon
One possibility is to replace the pure strategy S[k] = "push hand k and anything stronger" to allow strategies of the form S[k+9/10] = "push hand k 10%, and any hand k+1 or stronger." You can adjust the k+9/10 in smaller increments than you can adjust an integer k.

Aha! Maybe.

Originally posted by pzhon
Do you get a unique (or at least canonical) Nash equilibrium strategy for each equity model? I think it would be of interest if you determined the Nash equilibrium ranges for ICM and ICM+, and computed the transition probabilities for 0-3 ICM players versus the complementary numbers of ICM+ players, either for the simplified model with randomized positions or keeping track of positions. This would let you compute the tournament equities of each state, and you can compare the performance of the first player in a (15-10-5) state who has two ICM opponents if he follows the ICM equilibrium strategy versus the ICM+ equilibrium strategy. What you would hope to see is that ICM+ extracts more equity out of each situation than ICM regardless of how many opponents are playing by the ICM.

Yes, I seem to get a unique solution for each model. I tried the equilibrium ICM+ strategy versus two opponents playing the equilibrium ICM strategy, and didn't see any significant equity difference between the models (graphs on page 5 of this thread). That was the point of my post about rock-paper-scissors. I thought that I could only see whether ICM+ is a more accurate model of TEQ by seeing whether it could exploit players using ICM.

Hmm. A lot going on here! I think, I really have to get more knowledge on game theory to keep track of this.

But I’ll suggest something anyway :

Imo, it’s worth to find out if ICM under- or overestimates the short stacks. Maybe, this could be done by running two simulations:

- Play two ICM nash players off against one player who plays ICM nash but will add one hand to his calling range if his stack is 7bb or less and he's the shortest stack (ofc not for overcalls).
- Do the same, with the only difference that the deviating player calls one hand less than ICM nash suggests if his stack is 7bb or less and he's the shortest stack.

Ofc, this is a very simplified model, but I think it’s safe to say that if our first exploitative player beats ICM, then ICM overvalues the shortstack. If the second method wins, the opposite is probably true. If both fail, this could be an indication that ICM’s misjudging of TEQ doesn’t distort the nash equilibrium too much.

What do you think about this?

Well, since I'm not running anything at the moment, and that's easy to implement, I'll give it a try!

Originally posted by muebarek

Imo, it’s worth to find out if ICM under- or overestimates the short stacks. Maybe, this could be done by running two simulations:

- Play two ICM nash players off against one player who plays ICM nash but will add one hand to his calling range if his stack is 7bb or less and he's the shortest stack (ofc not for overcalls).
- Do the same, with the only difference that the deviating player calls one hand less than ICM nash suggests if his stack is 7bb or less and he's the shortest stack.

Plus or minus one hand seems to do slightly worse than ICM. I also tried +5 hands, which did much worse. I'm starting to think that, just because ICM doesn't get TEQ right doesn't necessarily mean it doesn't get very close to the right equilibrium ranges.

I think I can now get a sensible equilibrium for ICM+ v ICM trying to exploit each other, provided that my initial guess is the ICM equilibrium. A few trials suggest that the new equilibrium is nearby. I'll also try this for MW v ICM and report back.

Originally posted by jbpatzer

Originally posted by muebarek

Imo, it’s worth to find out if ICM under- or overestimates the short stacks. Maybe, this could be done by running two simulations:

- Play two ICM nash players off against one player who plays ICM nash but will add one hand to his calling range if his stack is 7bb or less and he's the shortest stack (ofc not for overcalls).
- Do the same, with the only difference that the deviating player calls one hand less than ICM nash suggests if his stack is 7bb or less and he's the shortest stack.

Plus or minus one hand seems to do slightly worse than ICM. I also tried +5 hands, which did much worse. I'm starting to think that, just because ICM doesn't get TEQ right doesn't necessarily mean it doesn't get very close to the right equilibrium ranges.

Ach! Just found a big bug in my code for ICM v XXX. Will redo!

Originally posted by jbpatzer
I'm starting to think that, just because ICM doesn't get TEQ right doesn't necessarily mean it doesn't get very close to the right equilibrium ranges.

yeah, that's what I was meaning with "stability" earlier. maybe the nash ranges really aren't that susceptible to small changes in TEQ predictions.

but we can't really know.

that's why I suggested to do it the other way round with the simulations you're running atm. It lets us see how the EV changes by making slight changes in the ranges. it's a bit like pertubation theory. if you don't understand the full system just assume it to be a known system (in our case the ICM nash ranges) with some added pertubations.

too bad there was a bug. I'm looking forward to the results since I'm optimistic that they might give some insight in this.

edit: btw i'm expecting +5 hands to be quite off. i mean, it's in your model like if you would widen the shortstack's calling range by 20% in Hold'em.

Originally posted by muebarek
that's why I suggested to do it the other way round with the simulations you're running atm. It lets us see how the EV changes by making slight changes in the ranges. it's a bit like pertubation theory. if you don't understand the full system just assume it to be a known system (in our case the ICM nash ranges) with some added pertubations.

too bad there was a bug. I'm looking forward to the results since I'm optimistic that they might give some insight in this.

The bug was in the ICM+ v ICM calculations, not in the calculations based on your suggestion.

Originally posted by jbpatzer
The bug was in the ICM+ v ICM calculations, not in the calculations based on your suggestion.

ok. icm seems to have the calling ranges of the shortstack right (at least with the accuracy N=25 can provide)

Thought this might be amusing. (0.65:0.35 payout structure)

A maniac pushes all in every hand.

Inspired by Gabinr1's very funny all in every hand video (in Romanian, but still entertaining with the sound off). He claims he only did this once.

In contrast, a wimp always folds.

Maniac > Wimp! (but not by much)

I expect an exploitative strategy would do significantly better against both.

And I'm struggling to find anything much better or worse than straightforward ICM atm.

dont lie, ICM + reads

.