The 3bet/4bet/5bet/shove game

    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      As I mentioned in my blog, I've been playing around with a model for the 3bet/4bet/5bet/shove game.

      Here are my assumptions

      i) UTG and button have 100bb stacks.
      ii) We ignore everyone else, but take into account the 1.5bb of dead money in the pot (Not sure this is v important though)
      iii) UTG open raises to 3.5bb with 12% of his hands.
      iv) Button 3bets to 10bb with KK+ balanced by some mid suited connectors (long tirade in my blog from various people telling me that flatting QQ and AK IP is more +EV than 3betting (I play NL10 rush)).
      v) UTG 4bets to 25bb with KK+ and also with some hands that will be bluffcatchers (e.g. JJ,QQ).
      vi) Button moves AI with KK+ and some fraction of his bluff range.
      vii) UTG calls with KK+ and some fraction of his bluffcatchers.
      viii) No card removal effects.

      At the river, there are four possibilities

      1) UTG value range v button value range = KK+ v KK+ = 0.5/0.5
      2) UTG value range v button bluff range = KK+ v e.g. 76s = 0.775/0.225
      3) UTG bluffcatchers v button bluff range = e.g. JJ v 76s = 0.775/0.225
      4) UTG bluffcatchers v button value range = e.g. JJ v KK+ = 0.1825/0.8175

      Some notation
      ------------------

      v1: proportion of hands that UTG opens with and has in his value range = 0.14 (12 combos out of 164)
      v2: proportion of value hands in button's 3bet range (1-v2 = bluffs)
      b1: UTG 4bets with v1+b1 of hands he raises, and folds 1-v1-b1
      b2: proportion of bluff hands that button 5bets (he raises v2+b2(1-v2) and folds (1-b2)(1-v2))
      b3: proportion of bluffcatchers that UTG calls AI with (calls (v1+b1b3)(v1+b1), folds (1-b3)b1/(v1+b1)

      After some algebra (ugh!), which I'd be very happy for someone to check, I get that the EV for each player is

      UTG = -3.5(1-v1-b1)+11.5(v1+b1)
      +b2(1-v2)(-11.5(v1+b1)-25(1-b3)b1+56.16(v1+b1b3))
      +v2(-11.5(v1+b1)-25-(1-b3)b1+0.75v1-63.23b1b3)

      Since UTG chooses b1 and b3, the best he can do is to choose b1=b3 = 0, i.e. value only.

      Button EV = 5(1-v1)-10v1(1-b2)(1-v2)+0.75v1v2-54.66v1b2(1-v2)
      +b1(-5-10(1-b2)(1-v2)+26.5(v2+b2(1-v2)))
      +b1b3(-26.5(v2+b2(1-v2))-54.66b2(1-v2)+64.73v2)

      Now we can make the button indifferent to UTG's choice of bluffcatching frequency by making the factors that multiply b1 and b1b3 zero. This gives v2 = 0.28, b2 = 0.18. This is the Nash equilibrium strategy, with which the button gets +2.5bb and UTG loses 1bb (the power of position, and don't forget the 1.5bb from the blinds).

      This means that the unexploitable way for the button to balance his 3betting range is to 3bet with KK+, which is 12 combos, and since v2 = 0.28, about 31 combos of midsuited connectors (e.g. 45s, 56s, 67s, 78s, 89s, oops, ran out, so maybe need some one gappers too!). UTG should only 4bet for value, and, since b2 = 0.18 the button should shove with KK+ and 18% of his bluffs (about 5 combos, e.g. 76s and 87h).

      Some questions?
      ---------------------

      1) Can anyone check this? I've been known to bugger this sort of thing up before!
      2) How can we change the assumptions to make things more realistic? Remember, this is mathematical modelling, so the name of the game is
      Model -> results -> compare with reality -> change model -> repeat until bored/satisfied.
      3) Can I be arsed to redo this with variable bet and stack sizes??
  • 17 replies
    • Fagin
      Fagin
      Bronze
      Joined: 06.05.2008 Posts: 544
      I am not sure about the range you ascribe to UTG here.

      You seem to insist on KK+ which would seem reasonable, however I have recently found myself 5bet shoving KK+ from MP2, MP3, CO and being called by UTG with AQos+, JJ+. The callers have, a couple of times, insisted that this is "standard" play despite being oop, "because position is irreleveant".

      It may be that you play a higher limit with better players - I have found these plays at NL10 recently.

      It amused me at the time despite losing a couple of these clashes.

      I have no idea of the math and am unsure if these plays I have seen are just a few isolated donks, but some further imput from others may help here.

      Cheers
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      I can try it again for different ranges and bet sizes. In my view, the important point is that this sort of game can be solved fairly easily. I'd be interested to know whether anyone else has tried this sort of calculation before.
    • muebarek
      muebarek
      Bronze
      Joined: 31.07.2008 Posts: 532
      I checked the algebra for UTG besides the numbers 56.16 and -63.23 (too lazy :f_biggrin: ) and it seems correct. I didn't check EV(BU) since it's
      EV(BU) = 1.5 - EV(UTG).

      But I think the underlying simplifications/assumptions are too restrictive. Imo, the biggest problem here is that UTG isn't rewarded at all for raising anything but the nuts. There is no term in the EV calculation that lets UTG win the 1.5bb in the pot if BU does not play. In this calculation, BU gets KK+ or suited connectors every hand and UTG will therefore get 3bet 100% of the time by a fairly strong range. Of course he's gonna lose since he's forced to play like 11,xx% more hands than he would normally in this setting.
      Therefore it would make more sense to start with a blind battle situation where getting a walk (for the BB) and "raise and take it" (for the SB) are taken into account [in UTG vs BU there are a lot of other players which perturb the game]
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      Originally posted by muebarek
      Therefore it would make more sense to start with a blind battle situation where getting a walk (for the BB) and "raise and take it" (for the SB) are taken into account [in UTG vs BU there are a lot of other players which perturb the game]
      Aha! Good idea. I'll try that. Later.
    • fusionpk
      fusionpk
      Bronze
      Joined: 21.01.2010 Posts: 1,692
      @Fagin: I don't think giving UTG a range of KK+ is all that bad here if we assume unknown tag, with reads for sure his range changes and for sure the argument about 3b'ing QQ for value starts again. (Altho I flat QQ most the time on BU, I still agree that vs many opps its more +EV to 3b QQ, I just think vs unknown who goes broke v tight and rarely flats (I think this is most tags) we own ourselves). Anyway not wanting to get into this argument just wanting to back jb's range assumptions up. The plrs u mention are bad/probably losing regs and therefore we should adapt to their ranges, i think we are just assuming taggy unknown?

      I like your analysis jb. Not that I completely understand all the math :f_confused:

      I guess its quite hard to mathematically model this, since we assume separate events. For example, if we assume we flat JJ/QQ and that we are 3b'ing KK+ and bluffs, we soon get exploited by UTG and it becomes mathematically wrong (UTG will start 4b'ing differently and flatting in some spots), which obviously we can't account for in the model.

      I think in 3b/4b/5b wars its pretty simple, we need to be one step ahead of the other person. For example, start off with a pretty sick tight value range & some bluffs. Once he adapts we re-adapt. If he starts 4b'ing wider range we start getting in slightly wider (we can now get QQ/AK in and be happy that its more +EV than flatting), if he starts 4b'ing alot of our 3b's (he is light) we can start adding alot of pairs to our 3b range and consider jamming them over his 4b's. If he begins to flat we can completely depolarise ourself and 3b a much bigger value range keeping our go broke range still tight.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      Originally posted by fusionpk
      if we assume we flat JJ/QQ and that we are 3b'ing KK+ and bluffs, we soon get exploited by UTG and it becomes mathematically wrong (UTG will start 4b'ing differently and flatting in some spots), which obviously we can't account for in the model.

      I think in 3b/4b/5b wars its pretty simple, we need to be one step ahead of the other person. For example, start off with a pretty sick tight value range & some bluffs. Once he adapts we re-adapt. If he starts 4b'ing wider range we start getting in slightly wider (we can now get QQ/AK in and be happy that its more +EV than flatting), if he starts 4b'ing alot of our 3b's (he is light) we can start adding alot of pairs to our 3b range and consider jamming them over his 4b's. If he begins to flat we can completely depolarise ourself and 3b a much bigger value range keeping our go broke range still tight.
      The point is that, within the context of this model, the strategy I gave above is unexploitable. It's a Nash equilibrium. If the button uses these ranges, UTG can't exploit him whatever he does. The button can do better if UTG starts bluffing, but then he opens himself up to being exploited, and so it goes on.

      Actually, a profit of the blinds + 1bb is pretty meagre for a strategy that involves making such big bets, and if I added in the effect of the rake, I doubt whether the button makes any money.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      Originally posted by muebarek
      There is no term in the EV calculation that lets UTG win the 1.5bb in the pot if BU does not play. In this calculation, BU gets KK+ or suited connectors every hand and UTG will therefore get 3bet 100% of the time by a fairly strong range.
      This is what happens when UTG opens, the button 3bets and the blinds fold. In other hands, UTG may well pick up the blinds, but the game models this subset of hands only.
    • arisko
      arisko
      Bronze
      Joined: 23.09.2009 Posts: 392
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      A world without maths.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      And of course card removal effects are HUGE in the game as I set it up. If both players have a value range of KK+, and one of them looks down at KK, the other's value range is 6 combos of AA and 1 of KK. Eek! More thought needed.
    • Waiboy
      Waiboy
      Bronze
      Joined: 18.09.2008 Posts: 4,877
      Originally posted by jbpatzer
      A world without maths.
      I'm pretty sure hairdressers and telephone hygienists have something pretty similar to this. Do you think telephone hygienists actually exist, or is this something that Douglas Adams made up because it's just an awesome concept?

      /derail.
    • muebarek
      muebarek
      Bronze
      Joined: 31.07.2008 Posts: 532
      Originally posted by jbpatzer
      This is what happens when UTG opens, the button 3bets and the blinds fold. In other hands, UTG may well pick up the blinds, but the game models this subset of hands only.
      yep. you're right. so you should have the correct bluffing frequencies for BU in this particular game. card removal does change the absolute amount of combos you want to bluff with but (if I'm not forgetting something important) won't have impact on BU's frequencies as they only depend on betsizes, deadmoney and all-in equities which (in your example) don't change by taking account of card removal.

      If you approach the problem by only considering a subset of hands, looking at the EV-value is ok to compare two strategies (within this game) with each other but the absolute value doesn't have much informative value since one picks a setting which puts BU in a favorable situation - so of course he's winning :s_cool:
      Finding a nash equilibrium in this special case allows you to be unexploitable vs Villain's changes of 3bet/fold-, 4bet/fold- and bluffshovingranges but you can be exploited by Villain playing another game (i.e. using different value3betranges/openraisingranges). In UTG vs BU, your model should be pretty good against regs who mainly 4bet or fold from UTG since the valueranges KK+ are usually exactly what they're getting in.
      But on the other hand, if UTG only value4bets with KK+ and mainly 4bets for value and folds otherwise [and sticks to this game's nash-ranges] - 3betting any other hand than AA for value wouldn't make sense for the BU.
      For UTG: against a BU-3betting range which is so polarized, I think, there's not need to have a 4betting range at all.

      My guess for the real nash equilibrium in a 3bet/4bet/5betshove game UTG vs BU is that it would have looser valueranges (only value-"getting it in" with KK+ is imo already an exploitative play adjusting to the fact that people are too tight [and so it will show a higher EV in reality than playing the actual nashEQ]), but determining the correct valueranges is difficult since it depends on all the other people to act after UTG as well.
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      Originally posted by muebarek
      If you approach the problem by only considering a subset of hands, looking at the EV-value is ok to compare two strategies (within this game) with each other but the absolute value doesn't have much informative value since one picks a setting which puts BU in a favorable situation - so of course he's winning :s_cool:
      Kind of. Actually he doesn't beat the rake. :f_mad:




      My guess for the real nash equilibrium in a 3bet/4bet/5betshove game UTG vs BU is that it would have looser valueranges (only value-"getting it in" with KK+ is imo already an exploitative play adjusting to the fact that people are too tight [and so it will show a higher EV in reality than playing the actual nashEQ]), but determining the correct valueranges is difficult since it depends on all the other people to act after UTG as well.
      I will have to redo the calculation with different ranges, and also see whether I can include the range as a variable. I think doing this for just two players would still be useful.

      I may have an MSc student doing a poker project with me this summer, so I could try to get him to work on this. Although it's a Scientific Computing MSc, so he'd have to crunch some numbers at some point. ICM models may be a better option for him.
    • muebarek
      muebarek
      Bronze
      Joined: 31.07.2008 Posts: 532
      Originally posted by jbpatzer
      Kind of. Actually he doesn't beat the rake. :f_mad:
      Yeah. Tough to win money without having a call button available ;)


      Originally posted by jbpatzer
      I will have to redo the calculation with different ranges, and also see whether I can include the range as a variable. I think doing this for just two players would still be useful.
      That's the reason why your model is easy to solve (and therefore so nice :f_biggrin: ). Having the range as variable requires having an Equity(range1,range2) function which easily makes the calculations costly in terms of runtime.
      It'd definitely be useful to have results on this for two players.


      Originally posted by jbpatzer
      I may have an MSc student doing a poker project with me this summer, so I could try to get him to work on this. Although it's a Scientific Computing MSc, so he'd have to crunch some numbers at some point. ICM models may be a better option for him.
      Now that's a cool job for a MSc student ^^
      We need more profs in poker :s_biggrin:
    • sapheal
      sapheal
      Global
      Joined: 23.02.2008 Posts: 1,015
      jbplatzer :) ,

      First off, I agree with the statement about mathematical modelling. Still, it's only partially true, ie. we do have modelling (like in case of RAKE receivers of electromagnetic signals for mobile phones) of singals using the orthogonal sequences and the Parseval law, which is in fact just very exact (ideal) description of what really happens, ie. from this modelling we can deduce how experiments in life look like.

      Yes, when it comes to problems with significant code coverage (a lot of situations and variables, eg. poker), we in most cases have to analyze the game (and model the game) using the suggested iterative methodology. Therein, we can use the method for iterative improvement based on BER (error ratios). However, there are other approaches that include, for instance, the convolution function to find best adaption methodologies or pot distribution-approach (when it comes to poker) which allows adaption via good understanding of default pot sizes for different cases (hands).

      When it comes to your model, I really dislike that you called JJ-QQ bluffcatchers. These are value hands and this is not even close. If I didn't miss anything, this looks correct. Still, I think that the very modelling might be simplified, because using too many variables and e.g. counting derivatives would be too time-consuming. We can say that on certain stakes ppl have a tendency not to 5bet bluff too often or their range contains 60% value hand and 40% bluff hands and that their 5bet/value range depends on our W$SD and All-in W$SD.

      BTW, card removal also (range vs range) could be used to simplify the model by just adding ca. 10% for FE preflop. Also, the ranges you calculated I dislike a bit, ie. the numbers you provided. Too big generalization.

      And one advice (a practical one): I'd use mathematics in poker as a tool to gain knowledge and analyze situations better rather than go very deep into modelling unless you have VERY ACCURATE data and want to create a small edge.


      Looks okay, but still, haven't made it myself, so could have overlooked something.

      cheers,
      Michal
    • jbpatzer
      jbpatzer
      Bronze
      Joined: 22.11.2009 Posts: 6,947
      Originally posted by sapheal
      jbplatzer :) ,

      First off, I agree with the statement about mathematical modelling. Still, it's only partially true, ie. we do have modelling (like in case of RAKE receivers of electromagnetic signals for mobile phones) of singals using the orthogonal sequences and the Parseval law, which is in fact just very exact (ideal) description of what really happens, ie. from this modelling we can deduce how experiments in life look like.

      Yes, when it comes to problems with significant code coverage (a lot of situations and variables, eg. poker), we in most cases have to analyze the game (and model the game) using the suggested iterative methodology. Therein, we can use the method for iterative improvement based on BER (error ratios). However, there are other approaches that include, for instance, the convolution function to find best adaption methodologies or pot distribution-approach (when it comes to poker) which allows adaption via good understanding of default pot sizes for different cases (hands).

      When it comes to your model, I really dislike that you called JJ-QQ bluffcatchers. These are value hands and this is not even close. If I didn't miss anything, this looks correct. Still, I think that the very modelling might be simplified, because using too many variables and e.g. counting derivatives would be too time-consuming. We can say that on certain stakes ppl have a tendency not to 5bet bluff too often or their range contains 60% value hand and 40% bluff hands and that their 5bet/value range depends on our W$SD and All-in W$SD.

      BTW, card removal also (range vs range) could be used to simplify the model by just adding ca. 10% for FE preflop. Also, the ranges you calculated I dislike a bit, ie. the numbers you provided. Too big generalization.

      And one advice (a practical one): I'd use mathematics in poker as a tool to gain knowledge and analyze situations better rather than go very deep into modelling unless you have VERY ACCURATE data and want to create a small edge.


      Looks okay, but still, haven't made it myself, so could have overlooked something.

      cheers,
      Michal
      Not sure I understood quite what you meant, but I am going to try this again with more realistic (i.e. wider) ranges for HU. I think the best approach is to fix the bet sizes, but allow the ranges to be variable.
    • sapheal
      sapheal
      Global
      Joined: 23.02.2008 Posts: 1,015
      jbplatzer,

      Please, ask questions. It's better to have discussion rather than saying we don't understand something :) It's possibly my fault you're saying that, but I want to address your problem as much as possible.

      What I meant is that you could potentially use convolution function or analytic approach to solve game theory problems. And, that your approach is good but simplification would be necessary to make it effective. And, finally, more practical things: you need better range estimation. And, ofc, JJ-QQ are not bluffcatchers.

      And I really like your research :heart:

      Please, understand: I am always assuming that if I'm not understood (even if I talk mostly about advanced concepts) fast then it's my fault. This is my approach. I want to help. I want to be very clear and precise, and be able to talk about very advanced things in a simple and clear manner. :)

      Feel free to ask questions and discuss ;-)

      Cheers,
      Michal