Midlife Crisis? What Midlife Crisis?

Originally posted by jbpatzer
My business is creating mathematical models of the real world, and as such I'm forever trying to question my assumptions and compare to reality. I don't want to end up like those economists that Taleb laughs at. Taking randomness into account in deterministic models (uncertainty quantification as its called now) is a hot topic here at Nottingham and elsewhere. Personally, I've always tried to steer clear of stochastic models because I know fuck all about them, but poker has given me more of an interest. I also have a PhD student who is developing a stochastic model of an etching process, for which we're having to talk to somebody else because I'm not in the business of bullshitting my students. A lot of my colleagues are interested in problems like nuclear waste disposal in permeable reservoirs for which there's limited data on the permeability. In that context, stock market crash = poisoning people's water - best taken seriously.

When I was studying Geology there was a huge push to apply statistical methods to fault systems. 40 years on and they're no closer. With weather systems they've come a long way. Big problem is almost a complete lack of data on fault systems.

Although luck and randomness may play a role in markets, but the influence would be slight in the face of all the vested interests cooking the game to suit yourself.

It is rather like complaining about "variance" when you lose at poker when 3 of the 6 players are colluding.

Peace,
VS

How's the GTO project going?

Thanks for the link to that site, never heard of it and it seems like it might have some interesting things

Originally posted by VorpalF2F
When I was studying Geology there was a huge push to apply statistical methods to fault systems. 40 years on and they're no closer. With weather systems they've come a long way. Big problem is almost a complete lack of data on fault systems.

I think my colleagues have random fractures in their models, but I'm not inclined to live near a nuclear waste depository just yet thanks v much.

Originally posted by Yoghi
How's the GTO project going?

Thanks for the link to that site, never heard of it and it seems like it might have some interesting things

I did write about the current state of play in my last but one post. Basically, apart from a few minor things I'd like to change, I can find the GTO solution for a pair of ranges and a given flop and betting structure, as long as the tree isn't too big and the ranges aren't too wide, and by too wide I mean really very wide indeed.

Mr Sutherland knows his game theory and also his NLHE and can also write very clearly. It's a rare combination.

Nice to see you back around here. Are you still crushing FLHE? I keep meaning to write something to set up the fixed limit game tree in my code. It would be fairly straightforward, but I'm not that interested myself.

Don't think there's much new here, but this may interest. The first part is mostly an introduction. Towards the end they talk a little about the CFR they used. Heads up, limit though, I mean really...
Sciencemag: HEADS-UP LIMIT HOLD’EM POKER IS SOLVED

HEADS-UP LIMIT HOLD’EM POKER IS SOLVED

Poker is a family of games that exhibit imperfect information, where players do not have full knowledge of past events... blah blah blah... This result was enabled by a new algorithm, CFR+, which is capable of solving extensive-form games orders of magnitude larger than previously possible.

edit: here the authors talk a little about it
http://www.theverge.com/2015/1/8/7516219/Texas-Hold-Em-poker-solved-computer-program-cepheus

Originally posted by jules97
Don't think there's much new here, but this may interest. The first part is mostly an introduction. Towards the end they talk a little about the CFR they used. Heads up, limit though, I mean really...
Sciencemag: HEADS-UP LIMIT HOLD’EM POKER IS SOLVED

HEADS-UP LIMIT HOLD’EM POKER IS SOLVED

Poker is a family of games that exhibit imperfect information, where players do not have full knowledge of past events... blah blah blah... This result was enabled by a new algorithm, CFR+, which is capable of solving extensive-form games orders of magnitude larger than previously possible.

edit: here the authors talk a little about it
http://www.theverge.com/2015/1/8/7516219/Texas-Hold-Em-poker-solved-computer-program-cepheus/blockquote >

Ooooooo! CFR+. Like CFR,.......but +.....:sfishy:

Looks suspiciously easy to implement. I'll be like a rat up a drainpipe on this one.:pokerface:

May I ask a studenty noobish question here bossman?

Does being able to 'solve' fixed limit, to whatever degree, mean that an answer to nl is getting that much closer? I know about game theory (GT) and that stuff, but all that GT is useless at lower limits anyway imo ( am I a fish again? Sigh).

Further, would the solution to FL mean the downfall of NL in the long term, even given the complexity the solution to that.

Sorry if me question is posed badly, but I have no doubt whatsoever you have encountered those before - on the odd occasion - and can interpret .

Also,
http://www.theguardian.com/science/2015/jan/08/poker-program-cepheus-unbeatable

In that article, no claim that it is solved -- unless I missed it.

I suppose depends on definition of "solved"
VS

Chess has been solved because there is a finite number of moves. A few grandmasters can beat/draw the computers.
Poker has not been solved as far as I know. Even limit. They have gotten a lot closer but.......... Also are they playing cash or tournaments, full ring, 6 handed, heads-up? They all have different strategies. Heads-up-for-rolls is probably the closest to being solved but even Phil Laak beat the computer the last time he played it and he is not even in the top 10 of pros.

I only skimmed through it, but it does appear that they've found an approximation to the equilibrium solution of HU rake free FLHE that's so accurate that it is in effect 'the' solution. If that's your definition of 'solved', and I think it's a reasonable definition, then it's solved.

Playing exploitatively to maximise your expectation is much harder. Playing with more than two players means that it's not clear what 'solved' means any more. The three player FL, one street, AKQJ game has certainly not been solved.

To answer Mal's question, CFR+ is a very simple idea that seems to have massively speeded up the rate at which these algorithms converge. While it doesn't mean HU NLHE is going to be solved any time soon, the abstractions you can use can be more accurate, i.e. more hand buckets (closer to the actual game) and more possible bet sizes (closer to the actual decision tree). I'm going to implement it in my code and see what happens.

Details of an update at https://forums.pokerstrategy.com/forum/thread.php?postid=1982511#post1982511

I'm still not sure whether CFR+ works better in my code, because I screwed up the comparison. I should find out when I get back in to work tomorrow.

Looks like I've got some new hands to go over
Downloaded them all and will get to looking at them all over the next few days.

Looking at the 732 mono one now. Interesting to see no flop raising range from either of the players.

On a Tc turn, after the flop goes x/x. UTG check jams 6%! 80bb in your face. Take that...

Interestingly the x/jam range consists of nut flushes (duh), 20% of top of top set (ok no biggy) and then a good chunk of Q:sJ:s, J:sT:s & K:sQ:s and then for bluffs mostly made up of some % of A:sKx & A:xK:s, K:xQ:s.

I think those Q:sJ:s, J:sT:s & K:sQ:s type hands in a jam range is part of a pattern we're seeing. I'm fully sold on having merged raising ranges with plenty of money & streets behind. Because a) these GTO solutions demonstrate it and b) I believe the reason is, that it forces an opponent's bluffcatching/continuing range to be tighter (therefore less EV), forcing folds from hands that have some ok equity against a hero's value raises, but also are already ahead of most of villains "bluff" raises. (Does that make sense to you?! )
The same argument for having merged ranges in cbetting/betting ranges also makes sense to me.

Anyway after rambling that out. It is interesting that this solution also favours that kind of merged raising range when jamming, when there is no money and one street behind. Even if it is an appropriately narrow part of the jamming range.

Expounding on this line even further. Some kind of merged preflop 3b range is probably going to be optimal. Although balancing it with a cold call range in most spots makes things a bit tricky... Maybe snowies preflop ranges are starting to look better... With a mixed strategy of 3b&cc with all those middling hands that many often consider easy calls.

Here's snowies 3b/cc range BTN v CO fwiw.

Originally posted by jules97
and b) I believe the reason is, that it forces an opponent's bluffcatching/continuing range to be tighter (therefore less EV), forcing folds from hands that have some ok equity against a hero's value raises, but also are already ahead of most of villains "bluff" raises. (Does that make sense to you?! )

On this topic. I've been playing a more merged raising strategy lately and I'm seeing some VERY bad hero calls. Where I can only assume, villains are assuming I'm bluffing too much. And then bluffcatch with some real trash hands that would do fine against a polarized range (and great if I was/am bluffing too much), but make horrendous calls against a merged range.

I think that all makes sense.

Any flops I haven't done that you think would be particularly interesting?

I really badly want to look at Snowie's ranges, but I just can't get bloody fractional ranges to work. I am a complete doughnut.

The jury's also still out on CFR+ for my problem, mainly because I keep having to run another code on my work PC at the same time (Work! Pffff!). It looks like it does provide a significant speed up, but not the order of magnitude that those clever Alberta people got for FLHE. This may be because their set of decision nodes, per hand v hand match up, is an order of magnitude or two smaller than the ones I'm using for NLHE. On a similar note, when I tried to use CFR+ and average strategy sampling at the same time, which is pointless for FLHE, the whole thing went tits up, which I found rather mysterious.

I'm planning a post on my plans for the future soon, because I'm getting more and more hooked on the idea of being a gentleman amateur computer poker enthusiast. In particular, I find this very attractive, probably not this year but next. After all, I do keep banging on about multiplayer games. I'm trying to find out about learning algorithms for repeated games with imperfect information. Actually forget the 'planning a post' thing. That'll do for now.

New post on CFR+ at https://forums.pokerstrategy.com/forum/thread.php?postid=1983370#post1983370

Originally posted by jules97

On a Tc turn, after the flop goes x/x. UTG check jams 6%! 80bb in your face. Take that...

I was thinking about this on the way into work today. I think you have to be a bit careful about drawing conclusions from this because the bet sizes allowed are 2/3 pot and all in. When it check/jams 6% that might just be indicative of the fact that the optimal strategy is to bet rather bigger than 2/3 pot, and this is the best approximation given the constraints I've imposed. I should probably try it again with a couple of bigger bet sizes allowed and see what the solution is then. Even just allowing bets of 2/3 and 4/3 pot might give some indication of what's going on.

I just found out about the Lemonade Stand Game.

http://www.sigecom.org/exchanges/volume_10/1/ZINKEVICH.pdf

OMG. This is a deterministic three player game with perfect information. No cards. No dice. It's simpler than the three player AKQJ game (Kuhn poker). It's simpler than any multiplayer poker game. Nobody has any idea what the best strategy is.

EDIT: OK. After reading around it a bit, some people do have some good strategy ideas,

http://eprints.soton.ac.uk/271215/1/ECAI-604.PDF

but they're based on thinking carefully about what good strategies might be, not on brute force calculation of equilibria. This game has 53 pure strategy Nash Equilibria. Stick that in your GTO pipe and smoke it!

I find this encouraging and strangely comforting.

I'm still to go over the rest of those UTG v BTN hands you uploaded. I don't really have any good suggestions for flops. There seems to be a nice coverage now.

The only idea I can think of, is flops that favour the OOP player tend to be more interesting. Because if not the OOP player seems to be so defensive/passive and have massive checking and bugger all x/r ranges.

The poker bot competition sounds cool. I'm interested how you go about dealing with the problem of preflop play. And if your CFR bot will be much help with it. Looks like they only have heads up no limit at the moment. They should have a 6max bot competition ♦ They could even chuck some fishbots onto each table!

I was thinking about this on the way into work today. I think you have to be a bit careful about drawing conclusions from this because the bet sizes allowed are 2/3 pot and all in. When it check/jams 6% that might just be indicative of the fact that the optimal strategy is to bet rather bigger than 2/3 pot, and this is the best approximation given the constraints I've imposed. I should probably try it again with a couple of bigger bet sizes allowed and see what the solution is then. Even just allowing bets of 2/3 and 4/3 pot might give some indication of what's going on.

Yeah, that's a good reminder. I also didn't initially notice that these solutions only have a .6 pot or jam option. .6 is probably a bit small for a traditional flop raise and yeah missing that mid sized flop raise probably takes out a little of the raising ranges.

Honestly, I'm not sure it matters that much as a learning tool though. I assume the lessons learned from range construction will be similar anyway. And I think that's the most important part.

I seem to remember that lemonade game from an introductory game theory course I did years ago. It must have been different. All I remember is that I really enjoyed the course until it got all mathsy and hard

I was thinking a bit about these solutions. And the other big take away that doesn't seem to happen in human games nowhere near as much as these solutions, is checking strong hands behind IP. Some of the better players are starting to cotton on that being aggressive OOP is maybe not the best idea, but IP it's to a far lesser extent.

So I assume going absolutely fucking crazy (or just a little bit if that's your style ) as UTG v BTN (or more often BB v BTN/CO/MP/UTG) on the turns and rivers after flops go x/x is going to be hugely profitable against just about everyone at the moment.

Can you please post more about those ch/back nuts IP situations. What % does it ch/back only on dry boards or on wet as well? It should be mixed strategy I asume.

Originally posted by dooleslovs
Can you please post more about those ch/back nuts IP situations. What % does it ch/back only on dry boards or on wet as well? It should be mixed strategy I asume.

Can you not get my solution viewer software to work?

I will try.