HU Postflop GTO software (JBMCCFR)

• Bronze
Joined: 22.11.2009
I'm starting this thread so that I can refer to it when people ask me what I'm doing over in my blog.

I've given up playing much poker since July to work on a software idea that I had. The key points are:

1) Although HU poker must have a GTO solution, the same cannot be said about multiplayer games. I wrote an article about this:

http://www.modernpokereducation.com/preview/bonus_maths_5.pdf

2) If two players get HU to the flop, from that point onwards the game is two player and zero sum, and therefore must have a GTO solution. Specifically, for a given board and given ranges on the flop, we have a two player zero sum game that must have a GTO solution.

3) I have read the literature on solving games like this, and implemented the current method of choice, Monte Carlo Counter Factual Regret minimisation (MCCFR) in MATLAB, with a mex file to some C++ that does the main calculation. The bet sizes are restricted to a finite number of possible fractions of the pot at each decision point. I have implemented my own lossless bucketing algorithm. This algorithm is guaranteed to converge to the exact solution, with no loss of information.
EDIT: Actual, some information is lost.

I have just started making solutions available, along with some software to investigate it, at

http://tinyurl.com/JBMCCFR

Be sure to tick 'Create a Desktop Shortcut' during installation, because there seems to be an issue with running it from the command line.

EDIT: Latest version of the interface is smaller, to fit on a laptop screen.

-------------------------------------------

At the moment I'm working my way through the 10 example hands at the end of Applications of No Limit Holdem, currently computing a GTO solution to Hand 2. I'm not yet sure exactly how many iterations I need to get a good solution but, even though there's some indication that a couple of thousand is OK, I'm doing 10K sweeps through the game tree to be sure, which takes about 3 days on my PC. This software is not a quick fix.

My main objectives over the next month or so are:

1) Improve the efficiency of my data storage so that I can compute solutions on larger game trees (more bet sizes, wider ranges).
2) Read the literature on NLHE to work out how to sample the tree efficiently with a wider range of bet sizes. (Average Strategy Sampling looks interesting)
3) Improve the efficiency of the code as far as possible.

Note that my qualifications to do this are (i) I'm a mathematician, (ii) I love poker, but sadly NEITHER (iii) I'm an expert at computational game theory NOR (iv) I am a good poker player. Please bear this in mind when interrogating me!

EDIT: Significant progress since I first posted this. See later posts.
• 40 replies
• Bronze
Joined: 22.11.2009
I've just posted an update which fixes a few bugs and adds totals for each action on each street. It also cleans up the solution when a player is calling all in, so that he calls with the calculated percentage of hands, but with the strongest hands. This is a common problem when calculating what hands to call with in a GTO solution.
• Bronze
Joined: 22.11.2009
Apparently the latest version didn't upload from my laptop last night, and my laptop's at home. I'll have to upload it when I get home. In the meantime, I've uploaded the data for Janda hand 2 (7K iterations looked like it was plenty).
• Bronze
Joined: 22.11.2009
Originally posted by jbpatzer
Apparently the latest version didn't upload from my laptop last night, and my laptop's at home. I'll have to upload it when I get home. In the meantime, I've uploaded the data for Janda hand 2 (7K iterations looked like it was plenty).
OK. Latest version of JBMCCFR posted, but I've moved the files. You can now find them at

tinyurl.com/JBMCCFR
• Bronze
Joined: 10.07.2010
Originally posted by jbpatzer
Originally posted by jbpatzer
Apparently the latest version didn't upload from my laptop last night, and my laptop's at home. I'll have to upload it when I get home. In the meantime, I've uploaded the data for Janda hand 2 (7K iterations looked like it was plenty).
OK. Latest version of JBMCCFR posted, but I've moved the files. You can now find them at

tinyurl.com/JBMCCFR
• Bronze
Joined: 22.11.2009
Originally posted by SHAKA1992
Originally posted by jbpatzer
Originally posted by jbpatzer
Apparently the latest version didn't upload from my laptop last night, and my laptop's at home. I'll have to upload it when I get home. In the meantime, I've uploaded the data for Janda hand 2 (7K iterations looked like it was plenty).
OK. Latest version of JBMCCFR posted, but I've moved the files. You can now find them at

tinyurl.com/JBMCCFR
No. If you run the installer, it automatically downloads the MATLAB Runtime Environment (a chunky 450MB) as part of the installation. You can then run the solution viewer as an executable. Remember, I'm not releasing the code that generates the solutions (yet), just a solution viewer.
• Bronze
Joined: 10.07.2010
when imn running JBMCCFR.exe, the program show this error:

?throw_exception_v4@internal@tbb@@YAXW4exception_id@12@@Z in the tbb.dll
• Bronze
Joined: 22.11.2009
Originally posted by SHAKA1992
when imn running JBMCCFR.exe, the program show this error:

?throw_exception_v4@internal@tbb@@YAXW4exception_id@12@@Z in the tbb.dll
God knows what that is. I compiled the executable to run on a 64 bit Windows 7 machine. Is that what you're using? I'm no expert on this stuff.
• Super Moderator
Super Moderator
Joined: 02.09.2010
Originally posted by SHAKA1992
when imn running JBMCCFR.exe, the program show this error:

?throw_exception_v4@internal@tbb@@YAXW4exception_id@12@@Z in the tbb.dll
What OS and version?
All patches up to date?

Can you find tbb.dll on your hard drive?
If so, post its full path, date and size.

to make it faster, do you have any OTHER than:
C:\Program Files (x86)\Intel\OpenCL SDK\3.0\bin\x64\tbb\tbb.dll
2013-04-23 05:23
195K

C:\Program Files (x86)\Intel\OpenCL SDK\3.0\bin\x86\tbb\tbb.dll
2013-04-23 05:25
151K

If you do, it is *possible* the software is finding the wrong one.

Cheers,
--VS
• Bronze
Joined: 22.11.2009
Just added the data file for Janda Example Hand 3.

I will have an update this weekend that adds the hot and cold equity and the expected value of each player on the flop, and also a hand by hand breakdown. I've also changed the layout a little and added what % of each hand bucket has made it through to a given street. I think it'll make the interface a lot more usable.
• Bronze
Joined: 22.11.2009
Version 1.4 now uploaded to tinyurl.com/JBMCCFR. I think there's all the information that anybody could require there in the gui now. Apart from bug fixes, I'll probably not be updating it for a while as I'll be working on increasing the range of betsizes in the game tree.

I've started computing the solution for Janda example hand 4, which should be done by early next week.

Thanks,

JB
• Bronze
Joined: 22.11.2009
Solution for Janda Example Hand 4 uploaded.
• Bronze
Joined: 22.11.2009
Solution for Janda Hand 5 uploaded.

A quick update.

I've not been able to use the data structure that I thought I would for some tedious technical reasons, but I do now have a structure implemented that will allow me to work with bigger game trees. I also updated the river hand bucketing algorithm, which I realized to could be more aggressive without losing information. This may speed up convergence a little.

My plan is to redo one of the 3bet pot examples to check that my new code gives the same solution. I'm then going back to Janda Example Hand 1 and redoing it with 0.5x, 0.75x and 1x pot bets allowed on all streets. I think that will converge in a reasonable time. I also want to work out how to parallellize the MCCFR algorithm with the C++ part of the code. Apparently I'm going to need to use Boost.Thread and learn about multithreading. Fun, fun, fun.
• Bronze
Joined: 22.11.2009
I've posted the GTO solution of Janda Hand 1 with 50%, 75% and 100% pot raise sizes allowed, and also the solution to GTO Brainteaser #8 from the GTORangeBuilder blog at tinyurl.com/JBMCCFR. These use a new data storage format, so you'll need to update the software to version 1.5, an installer for which I've also added. The new version can still read the old data format, so it should work with any older data files you've copied. It has also been updated to allow solutions to start from turn or river, and I've made it a bit bigger because there's so much information in the multiple bet size solutions (e.g. fold, call, raise 0.5 pot, raise 0.75 pot, raise pot, or jam). Any ideas on the best way to display the data appreciated.

• Bronze
Joined: 22.11.2009
Originally posted by jbpatzer
I've posted the GTO solution of Janda Hand 1 with 50%, 75% and 100% pot raise sizes allowed, and also the solution to GTO Brainteaser #8 from the GTORangeBuilder blog at tinyurl.com/JBMCCFR. These use a new data storage format, so you'll need to update the software to version 1.5, an installer for which I've also added. The new version can still read the old data format, so it should work with any older data files you've copied. It has also been updated to allow solutions to start from turn or river, and I've made it a bit bigger because there's so much information in the multiple bet size solutions (e.g. fold, call, raise 0.5 pot, raise 0.75 pot, raise pot, or jam). Any ideas on the best way to display the data appreciated.

Oooooooo! I think I've found a bug in my indexing routine for the new Janda hand 1 data file. I'm going to have to recompute it. Treat with a pinch of salt!
• Bronze
Joined: 22.11.2009
Originally posted by jbpatzer
Originally posted by jbpatzer
I've posted the GTO solution of Janda Hand 1 with 50%, 75% and 100% pot raise sizes allowed, and also the solution to GTO Brainteaser #8 from the GTORangeBuilder blog at tinyurl.com/JBMCCFR. These use a new data storage format, so you'll need to update the software to version 1.5, an installer for which I've also added. The new version can still read the old data format, so it should work with any older data files you've copied. It has also been updated to allow solutions to start from turn or river, and I've made it a bit bigger because there's so much information in the multiple bet size solutions (e.g. fold, call, raise 0.5 pot, raise 0.75 pot, raise pot, or jam). Any ideas on the best way to display the data appreciated.

Oooooooo! I think I've found a bug in my indexing routine for the new Janda hand 1 data file. I'm going to have to recompute it. Treat with a pinch of salt!

I note that, when this file is loaded, it takes up 1.7GB of RAM, so beware if you don't have much RAM. This is clearly something I'm going to have to look into if I want to distribute larger solutions. The file is only 0.28GB, so it's clearly reasonably well compressed. I can already halve the RAM requirement by saving it as single rather than double precision numbers, but I'll have to work a bit harder than that if this is going to be practical.
• Bronze
Joined: 22.11.2009

which allows for 0.25, 0.5, 0.75, 1, 1.25 and 1.5 pot bets. I think the solution is quite interesting, as there are several spots where 0.25 pot bets are in the solution.

I can see that the last gui I released is a bit big for laptop screens, so I will release a smaller version some time soon.
• Bronze
Joined: 22.11.2009
After discussing hand bucketting with Alex Sutherland, I think I've done it wrong in the sense that I've lost some information. This doesn't mean that the results I have aren't useful, but there is some information loss in my bucketting algorithm. I basically haven't taken into account range v range card removal effects. I'm going to redo the bucketting algorithm and see what effect it has. Sadly, it wil for sure make the code slower.

That's beta testing for you....
• Bronze
Joined: 22.11.2009
Originally posted by jbpatzer
After discussing hand bucketting with Alex Sutherland, I think I've done it wrong in the sense that I've lost some information. This doesn't mean that the results I have aren't useful, but there is some information loss in my bucketting algorithm. I basically haven't taken into account range v range card removal effects. I'm going to redo the bucketting algorithm and see what effect it has. Sadly, it wil for sure make the code slower.

That's beta testing for you....
After further discussions, here's my thoughts and plan.

1) thoughts posted here Midlife Crisis? What Midlife Crisis?

2) Test this by calculating an equilibrium for a pair of ranges with maybe 40-50 combos, firstly with my bucketting and secondly without to see whether there's any significant change.

3) Calculate the epsilon distance of my abstracted solution within the unabstracted game. This is a measure of how close my solution is to the Nash equilibrium (epsilon = 0).

4) Drink lots of wine.

I think that's a good plan, especially the last bit.
• Bronze
Joined: 22.11.2009
I have implemented by new bucketting algorithm, and so far it seems to work, although I need to do some more testing. Unfortunately, for Janda hand 1, it increases the number of buckets by a factor of just over two, and the computation time by a factor of about six. This is not good. I will focus on this example for testing and work whether there's much difference in EV or equilibrium strategy between my old and new bucketting algorithms. I'll work out the exploitability of each as well - something I should have done ages ago really. That's all probably going to take while, so don't hold your breath.

I've added an installer for a smaller interface version that should make everything visible on a smaller laptop screen. If anyone still can't see the flop, let me know.

When installing the software, make sure you check 'create a desktop shortcut', or it may not work.