Data Science project: poker data set

Hi Andre,
I do believe that PT4 and HM2 have all the data that you need.

The problem is that the database used by both of them is postgresql, and the data dictionaries aren't published. So if you wanted to query the database directly, this might be a problem. PT4 allows you to write custom reports -- HM2 might as well, but it has been ages since I used it.

There is a free database called fpdb, with a well-known data structure that uses MySQL, sqlite or postgresql (your choice). I use this, along with a bunch of custom Excel worksheets with data links to the tables.

Next hurdle as I see it is -- where will you get the data?

Some random thoughts ---
Does room matter? How about game?
I'm a decent winning player at some games, not so much at others, and I have a decent win rate at cash, not so much SnGs, who knows at MTTs.

In some rooms I do really well, others BE, and I got crushed at iPoker rooms.

How will your analysis handle all these factors?

Your project sounds fascinating -- I hope you'll keep us posted.

All the best,
VS

I believe Pokertracker, HM2 have all the information (and then some) you'll ever need.
Pokertracker stores lots of information particulary and this is the reason it has slower importing compared to HM2.
While you can construct specific queries to database using the PT interface, i think you also use the PostgreSQL database directly using phpPgAdmin (and with some knowledge of SQL) and give the credentials of your Pokertracker database.
I don't really know what you'll find in the database though and how PT organizes it but if you find out you could do whatever you wish with the database and show anywhere like a HTML page, provided you know some PHP.

Now my question would be, whether anyone has an idea, how I could get a big enough data set or whether someone would even be as kind to provide their data?

I don't know what games are you interested but i found around 2 years ago a huge list of obfuscated hands (cash games of all limits and platforms), for such uses.
The link is http://www.outflopped.com/questions/286/obfuscated-datamined-hand-historie s" rel="nofollow ugc" target="_blank">this and i believe it has enough hands for your case.

Thank you very much to the both of you for your suggestions and thoughts. Also a big thank you for the link to the hand histories, vlzvl!
I admit I haven't completely thought everything through, since I am not 100% sure, yet, whether I will use poker data for my project or something else.
But here is some more information: I will be using the statistical language R to analyze the data. It would be best if I could get a file in CSV format. Like I said, it has been a long time since I have been using Pokertracker, so I am not sure, whether I remember this correctly: The hand histories can be imported into a tool like PokerTracker, which is then using a postgreSQL database. So would the statistical values like PFA, 3bet, etc. be stored in this database (based on some complicated pre-processing of the files with the hand histories)? So basically this means I would need PokerTracker (which I currently don't have anymore), so that I can import the hand histories and then try to access the postgreSQL database and get a CSV extract. Does anyone know, whether someone has provided a database extract instead of the hand histories file?

Regards,
Andre

If you use the hand histories from HandHQ.com (the link) you will need to import them into a database.

For that, I suggest using fpdb.
First of all it is free.
Next, the data tables are all well documented.
Finally, fpdb is open-source (python) so if you're at all adept in that language, you may even be able to modify it to do more than it already does.

I use fpdb for all non-holdem games, since PT4 and HM2 are just for Hold'em (and Omaha for an additional fee)

Cheers,
VS

As for the data, I think that most tools should have CSV export options for reports, which is all you need. I did look at the HM2 database tables a while ago. it's usable, but I wouldn't recommend using them just to get the data you need.

Winning and losing player are pretty bad classes though. Your target space is continuous, e.g. won/lost BB per 100 hands for cash games. Of course you can arbitrarily divide the classes at +/-0 but that doesn't make them useful. It will get you very random (=bad) results on the border of those classes. That is, a +0,01BB player and a -0,01BB player have very similar winrates, but are fed to the system in the training phase as two opposite classes. In contrast, a -0,01BB and a -100BB player are in the same class. Of course you can pre-select only big winners and losers.

More interesting would be to predict probability distributions for certain winrates, given player stats as input. Very similar approach, but way more useful in my opinion. Not sure if that fits your project though.