Will Tipton's Solving Poker With IPython And Fictitious Play
Anyone who has read Expert Heads Up No Limit Holdem knows that there is tons to learn from solving model games and analyzing the GTO strategies that result. The ability to solve large poker games was directly responsible for many of the insights in EHUNL and set it apart from poker advice available elsewhere. Unfortunately, this sort of work involves large computations, and software to perform these calculations was not publicly accessible -- until now.
This video series will walk you through the development of computer code to solve for maximally exploitative and equilibrium strategies of arbitrary HUNL decision trees, to visualize the trees, and to investigate the resulting strategies and the EVs of playing them. You will learn to leverage the powerful but user-friendly iPython (interactive python) programming enviroment and create a tool that allows you to perform many of the calculations described in the books. Moreover, in learning how to build it rather than how to use a pre-existing software package, you will gain the understanding necessary to go beyond published work and to perform game theoretic studies of your own imagination. It's an exciting time to be a poker player.
FAQ
Is prior programming experience necessary?
Absolutely not. The series was built to teach players without any programming knowledge at all, how to build the computer code to solve valuable situations. All you need is a willingness to learn.
How do I know if this series is for me?
If you're interested in solving poker situations to help you become a much stronger player, this series is for you. If you're still unsure, we've included the first three videos of the series below, free, to show you how simple this is. Watch the sample videos and get a full idea of what the video pack is all about.
Sample Videos
Video 1: Introduction
Video 2: Windows IPython Setup
Video 3: Functions
Video 9: The Shove/Fold Game
Table of Contents
- Intro
- iPython setup
- Intro to iPython and functions
- Hand-vs-hand equity calcuations
- Hand-vs-hand equities fast
- A bit of homework: ranges
- The Ranges class and hand-vs-range equities
- Equity distributions
- The shove/fold game
- Decision points and trees
- Visualizing trees with graphviz
- Strategies
- Max-EV play
- Maximally exploitative strategies and FP
- A turn and river spot and discussion
- What's next?
Technical FAQ
When writing my Range class, __repr_svg__ doesn't seem to work: trying to display a range gives something like <__main__.Range instance at 0x0638DD00>.
That text is the default representation of a range (the name of the class and its memory address). This happens because it can't use the image-generating display function we've written. There could be two reasons for this: (1) it can't find the function, or (2) there's an error when it tries to run it -- in this case, it silently falls back on the default representation.
To address (1), make sure you've named the function correctly: _repr_svg_. One underscore at the beginning and end and one in the middle. Also make sure that indentation/whitespace is correct as in the video. If it is incorrect, python may not know that that function should be associated with the Range class.
To address (2), run the function manually. In other words, if you have a range called bob, run: bob._repr_svg_(). This way, if an error occurs, it will be displayed, and you can debug.
Errata
Note: some errors were made in early videos and then tracked down and corrected in later ones. This gave us a chance to demonstrate the debugging process, which is itself a very important skill. Only bugs that were not found by the end of the series are listed here.
In setMaxExplEVsAtNatureDP() and setMaxExplEVsAtVillainDP(), the loop over all hands is over numCards twice, as usual. In the video, one of the loops was over numHands by mistake. (Thanks erdnase!)
In the getRange function of the StrategyPair class, "range[n]" should be "ranges[n]"
In the setRangeString function of the Range class, there's an issue with the case which covers unsuited (XYo) hands. As written, it only covers 6 of the 12 combos, e.g. AhKd but not AdKh. The following code fragment corrects this (only the 3rd and 4th lines changed). Thanks PN Houle!
else: # unsuited hands
for i in range(numSuits):
for j in range(numSuits):
if i != j:
self.setFrac(pe.string2card([rank1+suits[i], rank2+suits[j]]), value)
In video 11, we add a directory to the end of the Windows PATH variable. This directory should be the "bin/" subdirectory of the place where we installed GraphViz. What was shown on screen when I did this was correct, but I forgot the "bin/" part when describing it.
When renaming the argument to getEquityVsHand from "board" to "b", I forgot to change it everywhere in the function body! After making the change, every instance of "board" except one in the function should have been changed to "b". Furthermore, this led to incorrect EVs when solving the turn and river game. After 300 iterations, the correct numbers there are SB EV: 19.00 BB and BB EV: 21.04 BB. See this post and the surrounding discussion for more info. Keep in mind that if you were affected by this bug when you generated some EquityArrays, you'll need to delete and re-generate them!
Payment Methods: Paypal (Credit or Debit card without a Paypal account), Moneybookers/Skrill (Email for other options)
Release Date: July 1st, 2014
Release Price: $99.91
Length: Approximately 12 hours
Total File Size: 2 GB
File Type: .exe file
Protection system: This video pack is protected by a sharing protection system. These videos can be played on both your laptop and desktop, using Windows XP, Windows Vista, Windows 7, Windows 8 and / or Windows Server computers and both online and offline after a single entry of your serial key. There is also a Mac OS version to use with a Windows guest running in a VM on a Mac OS host. More information is available on the download page.
After purchasing you can download from here.
Was waiting this for a long time, awesome.
Is the CFR algorithm covered in this pack / used for fictitious play?
Hey, FP and CFRM are separate algorithms. FP is used here, mostly because it's simple and intuitive. It also requires us to write code to solve for maximally exploitative strategies, something that's useful to be able to do in its own right.
Hope you enjoy the vids!
Cheers,
Will
Anyone else having their ipython notebook not apply css in the browser?
What do you mean? and what browser are you using?
Well It looks as though no styling is being applied along with the html. It still works. I can create a new notebook and I can see the commands being executed in the command line console. There is a dropbox link attached to image of a screenshot of my browser that I just took. I tried it on firefox chrome and IE all with same result.
Hm weird, well if it does that on all browsers, it seems that something might be wrong with the ipython installation itself. Try re-downloading and reinstalling the anaconda software?
Ya I tried a few different things and cant get it to work. I am just going to use the spyder IDE or Eclipse and just manually install all the required libraries. Is this going to cause issues with following the video series you think?
Well there definitely are somes spots where you'll need to adjust some things. I use a couple ipython-specific features for visualizing ranges and decision trees. Nothing crazy, and you sound like you have some prior background, so it shouldn't keep you from following along. That said, ipython does do some cool stuff, and if I were you I'd try to install ipython without the anaconda distn before I gave up and tried to proceed in standard python.
I like the books and 1st video pack very much. They are great strategy wise and have helped me a lot to understand the game better.
But after watching these 3 introductory videos to pack 2 I don't really see the benefits of purchasing this new video pack.
I don't mean to be negative but I would like to understand better what are the benefits for a poker player for learning iPython programing language and spending lots of time designing our own software when there are many good software packages on the market, and they are relatively cheap.
I understand that this is intended for people with no programing experience or with little programing experience but lets not fool ourselves, to be able to create our own software people will need to spend hundreds of hours learning the programing language properly and even more time to design the software. I don't really see the point in this. Will this make me a better poker player?
"If you want to win, you must not lose!"
Hello,
Of course lots of other poker-related software is available, but I don't believe there are any other publicly available software packages that can solve for the equilibria of general model games. If you enjoyed the books, then there's a lot to learn from being able to do the same sort of calculations described therein.
This series does not attempt to be a complete introduction to programming -- I just cover what's necessary to get the job done -- but Python is quite user friendly, and it really doesn't take long at all to start doing useful things. Perhaps we'll post another video from the middle of the series or so to show how quickly we can start doing useful things, or perhaps some other players who are new to programming will be kind enough to post some reviews.
Cheers,
Will
Thanks for your answer.
I have some minimal programing experience but this is not something that I enjoy doing so I would prefer to buy finished software than to program my own.
I'll have to think some more if this is the right video series for me.
Posting a video where we could se an example of finished software could definitely help.
"If you want to win, you must not lose!"
Poker software on the market is pretty limited wrt solving games.
Haven't watched this video pack but sounds like it fills a gap for the lower end of advanced users.
2 problems I see:
- price point - the books which undoubtedly contain more info are only ~ $50. You definitely could have charged more for your books!
- end user market - the mass market is generally lazy poker players who want a quick fix. Sadly products marketed as an out-of-the-box took which will improve players games will tend to sell better than the likely more beneficial product which gives you the skills to build better tools yourself. This is compounded by the correlation of advanced users who are interested in putting in the required effort either already being able to program or being able to teach themselves quite easily.
yep, and those are the guys who write reviews on amazon like:
"although i am a good HU player, the book was of little use for me, because it is way too complex"
yes, that is an actual review on amazon (it´s in german, the above is my own translation).
well, poker is a complex game, so writing a "just add water" primer on how to crush nose bleeds games is kind of silly.
what i like about the books is that it is not limited to micro, low stakes, mid stakes, etc, but it teaches you to generally think about the game and find your own approach to it. also: the concepts are not just limited to NLH. you can extrapolate the information in the books to any variant. solving complex HU PLO situations is certainly a little more tedious, but possible using the concepts in the book.
Guys, calling lazy someone who doesn't want to learn programming language is not fair.
I understand that solving specific poker situation with software is beneficial to poker players and that strategies and ideas from the books can be studied more deeply with our own software. Learning new programming language and designing poker software could also be beneficial but I have some doubts about it. First, this is not very time efficient. Someone who is not very experienced or interested in programming will have to spend lots of time learning the programming language and designing the software. I don’t think that the gained benefits of such a process justify the time invested. There are much more time efficient ways to study poker and to analyze poker situations.
I’ll give an extreme example below:
“Knowing math and statistics is beneficial to poker players so for a new poker player it would be beneficial to have Mathematics and Statistics Degree. Degree in Economics and Psychology would also be beneficial. I almost forgot about the Computer Science degree. It would be even better to have a Masters degree or Doctoral degree in those 5 subjects. These would be perfect educational backgrounds for a poker player and I would recommend finish them before someone embarks on a poker career. Being active in sports and outdoor activities is also mandatory. Mountain climbing is proven to strengthen peoples will, mental toughness, self determination, perseverance and self-confidence, which are all mandatory skills for a poker player so I would recommend climbing Mount Everest once every 2 to 3 years to develop and maintain these valuable skills… ”
Nobody can deny that possessing the skills and knowledge mentioned above would be beneficial for playing poker but going through such a difficult and long learning process is not the best use of our time if we want to develop our poker skills.
Of course, if someone already knows how to program and enjoys this sort of things than this video pack will likely be a good addition to their knowledge.
"If you want to win, you must not lose!"
I'd like to reiterate that it doesn't take that much time to start doing useful things in Python. I definitely wouldn't have been able to make a series like this with something like C or java.
That said, I agree that this won't be the shortest path to improving everyone's game, and that goes esp for newer players for whom there is likely lower hanging fruit. On the other hand, there are certain sorts of advanced studies that can not be done any other way.
Finally, I've found that a bit of coding is a v useful life skill outside of poker. Here's a relevant/potentially interesting discussion --
http://forumserver.twoplustwo.com/29/news-views-gossip/why-poker-players...
From my posts it may seem that I'm against this pack or that I think that it is not good, but this is not the case. I actually think it is quite valuable but I didn't like the the fact that there is lot of programming involved.
I think I will actually buy this product, but not right now. After the summer I will have much more free time and I think I will be ready to learn iPython than.
Thanks for explaining that iPython is not that hard to learn.
"If you want to win, you must not lose!"
To follow up on this, we did go ahead and post another video: Vid9 (solving the shove/fold game). It's about the halfway point of the video series and gives a taste of some less introductory content.
i didn´t say that. i was merely pointing out that it´s silly to call a book on solving NLH too complex. NLH is a complex game, therefore it´s quite difficult, let alone impossible, to write a simple book about it. will has done a good job explaining the concepts with as little math as possible. i would merely subscribe to calling those players lazy who expect a "just add water" approach to the game and don´t really want to work in their game.
i definitely can understand if someone does not feel like learning to program, no matter how simple it may be. yet, once you reached a certain understanding of the game, you will have to find your own approach to it and modeling situations you find crucial for the game and with the help of such models design your own ranges, your own betting sequences, your own bet sizes, can hugely benefit your playing.
the programming skills required for the books are comparable to the kind of vocabulary you have to learn if you travel for a couple of days to a country where you don´t speak the language. after finishing the course, you will be able to order dinner, ask for a room with shower, will be able to buy stuff at a local market, understand how much money the bar keeper wants, etc. you will not be able to read a book on philosophy, but you will do fine with the most basic requirements of language.
cheers
s.
Anyone bought the book? What did you think? Looking for a review :)
Swedish
Student who eventually gonna crush. Just learning. PM if u need a analyze-buddy.
Hands down best poker strategy book on the market. I'm going to get rid of the other ~ 10 poker books in my library because they are now redundant.
I will caveat this with a warning though: the books are not easy to read and will require a decent amount of study & thought to get value from them. Additionally, the emphasis of the book is on pseudo GTO solutions to simplified games. The practicality of which is probably best summarised as: the book will lay the foundation for you to go:
- break down your opponents game
- design exploitative strategies to exploit imbalances in their game
- identify when the exploitative deviations from balanced indifference points you deliberately take in your game are being exploited and how to adjust
It took me roughly 2 days to follow along the entire video and get the software to work as demonstrated.
My first impression is that this is an extremely interesting video pack for someone motivated to learn to solve some games. It has the added benefit of showing the process of writing a non-trivial program.
Given that I have previous programming experience I would estimate that a new user might take a bit longer to go through the video. You can take it in small pieces (say, a chapter a day) and be done in 2 weeks.
Some general advice if you never programmed before:
- pay attention to detail: most compilers (in this case the Python compiler) are not smart enough to correct simple typos, so try to minimize those as they can be the source of errors that are simple to fix but might frustrate a newcomer. (hint: use TAB to complete the variable names, function names, etc.).
- pay attention to the indentation of the code shown in the video because in Python white space is meaningful.
- don't skip around the video chapters (at least initially) because Will also recorded the debugging process which ends up being a bit non-linear. You will frequently have to go back and correct previous mistakes which are found only later in the video. If you skip around you might miss these corrections and the code won't work.
Thanks for the review, erdnase. I agree with all your tips!
Hi there
Thanks for recording these videos. I've never programmed before, currently on Video 7, Range Class, please can someone help me find the problem with my code!
At 1 hr: 23 mins, when I write this code
bob = Range()
bob.setAllFracs(1.0)
bob.setFrac(pe.string2card(['Ah', 'Jh']), 0.0)
display(bob)
then for me, AJs changes from green to white - I can't get it to go a lighter shade of green which is what should actually happen.
Whereabouts should I be looking in my code to sort out this bug?
Can you post the code for your setFrac() and _repr_svg_(self) methods?
^this, and also getAmbigFrac() could be the source of the error.
Btw, there was an errata affecting the setRangeString() function that you should check out, although I don't think it could fully explain this issue.
Hello,
Where can i find the preflop.ea.npy file? Calculations are a bit long :p
Thanks
Hey, I forgot to include the file in the download originally, but it should be there now.
Cheers,
Will
I have programmed for 30 years and, for the last 8 years, specialized in python.
I have also played poker for over 30 years and try to stay up to date on the latest theory and practice.
I am very interested in the code discussed here but I don't have a Wndows computer and so an .exe file makes no sense for me. (I use Linux professionally and a Mac as my home computer.)
I managed to get pokereval lib installed on my Mac (although that was a real pain) and then I successfully completed the 3 videos. When the idea of a 'conflicts' function was introduced, I wrote a working version of the function before the narrator had even explained the notion of booleans.
Bottom line: I don't want to waste my time on a video that is teaching python. I don't want to download an exe file. I don't care for ipython's notebooks either and would rather just have access to the python files.
In the video the narrator says that this code is publicly available. So can you hook me up? Where is it? I don't mind shelling out some bucks for the code.
kidpython
The video doesn't attempt to teach python from scratch. It jumps right into coding the poker solver, piece by piece. I don't think you will be bored with extended beginner level explanations.
So I mentioned that I didn't like ipython or notebooks and just wanted the python code. After looking closer at what you provide in the videos that have already been posted, it finally became clear to me that asking for the python code directly is probably a non-starter because the code probably doesn't exist in that form. The inlining of the pylab code clearly ties this project to notebooks and ipython. I get it.
So I will back off of that request and simply ask if it is possible to get the videos in a form other than an '.exe' file given that I don't own a Windows computer.
Thanks!
Hi kidpython,
I just e-mailed you a possible solution that might work for you. Please check it out and let me know what you think.
st
Hi Will,
I am at the end Equity Array video. Excellent video series so far. Thank you.
Glad you're enjoying it :)
How large game trees can be solved with modern personal computers like 4Ghz 4 core and 8-16gb ram within a day of computing time using this software? Is RAM or computing time the most limiting factor for a PC?
Do we need to restrict the number of options to simulate turn play? For example only allow one betsize with maximum two bets left.
Well if cpu time is an issue, you can always wait longer, whereas if you don't have enough memory, you're dead without doing some more work. For me, RAM was always the limiting factor, but I'm pretty patient.
If you have a game in mind, you can estimate how much memory it'll take to store the tree and solutions. Each EquityArray is about 56MB and each range and each set of EVs at a decision point are about 21 kB. As discussed in the vids, these are each a couple factors of 2 larger than they have to be because I favored convenience over efficiency, so they can be pared down if you really want it.
But without any optimizations, we're looking at something like 56 MB * (number of boards in the tree) + 3 * 21 kB * (number of actions in the tree). There are some other things that take memory too, but that'll get you pretty close.
Is there a way to get these videos to play faster? The videwing experience on youtube for the first 3 videos was much better for me because I could listen at 1.5-2x speed and follow along..
Hey ZhengHe, unfortunately I don't know of any way to do this. Sorry about that.
The built-in playback interface allows variable speeds. Just use the double-right or double-left arrows on each side of the play and stop buttons. 1.5x speed doesn't seem to be supported, though. Only factors of 2 increases or decrements in speed are.
Everything worked well for me until now. When I enter the command bob = Range() and bob.setAllFracs(1.0), only the range occurs as shown in the video, but it doesn't turn green even though I have triple checked everything. I have no idea how to fix that mistake. Any suggestions?
Hey KnutFH,
I can think of a few reasons this might happen -- the internal data of the range itself isn't set correctly, the function that calculates the fraction of ambiguous hand combos isn't correct, the _repr_svg_ function could be buggy. Could you please follow the procedure described at
http://forumserver.twoplustwo.com/showpost.php?p=44044601&postcount=16
to help us debug?
Thanks
Fixed it. Went through the Notebook over and over again - was a typo obv ... Thanks for support anyways!
i have a problem in the lesson equity array
when i programmed everything along with the recording, it was fine, but when i reopened it the next day and ran all cells, i get the following error report in the input "myEArray = EquityArray(board)"
AttributeError Traceback (most recent call last)
<ipython-input-49-72e875fb97fa> in <module>()
----> 1 myEArray = EquityArray(board)
<ipython-input-47-f1671884be97> in __init__(self, b)
11 self.eArray = numpy.zeros((numCards, numCards, numCards, numCards))
12 if os.path.isfile (self.getFilename()):
---> 13 self.eArray = numpy.load(self.Filename)
14 else:
15 self.makeArray()
AttributeError: EquityArray instance has no attribute 'Filename'
how can i resolve this?
thanks for helping
The error message indicates that the EquityArray class doesn't have any attribute -- that is, a data structure or a method --
called Filename. So, you must either define Filename two lines above with the code
Filename = self.getFilename()
or you can replace this occurrence of Filename directly with self.getFilename()
This will eventually be added in the errata.
yep, that´s it. thanks.
ran into another problem, it´s in the "shove/fold game" lesson:
TypeError Traceback (most recent call last)
<ipython-input-194-7183a48008b3> in <module>()
----> 1 doShoveFoldGame()
<ipython-input-191-d83b0207c1b8> in doShoveFoldGame()
23 hand = [i,j]
24 bb_call_freq = bbCallRange.getNumHandsWithoutConflicts(hand) / numVillainHands
---> 25 equity = getEquityVsHand(hand, bbCallRange, ea)
26 evJam = (1 - bb_call_freq) * (S+1) + (bb_call_freq) * equity * 2*S
27 evFold = S - 0.5
<ipython-input-193-e5461160bd16> in getEquityVsHand(hand, villainHand, b)
8
9 def getEquityVsHand(hand, villainHand, b):
---> 10 if conflicts(hand,villainHand) or conflicts(hand,board) or conflicts(board,villainHand):
11 return -1
12 peresult = pe.poker_eval(game = 'holdem', pockets = [hand, villainHand], board = b)
<ipython-input-163-a5fde56e5e43> in conflicts(cards1, cards2)
11 def conflicts(cards1, cards2):
12 for i in cards1:
---> 13 for j in cards2:
14 if i == j and i < numCards:
15 return True
TypeError: iteration over non-sequence
as far as i can see, i wrote the code just like it is in the tutorial, so i don´t really get why "for j in cards2" should be a problem here.
thanks for any help
In line 10 of getEquityVsHand(), the two occurrences of 'board' should be 'b'. Else you are using the global variable board rather than the input b of the function. Depending how you earlier defined board, this may fully explain the error. This error in the definition of getEquityVsHand() was fixed later in the video.
i already tried that (sorry, i didn´t upload that). unfortunately it didn´t resolve the problem.
Here are some thoughts on debugging:
http://forumserver.twoplustwo.com/showpost.php?p=44044601&postcount=16
and this seems like a perfect spot for it. Please give that a try and provide the information it asks for to help us debug.
thanks for the guideline, will.
i found the problem:
in line 25, instead of getEquityVsHand, i should have written getEquityVsRange.
i shouldn´t do that kind of stuff at 3 o´clock in the morning :)
thanks for helping, everybody.
That wasn't the only error in your code. Check my previous reply.
Pages