Everything so far answered "what are the odds, and how should I bet?" against a fixed world. But poker has an opponent who adapts. If you only ever bet your strong hands, a thinking opponent folds every time you bet — you've become exploitable. The fix is unsettling and beautiful: play a mixed strategy, bluffing with exactly the right frequency, so that no opponent can do better than break even against you. That's a Nash equilibrium.
The astonishing part: a program can discover that strategy on its own, with no poker knowledge, just by playing millions of hands against itself and regretting its mistakes. The algorithm is Counterfactual Regret Minimization (CFR) — the exact idea behind the bots (Libratus, Pluribus) that crushed professional players. You're going to build it.
Full poker is enormous, so we use the smallest game that still has real bluffing — Kuhn poker:
Tiny — but it has value betting, bluffing, and folding. It has a known solved answer, so you can check your solver is right.
Play the game against yourself, over and over. At each decision, track regret: for every action you didn't take, how much better off you'd have been if you always had. Then make your future strategy proportional to positive regret — do more of what you wish you'd done. Average your strategy over all iterations, and that average provably converges to the equilibrium.
| check, check | showdown for the antes — higher card wins +1, loser −1 |
| bet, fold (or check, bet, fold) | the folder gives up the pot — bettor +1 |
| bet, call (or check, bet, call) | showdown for the bigger pot — higher card +2, loser −2 |
import random # Kuhn poker. Cards J,Q,K = 0,1,2. Actions: 'p' = pass (check/fold), 'b' = bet (call). # A history is a string of actions, e.g. "" (P1 to act), "pb" (P1 facing a bet). class Node: def __init__(self): self.regret_sum = [0.0, 0.0] # regret for [pass, bet] self.strategy_sum = [0.0, 0.0] # running total, to average later def strategy(self): # regret matching: probabilities proportional to POSITIVE regret r = [max(x, 0.0) for x in self.regret_sum] total = sum(r) # TODO: if total > 0 return the normalised r; otherwise return [0.5, 0.5] ... def average_strategy(self): # TODO: normalise strategy_sum (or [0.5,0.5] if it's all zero) ... nodes = {} # infoset key -> Node. key = str(card) + history, e.g. "2pb" def cfr(cards, history, p0, p1): """Expected value of this node TO THE PLAYER ABOUT TO ACT. p0, p1 = the reach probabilities of player 0 and player 1.""" player = len(history) % 2 # 0 or 1 # 1) TERMINAL? if the game has ended, return the payoff to `player` # (use the table above; remember 'cards[player]' vs 'cards[1-player]') # 2) infoset node info = str(cards[player]) + history node = nodes.setdefault(info, Node()) strat = node.strategy() # 3) try each action, recursing. NOTE the value comes back from the # OTHER player's point of view, so negate it. util = [0.0, 0.0] for a in (0, 1): next_history = history + ('p' if a == 0 else 'b') # TODO: recurse. if player==0, this player's reach is p0 and you pass # (p0*strat[a], p1); if player==1 pass (p0, p1*strat[a]). # util[a] = -cfr(cards, next_history, ...) ... node_util = sum(strat[a] * util[a] for a in (0, 1)) # 4) update regrets (weighted by the OPPONENT's reach) and strategy_sum # (weighted by THIS player's reach) opp_reach = p1 if player == 0 else p0 my_reach = p0 if player == 0 else p1 for a in (0, 1): # TODO: node.regret_sum[a] += opp_reach * (util[a] - node_util) # TODO: node.strategy_sum[a] += my_reach * strat[a] ... return node_util def train(iterations=200000): deck = [0, 1, 2]; value = 0.0 for _ in range(iterations): random.shuffle(deck) value += cfr(deck[:2], "", 1.0, 1.0) return value / iterations # average game value to Player 1
Terminal detection. A history ends when the last action is a fold, or when both have acted without a fresh bet outstanding. The finished histories are: pp, bp, bb, pbp, pbb. For pp and pbb/bb it's a showdown (higher card wins 1 or 2); for bp/pbp someone folded (the player who just acted wins 1, because their opponent folded).
The sign flip. cfr returns value to whoever is about to act. When you recurse, the child node is the other player acting, so their gain is your loss — hence util[a] = -cfr(...).
Regret matching by hand: if regrets are [−2, 5], positive parts are [0, 5], so play [0, 1] — always the second action. If both are ≤ 0, play [0.5, 0.5].
Why average? The current strategy bounces around; it's the average strategy over all iterations that converges to equilibrium. That's what strategy_sum / average_strategy is for.
Start with ~200k iterations. If your game value comes out near −0.056, you've almost certainly got it right.
Train your solver, read off the average strategy, and enter these. (There's a family of equilibria, so your exact bluff rate can vary — but these signatures always hold.)
Scale CFR up (with abstraction and sampling) and you get the poker superhuman AIs. The same regret-minimisation idea underpins parts of modern game-playing and even some approaches to training agents. You've touched the real thing. If you want the next rung after this, ask Smee — there's a market-making challenge with your name on it.