root: Junk rut (R = +1)
▼ choose an action ▼
Indulge
visits N = 0
value —
UCB ∞
Invest
visits N = 0
value —
UCB ∞
Each iteration: select an action by its UCB score, expand to the
next state the model samples, simulate by rolling out that strategy (keep taking the chosen
action) for its discounted return, then back up that return into the action's running value.
Invest's rollouts reach Healthy and pull ahead; Indulge's stay in the rut.