https://josephausterweil.github.io/probintro/decisions/index.htmlChibany learns to act. Believing is not enough: autumn brings the health kick, and every belief now has to cash out as a choice. This Part goes from beliefs to actions — losses and decisions, planning across time (MDPs), learning to act without a map (Q-learning), and then the great reversal: watching someone else act and inferring their goal (inverse RL, theory of mind), acting when the world itself is hidden (POMDPs and the Tiger problem), and the modern frontier where these ideas train today’s largest models.Hugoen-usThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/decisions/decision-theory/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/decisions/decision-theory/index.htmlFrom Beliefs to Actions For nineteen chapters we have asked one question, over and over, in a dozen disguises: given what I have seen, what should I believe? We built priors, turned data into likelihoods, and read off posteriors — the whole machinery of Bayesian learning. But a belief is not yet a choice. At some point Chibany has to put down the calculator and eat the bento. Jamal: “Okay, you’ve computed that the bento is probably fresh. Probably. So — do you eat it?”https://josephausterweil.github.io/probintro/decisions/mdps/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/decisions/mdps/index.htmlFrom One Decision to a Sequence Chapter 20 taught Chibany to make one good decision: weigh the loss, average over the belief, act. But life is not one decision. Eating the bento changes how hungry he is tomorrow; skipping the gym today makes going tomorrow harder. Actions reshape the world that the next action faces. The moment choices have consequences that ripple forward, “pick the best action” is no longer enough — you have to pick the best sequence.https://josephausterweil.github.io/probintro/decisions/q-learning/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/decisions/q-learning/index.htmlWhen You Don’t Know the Model Chapter 21 solved the Chibany MDP exactly — but only because we knew the whole world: every transition probability $T(s'\mid s,a)$ and every reward $R(s)$. Value iteration reads those numbers off the model. Take the model away and the Bellman backup has nothing to read. Jamal: “But Chibany doesn’t have the transition matrix of his own life. Nobody does. You just… try things and see what happens.”https://josephausterweil.github.io/probintro/decisions/inverse-rl/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/decisions/inverse-rl/index.htmlRunning the Camera Backwards The last three chapters all ran the same direction. A goal (or a reward) defined an MDP; value iteration turned it into a value; the value became a policy; the policy produced actions. Goal → behavior. That direction — forward planning — is what Chapters 20–22 built: even when the model is unknown, Chapter 22’s Q-learning still learns the goal-driven policy. Now watch the arrow turn around. You see someone in a café get up, walk past the pastry case, past the coffee bar, and straight to the door. You instantly form a belief: they’re leaving. You never saw their goal — you saw three footsteps and ran the planner backwards, asking which goal would have made those footsteps sensible.https://josephausterweil.github.io/probintro/decisions/pomdps-belief/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/decisions/pomdps-belief/index.htmlWhen the Agent Can’t See the State Chapter 23 inferred a hidden goal from behavior. But it quietly assumed the agent itself could see everything — it knew exactly which cell it was in. Real agents are not so lucky. A doctor cannot see the disease, only the symptoms; a robot’s camera is noisy; you cannot see the tiger behind the door. The state is hidden, and the agent must act anyway — interleaving cheap information-gathering with an eventual commitment, never fully certain when it acts.https://josephausterweil.github.io/probintro/decisions/modern-rl-world-models/index.htmlThu, 02 Jul 2026 00:00:00 +0000https://josephausterweil.github.io/probintro/decisions/modern-rl-world-models/index.htmlThe Same Inversion, at Frontier Scale The last two chapters built one machine and pointed it at smaller and smaller hidden things: a goal, a belief, a reward, another mind. All of it was hand-sized — three grid cells, two doors. This chapter is the same machine at the scale of the systems making headlines: aligning large language models, planning by imagining, and the genuinely unsettled question of whether those models have minds to read at all.