Q-learning vs Dyna on the Chibany MDP

Dyna planning steps 10 α · learning rate 0.10

real steps 0

exact answer — value iteration (Ch 21):
V* = Junk 25.6, Trying 28.4, Healthy 39.8 · optimal = Invest everywhere

Q-learning · model-free

one TD update per real step

same TD update plus planning backups from a learned model