Current policy π(a | s)

Episode 1
the move just taken made more likely made less likely (after each episode)

θ(st, a) ← θ(st, a) + α[Gt − b](1[a = at] − π(a | st))