max Q(s, a) at each point

Episode 1
−1 → 0 → +1 this episode's path

w ← w + α[R(s) + γ maxa′ Q(s′, a′) − Q(s, a)] ∇wQ(s, a)