Policy gradient gridworld (REINFORCE)
Settings
Settings
World
Learning rate α
0.20
Discount γ
0.90
Noise
0.20
Living reward
0.00
Speed
2 steps/s
Grid size
100%
Show
Probabilities of each move
Most likely move
Baseline b (subtract the average return so far)
Show the most likely move in each square (most-likely view)
Show the last move
Current policy π(a | s)
Episode 1
the move just taken
made more likely
made less likely (after each episode)
θ(s
t
, a) ← θ(s
t
, a) + α[
G
t
− b
](1[a = a
t
] − π(a | s
t
))
Step
Run continuously
Run 100 episodes
Reset