Continuous policy gradient gridworld (REINFORCE)
Settings
Settings
World
Hidden layers
8 units
8 + 8 units
16 + 16 units (better for the maze)
Learning rate α
0.010
Discount γ
0.90
Entropy bonus β
0.010
Step length
0.50
Length noise
0.10
Angle noise
10°
Living reward
-0.05
Speed
4 steps/s
Plot size
100%
Baseline b (subtract the average return so far)
Show the most likely move at a grid of points
Most likely move, shaded by how likely (max π)
Episode 1
less → more likely
this episode's path
θ ← θ + α Σ
t
[
G
t
− b
] ∇
θ
log π(a
t
| s
t
)
Step
Run continuously
Run 100 episodes
Reset