Continuous gridworld
Settings
Settings
World
Hidden layers
8 units
8 + 8 units
16 + 16 units (better for the maze)
Learning rate α
0.030
Discount γ
0.90
Exploration ε
0.20
Step length
0.50
Length noise
0.10
Angle noise
10°
Living reward
-0.05
Speed
4 steps/s
Plot size
100%
Show the best move at a grid of points
max Q(s, a) at each point
Episode 1
−1 → 0 → +1
this episode's path
w ← w + α[
R(s) + γ max
a′
Q(s′, a′)
− Q(s, a)] ∇
w
Q(s, a)
Step
Run continuously
Run 100 episodes
Reset