Q-network gridworld
Settings
Settings
World
Inputs
(x, y) position
one-hot square (9 inputs)
Hidden units
2
4
8
Learning rate α
0.050
Discount γ
0.90
Exploration ε
0.30
Noise
0.20
Living reward
0.00
Speed
2 steps/s
Grid size
100%
Show
Q-values
Values and policy
Show the best move in each square (values view)
Show the last move
Network's Q-values
Episode 1
Q(s, a) being updated
max
a′
Q(s′, a′): the largest value where it landed (an exit: its reward), used in the target
other values the same step moved
w ← w + α[
R(s) + γ max
a′
Q(s′, a′)
− Q(s, a)] ∇
w
Q(s, a)
Step
Run continuously
Run 100 episodes
Reset