V-table gridworld
Settings
Settings
World
Learning rate α
0.50
Discount γ
0.90
Exploration ε
0.30
Noise
0.00
Living reward
0.00
Speed
2 steps/s
Grid size
100%
Show the best move in each square
Show the last move
Compare with
nothing
Vπ: true value of the current policy
V*: best possible value
Colour squares by error (V − the value compared with)
Current values
Episode 1
V(s) being updated
V(s′): the value where it landed, used in the target
↑
best move from each square
V(s) ← V(s) + α[
R(s) + γV(s′)
− V(s)]
Step
Run continuously
Run 100 episodes
Reset