PPO cart
Settings
Settings
Policy learning rate
0.0010
Discount γ
0.980
Entropy bonus β
0.010
Episodes per batch
8
Clip ε
0.20
Update epochs
4
GAE λ
0.95
Value learning rate
0.0030
Reward
+1 catch · −d/10 miss
+1 catch · −1 miss
Speed
1.0×
Game size
100%
Fast-forward (skip animation)
Show where the ball will land
Game
Episode 1
← Left
Coast
Right →
L = mean min(ρA, clip(ρ, 1−ε, 1+ε)A) + βH(π)
ρ = π(a | s) / π
old
(a | s) · maximize L
Step
Run continuously
Run one batch
Reset