RL code transformer
REINFORCE
GRPO
Reward
Reward
Preset
Checklist (partial credit)
Tests only
All tests or nothing
Parses only
Custom
Parses
0.10
Runs
0.20
Right type
0.20
Tests
0.50
Tests: all or nothing
Training
Training
Adam rate
Batch
programs per update, in batches
Baseline b: subtract the average reward so far
Entropy bonus
Max length
tokens
Prompts
Prompts
Prompt
A random TinyCode program each time
Keep the one on screen
Pool
0 programs
Model
Model
Start from
Pretrained on TinyCode
Untrained
Network
transformer · window 32 · dimension 32 · 4 heads
View
View
Network connections
Loading TinyCode…
Run code
The checklist appears when the program is finished.
θ ← θ + α · (R − b) · Σ
t
∇
θ
log π(a
t
| s
t
)
Each finished program updates the model.
Live
Edit
Fast
Initializing WebGPU…
Run
Finish and update
Speed
2 tokens/s
5 tokens/s
20 tokens/s
Batches, as fast as possible
Updates
0
Programs
0
Tests passed
—
Reward
—