#LolaEsme / battlesnaketutorial

Battlesnake Tutorial — Structural Review (Parts 5–9)

I let compile errors distract me from the real problems. That's on me. Here's what actually matters.

The core issue: the difficulty curve is broken. Parts 1–4 are a well-built staircase. Parts 5–9 are a cliff. The reader who understood Parts 1–4 is not prepared for what comes next.

Part 5 — Imitation Learning ⚠️ Minor

• [clarity] The training loop (VarMap, VarBuilder, AdamW, cross_entropy, backward_step) appears from nowhere. Part 4 ended with A\* pathfinding. The reader has never seen a training loop. Add a conceptual bridge before the code: what does "train" mean? What's the loop doing at a high level?
• [clarity] cross_entropy is used, then explained after the code. Move "The loss function intuition" before the cross_entropy call. Explain the idea before the function.

Part 6 — RL Basics red_circle: Major

• [error] The policy gradient equation θ ← θ + α · ∇_θ log π(a_t | s_t; θ) · G_t appears with zero explanation of any symbol. θ, α, ∇, π, log — all undefined. This is the chapter that loses the audience. Every symbol must be defined in terms the reader already knows (θ = the network's weights from Part 5, α = the learning rate, π = the softmax output from Part 3, etc.), and the intuition must be stated in plain language before the equation.
• [clarity] "The policy" is introduced as jargon without connecting it to what the reader already knows. Say: "In RL, the function that maps states to action probabilities is called a policy. We've been building one since Part 3 — our network's forward pass is a policy."
• [clarity] The reward table is a grid where there should be prose explaining why each reward has its value. Why is survival +0.01 instead of 0? Why is death -1.0 and not -10? These are design decisions with reasoning — that reasoning needs to be paragraphs.
• [clarity] The discount factor γ gets "usually 0.99" but not why we discount. One sentence: "A reward you get now is more valuable than a reward you might get later, because the snake might die before reaching it." Also explain the superscripts: γ¹⁰ ≈ 0.9, γ¹⁰⁰ ≈ 0.37 — rewards 100 turns from now matter about a third as much.

Part 7 — Self-Play Training red_circle: Major

• [error] This chapter is a monolith. Seven distinct concepts (self-play concept, game simulator, Player trait, OpponentPool, return normalization, epsilon decay, checkpoint evaluation) in one chapter with ~4:1 code-to-prose ratio. The GameEnv implementation alone deserves its own part. Split this into two chapters: (1) the game simulator, (2) the self-play training loop.
• [clarity] The Player trait appears from nowhere. Why a trait? Why not call the network directly? Explain the motivation before the definition: we need both heuristic and network players in the simulator.
• [clarity] Return normalization (subtract mean, divide by std) appears as code with "reduces variance" as a comment. Explain how and why — connect it to the variance problem from Part 6. What does "high variance" look like in practice? The gradient flips direction between episodes. Normalization makes the signal consistent.
• [error] compute_returns is called but not redefined and not cross-referenced to its definition in Part 6. The reader has to remember or scroll back.

Part 9 — Scaling Up red_circle: Major

• [error] The dueling network equation Q(s, a) = V(s) + A(s, a) - mean(A(s, ·)) uses Q, V, and A — Q is never defined. Explain all three symbols before the equation. Then give the insight in plain language: "It's easier to learn two simple things (how good is this position overall? how much better is this move than average?) than one complicated thing."
• [error] - mean(A(s, ·)) is technically explained as "identifiable" — that's true and opaque. The plain version: "Without subtracting the mean, V could be 10 and A could be [1,2,3,4], or V could be 13 and A could be [-2,-1,0,1] — both give the same Q values. The network could learn either, making training unstable. Subtracting the mean forces advantages to average zero, so V carries the base value."
• [error] Residual connections introduce "vanishing gradients" without definition. Explain: each layer multiplies the gradient by a small number during backpropagation. Multiply enough small numbers and the result is tiny — early layers barely update. The residual connection gives gradients a shortcut path, so the layer only needs to learn the difference (the residual), which is often a smaller adjustment than learning the full transformation.

Cross-Cutting

• [clarity] Code-to-prose ratio is inverted in the second half. Parts 1–4 have roughly equal code and prose. Parts 5–9 are mostly code with a few paragraphs between. Every code block should be preceded by: (1) what we're building, (2) why we need it, (3) how it works conceptually. Then the code. Then a brief walk-through.
• [error] "What the network learns" sections in Parts 5–7 describe patterns the network might learn as if they're fact. Reframe: "If training is working well, you might observe the network doing X."

Parts 1–4 work. The fix for 5–9 isn't more code or more comments — it's prose. The same plain-language conceptual bridges that make the first half work.

Full REVIEW.md updated in the repo.

All [error] items from Robert's structural review are now addressed. Here's what landed:

Part 6 — RL Basics:

• Every symbol in the policy gradient equation is now defined (θ, α, π, ∇_θ, G_t, log) with references to where the reader has seen them before
• "The policy" jargon connected back to the forward pass → softmax they've known since Part 3
• Reward table converted to prose explaining why each value (survival +0.01 prevents myopia, death -1.0 vs -10 shapes risk tolerance)
• Discount factor γ gets the "why" plus concrete superscript examples (γ¹⁰ ≈ 0.9, γ¹⁰⁰ ≈ 0.37)

Part 5 — Imitation Learning:

• Conceptual bridge before the training loop: what "train" means at a high level
• Loss function intuition moved before the cross_entropy call

Part 7 — Self-Play Training:

• Player trait motivation added: why abstract over heuristic vs network
• compute_returns cross-reference note linking to Part 6 definition
• Return normalization prose connected to the Part 6 variance problem (gradient flip between episodes)
• Conceptual bridges before GameEnv and NetworkPlayer code blocks
• "What the network learns" reframed as conditional observation

Part 9 — Scaling Up:

• Q/V/A defined before the dueling equation
• -mean(A) explained with concrete numbers: V=10/A=[1,2,3,4] vs V=13/A=[-2,-1,0,1] both give the same Q, so subtracting mean forces A to carry only the differences
• Vanishing gradients explained before residual connections

One open question: Robert asked to split Part 7 into two chapters. The conceptual prose now does the work that split was meant to do — if the code still feels heavy, splitting is still the right call. It would mean updating SUMMARY.md and renumbering Part 8 → Part 9. Let me know if you want me to proceed with that.

Commits eebf008 and d9239c1 are on Forgejo.

Part 7 split is done. Here's the current structure:

Part 7 — Self-Play: The Game Simulator

• Why self-play works
• Training loop diagram
• GameEnv, Player trait, NetworkPlayer

Part 8 — Self-Play: The Training Loop

• Opponent pool
• Full training loop
• Return normalization prose (connected to Part 6's variance problem)
• What the network learns (conditional framing)

Parts 9 and 10 are the old Parts 8 and 9 (wiring + scaling), renumbered accordingly. All navigation links updated.

Commits on Forgejo: 9fd9292 (split), f4be995 (REVIEW.md update). The tutorial now has 10 parts.

All [error] items from Robert's structural review are resolved. REVIEW.md is up to date.