RL scales when its task does

Reinforcement learning does improve with a bigger model and more training, but every documented case changes the task or the optimizer along with the size, and the plain version of the idea (widen the actor, train longer on the same task) has no support for blind locomotion in the literature or, so far, in Shappy's runs. RL scaling follows power laws in model size and interactions on Procgen, Dota 2 and a toy task, with constants that differ by orders of magnitude between tasks (Hilton et al.), and it pays when the target is broad: growing a humanoid tracker from 1.2 M to 42 M parameters, with up to 100 M frames of motion capture, lifted unseen-motion success from 98.0% to 99.6%, while a velocity-tracking specialist in the same study stopped improving beyond 8 GPUs (SONIC). Naive widening is the riskier move: PPO on ANYmal collapsed as its MLP grew from 2.4 M to 80 M parameters (Castanyer et al.), and Shappy's actor, widened twofold and continued for 400 M steps at an unchanged learning rate, collapsed too (trained-lane pass share 41.9% against 83.1%). Behaviour cloning scales cleanly because its labels are fixed and dense, and an RL-teacher-then-student pipeline inherits that without passing the teacher: Shappy's same-size student reached 0.825 against 0.831. Continuing a converged policy gained nothing: plain continuations finished 2.9, 7.6 and 12.3 points under their start, and keeping the critic and curriculum cut the loss to 1.5 points without producing a gain; the training logs also show plain continuations breaking in single updates (four of seven runs had one or more updates after which more than half of the training robots fell), where the state-kept run and the three narrow-bell runs had none. The small-command dead zone (commands the robot ignores) is a reward-geometry problem that scale does not move: narrowing the width of the forward tracking reward (the "bell", σ = 0.25 in training, 0.05 in the probes) lowered the translation thresholds by only 0.05–0.10 m/s, the size of run-to-run drift, and the turn threshold fell from 0.4 to 0.22–0.30 rad/s only when a narrow yaw bell and a 20% share of small commands were combined, at the cost of the translation responses. The decisive runs (a full-length wide actor, replicate controls, a continuation that waits while the optimizer warms up, the three bells with small commands on a state-kept start) are pending.

Fixed-task RL flattens by 0.1–3 B steps, and growing the task keeps paying

The default recipe for massively parallel locomotion is a wall-clock compromise, not an asymptote. Rudin et al. used a 98,304-sample batch for 1,500 updates (about 147 M environment steps, arithmetic; under 20 minutes), and their own ablation found that a 1 M-sample batch gave the best policy at a much longer training time (Rudin et al.). Shipped recipes stay between 50 M and 2.58 B steps (Zakka et al.; Kumar et al.; Margolis and Agrawal); only open-ended distributions run to 10^10–10^12. No source read shows a fixed-terrain blind policy improving after its plateau by continuing an unchanged recipe. SONIC trains 50 k iterations "when performance usually plateaus", and its velocity-tracking baseline OpenHomie peaked at 8 GPUs and did not improve at 32, which the authors attribute to a task reward that saturates once the behaviour is mastered (Luo et al.). Under a promote-and-demote curriculum a flat evaluation reward is also the expected signature of a moving target, because a better policy meets harder terrain (Rudin et al.; inference).

Plateaus at a fixed recipe are partly optimizer artifacts. Beukman et al. trace PPO stagnation to an outer step that is too large for the noise in sample-based loss estimates, and report monotone gains to a trillion transitions in an open-ended 2D task by raising parallel environments past one million, where defaults plateau below ten billion (Beukman et al.); re-running an earlier saturation study with a minibatch of 16,384 instead of 98,304 made vanilla PPO "much better across the board" (Singla et al., re-run in Beukman et al.). The sampler matters as much as the count: frontier-focused sampling reached 73% success against 54% for prioritized level replay at 1 M environments on 13 ANYmal-D terrains, where uniform baselines at 4 K environments did not learn (Zhang et al.). Scaling laws for RL belong to the task: Hilton et al. find optimal-size exponents of 0.40–0.80 (about 0.50 for language models) and a 2,000-fold difference in the training a model needs between Dota 2 and MNIST (Hilton et al.).

Where robot learning shows scale paying, what grew was breadth. SONIC scaled parameters (1.2 M to 42 M; unseen-motion tracking error 27.7 to 23.8 mm), motion-capture frames (4 M to 100 M) and compute (2 k to 21 k GPU-hours; more GPUs gave a better asymptote at equal iterations) against a dense per-frame target (Luo et al.); across about 1,000 generated robots, unseen-body reward doubled from 5% to 100% of the training bodies, quadrupeds saturated near 100 bodies, and extra data on 5% of the bodies saturated at 4× (Ai et al.).

Wider PPO networks destabilise before they help

No ablation in the sources read stands behind the (512, 256, 128) actor that legged_gym, Playground and ETH-lineage code share. The largest controlled PPO study (more than 250,000 agents, five MuJoCo tasks, 1–2 M steps) found the best policy width task-dependent: Humanoid rose from about 1,420 at 16 units to about 2,850 at 512, while HalfCheetah lost 46% (values read from the authors' figures); it recommended a wide value network, which showed no downside (Andrychowicz et al.). At larger sizes the failure is documented more often than the success. On Isaac Gym's ANYmal, baseline PPO "collapses as model size increases" across MLPs of 2.39 M to 80.15 M parameters (30 M steps, 4,096 environments), while multi-skip residual connections with a Kronecker-factored optimizer keep it stable (Castanyer et al.); off-policy, a plain-MLP SAC degrades from 0.1 M to 17 M parameters (SimBa) and "scaling actor is not effective" (BRO). None of the proposed causes (gradient pathologies, dormant neurons, rank collapse) has been tested on legged PPO with a privileged critic. One PPO cause does not involve capacity: when the outer step is large relative to update noise the policy stalls at any size (Beukman et al.), and a wider layer moves its output further per Adam step, which is why Hilton et al. scaled Adam's learning rate with initialization scale, roughly 1/√width (Hilton et al.).

Where larger RL-trained controllers help, the target is rich or data-limited. A [1024, 512, 256] MLP beat [512, 256, 128] by 6.0 points with 10% of a humanoid tracker's data and by 0.23 with all of it (Lu et al.), and wider PPO actors learn faster but finish at similar or slightly lower return (Bian et al.). Otherwise small actors suffice: single hidden layers of 32–64 units reached 80% of the best reward on Brax locomotion (Hegde et al.), and a Cassie stepping policy was recovered by supervised cloning into a (16, 16) network from 600 samples (Xie et al.). The pattern (inference) is that capacity buys speed on easy targets and accuracy only where the target holds more structure than a small network can fit. The critic is the usual place to add parameters, but the evidence for that is off-policy (BRO and SimBa above) and transfers to PPO only by analogy.

Behaviour cloning scales on a fixed target and stops at its teacher

The premise needs a qualifier: BC scales smoothly where labels are abundant, dense and fixed, and flattens when diversity runs out. In NetHack imitation the loss-optimal model size grows as C^0.61 and data as C^0.39, return tracks loss, and Atari laws "converge exactly to the mean return of the expert" (Tuyls et al.). In robot imitation the scaling axis is diversity: no clear power law in demonstration count at fixed environments (Lin et al.); removing 25% of RT-1's tasks cost as much as removing 49% of its data (Brohan et al.); a transformer distilled from three RL specialists performed alike on 10% and 100% of 17,636 episodes (Caluwaerts et al.); and across about 1,000 generated robots, data per body saturated at 4× the smallest scale while adding bodies doubled unseen-body reward (Ai et al.). For hidden-state skills the scaled quantity is memory: a parkour MLP student scored 0% and 1% on climbing and leaping where a GRU student scored 86% and 80% (Zhuang et al.).

Locomotion uses RL-then-student because direct RL on the student's observation fails or is slow: OmniH2O's student reached 94.10% by DAgger on a 25-step history and 47.11% when trained by RL on the same history (He et al.). Students then land at 90–100% of their teachers and no further:

A student exceeds its teacher only through something other than size: pooled data sources (Radosavovic et al.), a comparison with RL specialists rather than with its own oracle (HOVER), or an RL stage after distillation (the last table row). The imitation-gap work explains the ceiling: information the teacher uses and the student lacks is marginalised during imitation (Weihs et al.), and a behaviour-cloning plateau at 67% rose to 83% only with RL refinement (FetchMan). A student also copies what its teacher does as a function of inputs the student sees, so a flat response to small commands should transfer (inference); no source tests command dead zones under distillation.

A continuation pays an unlearning cost that critic, optimizer and step size decide

Every source read that resumes a trained policy with RL reports an early drop unless value function, step size and state coverage are managed. Zhou et al. trace the drop to value-function divergence at the onset of fine-tuning (Zhou et al.); Policy Decorator blames a randomly initialised critic that "cannot provide meaningful gradients" (Yuan et al.); Wołczyk et al. show that fine-tuning degrades the part of the state space it stops visiting, which for a terrain curriculum means the hard levels a restart removes (Wołczyk et al.). The optimizer is a third cause: Adam's first bias-corrected update moves every parameter by exactly its learning rate and decays to about 0.16 of that after roughly 40 iterations, so OpenAI Five restarted after surgery at a learning rate of 0, and Ma and Yarats recommend warm-up over 2/(1−β₂) = 2,000 steps (Ma and Yarats; OpenAI et al.). Brax's PPO restores the policy and optionally the critic but not optimizer state, and neither Brax nor rsl_rl checkpoints hold curriculum levels (code read on main on 2026-11-03, not necessarily the pinned commit; Brax).

The closest controlled ablation, ADEPT (massively parallel PPO on dexterous hands, 4,096 environments, five seeds per row), separates the factors: every actor learning rate of 1e-3 collapsed (0 of 5 seeds reached the final level) while every 1e-5 row held 27–47% success, and a critic warm-up under a frozen actor added 17.6 points (ADEPT). OpenAI Five's surgery-based run absorbed more than 20 changes but ended weaker than a from-scratch rerun (OpenAI et al.). No source continues a thousand-environment locomotion policy or separates critic, optimizer state and coverage; the project's runs are the nearest factorial.

Dead zones, drift and asymmetry belong to the reward and the loop, not the weights

legged_gym and its descendants zero any planar command of norm 0.2 m/s or less at sampling time, so those policies never see a slow command (legged_gym); Pupper's pipeline zeroes nothing, so its 0.2 m/s cliff is not that rule. What the code bases share is a tracking kernel exp(−e²/0.25) beside effort penalties, under which standing instead of tracking a command c forfeits only 1 − exp(−c²/σ) of one term: 0.148 at c = 0.2 and 0.302 at 0.3 for σ = 0.25, and 0.551 and 0.835 for σ = 0.05 (arithmetic). The cliff recurs across systems. MuJoCo Playground issue 361 reports an Op3 policy that realises 0.003 m/s of a 0.1 m/s command and 0.105 of 0.2 in MJX, but 0.000 and 0.001 in C MuJoCo, and reads it as standing earning 0.85 of the tracking reward at 0.2 m/s (issue 361). A benchmark of 34 open-source G1 policies finds three that "will not turn until asked for 0.4–0.6 rad/s" (write-up; single author, and its statements about third-party code are unverified), and Fu et al. clip the yaw command to at least 0.4 rad/s at low speed (Fu et al.). The same yaw cliff on different robots and rewards suggests a structural cause on top of the incentive gap (inference): turning in place needs a different foot pattern from stepping forward. Every shipped fix sits outside the policy (minimum-command clips, a four-foot stand at small commands, bias subtraction, estimator feedback); the training-side fixes (command-scaled σ, a minimum non-zero training command) are proposed in the issue and untested there.

The stand/step switch is itself a design choice. Free-running gait clocks make a zero command mean stepping in place, which fights a tracking term that pays for standing (the Playground maintainers' account of the G1 at zero command, issue 77); the shipped remedies are a phase frozen at zero command (Playground's T1 recipe), a state machine that stops the clock after 0.5 s of zero command (Lee et al.), or standing as its own command category (van Marum et al.). None was shown to remove a small-command dead zone, and in Walk These Ways the gait-free policy was slightly better at the lowest speeds (Margolis and Agrawal). Shappy's mt_c has no clock, so its switch is learned.

On symmetry, Mittal et al. report that PPO with symmetry augmentation "obtains the highest return and fastest convergence", while optimizing a mirror loss "comes at the cost of performance and slower convergence" because it ties equivalent states together whatever their rewards; perfectly symmetric policies also struggle at neutral states, since a quadruped starting from a stance cannot lift one front foot if its symmetric partner must lift too (Mittal et al.). Su et al. rank strict equivariance above augmentation above vanilla PPO and drop the loss because augmentation outperformed it, yet their augmented policy scored 6% below PPO on door pushing, and randomising joint properties breaks the invariance the methods assume (Su et al.). Drift under neutral commands persists with symmetric training, and an outer heading-and-lateral loop cut a G1's lateral RMSE from 0.339 to 0.022 m (RGB). No source ties symmetry to small-command dead zones.

What our measurements say: weights with slack, steps without gain, dead zones no single change closes

These are Shappy's own measurements, from the command-map README and the project logs it cites. The actor is the official size (720 inputs, 20 steps of 36; hidden layers 1024-512-512-512; 1.8 M weights). mt_c is the multi-terrain walking policy on the robot, trained 1 B steps (about an hour) from the stair policy own3_h1. "Pass share" is the fraction of 100 trials that cross a lane, averaged over the lanes of the standard test course that training covers (trained) or holds out (held-out); identical evaluations differ by up to 0.7 points. "Replica" is the simulated copy of the printed stair course, crossed in 100 trials per rise. The "dip" of a run is its evaluation reward at step 0 minus its lowest later value. A "dead zone" is the range of small commands the robot ignores: for mt_c a forward command of 0.15 m/s gives 0.000 m/s and 0.20 gives 0.178.

The actor has slack on flat ground, except at the stand-to-step switch

On flat-ground data (10 command modes × 8 trials, 34,000 inputs), replacing each weight matrix of layers 2–4 by its rank-r SVD without retraining left walking nearly intact at half rank and cost about a fifth at a quarter: mt_c walked 0.424 m/s at a 0.5 m/s command at full rank, 0.416 at rank 256 and 0.329 at rank 128 (own3_h1 0.427, 0.395, 0.330), with no falls. Ninety percent of each layer's activation variance sits in 20–34 directions, and 66–123 of 512 last-layer units (13–24%) hardly move. The turn from standing is the first function lost: at a 0.5 rad/s command mt_c fell from 0.485 to 0.465 (rank 256) and 0.208 (rank 128), own3_h1 from 0.415 to 0.337 and −0.002. The probe covers flat ground only and truncates without retraining, so it shows slack, not that a larger network could not do better; and idle units are defined over the sampled inputs, so units serving stairs can look idle on a flat batch (inference from Sokar et al.).

A twofold wider actor broke the continuation at an unchanged step size

ct_w widened every actor hidden layer twofold (5.70 M weights, 3.16× the multiply-adds) without changing the function (new units with random incoming and zero outgoing weights) and continued mt_c for 400 M steps at the recipe's learning rate of 3e-5. Its evaluation reward started at 18.05, fell to 5.1 at 120 M steps and ended at 4.33, against 17.59 → 12.83 → 17.19 for the plain continuation ct_c; mean KL per update ran 0.047–0.078 against 0.034, with a spike to 73,235 at 290 M steps. The final policy spins and has no forward or sideways response. On the standard course it passed 41.9% of trained-lane trials and 25.8% of held-out ones against 83.1% and 85.5% for mt_c (read 2026-11-03 01:26; the earlier prediction that widening would change nothing was wrong). By the reading written before the run this says nothing about capacity. The direct test, mt_w1, retrains the full 1 B steps with the actor twice as wide at half the learning rate (1.5e-5; Hilton et al.'s 1/√width heuristic would give 2.1e-5) and is pending. The wide actor trains at 176 k steps/s against 303 k, so widening costs 1.7× the wall-clock for 3.16× the multiply-adds, as Hilton et al.'s rule predicts for a model cheaper than its simulator (inference).

More steps from the same policy lost ground, and the restart explains most of it

Three plain continuations finished 2.9 (mt_cc), 7.6 (mt_sc) and 12.3 (ct_c) points under mt_c on the trained lanes, on code that changed between them, so one run cannot rank a course effect smaller than about nine points. The loss falls on hard lanes (the 4 cm replica fell from 82 to 9 crossings for ct_c), and ct_c's curriculum ended near half of mt_c's levels (stairs-up level 2.0 against 5.0). A restart returns robots to easy terrain (every plain continuation starts at level 1.5, ct_v at 4.1), but ct_c had climbed to 4.0 by 223 M steps before a collapse (below) threw it back to 1.4, so the final level is that of the survivor of its last collapse, not of the restart alone. ct_v kept the critic and the levels, ended 1.5 points under mt_c and 10.7 above ct_c in the same job: the restart, not the continuing, costs the ground. It did not gain, and it kept one early shock (reward 18.10 → 10.87 within 40 M steps, back to 16.5 by 80 M). The literature explains the slow loss (a reinitialised critic, lost coverage) and leaves the shock to the optimizer's first steps, untested; ct_v's learning rate of 3e-5 sits nearer ADEPT's safe 1e-5 than its collapsing 1e-3 on a log scale, in a different task (inference). ct_vf, whose policy waits while the optimizer warms up, is pending.

Training length has diminishing but real effect: mt_c's evaluation reward moved only from 16.78 to 17.20 over 1 B steps while its stairs-up level rose from 1.5 to 3.8, 4.4, 4.8 and 5.0 at 254, 503, 752 and 1,001 M steps (+0.6, +0.4, +0.2 per 250 M). The curriculum's rule of three passes up and one failure down pins the pass rate near 0.5^(1/3) = 0.79 (arithmetic), so level, not reward, is the progress measure. A continuation helped only where the reward changed: sb_a (mt_c continued with a narrower forward tracking reward) raised following of a 0.5 m/s command from 81% to 93% at the price of the 3 cm replica crossings (96–98 to 74–79 of 100).

Plain continuations break in single updates (read afterwards from the logs, not a registered reading)

Drawing the curriculum levels showed ct_c's stairs-up level falling from 4.0 to 1.4 within one logging step. Counting the steps at which more than half of the training episodes (averaged over the seven terrain families) ended in a fall, against 0.01–0.14 otherwise, gives these events, each with a largest update KL of 25–150 against a typical 0.1–0.5 (progress logs on the GPU server):

Plain-bell continuations that restart the critic collapsed in 4 of 7 runs (mt_cc, ct_c, sm_a and ct_w did; mt_g did not, and mt_sc and mt_sym ran only 200 M steps); the three narrow-bell continuations (yw_a, ys_a, sb_a) collapsed in 0 of 3 and the state-kept continuation in 0 of 1; mt_c's own 1 B steps had none and the older mt_a and mt_b two each. The level then climbs back from near zero, so the final course share of a plain continuation is that of its last survivor. Collapses and losses go together without lining up (ct_c 2 collapses, 12 points under; sm_a 1, 12; mt_cc 1, 3; ct_w 4, 41; against yw_a 0, 8; ys_a 0, 7; ct_v 0, 1.5). Nothing here says why the narrow bells did not collapse, whether a plain continuation collapses reproducibly (ct_c2 and ct_c3 will say), or whether bounding the update (a KL-adaptive step, gradient clipping) removes the collapses without costing the course; that is the next lever and is not registered. The ADEPT result above (actor learning rate decides collapse versus no collapse) and Brax's missing optimizer-state restore point at the same place (inference).

The same-size student matched its teacher in its own simulator

uni_b, a 1024-512-512-512 MLP distilled by DAgger from a composed teacher (walking with commands, a fall-recovery policy and the walking controller's start-up), reached a trained-lane share of 0.825 against 0.831 for the walking-with-commands teacher in the same jobs, and 0.855 against 0.861 held out (UNIFIED.md). It met all four behaviour commands, and its fall-and-walk chain was within 0.05 of the teacher's at every start (800 trials each). In the robot's ROS 2 simulator, with different contacts, it recovered and walked from 7 of 12 fall poses where the composed system managed all 12, so the match depends on the simulator the student learned in. Whether it inherits the teacher's dead zone has not been measured.

The turn threshold moved only when the reward bell and the practised commands changed together

A threshold is the smallest command whose realised response is at least half of itself, from a standing start (10 trials per condition, the training model in plain MuJoCo).

Below its threshold the robot does not move (mt_c turns ±0.007 rad/s at ±0.22 and +0.399 at +0.40), but while walking at 0.25 m/s there is no band (a 0.1 rad/s turn is followed), so the dead zone belongs to the standing posture, where the policy switches between a stand mode and a step mode at a command size. The one controlled pair, sb_a against mt_cc, differs only in the forward bell and moved the forward threshold from 0.20 to 0.15, backward from 0.30 to 0.20 and sideways from 0.30 to 0.20 (all four down, the turn left at 0.5), which fits the reward arithmetic and issue 361's reading; but shifts of 0.05–0.10 are the size of the drift among plain continuations (forward 0.15 to 0.25 for mt_cc, ct_c and ct_v), so the replicate controls decide how much of it is the bell. The yaw axis does not fit it alone. Narrowing the yaw bell (yw_a) raised the turn threshold to 0.5 on the left, with no response on the right up to 0.5, and lost the small right turns while walking (−0.10 → −0.006, against −0.116 for mt_c). Replacing 20% of commands with small single-axis ones (sm_a; about 26 M steps of pure small turns) moved it by 0.1 at most, the drift between plain continuations, consistent with standing earning 91% of the yaw term at 0.15 rad/s. Together (ys_a, read 2026-11-03 01:51) they moved the turn to 0.22 left and 0.30 right, the first run to do so, and paid with the translations (forward 0.40; backward and sideways unresponsive up to 0.30). The issue's untested fixes (a narrower or command-scaled bell, a minimum non-zero training command) have analogues here, and the analogues work only together, at a cost to the other axes. The course jobs of sm_a and ys_a are in (read 2026-11-03): trained share 71.5 and 76.1 against the control's 71.7 and 70.9, 4 cm replica crossings 35 and 40 against 9 and 6 (mt_c 80), so neither arm ends near mt_c, and the narrow-bell arm sits 5 points above the control, which had two collapses where the arm had none (previous section). The mixed-bell continuation yl_v is pending.

Outside the policy the symptom is fixable. With mt_c on the training model, upstream's person follower (a proportional law that turns the robot toward a detected person) centred a person 8–30° off in 16 of 48 trials, a minimum turn command of 0.45 rad/s centred 48 of 48 (mean overshoot 3.2°) and a pulsed command 48 of 48 (2.7°); simply raising the follower's gain from 2 to 6 centred 47 of 48 (overshoot 8.6°, median 2.2 s against 0.9 s and 0.6 s, no reversals), the cheapest follower change, run on mt_c only. The minimum must sit just above the policy's own threshold: 0.45 centred ct_c and ct_v but only 33 of 48 for yw_a, 0.6 centred all three (5–6° overshoot), and 0.8 hunted (up to 30 sign reversals). In the ROS 2 simulator upstream's follower left a perceived offset of 5–11° standing, as in the robot's own follow-me recording (stuck 23 s at 8.4°), while a 0.6 rad/s minimum and a 0.9 rad/s pulse centred the three cases needing a turn in 2.3–9.1 s and 1.8–4.7 s (a reading made after the fact). The level is the policy's threshold plus a margin, to be measured on the robot with its IMU; that has not been done.

The mirror loss moved lanes at an unchanged aggregate share

mt_sym added the squared difference between the policy's mean action and its mirror image, weighted 1, to the PPO loss for 200 M steps from mt_c; the control mt_sc is the same continuation without it (STRAIGHT.md). Symmetry error fell from 0.90 to 0.014 and the yaw drift of a straight forward command fell 81% at 0.5 m/s (+0.80 → +0.15 deg/s) and 44% at 0.75 m/s against the control, without reaching zero. The trained-lane share did not change (0.755 against 0.753), but four lanes fell by 17–26 passes (3 cm stairs 86 → 60, 7 cm curb 33 → 7, 5 cm boxes 25 → 8) and three rose by 19–42 (25° slope 1 → 43, 20° slope 68 → 94). Separately, 200 M further steps without the loss flipped the turn from −1.59 to +0.80 deg/s: more training redraws the asymmetry instead of averaging it out.

Where the literature and the measurements agree

Controls first, then width, then the levers that cost minutes

Position: do not widen the deployed actor on current evidence; finish the replicate controls before reading any course effect, because three plain continuations already differ by up to 9.4 points; treat continuations as probes of whether a change moves its target, never as the policy for the robot; and spend the cheap CPU and offline measurements before another GPU hour. Registered runs carry readings written before they ran; none has a result yet.

The additions below follow from the literature; they are proposals, not registered runs. First, the command map of the student and its teacher (CPU, 32 s per policy): thresholds within 0.1 of the teacher's mean the dead zone is inherited, as the imitation-gap argument predicts, so the fix belongs in the teacher or the follower; lower by more than 0.1 means distillation smooths the boundary and is itself a lever; higher means the boundary is the first thing a student drops. Prediction: inheritance, about 60%. Second, a retrain with twice the samples per update at the same minibatch size and learning rate (2 B steps, about 1.8 h), Beukman et al.'s remedy: hard lanes 5 passes up, or 3 points of share, mean the plateau was the step-to-noise ratio and the batch should scale before the network; no change means the ceiling is the task or the blind observation, and a 50-step history or a privileged-height actor is the next test. Third, a critic-only widening, free at deployment: a gain of 3 points means value-side capacity limits; no change (predicted at about 60%, because the strong evidence is off-policy) means it does not; a loss means critic gradients destabilise. Fourth, mirror augmentation in place of the loss (200 M steps from mt_c against mt_sc): drift at or under 0.3 deg/s with no lane more than 15 passes under mt_sc means symmetry is free in that form; the turn removed but lanes lost again means the constraint itself costs terrain; the turn unchanged means the body is the source and the heading hold (an outer loop that steers back to the commanded heading) stays. Last, weight norm, terrain-stratified dormancy and last-layer rank from the retrains' checkpoints: rising values mean plasticity limits training length, flat curves that the task or the step does.

One conditional run bridges scale and the dead zone. The turn from standing was the first function lost under compression, and ys_a showed that moving one axis's boundary costs the others. If yl_v fails by that trade, repeat its changes with the actor twice as wide at the learning rate that keeps mt_w1 stable: all thresholds at most 0.2 only in the wide net would make this boundary the first function on the robot limited by its weights; the same trade in the wide net would place the limit in the reward.

Conclusion

The contrast the question draws is better stated as a difference in what each method's signal contains than as BC scaling and RL not. Both flatten when the signal's diversity runs out, BC when more demonstrations come from the same environments and RL when the reward is already earned, and both resume when the target broadens (bodies, motion clips, frontier terrain). On that reading no weight-shaped ceiling has shown itself in Shappy's data so far: the actor has slack and a same-size student holds three skill groups. Its weakest behaviour sits where the reward is flat (standing earns 91% of the yaw term at 0.15 rad/s), and scale cannot buy a gradient the reward does not give. A dense target can, which is what SONIC's motion capture and the teacher-student route supply, so the dead zone is the one failure for which the BC route may offer a lever that a bigger RL run does not (inference; no source tests it).

In practice (inference), parameters go first to the critic and only with a smaller step; steps go to a changed task or a larger batch per update, not to the same recipe; breadth goes to teachers and bodies; and the dead zone is handled by a follower law calibrated to each policy's measured threshold until one policy holds all three axes at once.

What stands, what fell, what is unmeasured

What stands

• RL scaling is real and task-bound: success 98.0% to 99.6% from 1.2 M to 42 M parameters on motion capture; unseen-body reward doubled with more bodies while more data per body saturated.
• The actor has slack on flat ground: half the rank cost 0–8% of walking speed (mt_c 0.424 to 0.416 m/s); the turn from standing was the first function lost.
• A same-size student reproduced its teacher on terrain, 0.825 against 0.831.
• The restart, not the continuing, costs ground: ct_v ended at 81.7% trained share against 71.0% for ct_c, 1.5 points under mt_c.
• Bell and data together lowered the turn threshold to 0.22 and 0.30 rad/s (plain continuations 0.4–0.5; either change alone 0.3–0.6).
• A follower minimum command or pulse just above the policy's threshold centred 48 of 48 trials against 16 of 48 for the upstream law; a gain of 6 centred 47 of 48 on mt_c.
• The state-kept continuation and the three narrow-bell continuations had no collapse; four of seven plain continuations had one to four.
• The mirror loss lowered asymmetry (0.90 to 0.014; turn down 81% at 0.5 m/s).

What fell

• Widening the actor and continuing at the same step size: reward 18.05 to 4.33; 41.9% against 83.1% on the trained lanes.
• Continuing a converged policy for more steps: 2.9, 7.6 and 12.3 points under the start; no continuation gained.
• A narrower yaw bell alone as the turn fix: the threshold rose from 0.4 to 0.5 rad/s on the left, with none up to 0.5 on the right.
• Small-command data alone as the turn fix: the turn threshold moved by 0.1 or less and the forward threshold rose from 0.20 to 0.30 m/s.
• Reading a course effect from one continuation: plain continuations spread over 9.4 points.
• The critic and the levels as the cause of the early shock: ct_v kept both and still lost 7.23 reward units.
• The restart as the whole story of a plain continuation's end state: ct_c reached level 4.0 before its collapse threw it back.
• The mirror loss as free on terrain: four lanes fell by 17 to 26 passes.

What is unmeasured

• Width at full length: mt_w1 against mt_r0 and mt_r1.
• The noise of continuations and of retrains, and with it whether the forward bell, not drift, lowered the translation thresholds by 0.05–0.10 m/s: ct_c2, ct_c3, and the gap between mt_r0 and mt_r1.
• Whether the optimizer's first steps cause the early shock: ct_vf.
• Whether the turn threshold can fall without losing the translations, and whether width is what the trade consumes: yl_v, then the same changes at twice the width.
• Whether plain continuations collapse reproducibly, why the narrow-bell ones did not, and whether a bounded update prevents it: ct_c2 and ct_c3 (count the collapses in their logs), then a KL-bounded continuation.
• Whether the student inherits the dead zone: the command map of uni_b and its teacher.
• Whether samples per update, not steps, limit the plateau: a retrain with twice the samples per update.
• Whether the critic's width matters: a critic-only widening against the two retrains.
• Whether mirror augmentation avoids the lane cost: augmentation against mt_sc.
• Whether the weights drift over 1 B steps: norm, dormancy and rank from the retrains' checkpoints.
• Whether information, not weights, limits the hard lanes: a 50-step history or a privileged-height actor against mt_r0.
• The robot's own thresholds and lag: the open-loop sweep with its IMU.

Source notes, scripts and their outputs: notes/research_notes/Scaling RL policies versus BC/ in the Shappy repository.