Behaviour commands: the step reward clipped at zero

Read every 5 M steps, the training logs of all behaviour runs show robots falling on flat ground far more often than the multi-terrain run. Most commanded-behaviour steps earned nothing, so a fall cost nothing. Two probes changed how the step reward is computed; the falls dropped, the terrain loss did not.

• Falls on flat ground during training (median, 40–200 M steps): ten behaviour runs 8–43 %, the multi-terrain run without behaviours 3 %
• Where behaviour training starts, 59 % of commanded-behaviour steps earn zero reward (the step reward is clipped at zero and the behaviour penalties exceed the rest); default-walk steps 0 %
• Probe 1, behaviour penalties scale the reward instead of being subtracted (as Walk These Ways does): training falls 23 → 7 %. Probe 2, the fall penalty kept after the clip: 10 %, with unstable updates
• Default walk, 100 trials, 4 cm stairs / 20° slope: probe 1 17 / 52, probe 2 20 / 38, previous probe 44 / 69, multi-terrain run without behaviours 83 / 98
• Probe 1 follows commanded behaviours on 3 cm stairs and a 15° slope in 6 of 8 tests (previous probe 0 of 8), but stands 14 cm high with a leg splayed out
• Untested causes of the terrain loss: the default walk is 25 % of training in every behaviour run, commanded behaviours on terrain, commanded behaviours sharing the network