Walking straight: a heading hold

Told to walk straight, every policy turns slowly, the same way every time. The policies are not exactly left-right symmetric, and training never penalizes a slow turn, so each one keeps its own steady turn; its direction even differs between the training simulator and the robot's control software. A heading hold now sits between the command input and the walking policy. While the command has no turn, it steers back to the heading the robot had when the straight command began, read from the IMU. In the robot's control software the turn falls from up to 40 degrees in 20 s to about 1 degree.

• Robot's control software (ROS 2 simulation), 0.5 m/s for 20 s, three walks each, heading change without / with the hold: multi-terrain policy +18 to +19 / under 1 degree (2.0 / 0.7 m to the side), walking policy -9 to -12 / under 1 degree, Stanford's policy -34 to -40 / about -1 degree; walking speed unchanged, no emergency stops
• Training simulator, 32 walks per policy: the multi-terrain policy turns right there (1.6 degrees per second) but left in the control software (0.9-1.0); Stanford's turns left there and right in the control software. A copy of each policy with left and right swapped turns the other way by the same amount, and the robot model is symmetric, so the turn is in the policy's weights
• Why training leaves it: a steady turn of 1.6 degrees per second keeps 99.7 % of the yaw reward, and the policy does not see its heading. The turn also changes with the motor gains, the speed and further training (200 M more steps reversed it)
• What the hold leaves: the heading stays, but the robot drifts sideways 0.3-0.8 m in 20 s
• Making the policy itself symmetric was tested two ways and not adopted: averaging it with its mirror image keeps the walking policy's terrain results but costs the multi-terrain policy its 5 cm stairs (47 to 2 of 100); a symmetry term in training cut the turn by 44-81 % and cost four test lanes
• In the simulator the driving page turns the hold on (L switches it). Next: the same hold on the robot, after checking that the IMU's heading stays steady near the running motors, and a 5 m straight-line test with the hold off and on