Survey: choosing footholds with cameras and depth sensors

Whether seeing the ground and choosing where to step would help Pupper. Explicit footholds pay mainly on sparse terrain such as gaps and stepping stones; on stairs a policy that sees the terrain does as well, and at Pupper's size the errors of published systems are as large as the margin an 80 mm tread leaves.

• Explicit footholds win on sparse terrain (DTC crossed a 0.1 m gap 10 of 10 times where an end-to-end policy failed) and tie on stairs once both see the terrain (RLOC 93.0 % against 93.2 %)
• At Pupper's size it has been done with motion capture and known terrain (LittleDog, whose legs match Pupper's, and Solo-12), and once with a camera on the robot
• A 40 mm foot on an 80 mm tread leaves its centre a ±2 cm band. Published foothold trackers reach their targets within 2.3–5.3 cm, and 100 ms of latency at 0.5 m/s is 5 cm
• Pupper's head camera sees each front foot's next foothold at about 1 mm per pixel, but it gives no depth, loses the touchdown spot 20–70 ms before contact, and last sees a hind foot's landing ground 0.56–0.61 s before it lands, longer than the policy's 0.4 s history
• Proposed order: first give the policy the height scan its critic already receives and compare it with a blind twin on 5 cm stairs, 4–6 cm boxes and sparse lanes; model a real sensor only if the scan helps