Shappy · multi-terrain training
Results of the 1 B-step runs
Passes of 100 per test, mean of two evaluations; the stair policy is measured in the same job. A run replaces the stair policy if it is 5 points above it on trained tests, loses no trained test by more than 10 passes, keeps the printed-course 3 cm within 5 and stands at 0.17 m or higher.
18 tests every policy passes 97 times or more
| Stair policy (own3_h1) | C · from the stair policy | B · from the flat policy | A · from the flat policy, plain critic | |
|---|---|---|---|---|
| trained / held-out tests with pushes | 83.5% / 83.9% | 82.6% / 83.6% | 78.0% / 80.0% | 74.6% / 70.1% |
| printed course 3 / 4 cm | 99 / 82 | 99 / 77 | 98 / 63 | 95 / 12 |
| standing height | 0.181 m | 0.158 m | 0.166 m | 0.172 m |
Curriculum level at the end of training (0–13)
| family | C · from the stair policy | B · from the flat policy | A · from the flat policy, plain critic |
|---|---|---|---|
| flat | 8.1 | 7.6 | 7.9 |
| rough | 8.3 · ±1.4 cm | 7.2 · ±1.2 cm | 8.0 · ±1.3 cm |
| slope up | 6.2 · 10.6° | 4.5 · 8.2° | 4.6 · 8.4° |
| slope down | 8.5 · 13.7° | 7.8 · 12.8° | 7.8 · 12.8° |
| stairs up | 5.0 · 2.9 cm | 3.7 · 2.4 cm | 3.5 · 2.4 cm |
| stairs down | 8.1 · 4.1 cm | 6.8 · 3.6 cm | 7.4 · 3.9 cm |
| boxes | 4.9 · ±2.5 cm | 3.6 · ±2.1 cm | 3.7 · ±2.1 cm |
Training field
32 x 32 m heightfield, 2 cm grid. Columns: flat 1 · rough 2 · slope up 1 · slope down 1 · stairs up 3 · stairs down 3 · boxes 3 (of 14). Rows: level 0 (bottom) to 13. Flat outer ring.
Families × levels
Robots start at the tile centre: "up" tiles low, "down" tiles high. MuJoCo renders, robot in the default pose for scale.
flat
rough
slope up
slope down
stairs up
stairs down
boxes
Test course
38 lanes, 0.8 m wide, walked at 0.5 m/s from the triangle to the goal line (dashed in the profiles). * = held out: a terrain or value training never shows.
Baseline on the test course
Passes of 100 trials per lane. The stair policy was trained on flat ground and 2–4 cm stairs only. Measured on the first 34 lanes; the 4 lanes added afterwards are in the results above.
9 tests every policy passes 97 times or more
0.5 m/s with heading and sideways correction, 20 s limit; pass = goal line before a fall or leaving the lane.
Curriculum rule
Up a level after n crossings in a row, down after one failure: the level settles where pn = 0.5.
Standing still under a 0.5 m/s command keeps 66% of the tracking reward (1.5·e−1 + 0.8 + 0.5 of 2.8).
Our stair runs: a 3–5 cm field passed about 63% at the start and the policy learned to stop; a 2–4 cm field passed about 92% and kept climbing.
10% of episodes replay a random lower level and do not move the level.
Critic input
| feature (simulator truth) | size |
|---|---|
| actor input: 36 per step × 20 steps | 720 |
| base linear velocity (body) | 3 |
| base angular velocity (body) | 3 |
| gravity direction (body) | 3 |
| base height above terrain | 1 |
| foot clearance above terrain | 4 |
| foot contact | 4 |
| friction | 1 |
| push velocity | 2 |
| height scan 15 x 7, 3 cm grid | 105 |
| critic input | 846 |
Run A: the critic sees the actor input only. Runs B and C: the critic also sees the features on the left. A and B start from the flat policy, C from the stair policy.
The actor and the exported policy have the same form in every run; every critic starts fresh.
Status
| Field, test course, environment, evaluation and training code | ● done | multi_terrain.py · multi_env.py · eval_terrain.py · train_terrain.py |
| Height lookup against MuJoCo's surface | ● done | max error 0.0003 mm, 4,000 rays each on field and course |
| MuJoCo renders of the field and course | ● done | rendered on the Mac, 27 pictures |
| Baseline on the test course | ● done | first 26 lanes · 4 policies · 100 trials per lane |
| Training smoke run | ● done | 20 M and 8 M steps: per-family metrics logged, policy exported, 304 k steps/s |
| Baseline on the extended course, with and without pushes | ● done | 34 lanes · table above; pushes change the shares by under 2 points |
| Three training runs (1 B steps each) | ● done | none beats the stair policy: trained tests 82.9 / 78.8 / 75.4 % for C / B / A against 83.6 % |
| Course walked backward and sideways | ○ waiting | proposed: does the command mix hold the curriculum at 2.4–2.9 cm stairs? |