Final policy, seven commands in turn · 11 sFrom 0 to 1 billion steps · 2 min 4 s
The upstream Pupper v3 MJX pipeline, run on one lab GPU. One policy stands, walks, strafes and turns.
- 1 billion steps
- 1 h 41 min on an RTX 3080 Ti
- Reward 0.77 → 19.19
- Episode length 262 → 476 of 500
Policy and reward
Inputs and outputs
| Body rotation rate | 3 |
| Gravity direction | 3 |
| Velocity command | 3 |
| Orientation target | 3 |
| Joint angles | 12 |
| Previous action | 12 |
| Per step × 4 steps | 36 × 4 = 144 |
| Output | 12 joint targets, q0 + 0.75 a |
| Motor PD | kp 5.0, kd 0.25 |
Network
| Ours | Official |
|---|
| Input history | 4 steps | 20 steps |
| Hidden layers | 256, 128, 128, 128 | 1024, 512, 512, 512 |
| Parameters | 0.10 M | 1.79 M |
| Action range | ±0.75 rad | ±1.0 rad |
Reward weights
| Forward and sideways speed | 1.5 |
| Yaw rate | 0.8 |
| Upright body | 0.5 |
| Knee contact | −10 |
| Foot slip | −0.2 |
| Action change | −0.1 |
| Torque | −0.025 |
| Roll and pitch rate | −0.002 |