Bridging HPSTM and SKEL
Published:
Feeding 46 biomechanical angles into the kinematic layer is the moment HPSTM truly shakes hands with SKEL.
1. Background: Why SKEL over SMPL?
While SMPL dominates 3‑D human‐body pipelines, it naively treats every joint as a 3‑DoF ball‑and‑socket. SKEL [Xia et al., 2025, arXiv:2503.21751] pares those 72 DoF down to 46 biomechanically valid DoF, each bounded by anatomical limits. For HPSTM—whose downstream goal is safe motion transfer to robots—“no joint hyper‑extension” matters more than a few millimetres of MPJPE.
2. The Core Challenge: skel_euler_to_rot_mats
HPSTM’s transformer head predicts a 46‑D pose vector q. A naïve torch.euler_angles_to_matrix call fails because:
- Uneven DoF: the hip has 3 axes, the knee only 1.
- Custom axes: every DoF’s rotation axis is hand‑crafted in
joints_def.py, not a global X/Y/Z convention.
Miss either point and you will literally “ball‑joint” the knee—FK explodes.
3. Dissecting the HSMR Blueprint
HSMR—the first end‑to‑end SKEL regressor—shows the way:
| File | Purpose |
|---|---|
joints_def.py | maps 46 pose indices → joint names → custom axes |
osim_rot.py::exp_rot_vec | Rodrigues exponential map |
kin_skel.py | walks the joint tree and accumulates local T matrices |
The key idea: each DoF becomes an axis × angle rotation vector that is left‑multiplied into the joint’s existing local matrix.
4. Implementation Highlights
def skel_euler_to_rot_mats(q: torch.Tensor) -> torch.Tensor:
B = q.shape[0] # (B, 46)
R = torch.eye(3).repeat(B, 24, 1, 1) # 24 SKEL joints
for idx in range(46):
joint_id, axis = DOF_MAP[idx] # axis: (3,)
axis = axis.to(q.device)
theta = q[:, idx] # (B,)
rot_vec = axis * theta.unsqueeze(-1)
R_dof = exp_rot_vec(rot_vec) # (B,3,3)
R[:, joint_id] = R_dof @ R[:, joint_id] # left‑multiply
return R # (B,24,3,3)
Common pitfalls
- Multiplication order matters – left‑multiplication preserves the axis order (e.g., Z‑X‑Y for the hip).
- Use unit axes from
joints_def.py; defaulting to[1,0,0]will break elbows and ankles.
5. Outlook
skel_euler_to_rot_mats upgrades HPSTM from visually smooth to dynamically plausible.
Next steps:
- Flow‑matching diffusion to turn HPSTM into a generative motion prior while respecting SKEL limits.
References
[1] Yan Xia, Xiaowei Zhou, Etienne Vouga, Qixing Huang, Georgios Pavlakos.
Reconstructing Humans with a Biomechanically Accurate Skeleton.
Proceedings of CVPR 2025. arXiv:2503.21751. https://doi.org/10.48550/arXiv.2503.21751