BrutalCaesar/dpt_so101_cubcyl_recovery_tp48_unreg_emb256_3cam

DP-Transformer (unregularised) — SO-101, cube/cylinder pick-and-place

Diffusion Policy with the transformer backbone from Chi et al. §3.1 / Table 8, ported into lerobot 0.6.0 (which ships only the conv-UNet backbone). Trained on a 3-camera SO-101 arm.

Checkpoint: step 8,000 — the held-out eval_loss optimum of this run.

Architecture

The denoiser is TransformerForDiffusion, vendored unmodified from real-stanford/diffusion_policy (MIT). Backbone parameters close exactly at 9,020,934, matching the "9" in the paper's Table 8.

horizon / n_obs_steps / n_action_steps	48 / 2 / 24
n_layer / n_head / n_emb	8 / 4 / 256
conditioning	3 memory tokens (diffusion step k, obs t-1, obs t) via cross-attention
scheduler	DDIM, 100 train timesteps, 16 inference steps
encoder	3x ResNet-18 + SpatialSoftmax (32 keypoints), crop 216x288
action normalisation	MIN_MAX (measured variance 0.313, not 1.0)

Training

weight_decay 1e-6, attn_dropout 0.0 — the unregularised arm of a pre-registered A/B against the paper's Table 8 values (1e-3 / 0.3). 30,000 steps, batch 64, seed 1000, cosine schedule, 1x H200.

step	2k	4k	8k	12k	16k	24k	30k
eval_loss	0.0393	0.0311	0.0287	0.0288	0.0295	0.0310	0.0323

It bottoms at 8k and climbs monotonically after — this arm overfits, and it overfits early. The regularised arm reached only 0.0492 at 50k, so Table 8's recipe underfits a 120-episode dataset.

Bucketed evaluation

Held-out loss with the diffusion timestep k pinned rather than averaged, reported as variance explained (1 - MSE):

k	0	2	5	20	50	90	99
this model @30k	0.657	0.781	0.853	0.950	0.988	0.999	0.9998
regularised arm @30k	0.566	0.673	0.817	0.930	0.970	0.997	0.9975

This arm wins 15 of 15 buckets, and the margin is concentrated at low k (mean gap +0.078 at k<=5 versus +0.002 at k>=90) — the fine-action-detail regime.

Status and honest limitations

• NOT YET ROLLED OUT ON THE ARM. No task-success number exists for this checkpoint.
• eval_loss is unvalidated as a model-selection metric on this task. Exactly one diffusion checkpoint has ever been rolled out here, at n=12 episodes (~29% power), so there is no evidence either way that ranking checkpoints by it predicts task success.
• Steps 8k / 12k / 14k sit within 0.0001 of each other. "Best at 8k" is a coin-flip among those three.
• Trained on 120 episodes of a single operator's demonstrations, one table, one lighting condition. Expect no out-of-distribution robustness.

Use

from lerobot.policies.factory import get_policy_class
import phi.policies.dp_transformer  # registers "diffusion_transformer"

policy = get_policy_class("diffusion_transformer").from_pretrained(
    "BrutalCaesar/dpt_so101_cubcyl_recovery_tp48_unreg_emb256_3cam")

The diffusion_transformer policy type is not in upstream lerobot; it comes from src/phi/policies/dp_transformer.py in the phi repo, which must be on PYTHONPATH.