Echo-1.58: Ternary Masked-Diffusion LMs
Collection
Native ternary (1.58-bit) QAT for masked-diffusion language models, paired with FP16 twins. Research artifacts for the paper; not production models. • 5 items • Updated
Echo-1.58 27M Research Ladder
A bundle of 27M-parameter checkpoints (d_model 448, 8 layers) that back the paper's
mechanistic results: the recovery ablation, the multi-epoch repetition ladder (1 to 20
epochs on a fixed 75M-token pool), the technique arms, and the fresh-data baselines. Each
file is bfloat16, paired with its FP16 twin where relevant. These are provenance artifacts for
reproducing the tables and figures, not standalone models. See
https://github.com/lupodevelop/echo-1.58 for the code that consumes them and
results/RESULTS.md for the numbers.
Contents
fp16_e{1,5,10,20}.pt,tern_e{5,10,20}.pt,rec_e{5,10}.ptrepetition ladderrec_e5_lr3.pt,tern_e5_lr3.pt,rec_e5_vsched.pt,rec_e5_lr8.pt,rec_e5_rewarm.pttechnique and learning-rate arms