2026-09-23 21:35:22,836 INFO [train.py:1260] (1/2) Training started 2026-09-23 21:35:22,836 INFO [train.py:1270] (1/2) Device: cuda:1 2026-09-23 21:35:22,836 INFO [lexicon.py:168] (1/2) Loading pre-compiled data/lang_phone/Linv.pt 2026-09-23 21:35:22,837 INFO [train.py:1298] (1/2) Using dtype=torch.float16 2026-09-23 21:35:22,837 INFO [train.py:1299] (1/2) Use AMP=True 2026-09-23 21:35:22,838 INFO [train.py:1303] (1/2) About to create model 2026-09-23 21:35:22,991 INFO [train.py:1307] (1/2) Number of model parameters: 22780647 2026-09-23 21:35:23,014 INFO [train.py:1329] (1/2) Using DDP 2026-09-23 21:35:23,501 INFO [multidataset.py:43] (1/2) About to get train cuts from all datasets 2026-09-23 21:35:23,501 INFO [multidataset.py:46] (1/2) Loading Spanish Common Voice in lazy mode 2026-09-23 21:35:23,501 INFO [multidataset.py:52] (1/2) Loading SLR72 dataset in lazy mode 2026-09-23 21:35:23,501 INFO [multidataset.py:57] (1/2) Loading TinyVox Spanish train split in lazy mode 2026-09-23 21:35:23,502 INFO [multidataset.py:62] (1/2) Combining all training datasets 2026-09-23 21:35:23,502 INFO [asr_datamodule.py:220] (1/2) Enable MUSAN 2026-09-23 21:35:23,502 INFO [asr_datamodule.py:221] (1/2) About to get Musan cuts 2026-09-23 21:35:24,135 INFO [asr_datamodule.py:223] (1/2) About to get Hallway noise cuts 2026-09-23 21:35:24,144 INFO [asr_datamodule.py:253] (1/2) Enable SpecAugment 2026-09-23 21:35:24,144 INFO [asr_datamodule.py:254] (1/2) Time warp factor: 80 2026-09-23 21:35:24,144 INFO [asr_datamodule.py:264] (1/2) Num frame mask: 10 2026-09-23 21:35:24,144 INFO [asr_datamodule.py:277] (1/2) About to create train dataset 2026-09-23 21:35:24,144 INFO [asr_datamodule.py:307] (1/2) Using DynamicBucketingSampler. 2026-09-23 21:35:24,421 INFO [asr_datamodule.py:322] (1/2) About to create train dataloader 2026-09-23 21:35:24,422 INFO [multidataset.py:67] (1/2) About to get validation cuts from all datasets 2026-09-23 21:35:24,422 INFO [multidataset.py:70] (1/2) Loading Spanish Common Voice test set as validation 2026-09-23 21:35:24,422 INFO [multidataset.py:75] (1/2) Loading SLR72 dataset test set as validation 2026-09-23 21:35:24,422 INFO [multidataset.py:80] (1/2) Loading TinyVox Spanish validation split 2026-09-23 21:35:24,422 INFO [multidataset.py:85] (1/2) Combining all validation datasets 2026-09-23 21:35:24,422 INFO [asr_datamodule.py:353] (1/2) About to create dev dataset 2026-09-23 21:35:24,672 INFO [asr_datamodule.py:370] (1/2) About to create dev dataloader 2026-09-23 21:35:24,672 INFO [train.py:1520] (1/2) Sanity check -- see if any of the batches in epoch 1 would cause OOM. 2026-09-23 21:35:50,774 INFO [scaling.py:1024] (1/2) Whitening: name=None, num_groups=1, num_channels=256, metric=55.18 vs. limit=7.5 2026-09-23 21:35:51,046 INFO [train.py:1551] (1/2) Maximum memory allocated so far is 10859MB 2026-09-23 21:35:51,603 INFO [train.py:1551] (1/2) Maximum memory allocated so far is 10859MB 2026-09-23 21:35:52,292 INFO [train.py:1551] (1/2) Maximum memory allocated so far is 10859MB 2026-09-23 21:35:53,050 INFO [train.py:1551] (1/2) Maximum memory allocated so far is 10859MB 2026-09-23 21:35:53,796 INFO [train.py:1551] (1/2) Maximum memory allocated so far is 10859MB 2026-09-23 21:35:54,540 INFO [train.py:1551] (1/2) Maximum memory allocated so far is 10859MB 2026-09-23 21:35:59,519 INFO [train.py:1192] (1/2) Epoch 1, batch 0, loss[loss=3.846, simple_loss=3.474, pruned_loss=3.721, over 24589.00 frames. ], tot_loss[loss=3.846, simple_loss=3.474, pruned_loss=3.721, over 24589.00 frames. ], batch size: 137, lr: 2.00e-02, grad_scale: 1.0 2026-09-23 21:35:59,519 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 21:36:11,132 INFO [train.py:1224] (1/2) Epoch 1, validation: loss=3.681, simple_loss=3.318, pruned_loss=3.621, over 2564189.00 frames. 2026-09-23 21:36:11,133 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 10933MB 2026-09-23 21:36:11,903 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.86 vs. limit=7.5 2026-09-23 21:36:13,194 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=0.0, ans=0.3 2026-09-23 21:36:13,919 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.94 vs. limit=5.0 2026-09-23 21:36:18,338 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 6.086e+02 7.883e+02 1.927e+03 5.224e+03 6.687e+03, threshold=7.709e+03, percent-clipped=0.0 2026-09-23 21:36:19,090 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.18 vs. limit=5.008333333333334 2026-09-23 21:36:19,416 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=33.333333333333336, ans=0.2005 2026-09-23 21:36:20,856 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.89 vs. limit=7.525 2026-09-23 21:36:23,697 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 4.520e+02 6.170e+02 8.872e+02 3.026e+03 7.859e+03, threshold=3.549e+03, percent-clipped=0.0 2026-09-23 21:36:25,673 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=8.08 vs. limit=7.55 2026-09-23 21:36:28,920 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=100.0, ans=0.299 2026-09-23 21:36:29,016 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.76 vs. limit=7.5375 2026-09-23 21:36:34,734 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=49.86 vs. limit=5.066666666666666 2026-09-23 21:36:35,025 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 3.927e+02 5.130e+02 6.086e+02 8.872e+02 7.859e+03, threshold=2.434e+03, percent-clipped=0.0 2026-09-23 21:36:36,998 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=34.62 vs. limit=7.55 2026-09-23 21:36:38,938 INFO [train.py:1192] (1/2) Epoch 1, batch 50, loss[loss=1.04, simple_loss=0.9171, pruned_loss=1.092, over 24274.00 frames. ], tot_loss[loss=1.888, simple_loss=1.716, pruned_loss=1.66, over 1081387.08 frames. ], batch size: 125, lr: 2.20e-02, grad_scale: 0.25 2026-09-23 21:36:39,456 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=119.74 vs. limit=5.083333333333333 2026-09-23 21:36:40,375 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff3.min_abs, batch_count=166.66666666666666, ans=0.008333333333333333 2026-09-23 21:36:40,479 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=111.32 vs. limit=7.5625 2026-09-23 21:36:44,280 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=77.35 vs. limit=7.575 2026-09-23 21:36:49,554 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=149.60 vs. limit=7.575 2026-09-23 21:36:51,079 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=233.33333333333334, ans=0.236875 2026-09-23 21:36:52,674 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=233.33333333333334, ans=0.09475 2026-09-23 21:36:54,034 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=36.93 vs. limit=7.675 2026-09-23 21:37:02,744 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=18.64 vs. limit=5.075 2026-09-23 21:37:04,632 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=28.58 vs. limit=7.725 2026-09-23 21:37:07,235 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=15.43 vs. limit=4.133333333333334 2026-09-23 21:37:07,588 INFO [train.py:1192] (1/2) Epoch 1, batch 100, loss[loss=1.035, simple_loss=0.9007, pruned_loss=1.076, over 24594.00 frames. ], tot_loss[loss=1.448, simple_loss=1.297, pruned_loss=1.364, over 1915537.85 frames. ], batch size: 154, lr: 2.40e-02, grad_scale: 0.5 2026-09-23 21:37:08,411 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=12.57 vs. limit=5.083333333333333 2026-09-23 21:37:09,322 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 9.555e+01 1.958e+02 3.763e+02 5.587e+02 7.859e+03, threshold=7.526e+02, percent-clipped=0.0 2026-09-23 21:37:18,971 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=31.38 vs. limit=7.65 2026-09-23 21:37:23,007 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=105.73 vs. limit=7.65 2026-09-23 21:37:23,841 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=400.0, ans=0.48125 2026-09-23 21:37:23,973 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=57.87 vs. limit=5.2 2026-09-23 21:37:28,012 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=79.41 vs. limit=7.6625 2026-09-23 21:37:30,004 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=50.48 vs. limit=7.6625 2026-09-23 21:37:33,541 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=466.6666666666667, ans=5.291666666666667 2026-09-23 21:37:35,451 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=466.6666666666667, ans=0.04854166666666667 2026-09-23 21:37:35,699 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=82.60 vs. limit=7.675 2026-09-23 21:37:38,205 INFO [train.py:1192] (1/2) Epoch 1, batch 150, loss[loss=0.8963, simple_loss=0.7692, pruned_loss=0.9299, over 24263.00 frames. ], tot_loss[loss=1.267, simple_loss=1.123, pruned_loss=1.229, over 2560614.83 frames. ], batch size: 125, lr: 2.60e-02, grad_scale: 0.5 2026-09-23 21:37:38,467 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=103.32 vs. limit=7.6875 2026-09-23 21:37:41,722 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=500.0, ans=0.755 2026-09-23 21:37:41,789 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=34.98 vs. limit=7.6875 2026-09-23 21:37:49,797 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=9.76 vs. limit=4.213333333333333 2026-09-23 21:37:53,357 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=566.6666666666666, ans=7.925 2026-09-23 21:37:56,058 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=54.81 vs. limit=5.141666666666667 2026-09-23 21:37:56,558 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=43.44 vs. limit=7.7125 2026-09-23 21:37:58,891 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=600.0, ans=0.471875 2026-09-23 21:37:59,130 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=31.61 vs. limit=7.725 2026-09-23 21:37:59,667 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=71.20 vs. limit=7.725 2026-09-23 21:38:00,390 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.89 vs. limit=7.95 2026-09-23 21:38:00,404 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.33 vs. limit=7.95 2026-09-23 21:38:01,260 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=600.0, ans=0.756 2026-09-23 21:38:03,022 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=29.14 vs. limit=7.725 2026-09-23 21:38:04,903 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=22.16 vs. limit=7.7375 2026-09-23 21:38:10,609 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=633.3333333333334, ans=0.08575 2026-09-23 21:38:10,969 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.52 vs. limit=7.975 2026-09-23 21:38:11,806 INFO [train.py:1192] (1/2) Epoch 1, batch 200, loss[loss=1.109, simple_loss=0.9487, pruned_loss=1.09, over 24241.00 frames. ], tot_loss[loss=1.177, simple_loss=1.032, pruned_loss=1.156, over 3059395.90 frames. ], batch size: 257, lr: 2.80e-02, grad_scale: 1.0 2026-09-23 21:38:13,634 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 9.752e+01 1.933e+02 2.630e+02 3.377e+02 7.630e+02, threshold=5.261e+02, percent-clipped=1.0 2026-09-23 21:38:14,013 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=18.47 vs. limit=7.75 2026-09-23 21:38:15,209 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.52 vs. limit=5.333333333333333 2026-09-23 21:38:18,253 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.23 vs. limit=8.025 2026-09-23 21:38:19,026 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=15.68 vs. limit=5.175 2026-09-23 21:38:19,764 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=35.70 vs. limit=7.7625 2026-09-23 21:38:24,647 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=733.3333333333334, ans=0.09541666666666668 2026-09-23 21:38:24,692 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=733.3333333333334, ans=0.465625 2026-09-23 21:38:28,415 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=733.3333333333334, ans=0.17250000000000001 2026-09-23 21:38:32,076 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=766.6666666666666, ans=0.21150000000000002 2026-09-23 21:38:36,190 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=149.73 vs. limit=7.7875 2026-09-23 21:38:42,021 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=800.0, ans=0.872 2026-09-23 21:38:42,276 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.38 vs. limit=7.8 2026-09-23 21:38:42,655 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=800.0, ans=0.4625 2026-09-23 21:38:44,150 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.87 vs. limit=5.2 2026-09-23 21:38:45,290 INFO [train.py:1192] (1/2) Epoch 1, batch 250, loss[loss=1.107, simple_loss=0.9337, pruned_loss=1.091, over 24394.00 frames. ], tot_loss[loss=1.126, simple_loss=0.9777, pruned_loss=1.112, over 3444567.57 frames. ], batch size: 225, lr: 3.00e-02, grad_scale: 1.0 2026-09-23 21:38:49,648 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.48 vs. limit=4.333333333333333 2026-09-23 21:38:50,496 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=95.98 vs. limit=7.8125 2026-09-23 21:38:51,156 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.58 vs. limit=7.8125 2026-09-23 21:38:51,740 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.66 vs. limit=5.433333333333334 2026-09-23 21:39:01,502 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=10.83 vs. limit=8.175 2026-09-23 21:39:03,906 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.82 vs. limit=5.225 2026-09-23 21:39:04,013 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.04 vs. limit=3.135 2026-09-23 21:39:06,319 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=42.22 vs. limit=5.233333333333333 2026-09-23 21:39:07,325 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=933.3333333333334, ans=0.24066666666666667 2026-09-23 21:39:08,177 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=10.93 vs. limit=8.2 2026-09-23 21:39:11,085 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=966.6666666666666, ans=0.37916666666666665 2026-09-23 21:39:16,750 INFO [train.py:1192] (1/2) Epoch 1, batch 300, loss[loss=1.094, simple_loss=0.911, pruned_loss=1.07, over 24553.00 frames. ], tot_loss[loss=1.097, simple_loss=0.9422, pruned_loss=1.085, over 3757401.42 frames. ], batch size: 204, lr: 3.20e-02, grad_scale: 2.0 2026-09-23 21:39:16,867 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=1000.0, ans=0.453125 2026-09-23 21:39:17,085 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.96 vs. limit=7.875 2026-09-23 21:39:18,852 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 1.125e+02 2.100e+02 2.680e+02 3.774e+02 1.103e+03, threshold=5.360e+02, percent-clipped=10.0 2026-09-23 21:39:25,614 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=1033.3333333333333, ans=3.155 2026-09-23 21:39:32,918 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.12 vs. limit=7.9 2026-09-23 21:39:32,938 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.18 vs. limit=8.3 2026-09-23 21:39:41,219 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=1133.3333333333333, ans=0.446875 2026-09-23 21:39:42,539 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.77 vs. limit=7.925 2026-09-23 21:39:43,917 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=1133.3333333333333, ans=0.446875 2026-09-23 21:39:46,070 INFO [train.py:1192] (1/2) Epoch 1, batch 350, loss[loss=0.9312, simple_loss=0.7662, pruned_loss=0.9004, over 24563.00 frames. ], tot_loss[loss=1.08, simple_loss=0.919, pruned_loss=1.062, over 3998184.24 frames. ], batch size: 137, lr: 3.40e-02, grad_scale: 2.0 2026-09-23 21:39:47,577 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.39 vs. limit=7.9375 2026-09-23 21:39:52,895 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=1200.0, ans=0.155 2026-09-23 21:39:55,039 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=1200.0, ans=0.155 2026-09-23 21:40:00,409 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=6.82 vs. limit=7.9625 2026-09-23 21:40:02,862 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=16.24 vs. limit=5.616666666666666 2026-09-23 21:40:03,849 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.52 vs. limit=8.45 2026-09-23 21:40:05,950 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=1266.6666666666667, ans=0.07150000000000001 2026-09-23 21:40:08,143 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=1266.6666666666667, ans=0.1525 2026-09-23 21:40:12,159 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=1300.0, ans=0.4390625 2026-09-23 21:40:13,964 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.50 vs. limit=4.52 2026-09-23 21:40:14,457 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=1300.0, ans=0.4390625 2026-09-23 21:40:15,600 INFO [train.py:1192] (1/2) Epoch 1, batch 400, loss[loss=1.022, simple_loss=0.852, pruned_loss=0.9125, over 24596.00 frames. ], tot_loss[loss=1.063, simple_loss=0.8989, pruned_loss=1.031, over 4182594.36 frames. ], batch size: 170, lr: 3.60e-02, grad_scale: 4.0 2026-09-23 21:40:16,898 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=21.01 vs. limit=8.0 2026-09-23 21:40:17,315 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 1.156e+02 2.069e+02 2.740e+02 4.120e+02 1.261e+03, threshold=5.479e+02, percent-clipped=17.0 2026-09-23 21:40:17,449 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=1333.3333333333333, ans=0.5 2026-09-23 21:40:25,021 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=1366.6666666666667, ans=0.23633333333333334 2026-09-23 21:40:26,700 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=1366.6666666666667, ans=0.32916666666666666 2026-09-23 21:40:31,512 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=1400.0, ans=0.286 2026-09-23 21:40:32,631 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=1400.0, ans=0.14750000000000002 2026-09-23 21:40:42,679 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.58 vs. limit=8.6 2026-09-23 21:40:45,041 INFO [train.py:1192] (1/2) Epoch 1, batch 450, loss[loss=1.024, simple_loss=0.8616, pruned_loss=0.8557, over 24646.00 frames. ], tot_loss[loss=1.046, simple_loss=0.8829, pruned_loss=0.9858, over 4321809.51 frames. ], batch size: 175, lr: 3.80e-02, grad_scale: 4.0 2026-09-23 21:40:46,557 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=32.18 vs. limit=8.0625 2026-09-23 21:40:47,055 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=17.33 vs. limit=8.0625 2026-09-23 21:40:49,435 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=20.41 vs. limit=8.0625 2026-09-23 21:40:49,437 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=1500.0, ans=8.0625 2026-09-23 21:40:53,316 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.84 vs. limit=8.65 2026-09-23 21:40:59,972 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=1566.6666666666667, ans=0.4265625 2026-09-23 21:41:02,080 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=17.89 vs. limit=8.0875 2026-09-23 21:41:05,949 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.01 vs. limit=8.1 2026-09-23 21:41:07,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=1600.0, ans=0.425 2026-09-23 21:41:08,972 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.25 vs. limit=8.725 2026-09-23 21:41:14,883 INFO [train.py:1192] (1/2) Epoch 1, batch 500, loss[loss=1.008, simple_loss=0.8588, pruned_loss=0.79, over 24510.00 frames. ], tot_loss[loss=1.021, simple_loss=0.8631, pruned_loss=0.9294, over 4438802.26 frames. ], batch size: 218, lr: 4.00e-02, grad_scale: 8.0 2026-09-23 21:41:16,468 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=14.50 vs. limit=8.125 2026-09-23 21:41:16,640 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 1.720e+02 2.882e+02 3.963e+02 5.717e+02 1.217e+03, threshold=7.926e+02, percent-clipped=27.0 2026-09-23 21:41:27,045 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.42 vs. limit=4.693333333333333 2026-09-23 21:41:35,799 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.47 vs. limit=8.1625 2026-09-23 21:41:44,035 INFO [train.py:1192] (1/2) Epoch 1, batch 550, loss[loss=0.9529, simple_loss=0.8196, pruned_loss=0.7074, over 24271.00 frames. ], tot_loss[loss=0.9965, simple_loss=0.8456, pruned_loss=0.8731, over 4523514.58 frames. ], batch size: 257, lr: 3.99e-02, grad_scale: 8.0 2026-09-23 21:41:50,506 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=1866.6666666666667, ans=0.13 2026-09-23 21:41:50,514 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=1866.6666666666667, ans=0.4125 2026-09-23 21:41:52,089 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=1866.6666666666667, ans=0.13 2026-09-23 21:41:53,934 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=1.259e+01 2026-09-23 21:41:58,024 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.51 vs. limit=5.475 2026-09-23 21:42:03,114 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=1933.3333333333333, ans=0.05650000000000001 2026-09-23 21:42:08,660 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=1966.6666666666667, ans=0.4078125 2026-09-23 21:42:08,813 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=11.56 vs. limit=8.2375 2026-09-23 21:42:10,392 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=1966.6666666666667, ans=0.08770833333333333 2026-09-23 21:42:13,798 INFO [train.py:1192] (1/2) Epoch 1, batch 600, loss[loss=0.9674, simple_loss=0.8311, pruned_loss=0.7049, over 24306.00 frames. ], tot_loss[loss=0.9708, simple_loss=0.8277, pruned_loss=0.8179, over 4589097.94 frames. ], batch size: 234, lr: 3.99e-02, grad_scale: 8.0 2026-09-23 21:42:15,430 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 1.756e+02 2.946e+02 3.974e+02 5.357e+02 1.166e+03, threshold=7.947e+02, percent-clipped=7.0 2026-09-23 21:42:18,356 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=2000.0, ans=0.40625 2026-09-23 21:42:23,646 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.77 vs. limit=4.8133333333333335 2026-09-23 21:42:26,256 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=2066.6666666666665, ans=0.403125 2026-09-23 21:42:28,723 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=2066.6666666666665, ans=0.12250000000000001 2026-09-23 21:42:31,953 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=2100.0, ans=0.4015625 2026-09-23 21:42:32,495 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=2100.0, ans=0.05275 2026-09-23 21:42:38,151 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=2133.3333333333335, ans=0.4 2026-09-23 21:42:42,551 INFO [train.py:1192] (1/2) Epoch 1, batch 650, loss[loss=0.7957, simple_loss=0.7051, pruned_loss=0.5226, over 24593.00 frames. ], tot_loss[loss=0.9391, simple_loss=0.8054, pruned_loss=0.76, over 4654251.70 frames. ], batch size: 154, lr: 3.99e-02, grad_scale: 8.0 2026-09-23 21:42:43,655 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=2166.6666666666665, ans=0.3984375 2026-09-23 21:42:55,042 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=2233.3333333333335, ans=0.12437499999999999 2026-09-23 21:43:00,756 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.87 vs. limit=5.566666666666666 2026-09-23 21:43:03,485 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.47 vs. limit=9.2 2026-09-23 21:43:04,416 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=2266.6666666666665, ans=0.8206666666666667 2026-09-23 21:43:06,612 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=1.315e+00 2026-09-23 21:43:11,723 INFO [train.py:1192] (1/2) Epoch 1, batch 700, loss[loss=0.776, simple_loss=0.6925, pruned_loss=0.4933, over 24554.00 frames. ], tot_loss[loss=0.9156, simple_loss=0.7893, pruned_loss=0.7138, over 4690365.24 frames. ], batch size: 158, lr: 3.99e-02, grad_scale: 8.0 2026-09-23 21:43:13,409 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.090e+02 3.037e+02 4.027e+02 5.498e+02 1.629e+03, threshold=8.054e+02, percent-clipped=8.0 2026-09-23 21:43:18,748 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.79 vs. limit=6.183333333333334 2026-09-23 21:43:20,294 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=2366.6666666666665, ans=0.2763333333333333 2026-09-23 21:43:23,041 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=2400.0, ans=0.046 2026-09-23 21:43:30,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=2433.3333333333335, ans=0.08479166666666667 2026-09-23 21:43:31,856 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=2433.3333333333335, ans=0.22566666666666665 2026-09-23 21:43:32,850 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=2433.3333333333335, ans=0.3859375 2026-09-23 21:43:33,388 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=2433.3333333333335, ans=0.3859375 2026-09-23 21:43:40,328 INFO [train.py:1192] (1/2) Epoch 1, batch 750, loss[loss=0.8257, simple_loss=0.7321, pruned_loss=0.5277, over 24632.00 frames. ], tot_loss[loss=0.8879, simple_loss=0.7697, pruned_loss=0.6678, over 4726859.12 frames. ], batch size: 175, lr: 3.99e-02, grad_scale: 8.0 2026-09-23 21:43:40,990 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=2500.0, ans=0.084375 2026-09-23 21:43:49,611 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=2533.3333333333335, ans=0.08416666666666667 2026-09-23 21:43:50,353 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.28 vs. limit=8.45 2026-09-23 21:43:55,221 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=8.84 vs. limit=9.425 2026-09-23 21:44:04,882 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=2633.3333333333335, ans=0.17083333333333334 2026-09-23 21:44:08,282 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=8.95 vs. limit=9.475 2026-09-23 21:44:08,817 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.52 vs. limit=5.053333333333334 2026-09-23 21:44:09,720 INFO [train.py:1192] (1/2) Epoch 1, batch 800, loss[loss=0.673, simple_loss=0.6079, pruned_loss=0.4057, over 24531.00 frames. ], tot_loss[loss=0.8633, simple_loss=0.7525, pruned_loss=0.6276, over 4753131.05 frames. ], batch size: 137, lr: 3.99e-02, grad_scale: 16.0 2026-09-23 21:44:10,848 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=2666.6666666666665, ans=0.375 2026-09-23 21:44:11,237 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.531e+02 3.517e+02 4.281e+02 5.987e+02 1.290e+03, threshold=8.562e+02, percent-clipped=5.0 2026-09-23 21:44:12,455 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=2666.6666666666665, ans=0.375 2026-09-23 21:44:18,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=2700.0, ans=0.03925 2026-09-23 21:44:21,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=2733.3333333333335, ans=0.0385 2026-09-23 21:44:23,258 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.86 vs. limit=9.55 2026-09-23 21:44:26,412 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.69 vs. limit=8.5375 2026-09-23 21:44:28,438 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.57 vs. limit=9.575 2026-09-23 21:44:29,407 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.89 vs. limit=9.575 2026-09-23 21:44:30,173 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.67 vs. limit=8.5375 2026-09-23 21:44:34,713 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.12 vs. limit=8.55 2026-09-23 21:44:35,908 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.09 vs. limit=8.55 2026-09-23 21:44:38,065 INFO [train.py:1192] (1/2) Epoch 1, batch 850, loss[loss=0.8078, simple_loss=0.7189, pruned_loss=0.5007, over 24533.00 frames. ], tot_loss[loss=0.8393, simple_loss=0.7357, pruned_loss=0.5907, over 4772280.35 frames. ], batch size: 204, lr: 3.99e-02, grad_scale: 16.0 2026-09-23 21:44:42,266 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=2833.3333333333335, ans=0.17379166666666668 2026-09-23 21:44:42,844 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=2833.3333333333335, ans=0.3671875 2026-09-23 21:45:06,825 INFO [train.py:1192] (1/2) Epoch 1, batch 900, loss[loss=0.6513, simple_loss=0.5978, pruned_loss=0.3722, over 24514.00 frames. ], tot_loss[loss=0.8189, simple_loss=0.7217, pruned_loss=0.5592, over 4782748.04 frames. ], batch size: 137, lr: 3.99e-02, grad_scale: 16.0 2026-09-23 21:45:08,876 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.471e+02 3.813e+02 4.524e+02 5.453e+02 1.532e+03, threshold=9.047e+02, percent-clipped=3.0 2026-09-23 21:45:10,937 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=3.16 vs. limit=3.45 2026-09-23 21:45:18,411 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=3066.6666666666665, ans=0.031 2026-09-23 21:45:25,790 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=3100.0, ans=0.269 2026-09-23 21:45:26,778 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=3100.0, ans=0.5 2026-09-23 21:45:34,896 INFO [train.py:1192] (1/2) Epoch 1, batch 950, loss[loss=0.8085, simple_loss=0.68, pruned_loss=0.5495, over 11190.00 frames. ], tot_loss[loss=0.7986, simple_loss=0.7064, pruned_loss=0.5327, over 4715296.49 frames. ], batch size: 333, lr: 3.98e-02, grad_scale: 16.0 2026-09-23 21:45:34,964 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=3166.6666666666665, ans=0.21833333333333332 2026-09-23 21:45:44,542 INFO [train.py:1192] (1/2) Epoch 2, batch 0, loss[loss=0.6822, simple_loss=0.6251, pruned_loss=0.3893, over 24545.00 frames. ], tot_loss[loss=0.6822, simple_loss=0.6251, pruned_loss=0.3893, over 24545.00 frames. ], batch size: 137, lr: 3.91e-02, grad_scale: 32.0 2026-09-23 21:45:44,542 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 21:45:53,980 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([0.9715, 1.0467, 0.9905, 0.9660, 0.9175, 0.7935, 1.0416, 0.9894], device='cuda:1') 2026-09-23 21:45:55,880 INFO [train.py:1224] (1/2) Epoch 2, validation: loss=0.5047, simple_loss=0.4979, pruned_loss=0.2373, over 2564189.00 frames. 2026-09-23 21:45:55,881 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12446MB 2026-09-23 21:46:04,995 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=3226.6666666666665, ans=0.7870666666666667 2026-09-23 21:46:09,778 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=3260.0, ans=0.07775 2026-09-23 21:46:10,656 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=3260.0, ans=0.3471875 2026-09-23 21:46:10,678 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=3260.0, ans=0.3471875 2026-09-23 21:46:11,390 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.78 vs. limit=9.945 2026-09-23 21:46:15,339 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=3293.3333333333335, ans=0.7847333333333334 2026-09-23 21:46:16,911 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=3293.3333333333335, ans=0.7847333333333334 2026-09-23 21:46:17,421 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=3293.3333333333335, ans=0.7847333333333334 2026-09-23 21:46:19,795 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=3326.6666666666665, ans=0.3440625 2026-09-23 21:46:20,335 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=3326.6666666666665, ans=0.025149999999999992 2026-09-23 21:46:21,865 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.265e+02 3.080e+02 3.822e+02 5.393e+02 1.061e+03, threshold=7.645e+02, percent-clipped=3.0 2026-09-23 21:46:24,425 INFO [train.py:1192] (1/2) Epoch 2, batch 50, loss[loss=0.6426, simple_loss=0.587, pruned_loss=0.3676, over 24276.00 frames. ], tot_loss[loss=0.7423, simple_loss=0.6694, pruned_loss=0.4377, over 1081731.54 frames. ], batch size: 125, lr: 3.90e-02, grad_scale: 16.0 2026-09-23 21:46:28,089 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=3360.0, ans=0.07400000000000001 2026-09-23 21:46:28,090 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=3360.0, ans=0.3425 2026-09-23 21:46:33,958 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=3393.3333333333335, ans=0.02364999999999999 2026-09-23 21:46:36,875 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=3426.6666666666665, ans=0.07166666666666671 2026-09-23 21:46:40,341 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=3426.6666666666665, ans=0.09899494936611666 2026-09-23 21:46:43,258 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.72 vs. limit=10.095 2026-09-23 21:46:50,898 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=3493.3333333333335, ans=0.33625 2026-09-23 21:46:52,345 INFO [train.py:1192] (1/2) Epoch 2, batch 100, loss[loss=0.6537, simple_loss=0.6054, pruned_loss=0.3618, over 24633.00 frames. ], tot_loss[loss=0.7314, simple_loss=0.6643, pruned_loss=0.4236, over 1916537.19 frames. ], batch size: 154, lr: 3.90e-02, grad_scale: 16.0 2026-09-23 21:46:54,549 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=3526.6666666666665, ans=0.3346875 2026-09-23 21:46:54,619 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.60 vs. limit=6.763333333333334 2026-09-23 21:47:08,813 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.23 vs. limit=5.906666666666666 2026-09-23 21:47:14,979 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=3660.0, ans=0.26339999999999997 2026-09-23 21:47:17,959 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.567e+02 3.428e+02 4.005e+02 4.892e+02 1.099e+03, threshold=8.011e+02, percent-clipped=5.0 2026-09-23 21:47:20,060 INFO [train.py:1192] (1/2) Epoch 2, batch 150, loss[loss=0.5445, simple_loss=0.5192, pruned_loss=0.2826, over 24262.00 frames. ], tot_loss[loss=0.7057, simple_loss=0.6456, pruned_loss=0.4018, over 2561802.27 frames. ], batch size: 125, lr: 3.90e-02, grad_scale: 16.0 2026-09-23 21:47:28,284 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=3726.6666666666665, ans=0.07 2026-09-23 21:47:28,995 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=10.98 vs. limit=10.295 2026-09-23 21:47:37,796 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=3793.3333333333335, ans=0.035 2026-09-23 21:47:38,977 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=3793.3333333333335, ans=0.07629166666666667 2026-09-23 21:47:48,079 INFO [train.py:1192] (1/2) Epoch 2, batch 200, loss[loss=0.7923, simple_loss=0.7212, pruned_loss=0.4505, over 24229.00 frames. ], tot_loss[loss=0.6943, simple_loss=0.6376, pruned_loss=0.3913, over 3060531.52 frames. ], batch size: 257, lr: 3.90e-02, grad_scale: 16.0 2026-09-23 21:47:49,636 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=3860.0, ans=0.3190625 2026-09-23 21:47:53,822 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.10 vs. limit=10.42 2026-09-23 21:47:55,049 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.11 vs. limit=5.557333333333333 2026-09-23 21:47:57,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=3893.3333333333335, ans=0.7637333333333334 2026-09-23 21:47:58,399 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.28 vs. limit=10.42 2026-09-23 21:48:14,869 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.848e+02 4.117e+02 5.175e+02 6.960e+02 1.473e+03, threshold=1.035e+03, percent-clipped=14.0 2026-09-23 21:48:16,980 INFO [train.py:1192] (1/2) Epoch 2, batch 250, loss[loss=0.7085, simple_loss=0.6618, pruned_loss=0.3831, over 24413.00 frames. ], tot_loss[loss=0.6857, simple_loss=0.6318, pruned_loss=0.3831, over 3442773.72 frames. ], batch size: 225, lr: 3.90e-02, grad_scale: 16.0 2026-09-23 21:48:26,906 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.42 vs. limit=9.0225 2026-09-23 21:48:32,388 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=4093.3333333333335, ans=0.07441666666666667 2026-09-23 21:48:35,544 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=4126.666666666667, ans=0.30656249999999996 2026-09-23 21:48:42,341 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=4160.0, ans=0.305 2026-09-23 21:48:44,528 INFO [train.py:1192] (1/2) Epoch 2, batch 300, loss[loss=0.7075, simple_loss=0.6543, pruned_loss=0.3887, over 24554.00 frames. ], tot_loss[loss=0.6755, simple_loss=0.6254, pruned_loss=0.3734, over 3756551.30 frames. ], batch size: 204, lr: 3.90e-02, grad_scale: 16.0 2026-09-23 21:48:47,253 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=4193.333333333333, ans=0.7532333333333334 2026-09-23 21:48:57,711 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.20 vs. limit=6.0649999999999995 2026-09-23 21:49:05,875 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 21:49:07,514 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 21:49:10,536 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.695e+02 3.837e+02 4.969e+02 6.338e+02 1.168e+03, threshold=9.937e+02, percent-clipped=1.0 2026-09-23 21:49:12,571 INFO [train.py:1192] (1/2) Epoch 2, batch 350, loss[loss=0.5609, simple_loss=0.5368, pruned_loss=0.2899, over 24580.00 frames. ], tot_loss[loss=0.6672, simple_loss=0.6205, pruned_loss=0.3651, over 3997905.57 frames. ], batch size: 137, lr: 3.89e-02, grad_scale: 16.0 2026-09-23 21:49:22,928 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=4426.666666666667, ans=0.2925 2026-09-23 21:49:28,350 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.06 vs. limit=6.115 2026-09-23 21:49:30,509 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=4460.0, ans=0.0099 2026-09-23 21:49:38,335 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.86 vs. limit=6.123333333333333 2026-09-23 21:49:39,580 INFO [train.py:1192] (1/2) Epoch 2, batch 400, loss[loss=0.591, simple_loss=0.5736, pruned_loss=0.2983, over 24588.00 frames. ], tot_loss[loss=0.6565, simple_loss=0.6136, pruned_loss=0.3557, over 4183177.24 frames. ], batch size: 170, lr: 3.89e-02, grad_scale: 32.0 2026-09-23 21:49:39,944 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=8.61 vs. limit=9.1975 2026-09-23 21:49:40,189 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=4526.666666666667, ans=0.07 2026-09-23 21:49:51,666 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=4593.333333333333, ans=0.04752777777777778 2026-09-23 21:49:57,224 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=4626.666666666667, ans=10.97 2026-09-23 21:50:04,635 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.866e+02 3.994e+02 4.572e+02 6.150e+02 1.146e+03, threshold=9.145e+02, percent-clipped=1.0 2026-09-23 21:50:06,682 INFO [train.py:1192] (1/2) Epoch 2, batch 450, loss[loss=0.6322, simple_loss=0.6043, pruned_loss=0.328, over 24637.00 frames. ], tot_loss[loss=0.6496, simple_loss=0.6094, pruned_loss=0.3493, over 4318925.29 frames. ], batch size: 175, lr: 3.89e-02, grad_scale: 32.0 2026-09-23 21:50:09,431 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=4693.333333333333, ans=0.7357333333333334 2026-09-23 21:50:13,683 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.99 vs. limit=9.2725 2026-09-23 21:50:16,759 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=4726.666666666667, ans=0.2784375 2026-09-23 21:50:21,069 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=4760.0, ans=0.276875 2026-09-23 21:50:32,703 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=4826.666666666667, ans=0.009820289855072464 2026-09-23 21:50:32,810 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.13 vs. limit=6.206666666666667 2026-09-23 21:50:34,667 INFO [train.py:1192] (1/2) Epoch 2, batch 500, loss[loss=0.6699, simple_loss=0.632, pruned_loss=0.3548, over 24507.00 frames. ], tot_loss[loss=0.6397, simple_loss=0.603, pruned_loss=0.3409, over 4436544.55 frames. ], batch size: 218, lr: 3.89e-02, grad_scale: 32.0 2026-09-23 21:50:39,458 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.20 vs. limit=5.944 2026-09-23 21:50:40,306 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=4893.333333333333, ans=0.270625 2026-09-23 21:50:41,380 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=4893.333333333333, ans=9.335 2026-09-23 21:50:50,135 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.32 vs. limit=9.3475 2026-09-23 21:50:54,306 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=4960.0, ans=8.1 2026-09-23 21:50:57,068 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.12 vs. limit=6.248333333333333 2026-09-23 21:51:00,005 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 3.024e+02 3.952e+02 4.539e+02 5.350e+02 1.193e+03, threshold=9.079e+02, percent-clipped=2.0 2026-09-23 21:51:00,132 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=4993.333333333333, ans=0.25006666666666666 2026-09-23 21:51:02,059 INFO [train.py:1192] (1/2) Epoch 2, batch 550, loss[loss=0.7106, simple_loss=0.6613, pruned_loss=0.3836, over 24227.00 frames. ], tot_loss[loss=0.6328, simple_loss=0.5991, pruned_loss=0.3347, over 4521431.20 frames. ], batch size: 257, lr: 3.88e-02, grad_scale: 32.0 2026-09-23 21:51:03,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=5026.666666666667, ans=0.264375 2026-09-23 21:51:05,962 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.73 vs. limit=11.27 2026-09-23 21:51:13,667 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_na.min_abs, batch_count=5093.333333333333, ans=0.02 2026-09-23 21:51:22,899 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=5126.666666666667, ans=0.0 2026-09-23 21:51:29,341 INFO [train.py:1192] (1/2) Epoch 2, batch 600, loss[loss=0.6647, simple_loss=0.6264, pruned_loss=0.3524, over 24327.00 frames. ], tot_loss[loss=0.6259, simple_loss=0.5955, pruned_loss=0.3285, over 4589669.71 frames. ], batch size: 234, lr: 3.88e-02, grad_scale: 32.0 2026-09-23 21:51:33,153 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.17 vs. limit=6.077333333333334 2026-09-23 21:51:44,127 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=5260.0, ans=0.025 2026-09-23 21:51:53,527 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=5326.666666666667, ans=0.044472222222222225 2026-09-23 21:51:53,922 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.776e+02 3.843e+02 4.344e+02 5.247e+02 1.354e+03, threshold=8.688e+02, percent-clipped=5.0 2026-09-23 21:51:56,398 INFO [train.py:1192] (1/2) Epoch 2, batch 650, loss[loss=0.5503, simple_loss=0.5459, pruned_loss=0.2718, over 24613.00 frames. ], tot_loss[loss=0.615, simple_loss=0.5891, pruned_loss=0.3197, over 4654156.17 frames. ], batch size: 154, lr: 3.88e-02, grad_scale: 32.0 2026-09-23 21:52:00,058 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=5360.0, ans=0.0 2026-09-23 21:52:00,498 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=5360.0, ans=8.35 2026-09-23 21:52:04,708 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.50 vs. limit=6.348333333333333 2026-09-23 21:52:14,492 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=5460.0, ans=0.009682608695652174 2026-09-23 21:52:16,123 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=5460.0, ans=0.24406250000000002 2026-09-23 21:52:23,738 INFO [train.py:1192] (1/2) Epoch 2, batch 700, loss[loss=0.5515, simple_loss=0.5494, pruned_loss=0.2716, over 24555.00 frames. ], tot_loss[loss=0.6105, simple_loss=0.5871, pruned_loss=0.3156, over 4689102.68 frames. ], batch size: 158, lr: 3.88e-02, grad_scale: 32.0 2026-09-23 21:52:37,403 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=5593.333333333333, ans=0.23781249999999998 2026-09-23 21:52:41,561 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=5626.666666666667, ans=0.7030666666666667 2026-09-23 21:52:44,105 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=5660.0, ans=0.2346875 2026-09-23 21:52:46,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=5660.0, ans=0.04949747468305833 2026-09-23 21:52:47,767 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=5660.0, ans=0.2346875 2026-09-23 21:52:48,173 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 3.265e+02 4.700e+02 5.470e+02 6.470e+02 9.806e+02, threshold=1.094e+03, percent-clipped=5.0 2026-09-23 21:52:50,326 INFO [train.py:1192] (1/2) Epoch 2, batch 750, loss[loss=0.5874, simple_loss=0.5863, pruned_loss=0.2896, over 24652.00 frames. ], tot_loss[loss=0.6033, simple_loss=0.5828, pruned_loss=0.3101, over 4725398.75 frames. ], batch size: 175, lr: 3.87e-02, grad_scale: 32.0 2026-09-23 21:52:55,656 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.31 vs. limit=11.795 2026-09-23 21:52:58,025 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=5726.666666666667, ans=0.8072666666666667 2026-09-23 21:52:59,448 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=5726.666666666667, ans=0.2315625 2026-09-23 21:53:11,988 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=4.36 vs. limit=9.685 2026-09-23 21:53:13,054 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=5826.666666666667, ans=0.226875 2026-09-23 21:53:17,093 INFO [train.py:1192] (1/2) Epoch 2, batch 800, loss[loss=0.4756, simple_loss=0.4914, pruned_loss=0.2247, over 24548.00 frames. ], tot_loss[loss=0.5981, simple_loss=0.5798, pruned_loss=0.3061, over 4752098.11 frames. ], batch size: 137, lr: 3.87e-02, grad_scale: 32.0 2026-09-23 21:53:31,369 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 21:53:33,990 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=5960.0, ans=0.22062500000000002 2026-09-23 21:53:33,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=5960.0, ans=0.0 2026-09-23 21:53:41,967 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.936e+02 4.336e+02 5.358e+02 6.430e+02 1.287e+03, threshold=1.072e+03, percent-clipped=2.0 2026-09-23 21:53:43,799 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.88 vs. limit=9.76 2026-09-23 21:53:44,093 INFO [train.py:1192] (1/2) Epoch 2, batch 850, loss[loss=0.6261, simple_loss=0.6125, pruned_loss=0.3179, over 24546.00 frames. ], tot_loss[loss=0.5924, simple_loss=0.5767, pruned_loss=0.3019, over 4771106.58 frames. ], batch size: 204, lr: 3.87e-02, grad_scale: 32.0 2026-09-23 21:53:50,011 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=6060.0, ans=8.7875 2026-09-23 21:53:50,574 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=6060.0, ans=0.009552173913043478 2026-09-23 21:53:58,906 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=6093.333333333333, ans=12.07 2026-09-23 21:53:59,897 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=6126.666666666667, ans=0.21281250000000002 2026-09-23 21:54:02,938 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.19 vs. limit=12.094999999999999 2026-09-23 21:54:04,421 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=6126.666666666667, ans=0.04113888888888889 2026-09-23 21:54:05,124 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.52 vs. limit=6.54 2026-09-23 21:54:06,523 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=6160.0, ans=0.07 2026-09-23 21:54:10,814 INFO [train.py:1192] (1/2) Epoch 2, batch 900, loss[loss=0.49, simple_loss=0.5071, pruned_loss=0.2334, over 24566.00 frames. ], tot_loss[loss=0.5875, simple_loss=0.5744, pruned_loss=0.2981, over 4781481.45 frames. ], batch size: 137, lr: 3.86e-02, grad_scale: 32.0 2026-09-23 21:54:13,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=6193.333333333333, ans=0.23806666666666665 2026-09-23 21:54:16,003 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=6226.666666666667, ans=0.208125 2026-09-23 21:54:16,534 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=6226.666666666667, ans=0.208125 2026-09-23 21:54:18,149 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.22 vs. limit=8.113333333333333 2026-09-23 21:54:26,094 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=6260.0, ans=0.2374 2026-09-23 21:54:32,779 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=6326.666666666667, ans=0.23673333333333332 2026-09-23 21:54:36,146 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.813e+02 3.863e+02 4.821e+02 6.210e+02 1.366e+03, threshold=9.642e+02, percent-clipped=4.0 2026-09-23 21:54:37,675 INFO [train.py:1192] (1/2) Epoch 2, batch 950, loss[loss=0.7565, simple_loss=0.6304, pruned_loss=0.4452, over 11027.00 frames. ], tot_loss[loss=0.5858, simple_loss=0.5719, pruned_loss=0.2979, over 4708713.32 frames. ], batch size: 333, lr: 3.86e-02, grad_scale: 16.0 2026-09-23 21:54:38,427 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=6360.0, ans=0.8136 2026-09-23 21:54:50,006 INFO [train.py:1192] (1/2) Epoch 3, batch 0, loss[loss=0.5127, simple_loss=0.5326, pruned_loss=0.2445, over 24548.00 frames. ], tot_loss[loss=0.5127, simple_loss=0.5326, pruned_loss=0.2445, over 24548.00 frames. ], batch size: 137, lr: 3.67e-02, grad_scale: 32.0 2026-09-23 21:54:50,006 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 21:55:01,601 INFO [train.py:1224] (1/2) Epoch 3, validation: loss=0.3463, simple_loss=0.4172, pruned_loss=0.1341, over 2564189.00 frames. 2026-09-23 21:55:01,601 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 21:55:04,884 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.43 vs. limit=12.29 2026-09-23 21:55:05,206 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=6386.666666666667, ans=0.025 2026-09-23 21:55:12,603 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.14 vs. limit=12.34 2026-09-23 21:55:20,012 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=6486.666666666667, ans=9.932500000000001 2026-09-23 21:55:20,918 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.21 vs. limit=6.621666666666667 2026-09-23 21:55:28,190 INFO [train.py:1192] (1/2) Epoch 3, batch 50, loss[loss=0.4974, simple_loss=0.4963, pruned_loss=0.2489, over 24245.00 frames. ], tot_loss[loss=0.5759, simple_loss=0.5722, pruned_loss=0.2891, over 1080970.67 frames. ], batch size: 125, lr: 3.67e-02, grad_scale: 32.0 2026-09-23 21:55:38,966 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=6620.0, ans=0.0 2026-09-23 21:55:48,715 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.250e+02 3.740e+02 4.895e+02 5.677e+02 1.442e+03, threshold=9.791e+02, percent-clipped=3.0 2026-09-23 21:55:54,570 INFO [train.py:1192] (1/2) Epoch 3, batch 100, loss[loss=0.526, simple_loss=0.5427, pruned_loss=0.2546, over 24612.00 frames. ], tot_loss[loss=0.573, simple_loss=0.574, pruned_loss=0.2856, over 1915461.02 frames. ], batch size: 154, lr: 3.66e-02, grad_scale: 32.0 2026-09-23 21:55:55,872 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.16 vs. limit=8.36 2026-09-23 21:55:56,278 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=6720.0, ans=0.03866666666666667 2026-09-23 21:56:01,216 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=6753.333333333333, ans=0.03852777777777778 2026-09-23 21:56:01,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=6753.333333333333, ans=0.03852777777777778 2026-09-23 21:56:02,450 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten.whitening_limit, batch_count=6753.333333333333, ans=10.0325 2026-09-23 21:56:11,933 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.67 vs. limit=8.41 2026-09-23 21:56:19,295 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=6853.333333333333, ans=0.6601333333333333 2026-09-23 21:56:19,942 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.68 vs. limit=6.713333333333333 2026-09-23 21:56:20,704 INFO [train.py:1192] (1/2) Epoch 3, batch 150, loss[loss=0.397, simple_loss=0.4447, pruned_loss=0.1747, over 24251.00 frames. ], tot_loss[loss=0.5586, simple_loss=0.5634, pruned_loss=0.2766, over 2561045.84 frames. ], batch size: 125, lr: 3.66e-02, grad_scale: 32.0 2026-09-23 21:56:41,069 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.211e+02 4.076e+02 4.435e+02 5.203e+02 9.887e+02, threshold=8.870e+02, percent-clipped=1.0 2026-09-23 21:56:46,643 INFO [train.py:1192] (1/2) Epoch 3, batch 200, loss[loss=0.6354, simple_loss=0.6259, pruned_loss=0.3224, over 24248.00 frames. ], tot_loss[loss=0.5507, simple_loss=0.558, pruned_loss=0.2715, over 3060400.71 frames. ], batch size: 257, lr: 3.66e-02, grad_scale: 32.0 2026-09-23 21:56:46,774 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=7053.333333333333, ans=0.22946666666666665 2026-09-23 21:56:47,224 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=7053.333333333333, ans=0.22946666666666665 2026-09-23 21:56:47,287 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=7053.333333333333, ans=0.22946666666666665 2026-09-23 21:56:54,547 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=7086.666666666667, ans=0.03713888888888889 2026-09-23 21:56:56,711 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 21:56:57,722 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=7120.0, ans=0.037000000000000005 2026-09-23 21:57:00,434 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=7120.0, ans=0.2288 2026-09-23 21:57:03,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=7153.333333333333, ans=0.1646875 2026-09-23 21:57:13,124 INFO [train.py:1192] (1/2) Epoch 3, batch 250, loss[loss=0.5692, simple_loss=0.5857, pruned_loss=0.2764, over 24360.00 frames. ], tot_loss[loss=0.5466, simple_loss=0.5554, pruned_loss=0.2688, over 3443415.53 frames. ], batch size: 225, lr: 3.65e-02, grad_scale: 32.0 2026-09-23 21:57:23,528 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=7286.666666666667, ans=0.22713333333333333 2026-09-23 21:57:33,554 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.322e+02 4.274e+02 5.049e+02 6.099e+02 1.100e+03, threshold=1.010e+03, percent-clipped=5.0 2026-09-23 21:57:39,832 INFO [train.py:1192] (1/2) Epoch 3, batch 300, loss[loss=0.5559, simple_loss=0.5764, pruned_loss=0.2677, over 24524.00 frames. ], tot_loss[loss=0.5423, simple_loss=0.5528, pruned_loss=0.2658, over 3756123.41 frames. ], batch size: 204, lr: 3.65e-02, grad_scale: 32.0 2026-09-23 21:57:50,008 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=7453.333333333333, ans=0.009249275362318841 2026-09-23 21:57:59,169 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=7486.666666666667, ans=0.6379666666666667 2026-09-23 21:58:05,597 INFO [train.py:1192] (1/2) Epoch 3, batch 350, loss[loss=0.4411, simple_loss=0.4717, pruned_loss=0.2053, over 24585.00 frames. ], tot_loss[loss=0.5408, simple_loss=0.5529, pruned_loss=0.2643, over 3996783.59 frames. ], batch size: 137, lr: 3.65e-02, grad_scale: 32.0 2026-09-23 21:58:07,755 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=7553.333333333333, ans=0.1459375 2026-09-23 21:58:18,121 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.73 vs. limit=13.215 2026-09-23 21:58:19,773 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=7620.0, ans=0.034916666666666665 2026-09-23 21:58:25,886 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.871e+02 4.157e+02 4.789e+02 6.123e+02 1.192e+03, threshold=9.578e+02, percent-clipped=3.0 2026-09-23 21:58:25,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=7686.666666666667, ans=0.13968750000000002 2026-09-23 21:58:29,630 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=7686.666666666667, ans=0.07 2026-09-23 21:58:31,667 INFO [train.py:1192] (1/2) Epoch 3, batch 400, loss[loss=0.5597, simple_loss=0.5665, pruned_loss=0.2764, over 24581.00 frames. ], tot_loss[loss=0.5372, simple_loss=0.5508, pruned_loss=0.2618, over 4179520.75 frames. ], batch size: 170, lr: 3.64e-02, grad_scale: 32.0 2026-09-23 21:58:31,781 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=7720.0, ans=0.138125 2026-09-23 21:58:49,324 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=7820.0, ans=0.03408333333333334 2026-09-23 21:58:49,770 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=7820.0, ans=9.8875 2026-09-23 21:58:57,803 INFO [train.py:1192] (1/2) Epoch 3, batch 450, loss[loss=0.5618, simple_loss=0.577, pruned_loss=0.2733, over 24623.00 frames. ], tot_loss[loss=0.5337, simple_loss=0.5491, pruned_loss=0.2592, over 4318451.03 frames. ], batch size: 175, lr: 3.64e-02, grad_scale: 32.0 2026-09-23 21:59:01,260 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.53 vs. limit=5.577333333333334 2026-09-23 21:59:04,244 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=7920.0, ans=0.03366666666666667 2026-09-23 21:59:06,606 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=7920.0, ans=0.009147826086956521 2026-09-23 21:59:18,104 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.733e+02 3.882e+02 4.552e+02 5.545e+02 7.804e+02, threshold=9.105e+02, percent-clipped=0.0 2026-09-23 21:59:22,730 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=8020.0, ans=0.03325 2026-09-23 21:59:23,732 INFO [train.py:1192] (1/2) Epoch 3, batch 500, loss[loss=0.6205, simple_loss=0.6188, pruned_loss=0.3111, over 24504.00 frames. ], tot_loss[loss=0.5281, simple_loss=0.5452, pruned_loss=0.2555, over 4436634.58 frames. ], batch size: 218, lr: 3.64e-02, grad_scale: 32.0 2026-09-23 21:59:26,042 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=8053.333333333333, ans=0.6181333333333334 2026-09-23 21:59:30,209 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.80 vs. limit=10.5325 2026-09-23 21:59:43,911 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=8153.333333333333, ans=0.04949747468305833 2026-09-23 21:59:44,725 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=8186.666666666667, ans=0.125 2026-09-23 21:59:45,687 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=8186.666666666667, ans=0.21813333333333335 2026-09-23 21:59:46,120 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=8186.666666666667, ans=0.6134666666666666 2026-09-23 21:59:49,579 INFO [train.py:1192] (1/2) Epoch 3, batch 550, loss[loss=0.5884, simple_loss=0.5971, pruned_loss=0.2898, over 24268.00 frames. ], tot_loss[loss=0.5277, simple_loss=0.5451, pruned_loss=0.2551, over 4521761.77 frames. ], batch size: 257, lr: 3.63e-02, grad_scale: 32.0 2026-09-23 21:59:52,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=8220.0, ans=0.125 2026-09-23 21:59:54,367 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.73 vs. limit=7.055 2026-09-23 21:59:56,766 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=8253.333333333334, ans=0.6111333333333333 2026-09-23 21:59:58,450 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.34 vs. limit=10.595 2026-09-23 22:00:10,145 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.877e+02 4.060e+02 4.625e+02 5.385e+02 1.045e+03, threshold=9.250e+02, percent-clipped=2.0 2026-09-23 22:00:11,359 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.58 vs. limit=10.6325 2026-09-23 22:00:15,327 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.31 vs. limit=10.6325 2026-09-23 22:00:16,071 INFO [train.py:1192] (1/2) Epoch 3, batch 600, loss[loss=0.5627, simple_loss=0.5887, pruned_loss=0.2683, over 24333.00 frames. ], tot_loss[loss=0.5256, simple_loss=0.5444, pruned_loss=0.2534, over 4588709.90 frames. ], batch size: 234, lr: 3.63e-02, grad_scale: 32.0 2026-09-23 22:00:16,157 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=8386.666666666666, ans=0.0 2026-09-23 22:00:42,215 INFO [train.py:1192] (1/2) Epoch 3, batch 650, loss[loss=0.5091, simple_loss=0.5304, pruned_loss=0.2439, over 24610.00 frames. ], tot_loss[loss=0.5203, simple_loss=0.541, pruned_loss=0.2498, over 4653855.83 frames. ], batch size: 154, lr: 3.63e-02, grad_scale: 32.0 2026-09-23 22:00:42,331 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=8553.333333333334, ans=0.125 2026-09-23 22:00:44,779 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=8553.333333333334, ans=0.16446666666666665 2026-09-23 22:00:45,803 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.28 vs. limit=10.7075 2026-09-23 22:00:50,957 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=8586.666666666666, ans=0.21413333333333334 2026-09-23 22:00:52,747 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.68 vs. limit=10.7325 2026-09-23 22:00:58,289 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=8653.333333333334, ans=0.025 2026-09-23 22:01:02,476 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.648e+02 3.944e+02 4.523e+02 5.804e+02 9.829e+02, threshold=9.045e+02, percent-clipped=2.0 2026-09-23 22:01:08,251 INFO [train.py:1192] (1/2) Epoch 3, batch 700, loss[loss=0.486, simple_loss=0.518, pruned_loss=0.2271, over 24557.00 frames. ], tot_loss[loss=0.5199, simple_loss=0.5416, pruned_loss=0.2491, over 4689950.98 frames. ], batch size: 158, lr: 3.62e-02, grad_scale: 32.0 2026-09-23 22:01:21,809 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=8786.666666666666, ans=0.025 2026-09-23 22:01:33,921 INFO [train.py:1192] (1/2) Epoch 3, batch 750, loss[loss=0.488, simple_loss=0.532, pruned_loss=0.222, over 24643.00 frames. ], tot_loss[loss=0.5159, simple_loss=0.539, pruned_loss=0.2464, over 4726166.61 frames. ], batch size: 175, lr: 3.62e-02, grad_scale: 32.0 2026-09-23 22:01:40,570 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=8920.0, ans=0.3338 2026-09-23 22:01:46,411 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=8953.333333333334, ans=10.0 2026-09-23 22:01:54,438 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 3.161e+02 4.265e+02 5.051e+02 6.444e+02 1.055e+03, threshold=1.010e+03, percent-clipped=3.0 2026-09-23 22:01:57,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=9020.0, ans=0.2098 2026-09-23 22:01:58,940 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.22 vs. limit=4.353 2026-09-23 22:01:59,743 INFO [train.py:1192] (1/2) Epoch 3, batch 800, loss[loss=0.4073, simple_loss=0.4528, pruned_loss=0.1809, over 24546.00 frames. ], tot_loss[loss=0.5124, simple_loss=0.5368, pruned_loss=0.2441, over 4751856.40 frames. ], batch size: 137, lr: 3.61e-02, grad_scale: 32.0 2026-09-23 22:02:10,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=9120.0, ans=0.20879999999999999 2026-09-23 22:02:16,193 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.44 vs. limit=10.932500000000001 2026-09-23 22:02:24,137 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=9186.666666666666, ans=0.125 2026-09-23 22:02:24,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=9186.666666666666, ans=0.025 2026-09-23 22:02:25,517 INFO [train.py:1192] (1/2) Epoch 3, batch 850, loss[loss=0.5554, simple_loss=0.5761, pruned_loss=0.2673, over 24572.00 frames. ], tot_loss[loss=0.5088, simple_loss=0.5344, pruned_loss=0.2415, over 4770427.50 frames. ], batch size: 204, lr: 3.61e-02, grad_scale: 32.0 2026-09-23 22:02:30,371 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=9253.333333333334, ans=0.125 2026-09-23 22:02:32,310 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=9253.333333333334, ans=0.125 2026-09-23 22:02:35,550 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=9286.666666666666, ans=0.20713333333333334 2026-09-23 22:02:37,377 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=9286.666666666666, ans=0.5749666666666667 2026-09-23 22:02:37,389 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=9286.666666666666, ans=0.125 2026-09-23 22:02:45,546 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.677e+02 3.885e+02 4.421e+02 5.168e+02 9.776e+02, threshold=8.842e+02, percent-clipped=0.0 2026-09-23 22:02:50,921 INFO [train.py:1192] (1/2) Epoch 3, batch 900, loss[loss=0.4009, simple_loss=0.4582, pruned_loss=0.1718, over 24551.00 frames. ], tot_loss[loss=0.5073, simple_loss=0.5336, pruned_loss=0.2405, over 4781784.23 frames. ], batch size: 137, lr: 3.61e-02, grad_scale: 32.0 2026-09-23 22:02:55,388 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=9420.0, ans=0.008821739130434783 2026-09-23 22:02:56,134 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.89 vs. limit=14.565000000000001 2026-09-23 22:03:00,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=9420.0, ans=0.09899494936611666 2026-09-23 22:03:05,772 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=9486.666666666666, ans=0.027138888888888893 2026-09-23 22:03:07,366 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.82 vs. limit=9.743333333333332 2026-09-23 22:03:08,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=9486.666666666666, ans=0.125 2026-09-23 22:03:16,234 INFO [train.py:1192] (1/2) Epoch 3, batch 950, loss[loss=0.6621, simple_loss=0.5972, pruned_loss=0.3635, over 11332.00 frames. ], tot_loss[loss=0.5078, simple_loss=0.532, pruned_loss=0.2418, over 4711917.09 frames. ], batch size: 333, lr: 3.60e-02, grad_scale: 16.0 2026-09-23 22:03:17,244 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:03:18,678 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.85 vs. limit=9.776666666666667 2026-09-23 22:03:28,076 INFO [train.py:1192] (1/2) Epoch 4, batch 0, loss[loss=0.489, simple_loss=0.5191, pruned_loss=0.2294, over 24559.00 frames. ], tot_loss[loss=0.489, simple_loss=0.5191, pruned_loss=0.2294, over 24559.00 frames. ], batch size: 137, lr: 3.37e-02, grad_scale: 32.0 2026-09-23 22:03:28,076 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 22:03:39,678 INFO [train.py:1224] (1/2) Epoch 4, validation: loss=0.2977, simple_loss=0.3887, pruned_loss=0.1033, over 2564189.00 frames. 2026-09-23 22:03:39,678 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 22:03:40,757 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:03:51,645 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=9646.666666666666, ans=0.125 2026-09-23 22:03:53,276 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=9646.666666666666, ans=0.125 2026-09-23 22:03:55,997 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.607e+02 3.936e+02 5.033e+02 6.810e+02 2.340e+03, threshold=1.007e+03, percent-clipped=8.0 2026-09-23 22:04:01,318 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=9713.333333333334, ans=14.785 2026-09-23 22:04:03,010 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=9713.333333333334, ans=0.125 2026-09-23 22:04:05,754 INFO [train.py:1192] (1/2) Epoch 4, batch 50, loss[loss=0.4439, simple_loss=0.4782, pruned_loss=0.2048, over 24201.00 frames. ], tot_loss[loss=0.5181, simple_loss=0.5421, pruned_loss=0.2471, over 1081119.65 frames. ], batch size: 125, lr: 3.36e-02, grad_scale: 32.0 2026-09-23 22:04:11,089 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=9780.0, ans=0.2022 2026-09-23 22:04:19,121 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.58 vs. limit=9.906666666666666 2026-09-23 22:04:25,205 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.48 vs. limit=14.91 2026-09-23 22:04:28,934 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.50 vs. limit=4.482 2026-09-23 22:04:30,630 INFO [train.py:1192] (1/2) Epoch 4, batch 100, loss[loss=0.481, simple_loss=0.5111, pruned_loss=0.2254, over 24646.00 frames. ], tot_loss[loss=0.5091, simple_loss=0.5394, pruned_loss=0.2393, over 1916754.62 frames. ], batch size: 154, lr: 3.36e-02, grad_scale: 32.0 2026-09-23 22:04:37,488 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=9946.666666666666, ans=0.008707246376811594 2026-09-23 22:04:45,919 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=10013.333333333334, ans=0.09899494936611666 2026-09-23 22:04:46,886 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.831e+02 3.899e+02 4.639e+02 5.288e+02 7.699e+02, threshold=9.278e+02, percent-clipped=0.0 2026-09-23 22:04:52,103 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=10046.666666666666, ans=0.19953333333333334 2026-09-23 22:04:56,341 INFO [train.py:1192] (1/2) Epoch 4, batch 150, loss[loss=0.4085, simple_loss=0.453, pruned_loss=0.1819, over 24234.00 frames. ], tot_loss[loss=0.4956, simple_loss=0.529, pruned_loss=0.2311, over 2561940.35 frames. ], batch size: 125, lr: 3.36e-02, grad_scale: 32.0 2026-09-23 22:04:59,795 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=10080.0, ans=0.125 2026-09-23 22:05:00,667 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=10113.333333333334, ans=0.125 2026-09-23 22:05:01,415 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=10113.333333333334, ans=0.0 2026-09-23 22:05:11,872 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=10180.0, ans=0.125 2026-09-23 22:05:17,837 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.44 vs. limit=11.33 2026-09-23 22:05:21,816 INFO [train.py:1192] (1/2) Epoch 4, batch 200, loss[loss=0.5581, simple_loss=0.5914, pruned_loss=0.2624, over 24201.00 frames. ], tot_loss[loss=0.4915, simple_loss=0.526, pruned_loss=0.2285, over 3060288.22 frames. ], batch size: 257, lr: 3.35e-02, grad_scale: 32.0 2026-09-23 22:05:31,510 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=10313.333333333334, ans=0.0 2026-09-23 22:05:32,015 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=10313.333333333334, ans=0.125 2026-09-23 22:05:38,187 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.870e+02 3.865e+02 4.525e+02 5.215e+02 8.851e+02, threshold=9.051e+02, percent-clipped=0.0 2026-09-23 22:05:45,088 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=10380.0, ans=0.19619999999999999 2026-09-23 22:05:47,528 INFO [train.py:1192] (1/2) Epoch 4, batch 250, loss[loss=0.5578, simple_loss=0.5769, pruned_loss=0.2694, over 24386.00 frames. ], tot_loss[loss=0.4886, simple_loss=0.5238, pruned_loss=0.2267, over 3443845.62 frames. ], batch size: 225, lr: 3.35e-02, grad_scale: 32.0 2026-09-23 22:05:54,454 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=10446.666666666666, ans=0.023138888888888893 2026-09-23 22:06:00,253 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=10480.0, ans=0.025 2026-09-23 22:06:13,907 INFO [train.py:1192] (1/2) Epoch 4, batch 300, loss[loss=0.4723, simple_loss=0.5278, pruned_loss=0.2084, over 24573.00 frames. ], tot_loss[loss=0.4852, simple_loss=0.5214, pruned_loss=0.2245, over 3758204.04 frames. ], batch size: 204, lr: 3.34e-02, grad_scale: 32.0 2026-09-23 22:06:19,653 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=10613.333333333334, ans=0.00856231884057971 2026-09-23 22:06:22,041 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=10613.333333333334, ans=0.125 2026-09-23 22:06:29,738 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.647e+02 3.777e+02 4.403e+02 5.252e+02 8.106e+02, threshold=8.807e+02, percent-clipped=0.0 2026-09-23 22:06:34,622 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=10713.333333333334, ans=0.19286666666666666 2026-09-23 22:06:38,930 INFO [train.py:1192] (1/2) Epoch 4, batch 350, loss[loss=0.4143, simple_loss=0.4571, pruned_loss=0.1858, over 24592.00 frames. ], tot_loss[loss=0.4846, simple_loss=0.5213, pruned_loss=0.2239, over 3999292.61 frames. ], batch size: 137, lr: 3.34e-02, grad_scale: 32.0 2026-09-23 22:06:40,579 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=10746.666666666666, ans=0.125 2026-09-23 22:06:53,706 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=10846.666666666666, ans=0.125 2026-09-23 22:06:55,794 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.26 vs. limit=7.711666666666666 2026-09-23 22:06:57,119 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=10846.666666666666, ans=0.09899494936611666 2026-09-23 22:07:04,294 INFO [train.py:1192] (1/2) Epoch 4, batch 400, loss[loss=0.4551, simple_loss=0.509, pruned_loss=0.2006, over 24562.00 frames. ], tot_loss[loss=0.481, simple_loss=0.5191, pruned_loss=0.2215, over 4182051.99 frames. ], batch size: 170, lr: 3.34e-02, grad_scale: 32.0 2026-09-23 22:07:07,154 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=10913.333333333334, ans=0.125 2026-09-23 22:07:07,158 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=10913.333333333334, ans=0.19086666666666666 2026-09-23 22:07:07,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=10913.333333333334, ans=0.14086666666666664 2026-09-23 22:07:14,233 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff3.min_abs, batch_count=10980.0, ans=0.2 2026-09-23 22:07:17,420 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=10980.0, ans=0.8598 2026-09-23 22:07:17,837 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=10980.0, ans=0.19019999999999998 2026-09-23 22:07:20,156 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.811e+02 4.017e+02 4.684e+02 5.205e+02 7.719e+02, threshold=9.369e+02, percent-clipped=0.0 2026-09-23 22:07:23,129 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=11013.333333333334, ans=0.5145333333333334 2026-09-23 22:07:28,279 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.16 vs. limit=11.6425 2026-09-23 22:07:29,803 INFO [train.py:1192] (1/2) Epoch 4, batch 450, loss[loss=0.4937, simple_loss=0.5349, pruned_loss=0.2263, over 24643.00 frames. ], tot_loss[loss=0.4798, simple_loss=0.5185, pruned_loss=0.2205, over 4322107.94 frames. ], batch size: 175, lr: 3.33e-02, grad_scale: 32.0 2026-09-23 22:07:30,888 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=11080.0, ans=0.125 2026-09-23 22:07:33,727 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=11080.0, ans=0.5122 2026-09-23 22:07:33,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=11080.0, ans=0.5122 2026-09-23 22:07:36,516 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=11113.333333333334, ans=0.125 2026-09-23 22:07:55,434 INFO [train.py:1192] (1/2) Epoch 4, batch 500, loss[loss=0.5494, simple_loss=0.5764, pruned_loss=0.2612, over 24522.00 frames. ], tot_loss[loss=0.4763, simple_loss=0.5157, pruned_loss=0.2184, over 4439015.66 frames. ], batch size: 218, lr: 3.33e-02, grad_scale: 32.0 2026-09-23 22:08:05,393 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.98 vs. limit=7.828333333333333 2026-09-23 22:08:10,847 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=11346.666666666666, ans=0.025 2026-09-23 22:08:11,950 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.881e+02 3.912e+02 4.426e+02 4.981e+02 8.897e+02, threshold=8.851e+02, percent-clipped=0.0 2026-09-23 22:08:15,738 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.24 vs. limit=11.7675 2026-09-23 22:08:17,731 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=11380.0, ans=0.019250000000000003 2026-09-23 22:08:17,735 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=11380.0, ans=0.125 2026-09-23 22:08:21,094 INFO [train.py:1192] (1/2) Epoch 4, batch 550, loss[loss=0.5335, simple_loss=0.5644, pruned_loss=0.2513, over 24311.00 frames. ], tot_loss[loss=0.4747, simple_loss=0.5148, pruned_loss=0.2172, over 4523704.68 frames. ], batch size: 257, lr: 3.32e-02, grad_scale: 32.0 2026-09-23 22:08:25,674 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.23 vs. limit=11.780000000000001 2026-09-23 22:08:29,340 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.83 vs. limit=10.723333333333333 2026-09-23 22:08:46,463 INFO [train.py:1192] (1/2) Epoch 4, batch 600, loss[loss=0.4793, simple_loss=0.543, pruned_loss=0.2078, over 24345.00 frames. ], tot_loss[loss=0.4742, simple_loss=0.515, pruned_loss=0.2167, over 4590409.38 frames. ], batch size: 234, lr: 3.32e-02, grad_scale: 32.0 2026-09-23 22:08:48,875 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=11580.0, ans=0.01841666666666667 2026-09-23 22:08:50,324 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=11580.0, ans=0.025 2026-09-23 22:08:50,792 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=11580.0, ans=0.4947000000000001 2026-09-23 22:08:51,337 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=11613.333333333334, ans=0.125 2026-09-23 22:09:00,513 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=11646.666666666666, ans=0.125 2026-09-23 22:09:02,862 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:09:03,210 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.426e+02 3.617e+02 4.202e+02 4.992e+02 8.611e+02, threshold=8.403e+02, percent-clipped=0.0 2026-09-23 22:09:05,026 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=11680.0, ans=0.09196 2026-09-23 22:09:06,018 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=11680.0, ans=0.125 2026-09-23 22:09:12,190 INFO [train.py:1192] (1/2) Epoch 4, batch 650, loss[loss=0.4173, simple_loss=0.4706, pruned_loss=0.182, over 24593.00 frames. ], tot_loss[loss=0.4691, simple_loss=0.5116, pruned_loss=0.2133, over 4654984.64 frames. ], batch size: 154, lr: 3.32e-02, grad_scale: 32.0 2026-09-23 22:09:37,410 INFO [train.py:1192] (1/2) Epoch 4, batch 700, loss[loss=0.419, simple_loss=0.4763, pruned_loss=0.1808, over 24553.00 frames. ], tot_loss[loss=0.4686, simple_loss=0.5117, pruned_loss=0.2127, over 4690179.05 frames. ], batch size: 158, lr: 3.31e-02, grad_scale: 32.0 2026-09-23 22:09:41,238 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=11913.333333333334, ans=0.18086666666666668 2026-09-23 22:09:46,413 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.26 vs. limit=11.98 2026-09-23 22:09:49,311 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=11980.0, ans=0.0 2026-09-23 22:09:53,890 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.737e+02 3.838e+02 4.537e+02 5.531e+02 8.302e+02, threshold=9.074e+02, percent-clipped=0.0 2026-09-23 22:10:03,374 INFO [train.py:1192] (1/2) Epoch 4, batch 750, loss[loss=0.4285, simple_loss=0.4911, pruned_loss=0.1829, over 24633.00 frames. ], tot_loss[loss=0.4677, simple_loss=0.5107, pruned_loss=0.2124, over 4726543.72 frames. ], batch size: 175, lr: 3.31e-02, grad_scale: 32.0 2026-09-23 22:10:13,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=12146.666666666666, ans=0.025 2026-09-23 22:10:17,326 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=12146.666666666666, ans=0.008228985507246376 2026-09-23 22:10:21,927 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.02 vs. limit=11.09 2026-09-23 22:10:24,974 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=12213.333333333334, ans=0.008214492753623188 2026-09-23 22:10:29,562 INFO [train.py:1192] (1/2) Epoch 4, batch 800, loss[loss=0.3839, simple_loss=0.4438, pruned_loss=0.1619, over 24549.00 frames. ], tot_loss[loss=0.4665, simple_loss=0.5098, pruned_loss=0.2116, over 4751791.86 frames. ], batch size: 137, lr: 3.30e-02, grad_scale: 32.0 2026-09-23 22:10:34,764 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=12280.0, ans=0.0 2026-09-23 22:10:37,584 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.16 vs. limit=12.105 2026-09-23 22:10:43,582 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=12313.333333333334, ans=0.125 2026-09-23 22:10:45,945 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.897e+02 4.015e+02 4.569e+02 5.501e+02 1.043e+03, threshold=9.139e+02, percent-clipped=1.0 2026-09-23 22:10:55,550 INFO [train.py:1192] (1/2) Epoch 4, batch 850, loss[loss=0.4947, simple_loss=0.5394, pruned_loss=0.2251, over 24564.00 frames. ], tot_loss[loss=0.4634, simple_loss=0.508, pruned_loss=0.2094, over 4771110.46 frames. ], batch size: 204, lr: 3.30e-02, grad_scale: 32.0 2026-09-23 22:10:58,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=12413.333333333334, ans=0.17586666666666667 2026-09-23 22:11:04,460 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=12446.666666666666, ans=0.0 2026-09-23 22:11:07,935 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=12480.0, ans=0.125 2026-09-23 22:11:08,730 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=12480.0, ans=0.0 2026-09-23 22:11:12,643 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.32 vs. limit=12.192499999999999 2026-09-23 22:11:21,027 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.39 vs. limit=11.29 2026-09-23 22:11:21,290 INFO [train.py:1192] (1/2) Epoch 4, batch 900, loss[loss=0.4168, simple_loss=0.4668, pruned_loss=0.1834, over 24552.00 frames. ], tot_loss[loss=0.4614, simple_loss=0.5069, pruned_loss=0.208, over 4782521.34 frames. ], batch size: 137, lr: 3.29e-02, grad_scale: 32.0 2026-09-23 22:11:29,025 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=12613.333333333334, ans=0.125 2026-09-23 22:11:34,238 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.96 vs. limit=9.058666666666667 2026-09-23 22:11:36,035 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=12646.666666666666, ans=0.013972222222222226 2026-09-23 22:11:38,347 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.512e+02 3.730e+02 4.472e+02 5.508e+02 1.100e+03, threshold=8.944e+02, percent-clipped=1.0 2026-09-23 22:11:43,614 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=12713.333333333334, ans=0.125 2026-09-23 22:11:46,876 INFO [train.py:1192] (1/2) Epoch 4, batch 950, loss[loss=0.6247, simple_loss=0.5603, pruned_loss=0.3445, over 11139.00 frames. ], tot_loss[loss=0.4629, simple_loss=0.5057, pruned_loss=0.2101, over 4720315.53 frames. ], batch size: 333, lr: 3.29e-02, grad_scale: 32.0 2026-09-23 22:11:58,020 INFO [train.py:1192] (1/2) Epoch 5, batch 0, loss[loss=0.4285, simple_loss=0.4827, pruned_loss=0.1872, over 24581.00 frames. ], tot_loss[loss=0.4285, simple_loss=0.4827, pruned_loss=0.1872, over 24581.00 frames. ], batch size: 137, lr: 3.06e-02, grad_scale: 32.0 2026-09-23 22:11:58,020 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 22:12:09,817 INFO [train.py:1224] (1/2) Epoch 5, validation: loss=0.2774, simple_loss=0.3744, pruned_loss=0.09019, over 2564189.00 frames. 2026-09-23 22:12:09,817 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 22:12:09,919 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=12773.333333333334, ans=0.17226666666666665 2026-09-23 22:12:19,484 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=12840.0, ans=0.125 2026-09-23 22:12:21,467 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.03 vs. limit=9.136 2026-09-23 22:12:27,617 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=6.49 vs. limit=9.149333333333335 2026-09-23 22:12:28,458 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=12873.333333333334, ans=0.125 2026-09-23 22:12:31,013 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=12906.666666666666, ans=0.00806376811594203 2026-09-23 22:12:31,383 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=12906.666666666666, ans=0.0 2026-09-23 22:12:33,564 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=12906.666666666666, ans=0.125 2026-09-23 22:12:35,659 INFO [train.py:1192] (1/2) Epoch 5, batch 50, loss[loss=0.392, simple_loss=0.4405, pruned_loss=0.1718, over 24337.00 frames. ], tot_loss[loss=0.4679, simple_loss=0.5129, pruned_loss=0.2114, over 1081751.42 frames. ], batch size: 125, lr: 3.06e-02, grad_scale: 32.0 2026-09-23 22:12:36,322 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.56 vs. limit=12.3525 2026-09-23 22:12:38,770 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=12940.0, ans=0.125 2026-09-23 22:12:41,846 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.57 vs. limit=17.23 2026-09-23 22:12:44,255 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:12:48,346 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.520e+02 3.699e+02 4.389e+02 5.018e+02 1.310e+03, threshold=8.778e+02, percent-clipped=1.0 2026-09-23 22:12:52,465 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.51 vs. limit=12.39 2026-09-23 22:13:01,218 INFO [train.py:1192] (1/2) Epoch 5, batch 100, loss[loss=0.3945, simple_loss=0.4634, pruned_loss=0.1627, over 24603.00 frames. ], tot_loss[loss=0.4669, simple_loss=0.5148, pruned_loss=0.2095, over 1915486.74 frames. ], batch size: 154, lr: 3.05e-02, grad_scale: 32.0 2026-09-23 22:13:04,241 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=13106.666666666666, ans=0.125 2026-09-23 22:13:23,969 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=13240.0, ans=0.025 2026-09-23 22:13:26,874 INFO [train.py:1192] (1/2) Epoch 5, batch 150, loss[loss=0.3416, simple_loss=0.4065, pruned_loss=0.1384, over 24321.00 frames. ], tot_loss[loss=0.4573, simple_loss=0.5065, pruned_loss=0.204, over 2560441.83 frames. ], batch size: 125, lr: 3.05e-02, grad_scale: 32.0 2026-09-23 22:13:39,236 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.580e+02 3.450e+02 4.364e+02 5.078e+02 1.050e+03, threshold=8.729e+02, percent-clipped=2.0 2026-09-23 22:13:52,316 INFO [train.py:1192] (1/2) Epoch 5, batch 200, loss[loss=0.5504, simple_loss=0.578, pruned_loss=0.2614, over 24205.00 frames. ], tot_loss[loss=0.4526, simple_loss=0.5025, pruned_loss=0.2014, over 3059488.83 frames. ], batch size: 257, lr: 3.05e-02, grad_scale: 32.0 2026-09-23 22:13:55,889 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=13440.0, ans=0.1656 2026-09-23 22:14:12,104 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=13540.0, ans=0.010250000000000002 2026-09-23 22:14:12,833 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.94 vs. limit=9.429333333333334 2026-09-23 22:14:18,453 INFO [train.py:1192] (1/2) Epoch 5, batch 250, loss[loss=0.5235, simple_loss=0.5613, pruned_loss=0.2429, over 24401.00 frames. ], tot_loss[loss=0.4517, simple_loss=0.5016, pruned_loss=0.2009, over 3442769.17 frames. ], batch size: 225, lr: 3.04e-02, grad_scale: 32.0 2026-09-23 22:14:30,723 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_ff2.min_abs, batch_count=13673.333333333334, ans=0.1 2026-09-23 22:14:31,135 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.684e+02 3.685e+02 4.372e+02 5.234e+02 9.542e+02, threshold=8.745e+02, percent-clipped=1.0 2026-09-23 22:14:42,344 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=13740.0, ans=0.00941666666666667 2026-09-23 22:14:44,620 INFO [train.py:1192] (1/2) Epoch 5, batch 300, loss[loss=0.4879, simple_loss=0.5356, pruned_loss=0.2201, over 24567.00 frames. ], tot_loss[loss=0.4495, simple_loss=0.5001, pruned_loss=0.1995, over 3757222.89 frames. ], batch size: 204, lr: 3.04e-02, grad_scale: 32.0 2026-09-23 22:15:01,658 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.38 vs. limit=17.905 2026-09-23 22:15:02,605 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=13873.333333333334, ans=0.125 2026-09-23 22:15:06,770 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.68 vs. limit=12.715 2026-09-23 22:15:10,805 INFO [train.py:1192] (1/2) Epoch 5, batch 350, loss[loss=0.3497, simple_loss=0.4206, pruned_loss=0.1394, over 24586.00 frames. ], tot_loss[loss=0.4485, simple_loss=0.4999, pruned_loss=0.1986, over 3997847.92 frames. ], batch size: 137, lr: 3.03e-02, grad_scale: 32.0 2026-09-23 22:15:16,671 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=13973.333333333334, ans=0.025 2026-09-23 22:15:23,484 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.338e+02 3.709e+02 4.180e+02 5.205e+02 8.413e+02, threshold=8.359e+02, percent-clipped=0.0 2026-09-23 22:15:34,956 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=14073.333333333334, ans=0.4074333333333333 2026-09-23 22:15:37,107 INFO [train.py:1192] (1/2) Epoch 5, batch 400, loss[loss=0.4265, simple_loss=0.4902, pruned_loss=0.1814, over 24564.00 frames. ], tot_loss[loss=0.4471, simple_loss=0.4987, pruned_loss=0.1977, over 4178870.31 frames. ], batch size: 170, lr: 3.03e-02, grad_scale: 32.0 2026-09-23 22:15:45,032 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.82 vs. limit=6.827999999999999 2026-09-23 22:15:56,745 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=14206.666666666666, ans=0.09899494936611666 2026-09-23 22:16:03,114 INFO [train.py:1192] (1/2) Epoch 5, batch 450, loss[loss=0.4451, simple_loss=0.509, pruned_loss=0.1906, over 24620.00 frames. ], tot_loss[loss=0.4462, simple_loss=0.4983, pruned_loss=0.197, over 4318887.49 frames. ], batch size: 175, lr: 3.02e-02, grad_scale: 32.0 2026-09-23 22:16:10,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=14306.666666666666, ans=0.025 2026-09-23 22:16:15,358 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.89 vs. limit=18.255000000000003 2026-09-23 22:16:15,594 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.349e+02 3.590e+02 4.169e+02 4.919e+02 7.184e+02, threshold=8.339e+02, percent-clipped=0.0 2026-09-23 22:16:23,457 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=14406.666666666666, ans=0.00773768115942029 2026-09-23 22:16:28,212 INFO [train.py:1192] (1/2) Epoch 5, batch 500, loss[loss=0.4567, simple_loss=0.5251, pruned_loss=0.1942, over 24545.00 frames. ], tot_loss[loss=0.44, simple_loss=0.4938, pruned_loss=0.1931, over 4437174.50 frames. ], batch size: 218, lr: 3.02e-02, grad_scale: 16.0 2026-09-23 22:16:28,300 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=14440.0, ans=0.007730434782608696 2026-09-23 22:16:35,530 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=14473.333333333334, ans=0.09899494936611666 2026-09-23 22:16:47,493 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=16.37 vs. limit=18.405 2026-09-23 22:16:51,948 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=14573.333333333334, ans=0.125 2026-09-23 22:16:54,237 INFO [train.py:1192] (1/2) Epoch 5, batch 550, loss[loss=0.5092, simple_loss=0.5573, pruned_loss=0.2306, over 24247.00 frames. ], tot_loss[loss=0.441, simple_loss=0.4946, pruned_loss=0.1938, over 4521862.12 frames. ], batch size: 257, lr: 3.02e-02, grad_scale: 16.0 2026-09-23 22:16:55,351 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=14606.666666666666, ans=0.125 2026-09-23 22:16:56,298 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=14606.666666666666, ans=0.125 2026-09-23 22:17:02,271 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=14640.0, ans=0.15360000000000001 2026-09-23 22:17:02,723 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=14640.0, ans=0.125 2026-09-23 22:17:06,918 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.475e+02 3.784e+02 4.403e+02 5.598e+02 1.133e+03, threshold=8.806e+02, percent-clipped=4.0 2026-09-23 22:17:10,621 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=14706.666666666666, ans=0.0053888888888888875 2026-09-23 22:17:11,578 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=14706.666666666666, ans=0.125 2026-09-23 22:17:19,682 INFO [train.py:1192] (1/2) Epoch 5, batch 600, loss[loss=0.471, simple_loss=0.5334, pruned_loss=0.2043, over 24344.00 frames. ], tot_loss[loss=0.4393, simple_loss=0.4939, pruned_loss=0.1923, over 4588269.85 frames. ], batch size: 234, lr: 3.01e-02, grad_scale: 16.0 2026-09-23 22:17:27,482 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=14806.666666666666, ans=0.125 2026-09-23 22:17:35,571 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=14873.333333333334, ans=0.007636231884057971 2026-09-23 22:17:36,138 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=14873.333333333334, ans=0.125 2026-09-23 22:17:37,493 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=14873.333333333334, ans=0.04949747468305833 2026-09-23 22:17:45,413 INFO [train.py:1192] (1/2) Epoch 5, batch 650, loss[loss=0.3909, simple_loss=0.4588, pruned_loss=0.1615, over 24599.00 frames. ], tot_loss[loss=0.4362, simple_loss=0.4918, pruned_loss=0.1903, over 4653452.64 frames. ], batch size: 154, lr: 3.01e-02, grad_scale: 16.0 2026-09-23 22:17:48,425 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.85 vs. limit=18.705 2026-09-23 22:17:50,291 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=14940.0, ans=0.125 2026-09-23 22:17:55,001 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=14973.333333333334, ans=0.125 2026-09-23 22:17:55,008 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=14973.333333333334, ans=0.125 2026-09-23 22:17:58,443 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.307e+02 3.636e+02 4.065e+02 4.709e+02 8.553e+02, threshold=8.130e+02, percent-clipped=0.0 2026-09-23 22:17:58,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=15006.666666666666, ans=0.125 2026-09-23 22:18:01,581 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=15040.0, ans=0.125 2026-09-23 22:18:02,433 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=15040.0, ans=0.06088000000000002 2026-09-23 22:18:09,570 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:18:10,958 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=15073.333333333334, ans=0.003861111111111107 2026-09-23 22:18:11,839 INFO [train.py:1192] (1/2) Epoch 5, batch 700, loss[loss=0.3877, simple_loss=0.457, pruned_loss=0.1592, over 24553.00 frames. ], tot_loss[loss=0.4365, simple_loss=0.4924, pruned_loss=0.1903, over 4688939.92 frames. ], batch size: 158, lr: 3.00e-02, grad_scale: 16.0 2026-09-23 22:18:15,955 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=15106.666666666666, ans=0.125 2026-09-23 22:18:20,467 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=15140.0, ans=0.125 2026-09-23 22:18:28,200 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.83 vs. limit=8.801666666666666 2026-09-23 22:18:29,002 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=15206.666666666666, ans=0.125 2026-09-23 22:18:37,906 INFO [train.py:1192] (1/2) Epoch 5, batch 750, loss[loss=0.4186, simple_loss=0.485, pruned_loss=0.1761, over 24625.00 frames. ], tot_loss[loss=0.4342, simple_loss=0.4906, pruned_loss=0.1889, over 4722836.16 frames. ], batch size: 175, lr: 3.00e-02, grad_scale: 16.0 2026-09-23 22:18:47,036 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.70 vs. limit=18.98 2026-09-23 22:18:51,296 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.366e+02 3.748e+02 4.495e+02 5.532e+02 8.509e+02, threshold=8.989e+02, percent-clipped=3.0 2026-09-23 22:19:04,054 INFO [train.py:1192] (1/2) Epoch 5, batch 800, loss[loss=0.3779, simple_loss=0.4448, pruned_loss=0.1555, over 24555.00 frames. ], tot_loss[loss=0.4345, simple_loss=0.4904, pruned_loss=0.1893, over 4749368.20 frames. ], batch size: 137, lr: 2.99e-02, grad_scale: 32.0 2026-09-23 22:19:16,912 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=15506.666666666666, ans=0.007498550724637681 2026-09-23 22:19:23,573 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=15540.0, ans=0.007491304347826087 2026-09-23 22:19:29,796 INFO [train.py:1192] (1/2) Epoch 5, batch 850, loss[loss=0.4584, simple_loss=0.5185, pruned_loss=0.1992, over 24554.00 frames. ], tot_loss[loss=0.4326, simple_loss=0.4891, pruned_loss=0.188, over 4769335.18 frames. ], batch size: 204, lr: 2.99e-02, grad_scale: 32.0 2026-09-23 22:19:32,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=15606.666666666666, ans=0.025 2026-09-23 22:19:34,673 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.41 vs. limit=13.365 2026-09-23 22:19:35,081 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=15640.0, ans=0.3526 2026-09-23 22:19:39,486 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=15640.0, ans=0.0015000000000000013 2026-09-23 22:19:42,967 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.633e+02 3.828e+02 4.258e+02 5.330e+02 1.035e+03, threshold=8.516e+02, percent-clipped=1.0 2026-09-23 22:19:50,316 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=15740.0, ans=0.3491000000000001 2026-09-23 22:19:51,491 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.57 vs. limit=13.4025 2026-09-23 22:19:55,628 INFO [train.py:1192] (1/2) Epoch 5, batch 900, loss[loss=0.3839, simple_loss=0.4488, pruned_loss=0.1595, over 24568.00 frames. ], tot_loss[loss=0.4329, simple_loss=0.4894, pruned_loss=0.1882, over 4779796.06 frames. ], batch size: 137, lr: 2.98e-02, grad_scale: 32.0 2026-09-23 22:19:57,789 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=15773.333333333334, ans=0.0009444444444444422 2026-09-23 22:20:04,015 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:20:04,787 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=15.59 vs. limit=19.355 2026-09-23 22:20:06,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=15840.0, ans=0.0006666666666666696 2026-09-23 22:20:21,870 INFO [train.py:1192] (1/2) Epoch 5, batch 950, loss[loss=0.6266, simple_loss=0.5723, pruned_loss=0.3405, over 11529.00 frames. ], tot_loss[loss=0.4343, simple_loss=0.4885, pruned_loss=0.1901, over 4711984.07 frames. ], batch size: 333, lr: 2.98e-02, grad_scale: 32.0 2026-09-23 22:20:21,966 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=15940.0, ans=0.125 2026-09-23 22:20:33,966 INFO [train.py:1192] (1/2) Epoch 6, batch 0, loss[loss=0.4237, simple_loss=0.4758, pruned_loss=0.1857, over 24577.00 frames. ], tot_loss[loss=0.4237, simple_loss=0.4758, pruned_loss=0.1857, over 24577.00 frames. ], batch size: 137, lr: 2.78e-02, grad_scale: 32.0 2026-09-23 22:20:33,966 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 22:20:46,089 INFO [train.py:1224] (1/2) Epoch 6, validation: loss=0.2624, simple_loss=0.364, pruned_loss=0.08034, over 2564189.00 frames. 2026-09-23 22:20:46,089 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 22:20:55,658 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.670e+02 3.768e+02 4.556e+02 5.587e+02 1.060e+03, threshold=9.113e+02, percent-clipped=3.0 2026-09-23 22:21:01,077 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=16066.666666666666, ans=0.125 2026-09-23 22:21:03,039 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=16066.666666666666, ans=0.0 2026-09-23 22:21:12,021 INFO [train.py:1192] (1/2) Epoch 6, batch 50, loss[loss=0.3555, simple_loss=0.4186, pruned_loss=0.1462, over 24283.00 frames. ], tot_loss[loss=0.446, simple_loss=0.5002, pruned_loss=0.1958, over 1080820.44 frames. ], batch size: 125, lr: 2.78e-02, grad_scale: 32.0 2026-09-23 22:21:12,628 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=16133.333333333334, ans=0.125 2026-09-23 22:21:28,652 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.89 vs. limit=9.058333333333334 2026-09-23 22:21:32,078 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=16266.666666666666, ans=0.13733333333333334 2026-09-23 22:21:32,083 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=16266.666666666666, ans=0.09899494936611666 2026-09-23 22:21:33,062 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=16266.666666666666, ans=0.0 2026-09-23 22:21:35,491 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=16266.666666666666, ans=0.125 2026-09-23 22:21:36,775 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=16266.666666666666, ans=0.444 2026-09-23 22:21:37,639 INFO [train.py:1192] (1/2) Epoch 6, batch 100, loss[loss=0.4228, simple_loss=0.4814, pruned_loss=0.1821, over 24627.00 frames. ], tot_loss[loss=0.4413, simple_loss=0.4995, pruned_loss=0.1915, over 1915686.62 frames. ], batch size: 154, lr: 2.77e-02, grad_scale: 32.0 2026-09-23 22:21:37,743 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=16300.0, ans=0.125 2026-09-23 22:21:46,923 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.660e+02 3.443e+02 4.030e+02 4.669e+02 8.575e+02, threshold=8.059e+02, percent-clipped=0.0 2026-09-23 22:21:47,603 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_ff3.min_abs, batch_count=16366.666666666666, ans=0.2 2026-09-23 22:21:49,329 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=16366.666666666666, ans=0.0073115942028985515 2026-09-23 22:21:49,479 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.52 vs. limit=5.455 2026-09-23 22:21:49,824 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=16366.666666666666, ans=0.13633333333333333 2026-09-23 22:22:03,771 INFO [train.py:1192] (1/2) Epoch 6, batch 150, loss[loss=0.3459, simple_loss=0.4092, pruned_loss=0.1413, over 24301.00 frames. ], tot_loss[loss=0.4324, simple_loss=0.4913, pruned_loss=0.1867, over 2560955.91 frames. ], batch size: 125, lr: 2.77e-02, grad_scale: 32.0 2026-09-23 22:22:07,330 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=16466.666666666668, ans=0.0 2026-09-23 22:22:18,432 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=16533.333333333332, ans=0.125 2026-09-23 22:22:29,952 INFO [train.py:1192] (1/2) Epoch 6, batch 200, loss[loss=0.4873, simple_loss=0.5424, pruned_loss=0.2161, over 24228.00 frames. ], tot_loss[loss=0.4278, simple_loss=0.4877, pruned_loss=0.184, over 3059688.17 frames. ], batch size: 257, lr: 2.76e-02, grad_scale: 32.0 2026-09-23 22:22:39,186 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.471e+02 3.466e+02 4.197e+02 4.744e+02 7.854e+02, threshold=8.394e+02, percent-clipped=0.0 2026-09-23 22:22:44,119 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=16700.0, ans=0.125 2026-09-23 22:22:44,726 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.96 vs. limit=9.183333333333334 2026-09-23 22:22:55,342 INFO [train.py:1192] (1/2) Epoch 6, batch 250, loss[loss=0.5103, simple_loss=0.551, pruned_loss=0.2348, over 24370.00 frames. ], tot_loss[loss=0.4243, simple_loss=0.4847, pruned_loss=0.182, over 3444012.01 frames. ], batch size: 225, lr: 2.76e-02, grad_scale: 32.0 2026-09-23 22:22:58,507 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.53 vs. limit=13.4 2026-09-23 22:22:59,419 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=16800.0, ans=0.125 2026-09-23 22:23:10,000 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.65 vs. limit=10.746666666666666 2026-09-23 22:23:17,334 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=16933.333333333332, ans=0.00718840579710145 2026-09-23 22:23:17,818 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.88 vs. limit=13.466666666666665 2026-09-23 22:23:20,785 INFO [train.py:1192] (1/2) Epoch 6, batch 300, loss[loss=0.4542, simple_loss=0.5202, pruned_loss=0.1941, over 24572.00 frames. ], tot_loss[loss=0.4208, simple_loss=0.4823, pruned_loss=0.1796, over 3757697.71 frames. ], batch size: 204, lr: 2.76e-02, grad_scale: 32.0 2026-09-23 22:23:20,873 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:23:23,877 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=16966.666666666668, ans=0.1303333333333333 2026-09-23 22:23:25,147 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=16966.666666666668, ans=0.0 2026-09-23 22:23:30,250 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.608e+02 3.453e+02 3.959e+02 5.073e+02 7.733e+02, threshold=7.917e+02, percent-clipped=0.0 2026-09-23 22:23:40,007 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=17066.666666666668, ans=0.125 2026-09-23 22:23:40,380 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=17066.666666666668, ans=0.025 2026-09-23 22:23:46,443 INFO [train.py:1192] (1/2) Epoch 6, batch 350, loss[loss=0.3627, simple_loss=0.4315, pruned_loss=0.147, over 24563.00 frames. ], tot_loss[loss=0.4211, simple_loss=0.4829, pruned_loss=0.1797, over 3998082.92 frames. ], batch size: 137, lr: 2.75e-02, grad_scale: 32.0 2026-09-23 22:23:47,093 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=17133.333333333332, ans=0.12866666666666668 2026-09-23 22:23:51,565 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=17166.666666666668, ans=0.125 2026-09-23 22:23:52,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=17166.666666666668, ans=0.125 2026-09-23 22:23:56,591 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.84 vs. limit=20.375 2026-09-23 22:23:58,956 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=17200.0, ans=0.125 2026-09-23 22:24:05,352 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=17233.333333333332, ans=0.0 2026-09-23 22:24:06,882 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.20 vs. limit=13.962499999999999 2026-09-23 22:24:10,484 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=17266.666666666668, ans=0.12733333333333333 2026-09-23 22:24:12,997 INFO [train.py:1192] (1/2) Epoch 6, batch 400, loss[loss=0.4041, simple_loss=0.4697, pruned_loss=0.1693, over 24561.00 frames. ], tot_loss[loss=0.4195, simple_loss=0.4817, pruned_loss=0.1787, over 4180776.64 frames. ], batch size: 170, lr: 2.75e-02, grad_scale: 32.0 2026-09-23 22:24:13,571 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=17300.0, ans=0.2945 2026-09-23 22:24:14,810 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=17300.0, ans=0.125 2026-09-23 22:24:20,901 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=17333.333333333332, ans=0.125 2026-09-23 22:24:22,046 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.579e+02 3.534e+02 4.312e+02 5.095e+02 8.398e+02, threshold=8.624e+02, percent-clipped=1.0 2026-09-23 22:24:22,148 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=17333.333333333332, ans=0.125 2026-09-23 22:24:28,058 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=17400.0, ans=0.125 2026-09-23 22:24:35,385 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=4.54 vs. limit=14.0375 2026-09-23 22:24:36,957 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=17433.333333333332, ans=0.125 2026-09-23 22:24:38,384 INFO [train.py:1192] (1/2) Epoch 6, batch 450, loss[loss=0.4539, simple_loss=0.5077, pruned_loss=0.2, over 24630.00 frames. ], tot_loss[loss=0.4194, simple_loss=0.4814, pruned_loss=0.1787, over 4320400.16 frames. ], batch size: 175, lr: 2.74e-02, grad_scale: 32.0 2026-09-23 22:24:38,995 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.05 vs. limit=20.6 2026-09-23 22:24:43,770 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.40 vs. limit=14.0625 2026-09-23 22:24:46,315 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=17500.0, ans=0.125 2026-09-23 22:24:46,869 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=17500.0, ans=0.125 2026-09-23 22:25:02,554 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=17600.0, ans=0.125 2026-09-23 22:25:03,488 INFO [train.py:1192] (1/2) Epoch 6, batch 500, loss[loss=0.452, simple_loss=0.5207, pruned_loss=0.1916, over 24461.00 frames. ], tot_loss[loss=0.416, simple_loss=0.4787, pruned_loss=0.1767, over 4438237.65 frames. ], batch size: 218, lr: 2.74e-02, grad_scale: 16.0 2026-09-23 22:25:11,852 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=17666.666666666668, ans=0.0 2026-09-23 22:25:13,751 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.588e+02 3.510e+02 4.091e+02 4.746e+02 7.444e+02, threshold=8.182e+02, percent-clipped=0.0 2026-09-23 22:25:19,920 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=17733.333333333332, ans=0.125 2026-09-23 22:25:29,814 INFO [train.py:1192] (1/2) Epoch 6, batch 550, loss[loss=0.4915, simple_loss=0.548, pruned_loss=0.2175, over 24238.00 frames. ], tot_loss[loss=0.4176, simple_loss=0.4797, pruned_loss=0.1777, over 4522869.31 frames. ], batch size: 257, lr: 2.73e-02, grad_scale: 16.0 2026-09-23 22:25:42,907 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=17866.666666666668, ans=0.0 2026-09-23 22:25:49,003 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=17900.0, ans=0.0 2026-09-23 22:25:52,832 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=17933.333333333332, ans=0.0 2026-09-23 22:25:55,466 INFO [train.py:1192] (1/2) Epoch 6, batch 600, loss[loss=0.4387, simple_loss=0.5165, pruned_loss=0.1804, over 24329.00 frames. ], tot_loss[loss=0.4174, simple_loss=0.4799, pruned_loss=0.1775, over 4589435.48 frames. ], batch size: 234, lr: 2.73e-02, grad_scale: 16.0 2026-09-23 22:26:05,323 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.396e+02 3.556e+02 4.162e+02 5.274e+02 1.094e+03, threshold=8.324e+02, percent-clipped=2.0 2026-09-23 22:26:13,195 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.60 vs. limit=14.275 2026-09-23 22:26:18,952 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=18100.0, ans=0.125 2026-09-23 22:26:21,300 INFO [train.py:1192] (1/2) Epoch 6, batch 650, loss[loss=0.3879, simple_loss=0.4538, pruned_loss=0.161, over 24604.00 frames. ], tot_loss[loss=0.4132, simple_loss=0.477, pruned_loss=0.1747, over 4654260.56 frames. ], batch size: 154, lr: 2.73e-02, grad_scale: 16.0 2026-09-23 22:26:27,822 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=18166.666666666668, ans=0.006920289855072464 2026-09-23 22:26:31,869 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=18200.0, ans=0.0 2026-09-23 22:26:36,515 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=18200.0, ans=0.0 2026-09-23 22:26:41,818 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:26:43,221 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=18266.666666666668, ans=0.125 2026-09-23 22:26:44,701 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=18266.666666666668, ans=0.0 2026-09-23 22:26:47,622 INFO [train.py:1192] (1/2) Epoch 6, batch 700, loss[loss=0.3808, simple_loss=0.4544, pruned_loss=0.1536, over 24558.00 frames. ], tot_loss[loss=0.4135, simple_loss=0.478, pruned_loss=0.1745, over 4690768.23 frames. ], batch size: 158, lr: 2.72e-02, grad_scale: 16.0 2026-09-23 22:26:47,763 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=18300.0, ans=0.0 2026-09-23 22:26:48,888 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=18300.0, ans=0.125 2026-09-23 22:26:53,081 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=18333.333333333332, ans=0.125 2026-09-23 22:26:57,314 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.618e+02 3.557e+02 4.369e+02 5.425e+02 8.837e+02, threshold=8.739e+02, percent-clipped=2.0 2026-09-23 22:27:13,505 INFO [train.py:1192] (1/2) Epoch 6, batch 750, loss[loss=0.3794, simple_loss=0.4678, pruned_loss=0.1455, over 24633.00 frames. ], tot_loss[loss=0.4115, simple_loss=0.4764, pruned_loss=0.1733, over 4727210.74 frames. ], batch size: 175, lr: 2.72e-02, grad_scale: 16.0 2026-09-23 22:27:22,281 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=18500.0, ans=0.11500000000000002 2026-09-23 22:27:27,203 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=18533.333333333332, ans=0.006840579710144928 2026-09-23 22:27:32,704 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.87 vs. limit=14.4625 2026-09-23 22:27:34,178 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=18600.0, ans=0.125 2026-09-23 22:27:40,156 INFO [train.py:1192] (1/2) Epoch 6, batch 800, loss[loss=0.3422, simple_loss=0.419, pruned_loss=0.1327, over 24543.00 frames. ], tot_loss[loss=0.411, simple_loss=0.4759, pruned_loss=0.1731, over 4753412.27 frames. ], batch size: 137, lr: 2.71e-02, grad_scale: 32.0 2026-09-23 22:27:40,670 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=18633.333333333332, ans=0.125 2026-09-23 22:27:49,261 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=18666.666666666668, ans=0.07 2026-09-23 22:27:50,262 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.559e+02 3.295e+02 3.854e+02 4.496e+02 1.051e+03, threshold=7.707e+02, percent-clipped=1.0 2026-09-23 22:27:50,370 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=18700.0, ans=0.006804347826086956 2026-09-23 22:28:02,922 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=18766.666666666668, ans=0.125 2026-09-23 22:28:06,957 INFO [train.py:1192] (1/2) Epoch 6, batch 850, loss[loss=0.4346, simple_loss=0.4995, pruned_loss=0.1849, over 24545.00 frames. ], tot_loss[loss=0.4102, simple_loss=0.4753, pruned_loss=0.1726, over 4771849.86 frames. ], batch size: 204, lr: 2.71e-02, grad_scale: 32.0 2026-09-23 22:28:11,922 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=18833.333333333332, ans=0.0 2026-09-23 22:28:31,456 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.92 vs. limit=14.6 2026-09-23 22:28:31,990 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.70 vs. limit=14.6 2026-09-23 22:28:33,046 INFO [train.py:1192] (1/2) Epoch 6, batch 900, loss[loss=0.3288, simple_loss=0.411, pruned_loss=0.1233, over 24529.00 frames. ], tot_loss[loss=0.4107, simple_loss=0.4756, pruned_loss=0.1729, over 4782202.08 frames. ], batch size: 137, lr: 2.70e-02, grad_scale: 32.0 2026-09-23 22:28:37,108 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=6.59 vs. limit=9.741666666666667 2026-09-23 22:28:38,047 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.97 vs. limit=14.625 2026-09-23 22:28:43,023 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.416e+02 3.503e+02 4.182e+02 4.940e+02 9.030e+02, threshold=8.364e+02, percent-clipped=1.0 2026-09-23 22:28:58,672 INFO [train.py:1192] (1/2) Epoch 6, batch 950, loss[loss=0.6135, simple_loss=0.5536, pruned_loss=0.3367, over 11374.00 frames. ], tot_loss[loss=0.4129, simple_loss=0.4752, pruned_loss=0.1754, over 4715972.67 frames. ], batch size: 334, lr: 2.70e-02, grad_scale: 16.0 2026-09-23 22:29:10,806 INFO [train.py:1192] (1/2) Epoch 7, batch 0, loss[loss=0.3865, simple_loss=0.4545, pruned_loss=0.1592, over 24591.00 frames. ], tot_loss[loss=0.3865, simple_loss=0.4545, pruned_loss=0.1592, over 24591.00 frames. ], batch size: 137, lr: 2.53e-02, grad_scale: 32.0 2026-09-23 22:29:10,807 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 22:29:22,639 INFO [train.py:1224] (1/2) Epoch 7, validation: loss=0.2474, simple_loss=0.3531, pruned_loss=0.07088, over 2564189.00 frames. 2026-09-23 22:29:22,639 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 22:29:22,749 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=19160.0, ans=10.0 2026-09-23 22:29:22,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=19160.0, ans=0.07 2026-09-23 22:29:24,117 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=19160.0, ans=0.0 2026-09-23 22:29:29,872 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.72 vs. limit=14.596666666666666 2026-09-23 22:29:42,255 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.09 vs. limit=14.63 2026-09-23 22:29:48,120 INFO [train.py:1192] (1/2) Epoch 7, batch 50, loss[loss=0.3683, simple_loss=0.4294, pruned_loss=0.1536, over 24257.00 frames. ], tot_loss[loss=0.4231, simple_loss=0.4852, pruned_loss=0.1805, over 1081855.87 frames. ], batch size: 125, lr: 2.53e-02, grad_scale: 32.0 2026-09-23 22:29:53,665 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.228e+02 3.433e+02 4.092e+02 4.910e+02 9.076e+02, threshold=8.183e+02, percent-clipped=3.0 2026-09-23 22:30:00,118 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=19393.333333333332, ans=0.0 2026-09-23 22:30:01,719 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.77 vs. limit=22.045 2026-09-23 22:30:13,929 INFO [train.py:1192] (1/2) Epoch 7, batch 100, loss[loss=0.4194, simple_loss=0.4764, pruned_loss=0.1812, over 24622.00 frames. ], tot_loss[loss=0.4196, simple_loss=0.4851, pruned_loss=0.1771, over 1915399.44 frames. ], batch size: 154, lr: 2.52e-02, grad_scale: 32.0 2026-09-23 22:30:26,677 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.11 vs. limit=22.17 2026-09-23 22:30:39,891 INFO [train.py:1192] (1/2) Epoch 7, batch 150, loss[loss=0.3205, simple_loss=0.3995, pruned_loss=0.1207, over 24238.00 frames. ], tot_loss[loss=0.4104, simple_loss=0.4766, pruned_loss=0.1721, over 2560744.64 frames. ], batch size: 125, lr: 2.52e-02, grad_scale: 32.0 2026-09-23 22:30:39,968 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=19660.0, ans=0.125 2026-09-23 22:30:45,842 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.438e+02 3.557e+02 4.005e+02 4.673e+02 1.044e+03, threshold=8.010e+02, percent-clipped=1.0 2026-09-23 22:30:50,296 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=19726.666666666668, ans=0.0 2026-09-23 22:31:01,825 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=27.20 vs. limit=22.345 2026-09-23 22:31:06,307 INFO [train.py:1192] (1/2) Epoch 7, batch 200, loss[loss=0.4701, simple_loss=0.5312, pruned_loss=0.2045, over 24202.00 frames. ], tot_loss[loss=0.4068, simple_loss=0.474, pruned_loss=0.1698, over 3059918.73 frames. ], batch size: 257, lr: 2.51e-02, grad_scale: 32.0 2026-09-23 22:31:16,018 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=19893.333333333332, ans=0.006544927536231884 2026-09-23 22:31:29,428 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.04 vs. limit=22.47 2026-09-23 22:31:32,200 INFO [train.py:1192] (1/2) Epoch 7, batch 250, loss[loss=0.4855, simple_loss=0.5334, pruned_loss=0.2188, over 24373.00 frames. ], tot_loss[loss=0.4054, simple_loss=0.4726, pruned_loss=0.1691, over 3445413.72 frames. ], batch size: 225, lr: 2.51e-02, grad_scale: 32.0 2026-09-23 22:31:38,386 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.126e+02 3.140e+02 3.704e+02 4.295e+02 7.836e+02, threshold=7.407e+02, percent-clipped=0.0 2026-09-23 22:31:41,243 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:31:44,644 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=20060.0, ans=0.025 2026-09-23 22:31:50,721 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.61 vs. limit=8.0 2026-09-23 22:31:52,195 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=20126.666666666668, ans=0.125 2026-09-23 22:31:57,655 INFO [train.py:1192] (1/2) Epoch 7, batch 300, loss[loss=0.439, simple_loss=0.5148, pruned_loss=0.1817, over 24573.00 frames. ], tot_loss[loss=0.4035, simple_loss=0.4713, pruned_loss=0.1679, over 3758052.04 frames. ], batch size: 204, lr: 2.51e-02, grad_scale: 32.0 2026-09-23 22:32:03,351 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.24 vs. limit=22.5 2026-09-23 22:32:23,523 INFO [train.py:1192] (1/2) Epoch 7, batch 350, loss[loss=0.342, simple_loss=0.4113, pruned_loss=0.1364, over 24568.00 frames. ], tot_loss[loss=0.4034, simple_loss=0.4715, pruned_loss=0.1676, over 3998876.55 frames. ], batch size: 137, lr: 2.50e-02, grad_scale: 32.0 2026-09-23 22:32:23,616 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=20326.666666666668, ans=0.125 2026-09-23 22:32:29,790 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.460e+02 3.574e+02 4.263e+02 5.284e+02 1.007e+03, threshold=8.526e+02, percent-clipped=4.0 2026-09-23 22:32:32,277 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=20360.0, ans=0.0 2026-09-23 22:32:41,585 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=20426.666666666668, ans=0.0 2026-09-23 22:32:41,884 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.75 vs. limit=15.0 2026-09-23 22:32:48,471 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=20460.0, ans=0.006421739130434783 2026-09-23 22:32:49,309 INFO [train.py:1192] (1/2) Epoch 7, batch 400, loss[loss=0.3611, simple_loss=0.4438, pruned_loss=0.1392, over 24567.00 frames. ], tot_loss[loss=0.4009, simple_loss=0.4696, pruned_loss=0.166, over 4181291.05 frames. ], batch size: 170, lr: 2.50e-02, grad_scale: 32.0 2026-09-23 22:32:51,340 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=20493.333333333332, ans=0.0 2026-09-23 22:32:58,754 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=20526.666666666668, ans=0.125 2026-09-23 22:33:05,759 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=20593.333333333332, ans=0.1 2026-09-23 22:33:07,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=20593.333333333332, ans=0.125 2026-09-23 22:33:09,284 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.10 vs. limit=15.0 2026-09-23 22:33:15,873 INFO [train.py:1192] (1/2) Epoch 7, batch 450, loss[loss=0.4334, simple_loss=0.4925, pruned_loss=0.1872, over 24617.00 frames. ], tot_loss[loss=0.4023, simple_loss=0.4706, pruned_loss=0.167, over 4321080.77 frames. ], batch size: 175, lr: 2.49e-02, grad_scale: 16.0 2026-09-23 22:33:18,258 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=20660.0, ans=0.125 2026-09-23 22:33:22,351 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.542e+02 3.393e+02 3.897e+02 4.560e+02 8.616e+02, threshold=7.793e+02, percent-clipped=1.0 2026-09-23 22:33:22,432 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=20693.333333333332, ans=0.1 2026-09-23 22:33:24,211 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=20693.333333333332, ans=0.07 2026-09-23 22:33:29,029 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=20726.666666666668, ans=0.125 2026-09-23 22:33:41,212 INFO [train.py:1192] (1/2) Epoch 7, batch 500, loss[loss=0.4778, simple_loss=0.5355, pruned_loss=0.21, over 24519.00 frames. ], tot_loss[loss=0.3993, simple_loss=0.468, pruned_loss=0.1653, over 4438269.88 frames. ], batch size: 218, lr: 2.49e-02, grad_scale: 16.0 2026-09-23 22:33:41,318 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=20826.666666666668, ans=0.0 2026-09-23 22:33:42,573 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=20826.666666666668, ans=0.125 2026-09-23 22:33:47,122 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=20860.0, ans=0.006334782608695653 2026-09-23 22:33:59,541 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=20926.666666666668, ans=0.0 2026-09-23 22:34:01,933 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=20960.0, ans=0.2 2026-09-23 22:34:05,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=20960.0, ans=0.0 2026-09-23 22:34:06,362 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=20993.333333333332, ans=0.0 2026-09-23 22:34:06,699 INFO [train.py:1192] (1/2) Epoch 7, batch 550, loss[loss=0.4651, simple_loss=0.5288, pruned_loss=0.2007, over 24250.00 frames. ], tot_loss[loss=0.3989, simple_loss=0.4681, pruned_loss=0.1648, over 4523207.61 frames. ], batch size: 257, lr: 2.48e-02, grad_scale: 16.0 2026-09-23 22:34:07,736 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=20993.333333333332, ans=0.1 2026-09-23 22:34:12,701 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.497e+02 3.370e+02 3.713e+02 4.412e+02 6.548e+02, threshold=7.426e+02, percent-clipped=0.0 2026-09-23 22:34:17,665 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=21060.0, ans=0.125 2026-09-23 22:34:17,733 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=21060.0, ans=0.006291304347826087 2026-09-23 22:34:17,835 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.55 vs. limit=15.0 2026-09-23 22:34:21,115 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.96 vs. limit=15.0 2026-09-23 22:34:32,411 INFO [train.py:1192] (1/2) Epoch 7, batch 600, loss[loss=0.4232, simple_loss=0.5007, pruned_loss=0.1729, over 24332.00 frames. ], tot_loss[loss=0.3991, simple_loss=0.4685, pruned_loss=0.1648, over 4588710.85 frames. ], batch size: 234, lr: 2.48e-02, grad_scale: 16.0 2026-09-23 22:34:42,967 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=21226.666666666668, ans=0.015 2026-09-23 22:34:46,531 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=21226.666666666668, ans=0.125 2026-09-23 22:34:48,011 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=21260.0, ans=0.125 2026-09-23 22:34:49,122 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=21260.0, ans=0.125 2026-09-23 22:34:57,856 INFO [train.py:1192] (1/2) Epoch 7, batch 650, loss[loss=0.4226, simple_loss=0.4838, pruned_loss=0.1807, over 24585.00 frames. ], tot_loss[loss=0.396, simple_loss=0.4664, pruned_loss=0.1628, over 4653661.67 frames. ], batch size: 154, lr: 2.48e-02, grad_scale: 16.0 2026-09-23 22:34:59,336 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=21326.666666666668, ans=0.1 2026-09-23 22:35:02,699 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=21360.0, ans=0.125 2026-09-23 22:35:04,046 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.638e+02 3.513e+02 4.049e+02 4.785e+02 8.518e+02, threshold=8.098e+02, percent-clipped=6.0 2026-09-23 22:35:07,751 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=21393.333333333332, ans=0.125 2026-09-23 22:35:17,987 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=6.07 vs. limit=10.0 2026-09-23 22:35:18,816 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=21460.0, ans=0.0 2026-09-23 22:35:23,369 INFO [train.py:1192] (1/2) Epoch 7, batch 700, loss[loss=0.3557, simple_loss=0.4381, pruned_loss=0.1366, over 24557.00 frames. ], tot_loss[loss=0.3959, simple_loss=0.467, pruned_loss=0.1624, over 4689141.33 frames. ], batch size: 158, lr: 2.47e-02, grad_scale: 16.0 2026-09-23 22:35:28,173 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=21526.666666666668, ans=0.2 2026-09-23 22:35:36,203 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=21560.0, ans=0.125 2026-09-23 22:35:36,758 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=21560.0, ans=0.1 2026-09-23 22:35:46,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=21626.666666666668, ans=0.2 2026-09-23 22:35:49,343 INFO [train.py:1192] (1/2) Epoch 7, batch 750, loss[loss=0.4248, simple_loss=0.495, pruned_loss=0.1773, over 24620.00 frames. ], tot_loss[loss=0.3948, simple_loss=0.4659, pruned_loss=0.1618, over 4722688.30 frames. ], batch size: 175, lr: 2.47e-02, grad_scale: 16.0 2026-09-23 22:35:52,058 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.65 vs. limit=15.0 2026-09-23 22:35:56,130 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.721e+02 3.438e+02 3.905e+02 5.071e+02 8.170e+02, threshold=7.810e+02, percent-clipped=1.0 2026-09-23 22:36:01,102 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=21726.666666666668, ans=0.025 2026-09-23 22:36:15,761 INFO [train.py:1192] (1/2) Epoch 7, batch 800, loss[loss=0.3413, simple_loss=0.4156, pruned_loss=0.1335, over 24555.00 frames. ], tot_loss[loss=0.3944, simple_loss=0.4655, pruned_loss=0.1617, over 4750076.70 frames. ], batch size: 137, lr: 2.46e-02, grad_scale: 32.0 2026-09-23 22:36:17,211 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=21826.666666666668, ans=0.125 2026-09-23 22:36:18,747 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=21826.666666666668, ans=0.125 2026-09-23 22:36:24,461 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=21860.0, ans=0.1 2026-09-23 22:36:32,459 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=21926.666666666668, ans=0.006102898550724638 2026-09-23 22:36:41,623 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=21993.333333333332, ans=0.0 2026-09-23 22:36:41,927 INFO [train.py:1192] (1/2) Epoch 7, batch 850, loss[loss=0.4118, simple_loss=0.4895, pruned_loss=0.1671, over 24569.00 frames. ], tot_loss[loss=0.3935, simple_loss=0.4648, pruned_loss=0.1611, over 4770044.99 frames. ], batch size: 204, lr: 2.46e-02, grad_scale: 32.0 2026-09-23 22:36:45,096 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=21993.333333333332, ans=0.025 2026-09-23 22:36:49,250 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.337e+02 3.460e+02 3.975e+02 4.728e+02 8.722e+02, threshold=7.951e+02, percent-clipped=1.0 2026-09-23 22:36:58,094 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=22093.333333333332, ans=0.125 2026-09-23 22:37:08,211 INFO [train.py:1192] (1/2) Epoch 7, batch 900, loss[loss=0.3547, simple_loss=0.4304, pruned_loss=0.1395, over 24575.00 frames. ], tot_loss[loss=0.394, simple_loss=0.4651, pruned_loss=0.1615, over 4780916.31 frames. ], batch size: 137, lr: 2.46e-02, grad_scale: 32.0 2026-09-23 22:37:11,169 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.14 vs. limit=15.0 2026-09-23 22:37:26,309 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=22260.0, ans=0.0 2026-09-23 22:37:33,934 INFO [train.py:1192] (1/2) Epoch 7, batch 950, loss[loss=0.5473, simple_loss=0.5244, pruned_loss=0.2851, over 11169.00 frames. ], tot_loss[loss=0.3958, simple_loss=0.4641, pruned_loss=0.1637, over 4712552.76 frames. ], batch size: 334, lr: 2.45e-02, grad_scale: 32.0 2026-09-23 22:37:47,160 INFO [train.py:1192] (1/2) Epoch 8, batch 0, loss[loss=0.3428, simple_loss=0.4239, pruned_loss=0.1309, over 24572.00 frames. ], tot_loss[loss=0.3428, simple_loss=0.4239, pruned_loss=0.1309, over 24572.00 frames. ], batch size: 137, lr: 2.31e-02, grad_scale: 32.0 2026-09-23 22:37:47,160 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 22:37:58,738 INFO [train.py:1224] (1/2) Epoch 8, validation: loss=0.24, simple_loss=0.3477, pruned_loss=0.06622, over 2564189.00 frames. 2026-09-23 22:37:58,739 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 22:38:01,378 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.723e+02 3.734e+02 4.392e+02 5.202e+02 8.156e+02, threshold=8.783e+02, percent-clipped=1.0 2026-09-23 22:38:03,726 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=22386.666666666668, ans=0.025 2026-09-23 22:38:12,491 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.70 vs. limit=22.5 2026-09-23 22:38:23,097 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.23 vs. limit=15.0 2026-09-23 22:38:24,725 INFO [train.py:1192] (1/2) Epoch 8, batch 50, loss[loss=0.298, simple_loss=0.3815, pruned_loss=0.1073, over 24268.00 frames. ], tot_loss[loss=0.4036, simple_loss=0.4737, pruned_loss=0.1668, over 1082679.52 frames. ], batch size: 125, lr: 2.31e-02, grad_scale: 32.0 2026-09-23 22:38:25,230 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=22520.0, ans=0.125 2026-09-23 22:38:28,343 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.43 vs. limit=15.0 2026-09-23 22:38:50,850 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.91 vs. limit=15.0 2026-09-23 22:38:51,100 INFO [train.py:1192] (1/2) Epoch 8, batch 100, loss[loss=0.3608, simple_loss=0.4382, pruned_loss=0.1417, over 24603.00 frames. ], tot_loss[loss=0.4025, simple_loss=0.475, pruned_loss=0.165, over 1916440.83 frames. ], batch size: 154, lr: 2.30e-02, grad_scale: 32.0 2026-09-23 22:38:53,492 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.473e+02 3.371e+02 3.805e+02 4.450e+02 6.852e+02, threshold=7.610e+02, percent-clipped=0.0 2026-09-23 22:38:57,308 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=22720.0, ans=0.025 2026-09-23 22:39:16,974 INFO [train.py:1192] (1/2) Epoch 8, batch 150, loss[loss=0.3065, simple_loss=0.3842, pruned_loss=0.1144, over 24269.00 frames. ], tot_loss[loss=0.3964, simple_loss=0.4685, pruned_loss=0.1622, over 2560258.71 frames. ], batch size: 125, lr: 2.30e-02, grad_scale: 32.0 2026-09-23 22:39:20,019 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=22853.333333333332, ans=0.035 2026-09-23 22:39:30,272 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=22920.0, ans=0.0 2026-09-23 22:39:30,811 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=22920.0, ans=0.0 2026-09-23 22:39:32,925 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=22953.333333333332, ans=0.125 2026-09-23 22:39:42,294 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=22986.666666666668, ans=0.0 2026-09-23 22:39:43,721 INFO [train.py:1192] (1/2) Epoch 8, batch 200, loss[loss=0.46, simple_loss=0.5269, pruned_loss=0.1965, over 24220.00 frames. ], tot_loss[loss=0.3947, simple_loss=0.4665, pruned_loss=0.1614, over 3058813.35 frames. ], batch size: 257, lr: 2.29e-02, grad_scale: 32.0 2026-09-23 22:39:46,494 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.379e+02 3.268e+02 3.785e+02 4.404e+02 6.520e+02, threshold=7.571e+02, percent-clipped=0.0 2026-09-23 22:39:52,102 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.71 vs. limit=10.0 2026-09-23 22:39:52,936 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:40:02,206 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=23120.0, ans=0.125 2026-09-23 22:40:09,567 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=23186.666666666668, ans=0.005828985507246377 2026-09-23 22:40:09,925 INFO [train.py:1192] (1/2) Epoch 8, batch 250, loss[loss=0.4261, simple_loss=0.5045, pruned_loss=0.1738, over 24376.00 frames. ], tot_loss[loss=0.3924, simple_loss=0.4646, pruned_loss=0.1601, over 3443680.42 frames. ], batch size: 225, lr: 2.29e-02, grad_scale: 32.0 2026-09-23 22:40:13,471 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=23186.666666666668, ans=0.125 2026-09-23 22:40:22,932 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=23253.333333333332, ans=0.005814492753623188 2026-09-23 22:40:29,228 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=23286.666666666668, ans=0.125 2026-09-23 22:40:33,128 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=23320.0, ans=22.5 2026-09-23 22:40:35,626 INFO [train.py:1192] (1/2) Epoch 8, batch 300, loss[loss=0.4412, simple_loss=0.5109, pruned_loss=0.1857, over 24517.00 frames. ], tot_loss[loss=0.391, simple_loss=0.4635, pruned_loss=0.1593, over 3757751.11 frames. ], batch size: 204, lr: 2.29e-02, grad_scale: 32.0 2026-09-23 22:40:38,382 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.584e+02 3.199e+02 3.783e+02 4.797e+02 7.354e+02, threshold=7.566e+02, percent-clipped=0.0 2026-09-23 22:41:02,097 INFO [train.py:1192] (1/2) Epoch 8, batch 350, loss[loss=0.339, simple_loss=0.4094, pruned_loss=0.1343, over 24567.00 frames. ], tot_loss[loss=0.3907, simple_loss=0.4634, pruned_loss=0.1589, over 3998471.99 frames. ], batch size: 137, lr: 2.28e-02, grad_scale: 32.0 2026-09-23 22:41:07,979 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=23553.333333333332, ans=0.005749275362318841 2026-09-23 22:41:12,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=23586.666666666668, ans=0.2 2026-09-23 22:41:13,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=23586.666666666668, ans=0.025 2026-09-23 22:41:19,495 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=23620.0, ans=0.0 2026-09-23 22:41:19,508 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=23620.0, ans=0.125 2026-09-23 22:41:28,257 INFO [train.py:1192] (1/2) Epoch 8, batch 400, loss[loss=0.3773, simple_loss=0.4573, pruned_loss=0.1487, over 24576.00 frames. ], tot_loss[loss=0.3891, simple_loss=0.4623, pruned_loss=0.1579, over 4180363.42 frames. ], batch size: 170, lr: 2.28e-02, grad_scale: 32.0 2026-09-23 22:41:30,684 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.797e+02 3.606e+02 4.328e+02 5.043e+02 8.307e+02, threshold=8.655e+02, percent-clipped=3.0 2026-09-23 22:41:38,229 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=23753.333333333332, ans=0.1 2026-09-23 22:41:39,488 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.22 vs. limit=8.0 2026-09-23 22:41:46,193 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=23786.666666666668, ans=0.125 2026-09-23 22:41:53,883 INFO [train.py:1192] (1/2) Epoch 8, batch 450, loss[loss=0.3982, simple_loss=0.4771, pruned_loss=0.1597, over 24634.00 frames. ], tot_loss[loss=0.3891, simple_loss=0.4622, pruned_loss=0.158, over 4320258.85 frames. ], batch size: 175, lr: 2.28e-02, grad_scale: 32.0 2026-09-23 22:42:07,013 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=23920.0, ans=0.125 2026-09-23 22:42:07,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=23920.0, ans=0.125 2026-09-23 22:42:10,434 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=23953.333333333332, ans=0.0 2026-09-23 22:42:14,380 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=23953.333333333332, ans=0.1 2026-09-23 22:42:16,562 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=23986.666666666668, ans=0.005655072463768116 2026-09-23 22:42:20,492 INFO [train.py:1192] (1/2) Epoch 8, batch 500, loss[loss=0.4507, simple_loss=0.522, pruned_loss=0.1897, over 24532.00 frames. ], tot_loss[loss=0.388, simple_loss=0.461, pruned_loss=0.1575, over 4437813.25 frames. ], batch size: 218, lr: 2.27e-02, grad_scale: 32.0 2026-09-23 22:42:22,786 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.389e+02 3.416e+02 3.901e+02 4.503e+02 6.166e+02, threshold=7.802e+02, percent-clipped=0.0 2026-09-23 22:42:22,915 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:42:30,809 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=24086.666666666668, ans=0.0 2026-09-23 22:42:42,765 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.86 vs. limit=15.0 2026-09-23 22:42:43,139 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.03 vs. limit=6.0 2026-09-23 22:42:45,790 INFO [train.py:1192] (1/2) Epoch 8, batch 550, loss[loss=0.4059, simple_loss=0.488, pruned_loss=0.1619, over 24286.00 frames. ], tot_loss[loss=0.387, simple_loss=0.4607, pruned_loss=0.1567, over 4522651.26 frames. ], batch size: 257, lr: 2.27e-02, grad_scale: 32.0 2026-09-23 22:42:48,719 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=24186.666666666668, ans=0.125 2026-09-23 22:42:48,723 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=24186.666666666668, ans=0.125 2026-09-23 22:43:08,814 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=24320.0, ans=0.1 2026-09-23 22:43:12,066 INFO [train.py:1192] (1/2) Epoch 8, batch 600, loss[loss=0.4263, simple_loss=0.5091, pruned_loss=0.1718, over 24335.00 frames. ], tot_loss[loss=0.3872, simple_loss=0.4612, pruned_loss=0.1566, over 4590863.02 frames. ], batch size: 234, lr: 2.26e-02, grad_scale: 32.0 2026-09-23 22:43:13,678 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=24353.333333333332, ans=0.2 2026-09-23 22:43:14,633 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.462e+02 3.363e+02 3.921e+02 4.855e+02 7.268e+02, threshold=7.841e+02, percent-clipped=0.0 2026-09-23 22:43:24,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=24420.0, ans=0.125 2026-09-23 22:43:27,633 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=24453.333333333332, ans=0.0 2026-09-23 22:43:31,595 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=24453.333333333332, ans=0.0 2026-09-23 22:43:38,131 INFO [train.py:1192] (1/2) Epoch 8, batch 650, loss[loss=0.3767, simple_loss=0.4509, pruned_loss=0.1512, over 24597.00 frames. ], tot_loss[loss=0.3846, simple_loss=0.4593, pruned_loss=0.1549, over 4655309.05 frames. ], batch size: 154, lr: 2.26e-02, grad_scale: 32.0 2026-09-23 22:43:42,647 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=24553.333333333332, ans=0.125 2026-09-23 22:43:44,062 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=24553.333333333332, ans=0.125 2026-09-23 22:43:44,986 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=24553.333333333332, ans=0.125 2026-09-23 22:43:56,831 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=24620.0, ans=22.5 2026-09-23 22:44:03,746 INFO [train.py:1192] (1/2) Epoch 8, batch 700, loss[loss=0.3745, simple_loss=0.4516, pruned_loss=0.1487, over 24559.00 frames. ], tot_loss[loss=0.3856, simple_loss=0.4606, pruned_loss=0.1554, over 4689933.23 frames. ], batch size: 158, lr: 2.26e-02, grad_scale: 32.0 2026-09-23 22:44:06,126 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.679e+02 3.542e+02 4.120e+02 4.771e+02 6.786e+02, threshold=8.240e+02, percent-clipped=0.0 2026-09-23 22:44:12,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=24720.0, ans=15.0 2026-09-23 22:44:17,609 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=24753.333333333332, ans=0.0 2026-09-23 22:44:18,670 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=24786.666666666668, ans=0.1 2026-09-23 22:44:19,654 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=24786.666666666668, ans=0.125 2026-09-23 22:44:24,290 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=24820.0, ans=0.125 2026-09-23 22:44:25,870 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.40 vs. limit=6.0 2026-09-23 22:44:27,209 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=24820.0, ans=0.125 2026-09-23 22:44:29,780 INFO [train.py:1192] (1/2) Epoch 8, batch 750, loss[loss=0.3358, simple_loss=0.4352, pruned_loss=0.1183, over 24622.00 frames. ], tot_loss[loss=0.3852, simple_loss=0.4599, pruned_loss=0.1552, over 4726462.88 frames. ], batch size: 175, lr: 2.25e-02, grad_scale: 32.0 2026-09-23 22:44:33,547 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=24853.333333333332, ans=0.125 2026-09-23 22:44:39,318 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=24886.666666666668, ans=0.125 2026-09-23 22:44:45,552 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=24953.333333333332, ans=0.125 2026-09-23 22:44:56,098 INFO [train.py:1192] (1/2) Epoch 8, batch 800, loss[loss=0.3262, simple_loss=0.4062, pruned_loss=0.1231, over 24552.00 frames. ], tot_loss[loss=0.3848, simple_loss=0.4595, pruned_loss=0.155, over 4753209.28 frames. ], batch size: 137, lr: 2.25e-02, grad_scale: 32.0 2026-09-23 22:44:57,537 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=25020.0, ans=0.125 2026-09-23 22:44:58,378 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.531e+02 3.448e+02 4.026e+02 4.905e+02 8.713e+02, threshold=8.051e+02, percent-clipped=1.0 2026-09-23 22:45:09,556 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_abs, batch_count=25086.666666666668, ans=0.5 2026-09-23 22:45:21,923 INFO [train.py:1192] (1/2) Epoch 8, batch 850, loss[loss=0.4221, simple_loss=0.4961, pruned_loss=0.1741, over 24531.00 frames. ], tot_loss[loss=0.3834, simple_loss=0.4584, pruned_loss=0.1542, over 4770892.33 frames. ], batch size: 204, lr: 2.25e-02, grad_scale: 32.0 2026-09-23 22:45:22,037 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=25186.666666666668, ans=0.5 2026-09-23 22:45:24,133 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=25186.666666666668, ans=0.125 2026-09-23 22:45:35,418 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.65 vs. limit=15.0 2026-09-23 22:45:36,704 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=25253.333333333332, ans=0.0 2026-09-23 22:45:43,988 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=25320.0, ans=0.1 2026-09-23 22:45:48,073 INFO [train.py:1192] (1/2) Epoch 8, batch 900, loss[loss=0.3247, simple_loss=0.4095, pruned_loss=0.12, over 24538.00 frames. ], tot_loss[loss=0.3849, simple_loss=0.4595, pruned_loss=0.1551, over 4781962.67 frames. ], batch size: 137, lr: 2.24e-02, grad_scale: 32.0 2026-09-23 22:45:48,281 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.90 vs. limit=15.0 2026-09-23 22:45:50,754 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.644e+02 3.332e+02 3.860e+02 4.757e+02 7.777e+02, threshold=7.721e+02, percent-clipped=0.0 2026-09-23 22:45:51,916 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=25353.333333333332, ans=0.005357971014492754 2026-09-23 22:45:55,267 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=25386.666666666668, ans=0.125 2026-09-23 22:46:09,770 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.88 vs. limit=15.0 2026-09-23 22:46:10,504 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=25486.666666666668, ans=0.2 2026-09-23 22:46:10,950 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=25486.666666666668, ans=0.125 2026-09-23 22:46:11,532 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=25486.666666666668, ans=0.2 2026-09-23 22:46:12,193 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=8.72 vs. limit=12.0 2026-09-23 22:46:13,398 INFO [train.py:1192] (1/2) Epoch 8, batch 950, loss[loss=0.5083, simple_loss=0.4997, pruned_loss=0.2584, over 11051.00 frames. ], tot_loss[loss=0.386, simple_loss=0.4584, pruned_loss=0.1568, over 4715362.22 frames. ], batch size: 333, lr: 2.24e-02, grad_scale: 16.0 2026-09-23 22:46:14,627 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=14.66 vs. limit=15.0 2026-09-23 22:46:16,244 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=12.38 vs. limit=15.0 2026-09-23 22:46:25,543 INFO [train.py:1192] (1/2) Epoch 9, batch 0, loss[loss=0.3705, simple_loss=0.4449, pruned_loss=0.1481, over 24600.00 frames. ], tot_loss[loss=0.3705, simple_loss=0.4449, pruned_loss=0.1481, over 24600.00 frames. ], batch size: 137, lr: 2.12e-02, grad_scale: 32.0 2026-09-23 22:46:25,543 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 22:46:36,934 INFO [train.py:1224] (1/2) Epoch 9, validation: loss=0.2287, simple_loss=0.3401, pruned_loss=0.05867, over 2564189.00 frames. 2026-09-23 22:46:36,934 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 22:46:39,399 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=25546.666666666668, ans=0.125 2026-09-23 22:46:47,978 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.47 vs. limit=15.0 2026-09-23 22:46:56,405 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=25646.666666666668, ans=0.125 2026-09-23 22:46:59,055 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.30 vs. limit=15.0 2026-09-23 22:47:01,904 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.566e+02 3.336e+02 3.929e+02 5.395e+02 8.479e+02, threshold=7.858e+02, percent-clipped=1.0 2026-09-23 22:47:02,822 INFO [train.py:1192] (1/2) Epoch 9, batch 50, loss[loss=0.3141, simple_loss=0.3886, pruned_loss=0.1198, over 24304.00 frames. ], tot_loss[loss=0.3903, simple_loss=0.4649, pruned_loss=0.1579, over 1081961.25 frames. ], batch size: 125, lr: 2.11e-02, grad_scale: 32.0 2026-09-23 22:47:02,937 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=25713.333333333332, ans=0.07 2026-09-23 22:47:03,425 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=25713.333333333332, ans=0.005279710144927537 2026-09-23 22:47:13,445 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=25780.0, ans=0.0 2026-09-23 22:47:14,442 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=25780.0, ans=0.005265217391304347 2026-09-23 22:47:15,030 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.34 vs. limit=15.0 2026-09-23 22:47:21,554 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.70 vs. limit=15.0 2026-09-23 22:47:22,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=25813.333333333332, ans=0.0 2026-09-23 22:47:28,493 INFO [train.py:1192] (1/2) Epoch 9, batch 100, loss[loss=0.3295, simple_loss=0.4219, pruned_loss=0.1186, over 24607.00 frames. ], tot_loss[loss=0.3901, simple_loss=0.4667, pruned_loss=0.1568, over 1915947.78 frames. ], batch size: 154, lr: 2.11e-02, grad_scale: 32.0 2026-09-23 22:47:38,004 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=25913.333333333332, ans=0.125 2026-09-23 22:47:41,014 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=25946.666666666668, ans=0.125 2026-09-23 22:47:53,081 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.515e+02 3.359e+02 3.717e+02 4.379e+02 7.313e+02, threshold=7.434e+02, percent-clipped=0.0 2026-09-23 22:47:54,228 INFO [train.py:1192] (1/2) Epoch 9, batch 150, loss[loss=0.3158, simple_loss=0.3987, pruned_loss=0.1165, over 24250.00 frames. ], tot_loss[loss=0.3832, simple_loss=0.4596, pruned_loss=0.1534, over 2561021.91 frames. ], batch size: 125, lr: 2.11e-02, grad_scale: 32.0 2026-09-23 22:47:55,420 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=26046.666666666668, ans=0.125 2026-09-23 22:47:55,434 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=26046.666666666668, ans=0.125 2026-09-23 22:47:56,474 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=26046.666666666668, ans=0.125 2026-09-23 22:48:02,559 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=26080.0, ans=0.125 2026-09-23 22:48:03,520 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=26080.0, ans=0.125 2026-09-23 22:48:09,053 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=26113.333333333332, ans=0.125 2026-09-23 22:48:20,208 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=26180.0, ans=0.2 2026-09-23 22:48:21,177 INFO [train.py:1192] (1/2) Epoch 9, batch 200, loss[loss=0.4688, simple_loss=0.5361, pruned_loss=0.2007, over 24210.00 frames. ], tot_loss[loss=0.382, simple_loss=0.458, pruned_loss=0.153, over 3059732.49 frames. ], batch size: 257, lr: 2.10e-02, grad_scale: 32.0 2026-09-23 22:48:27,407 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.66 vs. limit=15.0 2026-09-23 22:48:27,997 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=15.01 vs. limit=22.5 2026-09-23 22:48:46,578 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.433e+02 3.475e+02 4.252e+02 4.980e+02 8.416e+02, threshold=8.504e+02, percent-clipped=4.0 2026-09-23 22:48:47,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=26380.0, ans=0.125 2026-09-23 22:48:47,546 INFO [train.py:1192] (1/2) Epoch 9, batch 250, loss[loss=0.4092, simple_loss=0.492, pruned_loss=0.1632, over 24380.00 frames. ], tot_loss[loss=0.3824, simple_loss=0.4583, pruned_loss=0.1532, over 3444530.68 frames. ], batch size: 225, lr: 2.10e-02, grad_scale: 32.0 2026-09-23 22:48:52,102 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=26380.0, ans=0.0 2026-09-23 22:48:58,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=26446.666666666668, ans=0.0 2026-09-23 22:48:59,303 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.87 vs. limit=15.0 2026-09-23 22:48:59,627 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=26446.666666666668, ans=0.125 2026-09-23 22:49:06,535 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=26480.0, ans=0.125 2026-09-23 22:49:12,083 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=26513.333333333332, ans=0.125 2026-09-23 22:49:13,458 INFO [train.py:1192] (1/2) Epoch 9, batch 300, loss[loss=0.3723, simple_loss=0.4646, pruned_loss=0.14, over 24532.00 frames. ], tot_loss[loss=0.3792, simple_loss=0.4559, pruned_loss=0.1513, over 3757664.01 frames. ], batch size: 204, lr: 2.10e-02, grad_scale: 32.0 2026-09-23 22:49:22,674 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=26580.0, ans=0.125 2026-09-23 22:49:23,254 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=26580.0, ans=0.125 2026-09-23 22:49:27,630 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=26613.333333333332, ans=0.125 2026-09-23 22:49:27,755 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.52 vs. limit=15.0 2026-09-23 22:49:28,048 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=26613.333333333332, ans=0.04949747468305833 2026-09-23 22:49:30,556 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=26646.666666666668, ans=0.0 2026-09-23 22:49:37,780 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=26680.0, ans=0.125 2026-09-23 22:49:39,518 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.408e+02 3.428e+02 3.972e+02 4.632e+02 6.980e+02, threshold=7.944e+02, percent-clipped=0.0 2026-09-23 22:49:39,621 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=26680.0, ans=0.125 2026-09-23 22:49:40,484 INFO [train.py:1192] (1/2) Epoch 9, batch 350, loss[loss=0.3348, simple_loss=0.411, pruned_loss=0.1293, over 24591.00 frames. ], tot_loss[loss=0.3799, simple_loss=0.4567, pruned_loss=0.1516, over 3998199.70 frames. ], batch size: 137, lr: 2.09e-02, grad_scale: 32.0 2026-09-23 22:49:43,289 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.21 vs. limit=10.0 2026-09-23 22:49:49,361 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=26746.666666666668, ans=0.09899494936611666 2026-09-23 22:49:51,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=26780.0, ans=0.005047826086956522 2026-09-23 22:49:57,501 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=26813.333333333332, ans=0.125 2026-09-23 22:49:59,359 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=26813.333333333332, ans=15.0 2026-09-23 22:50:03,466 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=26846.666666666668, ans=0.2 2026-09-23 22:50:06,328 INFO [train.py:1192] (1/2) Epoch 9, batch 400, loss[loss=0.3724, simple_loss=0.4479, pruned_loss=0.1484, over 24566.00 frames. ], tot_loss[loss=0.378, simple_loss=0.4553, pruned_loss=0.1503, over 4181053.67 frames. ], batch size: 170, lr: 2.09e-02, grad_scale: 32.0 2026-09-23 22:50:16,793 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=26946.666666666668, ans=0.005011594202898551 2026-09-23 22:50:24,776 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=26980.0, ans=0.125 2026-09-23 22:50:31,474 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.095e+02 3.561e+02 4.305e+02 5.055e+02 8.003e+02, threshold=8.611e+02, percent-clipped=0.0 2026-09-23 22:50:32,501 INFO [train.py:1192] (1/2) Epoch 9, batch 450, loss[loss=0.4009, simple_loss=0.4771, pruned_loss=0.1624, over 24612.00 frames. ], tot_loss[loss=0.3776, simple_loss=0.4552, pruned_loss=0.15, over 4321328.57 frames. ], batch size: 175, lr: 2.09e-02, grad_scale: 32.0 2026-09-23 22:50:37,542 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=27080.0, ans=0.125 2026-09-23 22:50:39,353 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.80 vs. limit=15.0 2026-09-23 22:50:43,152 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=27113.333333333332, ans=0.1 2026-09-23 22:50:53,536 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:50:58,331 INFO [train.py:1192] (1/2) Epoch 9, batch 500, loss[loss=0.4103, simple_loss=0.4875, pruned_loss=0.1665, over 24514.00 frames. ], tot_loss[loss=0.3757, simple_loss=0.4532, pruned_loss=0.1491, over 4438796.26 frames. ], batch size: 218, lr: 2.08e-02, grad_scale: 32.0 2026-09-23 22:51:00,370 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=27213.333333333332, ans=0.125 2026-09-23 22:51:06,084 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=27246.666666666668, ans=0.2 2026-09-23 22:51:18,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=27313.333333333332, ans=0.125 2026-09-23 22:51:23,574 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.367e+02 3.132e+02 3.739e+02 4.190e+02 8.139e+02, threshold=7.479e+02, percent-clipped=0.0 2026-09-23 22:51:24,488 INFO [train.py:1192] (1/2) Epoch 9, batch 550, loss[loss=0.3989, simple_loss=0.4919, pruned_loss=0.1529, over 24281.00 frames. ], tot_loss[loss=0.3755, simple_loss=0.4534, pruned_loss=0.1489, over 4523047.86 frames. ], batch size: 257, lr: 2.08e-02, grad_scale: 32.0 2026-09-23 22:51:24,944 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=3.97 vs. limit=5.0 2026-09-23 22:51:28,477 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=27380.0, ans=0.1 2026-09-23 22:51:42,896 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=27480.0, ans=0.0 2026-09-23 22:51:46,790 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=27513.333333333332, ans=0.125 2026-09-23 22:51:48,844 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=27513.333333333332, ans=0.125 2026-09-23 22:51:50,840 INFO [train.py:1192] (1/2) Epoch 9, batch 600, loss[loss=0.4336, simple_loss=0.5054, pruned_loss=0.1809, over 24318.00 frames. ], tot_loss[loss=0.3767, simple_loss=0.4545, pruned_loss=0.1494, over 4589538.08 frames. ], batch size: 234, lr: 2.08e-02, grad_scale: 32.0 2026-09-23 22:51:58,823 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=27580.0, ans=0.004873913043478261 2026-09-23 22:52:02,442 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=27613.333333333332, ans=0.07 2026-09-23 22:52:07,701 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=27646.666666666668, ans=0.125 2026-09-23 22:52:14,896 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=27680.0, ans=0.025 2026-09-23 22:52:15,352 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=27680.0, ans=0.004852173913043478 2026-09-23 22:52:15,744 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.679e+02 3.351e+02 3.948e+02 4.727e+02 8.005e+02, threshold=7.895e+02, percent-clipped=1.0 2026-09-23 22:52:15,873 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=27680.0, ans=0.2 2026-09-23 22:52:16,662 INFO [train.py:1192] (1/2) Epoch 9, batch 650, loss[loss=0.3162, simple_loss=0.4091, pruned_loss=0.1117, over 24604.00 frames. ], tot_loss[loss=0.3748, simple_loss=0.4529, pruned_loss=0.1484, over 4654178.23 frames. ], batch size: 154, lr: 2.07e-02, grad_scale: 32.0 2026-09-23 22:52:24,464 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=27746.666666666668, ans=0.125 2026-09-23 22:52:27,230 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=27780.0, ans=0.0048304347826086955 2026-09-23 22:52:40,818 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=27846.666666666668, ans=0.125 2026-09-23 22:52:43,732 INFO [train.py:1192] (1/2) Epoch 9, batch 700, loss[loss=0.3932, simple_loss=0.4644, pruned_loss=0.1609, over 24555.00 frames. ], tot_loss[loss=0.3753, simple_loss=0.4539, pruned_loss=0.1484, over 4689368.37 frames. ], batch size: 158, lr: 2.07e-02, grad_scale: 32.0 2026-09-23 22:52:51,866 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=27913.333333333332, ans=0.125 2026-09-23 22:53:08,732 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=28013.333333333332, ans=0.0 2026-09-23 22:53:09,685 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.443e+02 3.501e+02 4.089e+02 5.092e+02 1.078e+03, threshold=8.179e+02, percent-clipped=2.0 2026-09-23 22:53:10,787 INFO [train.py:1192] (1/2) Epoch 9, batch 750, loss[loss=0.3635, simple_loss=0.4486, pruned_loss=0.1392, over 24631.00 frames. ], tot_loss[loss=0.3738, simple_loss=0.4524, pruned_loss=0.1476, over 4726018.04 frames. ], batch size: 175, lr: 2.07e-02, grad_scale: 32.0 2026-09-23 22:53:15,306 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=28080.0, ans=0.125 2026-09-23 22:53:20,276 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=28080.0, ans=0.035 2026-09-23 22:53:22,839 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=28113.333333333332, ans=0.004757971014492754 2026-09-23 22:53:36,312 INFO [train.py:1192] (1/2) Epoch 9, batch 800, loss[loss=0.28, simple_loss=0.3753, pruned_loss=0.09237, over 24547.00 frames. ], tot_loss[loss=0.3716, simple_loss=0.451, pruned_loss=0.1461, over 4752085.88 frames. ], batch size: 137, lr: 2.06e-02, grad_scale: 32.0 2026-09-23 22:53:36,901 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=28213.333333333332, ans=0.125 2026-09-23 22:53:44,968 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=28246.666666666668, ans=0.04949747468305833 2026-09-23 22:53:52,406 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=28313.333333333332, ans=0.004714492753623189 2026-09-23 22:54:01,809 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.663e+02 3.282e+02 3.877e+02 4.476e+02 6.858e+02, threshold=7.755e+02, percent-clipped=0.0 2026-09-23 22:54:03,007 INFO [train.py:1192] (1/2) Epoch 9, batch 850, loss[loss=0.3718, simple_loss=0.4608, pruned_loss=0.1414, over 24536.00 frames. ], tot_loss[loss=0.3701, simple_loss=0.4498, pruned_loss=0.1452, over 4771371.06 frames. ], batch size: 204, lr: 2.06e-02, grad_scale: 32.0 2026-09-23 22:54:19,908 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.65 vs. limit=10.0 2026-09-23 22:54:23,630 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:54:26,400 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=28513.333333333332, ans=0.1 2026-09-23 22:54:29,022 INFO [train.py:1192] (1/2) Epoch 9, batch 900, loss[loss=0.2843, simple_loss=0.3852, pruned_loss=0.09171, over 24544.00 frames. ], tot_loss[loss=0.3703, simple_loss=0.45, pruned_loss=0.1453, over 4782121.75 frames. ], batch size: 137, lr: 2.06e-02, grad_scale: 32.0 2026-09-23 22:54:36,086 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=28580.0, ans=0.125 2026-09-23 22:54:47,156 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=28646.666666666668, ans=0.1 2026-09-23 22:54:53,812 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.503e+02 3.409e+02 3.820e+02 4.682e+02 1.064e+03, threshold=7.640e+02, percent-clipped=2.0 2026-09-23 22:54:54,329 INFO [train.py:1192] (1/2) Epoch 9, batch 950, loss[loss=0.5978, simple_loss=0.5613, pruned_loss=0.3172, over 11463.00 frames. ], tot_loss[loss=0.3732, simple_loss=0.4504, pruned_loss=0.148, over 4708049.23 frames. ], batch size: 334, lr: 2.05e-02, grad_scale: 16.0 2026-09-23 22:55:05,757 INFO [train.py:1192] (1/2) Epoch 10, batch 0, loss[loss=0.344, simple_loss=0.4344, pruned_loss=0.1268, over 24588.00 frames. ], tot_loss[loss=0.344, simple_loss=0.4344, pruned_loss=0.1268, over 24588.00 frames. ], batch size: 137, lr: 1.95e-02, grad_scale: 32.0 2026-09-23 22:55:05,757 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 22:55:07,395 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.9134, 2.7406, 3.4743, 1.8308], device='cuda:1') 2026-09-23 22:55:17,320 INFO [train.py:1224] (1/2) Epoch 10, validation: loss=0.2259, simple_loss=0.3365, pruned_loss=0.05759, over 2564189.00 frames. 2026-09-23 22:55:17,320 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 22:55:19,677 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=28740.0, ans=0.125 2026-09-23 22:55:24,667 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.90 vs. limit=22.5 2026-09-23 22:55:42,657 INFO [train.py:1192] (1/2) Epoch 10, batch 50, loss[loss=0.3076, simple_loss=0.3899, pruned_loss=0.1126, over 24210.00 frames. ], tot_loss[loss=0.383, simple_loss=0.4604, pruned_loss=0.1528, over 1082384.96 frames. ], batch size: 125, lr: 1.95e-02, grad_scale: 32.0 2026-09-23 22:55:44,049 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=28906.666666666668, ans=0.04949747468305833 2026-09-23 22:55:44,548 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=28906.666666666668, ans=0.125 2026-09-23 22:55:55,947 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=28973.333333333332, ans=0.0 2026-09-23 22:55:56,950 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=28973.333333333332, ans=0.125 2026-09-23 22:56:03,690 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.525e+02 3.405e+02 3.972e+02 4.546e+02 6.978e+02, threshold=7.944e+02, percent-clipped=0.0 2026-09-23 22:56:08,279 INFO [train.py:1192] (1/2) Epoch 10, batch 100, loss[loss=0.3533, simple_loss=0.4299, pruned_loss=0.1383, over 24614.00 frames. ], tot_loss[loss=0.3813, simple_loss=0.461, pruned_loss=0.1508, over 1914410.88 frames. ], batch size: 154, lr: 1.94e-02, grad_scale: 32.0 2026-09-23 22:56:21,192 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=29140.0, ans=0.0 2026-09-23 22:56:21,935 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.83 vs. limit=10.0 2026-09-23 22:56:30,467 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=29206.666666666668, ans=0.2 2026-09-23 22:56:32,390 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=29206.666666666668, ans=0.125 2026-09-23 22:56:34,220 INFO [train.py:1192] (1/2) Epoch 10, batch 150, loss[loss=0.2903, simple_loss=0.369, pruned_loss=0.1058, over 24236.00 frames. ], tot_loss[loss=0.3748, simple_loss=0.4545, pruned_loss=0.1476, over 2559764.82 frames. ], batch size: 125, lr: 1.94e-02, grad_scale: 32.0 2026-09-23 22:56:44,061 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.61 vs. limit=12.0 2026-09-23 22:56:47,746 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=29306.666666666668, ans=0.125 2026-09-23 22:56:55,828 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.371e+02 3.224e+02 3.571e+02 4.187e+02 7.105e+02, threshold=7.141e+02, percent-clipped=0.0 2026-09-23 22:56:57,865 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=29373.333333333332, ans=0.0 2026-09-23 22:57:00,263 INFO [train.py:1192] (1/2) Epoch 10, batch 200, loss[loss=0.4326, simple_loss=0.5101, pruned_loss=0.1776, over 24239.00 frames. ], tot_loss[loss=0.3729, simple_loss=0.4529, pruned_loss=0.1465, over 3059021.20 frames. ], batch size: 257, lr: 1.94e-02, grad_scale: 32.0 2026-09-23 22:57:01,995 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=29406.666666666668, ans=0.025 2026-09-23 22:57:08,059 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 22:57:26,439 INFO [train.py:1192] (1/2) Epoch 10, batch 250, loss[loss=0.3921, simple_loss=0.4844, pruned_loss=0.1499, over 24357.00 frames. ], tot_loss[loss=0.3723, simple_loss=0.452, pruned_loss=0.1463, over 3442566.73 frames. ], batch size: 225, lr: 1.93e-02, grad_scale: 32.0 2026-09-23 22:57:32,294 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=29606.666666666668, ans=0.1 2026-09-23 22:57:37,252 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=29640.0, ans=0.05 2026-09-23 22:57:40,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=29640.0, ans=0.125 2026-09-23 22:57:48,447 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.468e+02 3.253e+02 3.914e+02 4.692e+02 8.196e+02, threshold=7.827e+02, percent-clipped=1.0 2026-09-23 22:57:49,953 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=29706.666666666668, ans=0.125 2026-09-23 22:57:52,396 INFO [train.py:1192] (1/2) Epoch 10, batch 300, loss[loss=0.4074, simple_loss=0.4882, pruned_loss=0.1633, over 24541.00 frames. ], tot_loss[loss=0.3706, simple_loss=0.4505, pruned_loss=0.1453, over 3752442.34 frames. ], batch size: 204, lr: 1.93e-02, grad_scale: 16.0 2026-09-23 22:58:09,258 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=29840.0, ans=0.125 2026-09-23 22:58:16,658 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=29873.333333333332, ans=0.125 2026-09-23 22:58:18,452 INFO [train.py:1192] (1/2) Epoch 10, batch 350, loss[loss=0.3174, simple_loss=0.4035, pruned_loss=0.1156, over 24572.00 frames. ], tot_loss[loss=0.3692, simple_loss=0.4497, pruned_loss=0.1443, over 3994845.07 frames. ], batch size: 137, lr: 1.93e-02, grad_scale: 16.0 2026-09-23 22:58:22,409 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=29906.666666666668, ans=0.125 2026-09-23 22:58:25,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=29940.0, ans=0.125 2026-09-23 22:58:25,567 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=29940.0, ans=0.004360869565217392 2026-09-23 22:58:34,124 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=30006.666666666668, ans=0.125 2026-09-23 22:58:35,598 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=30006.666666666668, ans=0.05 2026-09-23 22:58:40,740 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.483e+02 3.273e+02 3.815e+02 4.612e+02 7.991e+02, threshold=7.629e+02, percent-clipped=1.0 2026-09-23 22:58:45,210 INFO [train.py:1192] (1/2) Epoch 10, batch 400, loss[loss=0.4157, simple_loss=0.481, pruned_loss=0.1751, over 24579.00 frames. ], tot_loss[loss=0.3694, simple_loss=0.4495, pruned_loss=0.1446, over 4177165.65 frames. ], batch size: 170, lr: 1.93e-02, grad_scale: 32.0 2026-09-23 22:58:46,746 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=30073.333333333332, ans=0.1 2026-09-23 22:58:48,692 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=30073.333333333332, ans=0.1 2026-09-23 22:58:51,557 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=30106.666666666668, ans=0.125 2026-09-23 22:58:56,302 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=30140.0, ans=0.125 2026-09-23 22:59:03,088 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=30173.333333333332, ans=0.0 2026-09-23 22:59:06,446 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=30206.666666666668, ans=0.0 2026-09-23 22:59:10,908 INFO [train.py:1192] (1/2) Epoch 10, batch 450, loss[loss=0.3716, simple_loss=0.4603, pruned_loss=0.1415, over 24618.00 frames. ], tot_loss[loss=0.3699, simple_loss=0.4498, pruned_loss=0.145, over 4317630.13 frames. ], batch size: 175, lr: 1.92e-02, grad_scale: 32.0 2026-09-23 22:59:22,128 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=30306.666666666668, ans=0.2 2026-09-23 22:59:24,011 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=30306.666666666668, ans=0.125 2026-09-23 22:59:26,880 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_positive, batch_count=30340.0, ans=0.05 2026-09-23 22:59:32,870 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.566e+02 3.169e+02 3.862e+02 4.320e+02 6.201e+02, threshold=7.724e+02, percent-clipped=0.0 2026-09-23 22:59:36,996 INFO [train.py:1192] (1/2) Epoch 10, batch 500, loss[loss=0.408, simple_loss=0.4923, pruned_loss=0.1619, over 24523.00 frames. ], tot_loss[loss=0.3676, simple_loss=0.448, pruned_loss=0.1437, over 4435724.01 frames. ], batch size: 218, lr: 1.92e-02, grad_scale: 32.0 2026-09-23 22:59:39,572 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.76 vs. limit=12.0 2026-09-23 22:59:47,419 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.63 vs. limit=15.0 2026-09-23 22:59:50,271 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=30473.333333333332, ans=0.1 2026-09-23 23:00:02,652 INFO [train.py:1192] (1/2) Epoch 10, batch 550, loss[loss=0.4388, simple_loss=0.5118, pruned_loss=0.1829, over 24293.00 frames. ], tot_loss[loss=0.3681, simple_loss=0.4485, pruned_loss=0.1438, over 4521375.65 frames. ], batch size: 257, lr: 1.92e-02, grad_scale: 32.0 2026-09-23 23:00:12,583 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=30606.666666666668, ans=0.125 2026-09-23 23:00:13,852 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.78 vs. limit=22.5 2026-09-23 23:00:20,460 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.30 vs. limit=15.0 2026-09-23 23:00:24,965 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.548e+02 3.379e+02 3.947e+02 4.533e+02 6.325e+02, threshold=7.895e+02, percent-clipped=0.0 2026-09-23 23:00:28,903 INFO [train.py:1192] (1/2) Epoch 10, batch 600, loss[loss=0.3772, simple_loss=0.4634, pruned_loss=0.1454, over 24328.00 frames. ], tot_loss[loss=0.367, simple_loss=0.448, pruned_loss=0.143, over 4587402.58 frames. ], batch size: 234, lr: 1.91e-02, grad_scale: 32.0 2026-09-23 23:00:39,717 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=30806.666666666668, ans=0.125 2026-09-23 23:00:46,239 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=30840.0, ans=0.125 2026-09-23 23:00:54,807 INFO [train.py:1192] (1/2) Epoch 10, batch 650, loss[loss=0.3824, simple_loss=0.4524, pruned_loss=0.1562, over 24587.00 frames. ], tot_loss[loss=0.3641, simple_loss=0.4459, pruned_loss=0.1411, over 4652703.06 frames. ], batch size: 154, lr: 1.91e-02, grad_scale: 32.0 2026-09-23 23:00:54,940 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=30906.666666666668, ans=0.125 2026-09-23 23:01:05,512 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=30973.333333333332, ans=0.2 2026-09-23 23:01:13,335 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=31006.666666666668, ans=0.1 2026-09-23 23:01:14,387 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=31006.666666666668, ans=0.04949747468305833 2026-09-23 23:01:16,593 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.511e+02 3.233e+02 3.609e+02 4.094e+02 7.383e+02, threshold=7.217e+02, percent-clipped=0.0 2026-09-23 23:01:20,541 INFO [train.py:1192] (1/2) Epoch 10, batch 700, loss[loss=0.3655, simple_loss=0.4445, pruned_loss=0.1433, over 24557.00 frames. ], tot_loss[loss=0.3634, simple_loss=0.446, pruned_loss=0.1404, over 4688394.72 frames. ], batch size: 158, lr: 1.91e-02, grad_scale: 32.0 2026-09-23 23:01:32,773 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=31140.0, ans=0.0041 2026-09-23 23:01:43,958 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=31206.666666666668, ans=0.004085507246376812 2026-09-23 23:01:44,436 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=31206.666666666668, ans=0.125 2026-09-23 23:01:46,766 INFO [train.py:1192] (1/2) Epoch 10, batch 750, loss[loss=0.391, simple_loss=0.4713, pruned_loss=0.1554, over 24640.00 frames. ], tot_loss[loss=0.3622, simple_loss=0.4448, pruned_loss=0.1398, over 4721470.68 frames. ], batch size: 175, lr: 1.90e-02, grad_scale: 32.0 2026-09-23 23:01:46,865 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=31240.0, ans=0.004078260869565218 2026-09-23 23:01:48,490 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=31240.0, ans=0.1 2026-09-23 23:01:48,937 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=31240.0, ans=0.1 2026-09-23 23:01:56,917 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.00 vs. limit=10.0 2026-09-23 23:01:58,064 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=31306.666666666668, ans=0.025 2026-09-23 23:02:08,439 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.729e+02 3.541e+02 4.303e+02 5.012e+02 8.664e+02, threshold=8.605e+02, percent-clipped=3.0 2026-09-23 23:02:10,265 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=31373.333333333332, ans=0.0 2026-09-23 23:02:12,653 INFO [train.py:1192] (1/2) Epoch 10, batch 800, loss[loss=0.2965, simple_loss=0.3894, pruned_loss=0.1018, over 24514.00 frames. ], tot_loss[loss=0.3622, simple_loss=0.4447, pruned_loss=0.1398, over 4748397.11 frames. ], batch size: 137, lr: 1.90e-02, grad_scale: 32.0 2026-09-23 23:02:19,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=31440.0, ans=0.1 2026-09-23 23:02:22,643 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=31473.333333333332, ans=0.2 2026-09-23 23:02:27,414 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:02:31,586 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=31506.666666666668, ans=0.0 2026-09-23 23:02:32,576 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=31540.0, ans=0.125 2026-09-23 23:02:38,480 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=31573.333333333332, ans=0.125 2026-09-23 23:02:38,833 INFO [train.py:1192] (1/2) Epoch 10, batch 850, loss[loss=0.411, simple_loss=0.4896, pruned_loss=0.1662, over 24549.00 frames. ], tot_loss[loss=0.3615, simple_loss=0.4442, pruned_loss=0.1394, over 4767572.72 frames. ], batch size: 204, lr: 1.90e-02, grad_scale: 16.0 2026-09-23 23:02:52,759 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=31640.0, ans=0.0 2026-09-23 23:02:53,680 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=31673.333333333332, ans=0.025 2026-09-23 23:02:54,801 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.97 vs. limit=10.0 2026-09-23 23:02:57,452 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=31673.333333333332, ans=0.125 2026-09-23 23:03:00,659 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 3.449e+02 3.836e+02 4.528e+02 6.517e+02, threshold=7.672e+02, percent-clipped=0.0 2026-09-23 23:03:04,646 INFO [train.py:1192] (1/2) Epoch 10, batch 900, loss[loss=0.3151, simple_loss=0.4049, pruned_loss=0.1127, over 24567.00 frames. ], tot_loss[loss=0.3617, simple_loss=0.4444, pruned_loss=0.1395, over 4779233.95 frames. ], batch size: 137, lr: 1.89e-02, grad_scale: 16.0 2026-09-23 23:03:07,880 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=31740.0, ans=0.125 2026-09-23 23:03:12,532 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=31773.333333333332, ans=0.125 2026-09-23 23:03:14,580 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.01 vs. limit=15.0 2026-09-23 23:03:29,379 INFO [train.py:1192] (1/2) Epoch 10, batch 950, loss[loss=0.5149, simple_loss=0.4979, pruned_loss=0.266, over 11581.00 frames. ], tot_loss[loss=0.3642, simple_loss=0.4441, pruned_loss=0.1421, over 4709688.41 frames. ], batch size: 333, lr: 1.89e-02, grad_scale: 16.0 2026-09-23 23:03:41,400 INFO [train.py:1192] (1/2) Epoch 11, batch 0, loss[loss=0.3152, simple_loss=0.4084, pruned_loss=0.111, over 24568.00 frames. ], tot_loss[loss=0.3152, simple_loss=0.4084, pruned_loss=0.111, over 24568.00 frames. ], batch size: 137, lr: 1.81e-02, grad_scale: 32.0 2026-09-23 23:03:41,400 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 23:03:53,080 INFO [train.py:1224] (1/2) Epoch 11, validation: loss=0.2194, simple_loss=0.3319, pruned_loss=0.05341, over 2564189.00 frames. 2026-09-23 23:03:53,080 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 23:03:56,890 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=17.72 vs. limit=22.5 2026-09-23 23:04:03,841 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=32000.0, ans=0.125 2026-09-23 23:04:05,829 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=32000.0, ans=0.125 2026-09-23 23:04:07,175 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=32000.0, ans=0.0 2026-09-23 23:04:11,359 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.597e+02 3.569e+02 4.127e+02 4.841e+02 1.254e+03, threshold=8.254e+02, percent-clipped=4.0 2026-09-23 23:04:18,729 INFO [train.py:1192] (1/2) Epoch 11, batch 50, loss[loss=0.291, simple_loss=0.3833, pruned_loss=0.09934, over 24238.00 frames. ], tot_loss[loss=0.3708, simple_loss=0.4522, pruned_loss=0.1447, over 1081445.50 frames. ], batch size: 125, lr: 1.80e-02, grad_scale: 32.0 2026-09-23 23:04:19,651 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=32100.0, ans=0.025 2026-09-23 23:04:25,374 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=32133.333333333332, ans=0.125 2026-09-23 23:04:32,253 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=32166.666666666668, ans=0.1 2026-09-23 23:04:33,478 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer_ff3.min_abs, batch_count=32166.666666666668, ans=0.2 2026-09-23 23:04:34,179 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.29 vs. limit=8.0 2026-09-23 23:04:39,298 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=32233.333333333332, ans=0.125 2026-09-23 23:04:44,054 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=32266.666666666668, ans=0.125 2026-09-23 23:04:44,364 INFO [train.py:1192] (1/2) Epoch 11, batch 100, loss[loss=0.3259, simple_loss=0.4162, pruned_loss=0.1178, over 24585.00 frames. ], tot_loss[loss=0.3707, simple_loss=0.4542, pruned_loss=0.1436, over 1915710.99 frames. ], batch size: 154, lr: 1.80e-02, grad_scale: 32.0 2026-09-23 23:04:45,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=32266.666666666668, ans=0.95 2026-09-23 23:04:46,210 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.93 vs. limit=15.0 2026-09-23 23:05:01,330 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=32366.666666666668, ans=0.125 2026-09-23 23:05:02,347 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.429e+02 3.284e+02 3.638e+02 4.173e+02 7.698e+02, threshold=7.277e+02, percent-clipped=0.0 2026-09-23 23:05:05,978 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff2.min_abs, batch_count=32400.0, ans=0.1 2026-09-23 23:05:10,202 INFO [train.py:1192] (1/2) Epoch 11, batch 150, loss[loss=0.2994, simple_loss=0.3821, pruned_loss=0.1084, over 24253.00 frames. ], tot_loss[loss=0.3654, simple_loss=0.4485, pruned_loss=0.1411, over 2560735.36 frames. ], batch size: 125, lr: 1.80e-02, grad_scale: 32.0 2026-09-23 23:05:11,359 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=32433.333333333332, ans=0.125 2026-09-23 23:05:12,675 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:05:18,464 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=32466.666666666668, ans=0.125 2026-09-23 23:05:23,864 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=32500.0, ans=0.125 2026-09-23 23:05:25,673 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=32533.333333333332, ans=0.09899494936611666 2026-09-23 23:05:34,099 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.56 vs. limit=10.0 2026-09-23 23:05:35,915 INFO [train.py:1192] (1/2) Epoch 11, batch 200, loss[loss=0.4328, simple_loss=0.506, pruned_loss=0.1798, over 24210.00 frames. ], tot_loss[loss=0.3625, simple_loss=0.4458, pruned_loss=0.1396, over 3059490.80 frames. ], batch size: 257, lr: 1.79e-02, grad_scale: 32.0 2026-09-23 23:05:43,034 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=32633.333333333332, ans=0.125 2026-09-23 23:05:54,621 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.499e+02 3.124e+02 3.551e+02 4.295e+02 6.642e+02, threshold=7.103e+02, percent-clipped=0.0 2026-09-23 23:05:55,807 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=32700.0, ans=0.125 2026-09-23 23:05:57,536 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=32733.333333333332, ans=0.1 2026-09-23 23:06:00,488 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=32733.333333333332, ans=0.1 2026-09-23 23:06:02,480 INFO [train.py:1192] (1/2) Epoch 11, batch 250, loss[loss=0.4083, simple_loss=0.4929, pruned_loss=0.1619, over 24380.00 frames. ], tot_loss[loss=0.363, simple_loss=0.4457, pruned_loss=0.1401, over 3442770.82 frames. ], batch size: 225, lr: 1.79e-02, grad_scale: 32.0 2026-09-23 23:06:02,596 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=32766.666666666668, ans=0.5 2026-09-23 23:06:06,731 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=32766.666666666668, ans=0.025 2026-09-23 23:06:10,563 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.08 vs. limit=15.0 2026-09-23 23:06:10,605 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.86 vs. limit=15.0 2026-09-23 23:06:17,546 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.73 vs. limit=15.0 2026-09-23 23:06:19,613 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.06 vs. limit=5.0 2026-09-23 23:06:25,664 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=32900.0, ans=0.125 2026-09-23 23:06:26,219 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=32900.0, ans=0.125 2026-09-23 23:06:28,101 INFO [train.py:1192] (1/2) Epoch 11, batch 300, loss[loss=0.3862, simple_loss=0.4751, pruned_loss=0.1486, over 24548.00 frames. ], tot_loss[loss=0.3618, simple_loss=0.4448, pruned_loss=0.1394, over 3756829.30 frames. ], batch size: 204, lr: 1.79e-02, grad_scale: 32.0 2026-09-23 23:06:29,191 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=32933.333333333336, ans=0.125 2026-09-23 23:06:31,044 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=32933.333333333336, ans=0.025 2026-09-23 23:06:39,115 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.69 vs. limit=15.0 2026-09-23 23:06:42,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=33000.0, ans=0.0036956521739130435 2026-09-23 23:06:46,882 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.558e+02 3.403e+02 3.902e+02 4.638e+02 6.729e+02, threshold=7.803e+02, percent-clipped=0.0 2026-09-23 23:06:46,978 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=33033.333333333336, ans=0.035 2026-09-23 23:06:52,789 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=33066.666666666664, ans=0.125 2026-09-23 23:06:53,481 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.69 vs. limit=15.0 2026-09-23 23:06:54,291 INFO [train.py:1192] (1/2) Epoch 11, batch 350, loss[loss=0.3431, simple_loss=0.4117, pruned_loss=0.1373, over 24589.00 frames. ], tot_loss[loss=0.3624, simple_loss=0.4455, pruned_loss=0.1397, over 3998908.41 frames. ], batch size: 137, lr: 1.78e-02, grad_scale: 32.0 2026-09-23 23:06:57,311 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.61 vs. limit=15.0 2026-09-23 23:06:59,424 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=33133.333333333336, ans=0.0 2026-09-23 23:07:01,029 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:07:09,588 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=33200.0, ans=0.125 2026-09-23 23:07:12,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=33200.0, ans=0.125 2026-09-23 23:07:14,159 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=33200.0, ans=0.0 2026-09-23 23:07:14,638 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=33200.0, ans=0.1 2026-09-23 23:07:16,102 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=33233.333333333336, ans=0.0036449275362318836 2026-09-23 23:07:20,417 INFO [train.py:1192] (1/2) Epoch 11, batch 400, loss[loss=0.3517, simple_loss=0.4452, pruned_loss=0.1291, over 24569.00 frames. ], tot_loss[loss=0.3613, simple_loss=0.4445, pruned_loss=0.139, over 4180245.49 frames. ], batch size: 170, lr: 1.78e-02, grad_scale: 32.0 2026-09-23 23:07:25,729 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.72 vs. limit=15.0 2026-09-23 23:07:30,928 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=33333.333333333336, ans=0.125 2026-09-23 23:07:33,721 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=33333.333333333336, ans=0.125 2026-09-23 23:07:39,253 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.379e+02 3.288e+02 3.676e+02 4.158e+02 6.374e+02, threshold=7.352e+02, percent-clipped=0.0 2026-09-23 23:07:41,963 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.21 vs. limit=15.0 2026-09-23 23:07:47,027 INFO [train.py:1192] (1/2) Epoch 11, batch 450, loss[loss=0.3841, simple_loss=0.4717, pruned_loss=0.1482, over 24623.00 frames. ], tot_loss[loss=0.3617, simple_loss=0.4448, pruned_loss=0.1393, over 4319640.49 frames. ], batch size: 175, lr: 1.78e-02, grad_scale: 32.0 2026-09-23 23:07:50,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=33433.333333333336, ans=0.0 2026-09-23 23:08:13,244 INFO [train.py:1192] (1/2) Epoch 11, batch 500, loss[loss=0.3883, simple_loss=0.475, pruned_loss=0.1508, over 24527.00 frames. ], tot_loss[loss=0.3586, simple_loss=0.4421, pruned_loss=0.1375, over 4437597.81 frames. ], batch size: 218, lr: 1.78e-02, grad_scale: 32.0 2026-09-23 23:08:19,001 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=33633.333333333336, ans=0.1 2026-09-23 23:08:21,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=33633.333333333336, ans=0.125 2026-09-23 23:08:21,998 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:08:31,384 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.556e+02 3.366e+02 3.820e+02 4.509e+02 7.829e+02, threshold=7.640e+02, percent-clipped=1.0 2026-09-23 23:08:31,586 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.92 vs. limit=10.0 2026-09-23 23:08:36,681 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=33733.333333333336, ans=0.2 2026-09-23 23:08:38,687 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=33766.666666666664, ans=0.125 2026-09-23 23:08:39,056 INFO [train.py:1192] (1/2) Epoch 11, batch 550, loss[loss=0.357, simple_loss=0.4529, pruned_loss=0.1306, over 24254.00 frames. ], tot_loss[loss=0.3598, simple_loss=0.4431, pruned_loss=0.1382, over 4522687.50 frames. ], batch size: 257, lr: 1.77e-02, grad_scale: 32.0 2026-09-23 23:08:44,832 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=33800.0, ans=0.0 2026-09-23 23:08:45,944 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.31 vs. limit=15.0 2026-09-23 23:08:50,585 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer_ff3.min_abs, batch_count=33833.333333333336, ans=0.2 2026-09-23 23:09:05,010 INFO [train.py:1192] (1/2) Epoch 11, batch 600, loss[loss=0.3939, simple_loss=0.4928, pruned_loss=0.1475, over 24319.00 frames. ], tot_loss[loss=0.3605, simple_loss=0.444, pruned_loss=0.1385, over 4588805.29 frames. ], batch size: 234, lr: 1.77e-02, grad_scale: 32.0 2026-09-23 23:09:07,666 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.93 vs. limit=15.0 2026-09-23 23:09:22,785 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:09:23,195 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.536e+02 3.381e+02 3.797e+02 4.395e+02 7.584e+02, threshold=7.594e+02, percent-clipped=0.0 2026-09-23 23:09:31,253 INFO [train.py:1192] (1/2) Epoch 11, batch 650, loss[loss=0.3338, simple_loss=0.4156, pruned_loss=0.1261, over 24587.00 frames. ], tot_loss[loss=0.3585, simple_loss=0.4424, pruned_loss=0.1373, over 4653490.49 frames. ], batch size: 154, lr: 1.77e-02, grad_scale: 32.0 2026-09-23 23:09:33,639 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.57 vs. limit=22.5 2026-09-23 23:09:44,882 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.88 vs. limit=15.0 2026-09-23 23:09:57,860 INFO [train.py:1192] (1/2) Epoch 11, batch 700, loss[loss=0.3129, simple_loss=0.4098, pruned_loss=0.108, over 24555.00 frames. ], tot_loss[loss=0.3592, simple_loss=0.4433, pruned_loss=0.1376, over 4689026.90 frames. ], batch size: 158, lr: 1.76e-02, grad_scale: 32.0 2026-09-23 23:10:10,489 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=34333.333333333336, ans=0.125 2026-09-23 23:10:14,667 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=34366.666666666664, ans=0.0 2026-09-23 23:10:16,093 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.478e+02 3.428e+02 3.960e+02 4.700e+02 7.348e+02, threshold=7.919e+02, percent-clipped=0.0 2026-09-23 23:10:17,543 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=34366.666666666664, ans=0.0033985507246376825 2026-09-23 23:10:23,502 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=34433.333333333336, ans=0.0 2026-09-23 23:10:23,798 INFO [train.py:1192] (1/2) Epoch 11, batch 750, loss[loss=0.3537, simple_loss=0.4468, pruned_loss=0.1303, over 24612.00 frames. ], tot_loss[loss=0.357, simple_loss=0.4417, pruned_loss=0.1362, over 4725096.66 frames. ], batch size: 175, lr: 1.76e-02, grad_scale: 32.0 2026-09-23 23:10:27,843 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.56 vs. limit=15.0 2026-09-23 23:10:39,049 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=34500.0, ans=0.2 2026-09-23 23:10:49,350 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.39 vs. limit=12.0 2026-09-23 23:10:50,130 INFO [train.py:1192] (1/2) Epoch 11, batch 800, loss[loss=0.3028, simple_loss=0.3896, pruned_loss=0.108, over 24537.00 frames. ], tot_loss[loss=0.3562, simple_loss=0.441, pruned_loss=0.1357, over 4750913.74 frames. ], batch size: 137, lr: 1.76e-02, grad_scale: 32.0 2026-09-23 23:10:50,728 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=34600.0, ans=0.95 2026-09-23 23:10:56,495 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=34633.333333333336, ans=0.125 2026-09-23 23:10:57,150 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.13 vs. limit=6.0 2026-09-23 23:10:57,399 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=34633.333333333336, ans=0.1 2026-09-23 23:10:58,408 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=34633.333333333336, ans=0.125 2026-09-23 23:11:07,131 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.12 vs. limit=15.0 2026-09-23 23:11:08,386 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.355e+02 3.357e+02 3.761e+02 4.428e+02 6.776e+02, threshold=7.523e+02, percent-clipped=0.0 2026-09-23 23:11:10,005 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=34700.0, ans=10.0 2026-09-23 23:11:10,485 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=34733.333333333336, ans=0.2 2026-09-23 23:11:16,144 INFO [train.py:1192] (1/2) Epoch 11, batch 850, loss[loss=0.3624, simple_loss=0.4555, pruned_loss=0.1347, over 24541.00 frames. ], tot_loss[loss=0.355, simple_loss=0.44, pruned_loss=0.135, over 4770140.02 frames. ], batch size: 204, lr: 1.76e-02, grad_scale: 32.0 2026-09-23 23:11:20,830 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=34800.0, ans=0.125 2026-09-23 23:11:21,262 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=34800.0, ans=10.0 2026-09-23 23:11:21,268 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=34800.0, ans=0.125 2026-09-23 23:11:31,382 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=34866.666666666664, ans=0.0 2026-09-23 23:11:34,443 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.14 vs. limit=15.0 2026-09-23 23:11:41,928 INFO [train.py:1192] (1/2) Epoch 11, batch 900, loss[loss=0.3013, simple_loss=0.3979, pruned_loss=0.1024, over 24558.00 frames. ], tot_loss[loss=0.3554, simple_loss=0.4404, pruned_loss=0.1353, over 4780725.98 frames. ], batch size: 137, lr: 1.75e-02, grad_scale: 32.0 2026-09-23 23:11:42,463 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=34933.333333333336, ans=0.0 2026-09-23 23:11:59,318 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.406e+02 3.306e+02 3.818e+02 4.396e+02 7.651e+02, threshold=7.635e+02, percent-clipped=1.0 2026-09-23 23:12:01,477 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=35066.666666666664, ans=0.125 2026-09-23 23:12:04,433 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=35066.666666666664, ans=0.125 2026-09-23 23:12:06,103 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=17.03 vs. limit=15.0 2026-09-23 23:12:06,815 INFO [train.py:1192] (1/2) Epoch 11, batch 950, loss[loss=0.5194, simple_loss=0.5238, pruned_loss=0.2575, over 11299.00 frames. ], tot_loss[loss=0.3571, simple_loss=0.4397, pruned_loss=0.1372, over 4710780.95 frames. ], batch size: 333, lr: 1.75e-02, grad_scale: 16.0 2026-09-23 23:12:08,986 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=35100.0, ans=0.035 2026-09-23 23:12:18,731 INFO [train.py:1192] (1/2) Epoch 12, batch 0, loss[loss=0.3126, simple_loss=0.4037, pruned_loss=0.1108, over 24557.00 frames. ], tot_loss[loss=0.3126, simple_loss=0.4037, pruned_loss=0.1108, over 24557.00 frames. ], batch size: 137, lr: 1.68e-02, grad_scale: 32.0 2026-09-23 23:12:18,732 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 23:12:21,918 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.7011, 2.7546, 3.1843, 2.7940], device='cuda:1') 2026-09-23 23:12:30,448 INFO [train.py:1224] (1/2) Epoch 12, validation: loss=0.216, simple_loss=0.3291, pruned_loss=0.05141, over 2564189.00 frames. 2026-09-23 23:12:30,448 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 23:12:30,480 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=35126.666666666664, ans=0.015 2026-09-23 23:12:32,028 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=35126.666666666664, ans=0.0032333333333333337 2026-09-23 23:12:35,694 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=35160.0, ans=0.2 2026-09-23 23:12:40,498 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=35193.333333333336, ans=0.0 2026-09-23 23:12:52,057 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:12:55,485 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35260.0, ans=0.1 2026-09-23 23:12:56,336 INFO [train.py:1192] (1/2) Epoch 12, batch 50, loss[loss=0.2706, simple_loss=0.3654, pruned_loss=0.0879, over 24240.00 frames. ], tot_loss[loss=0.3684, simple_loss=0.4506, pruned_loss=0.1431, over 1081909.91 frames. ], batch size: 125, lr: 1.67e-02, grad_scale: 32.0 2026-09-23 23:13:08,340 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=35360.0, ans=0.2 2026-09-23 23:13:09,872 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=35360.0, ans=0.125 2026-09-23 23:13:10,575 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.513e+02 3.441e+02 3.981e+02 5.014e+02 8.366e+02, threshold=7.962e+02, percent-clipped=2.0 2026-09-23 23:13:11,602 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=35393.333333333336, ans=0.125 2026-09-23 23:13:13,422 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35393.333333333336, ans=0.1 2026-09-23 23:13:21,568 INFO [train.py:1192] (1/2) Epoch 12, batch 100, loss[loss=0.3574, simple_loss=0.4351, pruned_loss=0.1398, over 24627.00 frames. ], tot_loss[loss=0.365, simple_loss=0.4507, pruned_loss=0.1396, over 1915769.74 frames. ], batch size: 154, lr: 1.67e-02, grad_scale: 32.0 2026-09-23 23:13:23,086 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.30 vs. limit=6.0 2026-09-23 23:13:24,902 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.71 vs. limit=15.0 2026-09-23 23:13:33,641 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=35526.666666666664, ans=0.07 2026-09-23 23:13:47,086 INFO [train.py:1192] (1/2) Epoch 12, batch 150, loss[loss=0.2694, simple_loss=0.3621, pruned_loss=0.08839, over 24255.00 frames. ], tot_loss[loss=0.3554, simple_loss=0.4421, pruned_loss=0.1343, over 2560832.95 frames. ], batch size: 125, lr: 1.67e-02, grad_scale: 32.0 2026-09-23 23:13:51,704 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=35626.666666666664, ans=0.125 2026-09-23 23:13:56,703 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:14:01,840 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.479e+02 3.158e+02 3.739e+02 4.407e+02 7.030e+02, threshold=7.477e+02, percent-clipped=0.0 2026-09-23 23:14:02,484 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=35726.666666666664, ans=0.1 2026-09-23 23:14:03,530 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=35726.666666666664, ans=0.125 2026-09-23 23:14:05,090 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=35726.666666666664, ans=0.125 2026-09-23 23:14:08,737 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.55 vs. limit=15.0 2026-09-23 23:14:13,307 INFO [train.py:1192] (1/2) Epoch 12, batch 200, loss[loss=0.4393, simple_loss=0.5149, pruned_loss=0.1819, over 24242.00 frames. ], tot_loss[loss=0.3536, simple_loss=0.4401, pruned_loss=0.1335, over 3059502.50 frames. ], batch size: 257, lr: 1.67e-02, grad_scale: 32.0 2026-09-23 23:14:13,440 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=35793.333333333336, ans=0.125 2026-09-23 23:14:24,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=35860.0, ans=0.125 2026-09-23 23:14:27,376 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=35860.0, ans=0.5 2026-09-23 23:14:33,239 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=35893.333333333336, ans=0.125 2026-09-23 23:14:34,299 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=35926.666666666664, ans=0.07 2026-09-23 23:14:36,284 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=35926.666666666664, ans=0.125 2026-09-23 23:14:40,067 INFO [train.py:1192] (1/2) Epoch 12, batch 250, loss[loss=0.3996, simple_loss=0.4831, pruned_loss=0.1581, over 24372.00 frames. ], tot_loss[loss=0.3547, simple_loss=0.4406, pruned_loss=0.1344, over 3443073.81 frames. ], batch size: 225, lr: 1.66e-02, grad_scale: 32.0 2026-09-23 23:14:41,351 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:14:44,592 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=35993.333333333336, ans=0.1 2026-09-23 23:14:54,015 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.274e+02 3.369e+02 3.869e+02 4.652e+02 8.346e+02, threshold=7.738e+02, percent-clipped=2.0 2026-09-23 23:14:55,971 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=36060.0, ans=0.05 2026-09-23 23:15:05,550 INFO [train.py:1192] (1/2) Epoch 12, batch 300, loss[loss=0.3769, simple_loss=0.4699, pruned_loss=0.142, over 24541.00 frames. ], tot_loss[loss=0.3527, simple_loss=0.4391, pruned_loss=0.1332, over 3756221.00 frames. ], batch size: 204, lr: 1.66e-02, grad_scale: 32.0 2026-09-23 23:15:06,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=36126.666666666664, ans=0.1 2026-09-23 23:15:07,956 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:15:10,007 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.87 vs. limit=6.0 2026-09-23 23:15:15,090 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=20.05 vs. limit=22.5 2026-09-23 23:15:20,908 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.09 vs. limit=22.5 2026-09-23 23:15:23,697 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=36226.666666666664, ans=0.125 2026-09-23 23:15:23,713 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=36226.666666666664, ans=0.125 2026-09-23 23:15:31,350 INFO [train.py:1192] (1/2) Epoch 12, batch 350, loss[loss=0.3144, simple_loss=0.3936, pruned_loss=0.1176, over 24537.00 frames. ], tot_loss[loss=0.3521, simple_loss=0.439, pruned_loss=0.1326, over 3993080.63 frames. ], batch size: 137, lr: 1.66e-02, grad_scale: 32.0 2026-09-23 23:15:33,378 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=36293.333333333336, ans=0.1 2026-09-23 23:15:33,399 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=36293.333333333336, ans=0.07 2026-09-23 23:15:42,267 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=36360.0, ans=0.95 2026-09-23 23:15:43,257 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=36360.0, ans=0.2 2026-09-23 23:15:45,545 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 3.376e+02 3.785e+02 4.280e+02 7.656e+02, threshold=7.571e+02, percent-clipped=0.0 2026-09-23 23:15:46,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=36393.333333333336, ans=0.125 2026-09-23 23:15:52,939 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.98 vs. limit=15.0 2026-09-23 23:15:56,588 INFO [train.py:1192] (1/2) Epoch 12, batch 400, loss[loss=0.3417, simple_loss=0.4335, pruned_loss=0.1249, over 24554.00 frames. ], tot_loss[loss=0.3497, simple_loss=0.437, pruned_loss=0.1312, over 4176817.18 frames. ], batch size: 170, lr: 1.66e-02, grad_scale: 32.0 2026-09-23 23:16:22,353 INFO [train.py:1192] (1/2) Epoch 12, batch 450, loss[loss=0.3436, simple_loss=0.4389, pruned_loss=0.1241, over 24624.00 frames. ], tot_loss[loss=0.3506, simple_loss=0.4375, pruned_loss=0.1318, over 4317399.94 frames. ], batch size: 175, lr: 1.65e-02, grad_scale: 32.0 2026-09-23 23:16:22,472 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=36626.666666666664, ans=0.2 2026-09-23 23:16:27,025 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:16:27,053 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=36626.666666666664, ans=0.0029072463768115946 2026-09-23 23:16:33,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=36693.333333333336, ans=0.125 2026-09-23 23:16:33,991 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=36693.333333333336, ans=0.125 2026-09-23 23:16:37,001 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.276e+02 3.307e+02 3.859e+02 4.571e+02 7.199e+02, threshold=7.718e+02, percent-clipped=0.0 2026-09-23 23:16:45,058 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=36760.0, ans=0.125 2026-09-23 23:16:48,549 INFO [train.py:1192] (1/2) Epoch 12, batch 500, loss[loss=0.3664, simple_loss=0.4564, pruned_loss=0.1382, over 24520.00 frames. ], tot_loss[loss=0.3498, simple_loss=0.4364, pruned_loss=0.1316, over 4436338.88 frames. ], batch size: 218, lr: 1.65e-02, grad_scale: 32.0 2026-09-23 23:16:49,717 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=36793.333333333336, ans=0.1 2026-09-23 23:16:57,131 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=36826.666666666664, ans=0.125 2026-09-23 23:16:57,174 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=36826.666666666664, ans=0.125 2026-09-23 23:17:05,543 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=36893.333333333336, ans=0.125 2026-09-23 23:17:14,961 INFO [train.py:1192] (1/2) Epoch 12, batch 550, loss[loss=0.3845, simple_loss=0.4754, pruned_loss=0.1468, over 24285.00 frames. ], tot_loss[loss=0.3497, simple_loss=0.4365, pruned_loss=0.1315, over 4521738.15 frames. ], batch size: 257, lr: 1.65e-02, grad_scale: 32.0 2026-09-23 23:17:29,877 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.741e+02 3.365e+02 3.815e+02 4.468e+02 6.543e+02, threshold=7.630e+02, percent-clipped=0.0 2026-09-23 23:17:34,715 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=37060.0, ans=0.2 2026-09-23 23:17:39,279 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=7.04 vs. limit=10.0 2026-09-23 23:17:41,216 INFO [train.py:1192] (1/2) Epoch 12, batch 600, loss[loss=0.3663, simple_loss=0.4658, pruned_loss=0.1334, over 24326.00 frames. ], tot_loss[loss=0.351, simple_loss=0.4375, pruned_loss=0.1323, over 4588611.98 frames. ], batch size: 234, lr: 1.65e-02, grad_scale: 32.0 2026-09-23 23:17:46,545 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=37160.0, ans=0.125 2026-09-23 23:17:48,622 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.44 vs. limit=15.0 2026-09-23 23:17:57,367 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:18:03,632 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=37260.0, ans=0.125 2026-09-23 23:18:07,215 INFO [train.py:1192] (1/2) Epoch 12, batch 650, loss[loss=0.3315, simple_loss=0.4203, pruned_loss=0.1214, over 24595.00 frames. ], tot_loss[loss=0.3489, simple_loss=0.436, pruned_loss=0.131, over 4653462.33 frames. ], batch size: 154, lr: 1.64e-02, grad_scale: 32.0 2026-09-23 23:18:22,116 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.397e+02 3.206e+02 3.726e+02 4.417e+02 8.115e+02, threshold=7.451e+02, percent-clipped=2.0 2026-09-23 23:18:23,252 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=37393.333333333336, ans=0.125 2026-09-23 23:18:26,240 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=37393.333333333336, ans=0.125 2026-09-23 23:18:33,711 INFO [train.py:1192] (1/2) Epoch 12, batch 700, loss[loss=0.3305, simple_loss=0.4261, pruned_loss=0.1175, over 24557.00 frames. ], tot_loss[loss=0.3499, simple_loss=0.4371, pruned_loss=0.1314, over 4689776.62 frames. ], batch size: 158, lr: 1.64e-02, grad_scale: 32.0 2026-09-23 23:18:44,982 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=37526.666666666664, ans=0.0 2026-09-23 23:18:44,982 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=37526.666666666664, ans=0.0027115942028985516 2026-09-23 23:18:52,548 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_positive, batch_count=37560.0, ans=0.05 2026-09-23 23:18:56,426 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.13 vs. limit=12.0 2026-09-23 23:18:59,038 INFO [train.py:1192] (1/2) Epoch 12, batch 750, loss[loss=0.3501, simple_loss=0.4427, pruned_loss=0.1287, over 24651.00 frames. ], tot_loss[loss=0.3493, simple_loss=0.4366, pruned_loss=0.131, over 4726292.53 frames. ], batch size: 175, lr: 1.64e-02, grad_scale: 32.0 2026-09-23 23:19:04,074 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=37660.0, ans=0.07 2026-09-23 23:19:13,555 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.581e+02 3.554e+02 3.988e+02 5.020e+02 7.837e+02, threshold=7.976e+02, percent-clipped=1.0 2026-09-23 23:19:25,089 INFO [train.py:1192] (1/2) Epoch 12, batch 800, loss[loss=0.2906, simple_loss=0.3817, pruned_loss=0.09972, over 24578.00 frames. ], tot_loss[loss=0.3482, simple_loss=0.4358, pruned_loss=0.1303, over 4752735.64 frames. ], batch size: 137, lr: 1.64e-02, grad_scale: 32.0 2026-09-23 23:19:38,546 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=37860.0, ans=0.2 2026-09-23 23:19:47,270 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=37926.666666666664, ans=0.125 2026-09-23 23:19:51,289 INFO [train.py:1192] (1/2) Epoch 12, batch 850, loss[loss=0.3888, simple_loss=0.4755, pruned_loss=0.1511, over 24550.00 frames. ], tot_loss[loss=0.348, simple_loss=0.4354, pruned_loss=0.1303, over 4771266.61 frames. ], batch size: 204, lr: 1.63e-02, grad_scale: 32.0 2026-09-23 23:19:55,164 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=37960.0, ans=0.035 2026-09-23 23:19:59,731 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff2.min_abs, batch_count=37993.333333333336, ans=0.1 2026-09-23 23:20:01,631 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=38026.666666666664, ans=0.2 2026-09-23 23:20:05,926 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.500e+02 3.277e+02 3.882e+02 4.474e+02 7.001e+02, threshold=7.765e+02, percent-clipped=0.0 2026-09-23 23:20:10,328 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=38060.0, ans=0.125 2026-09-23 23:20:17,451 INFO [train.py:1192] (1/2) Epoch 12, batch 900, loss[loss=0.3002, simple_loss=0.3891, pruned_loss=0.1057, over 24552.00 frames. ], tot_loss[loss=0.3476, simple_loss=0.4352, pruned_loss=0.13, over 4781520.49 frames. ], batch size: 137, lr: 1.63e-02, grad_scale: 32.0 2026-09-23 23:20:23,304 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=38160.0, ans=0.125 2026-09-23 23:20:40,635 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=38260.0, ans=0.2 2026-09-23 23:20:43,453 INFO [train.py:1192] (1/2) Epoch 12, batch 950, loss[loss=0.5013, simple_loss=0.5074, pruned_loss=0.2476, over 11330.00 frames. ], tot_loss[loss=0.3499, simple_loss=0.435, pruned_loss=0.1324, over 4715684.72 frames. ], batch size: 333, lr: 1.63e-02, grad_scale: 16.0 2026-09-23 23:20:55,025 INFO [train.py:1192] (1/2) Epoch 13, batch 0, loss[loss=0.2917, simple_loss=0.3892, pruned_loss=0.09712, over 24578.00 frames. ], tot_loss[loss=0.2917, simple_loss=0.3892, pruned_loss=0.09712, over 24578.00 frames. ], batch size: 137, lr: 1.56e-02, grad_scale: 32.0 2026-09-23 23:20:55,025 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 23:21:06,513 INFO [train.py:1224] (1/2) Epoch 13, validation: loss=0.213, simple_loss=0.3279, pruned_loss=0.04906, over 2564189.00 frames. 2026-09-23 23:21:06,513 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 23:21:07,853 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=38320.0, ans=0.0 2026-09-23 23:21:08,895 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=38320.0, ans=0.025 2026-09-23 23:21:15,328 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=38353.333333333336, ans=0.125 2026-09-23 23:21:16,539 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.533e+02 3.165e+02 3.581e+02 4.158e+02 9.000e+02, threshold=7.163e+02, percent-clipped=2.0 2026-09-23 23:21:32,016 INFO [train.py:1192] (1/2) Epoch 13, batch 50, loss[loss=0.2763, simple_loss=0.367, pruned_loss=0.09282, over 24240.00 frames. ], tot_loss[loss=0.3519, simple_loss=0.4395, pruned_loss=0.1322, over 1082119.77 frames. ], batch size: 125, lr: 1.56e-02, grad_scale: 32.0 2026-09-23 23:21:32,126 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=38486.666666666664, ans=0.07 2026-09-23 23:21:38,872 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=38520.0, ans=0.025 2026-09-23 23:21:46,705 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=38586.666666666664, ans=0.09899494936611666 2026-09-23 23:21:51,595 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=38586.666666666664, ans=0.002481159420289855 2026-09-23 23:21:52,929 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=38620.0, ans=0.2 2026-09-23 23:21:52,952 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=38620.0, ans=0.125 2026-09-23 23:21:57,489 INFO [train.py:1192] (1/2) Epoch 13, batch 100, loss[loss=0.3719, simple_loss=0.4355, pruned_loss=0.1542, over 24594.00 frames. ], tot_loss[loss=0.3554, simple_loss=0.4437, pruned_loss=0.1335, over 1914854.88 frames. ], batch size: 154, lr: 1.56e-02, grad_scale: 32.0 2026-09-23 23:21:57,573 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=38653.333333333336, ans=0.0 2026-09-23 23:21:59,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=38653.333333333336, ans=0.125 2026-09-23 23:22:08,233 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.515e+02 3.237e+02 3.577e+02 4.076e+02 7.676e+02, threshold=7.155e+02, percent-clipped=1.0 2026-09-23 23:22:09,258 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=38720.0, ans=0.1 2026-09-23 23:22:11,453 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.55 vs. limit=15.0 2026-09-23 23:22:15,024 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=38753.333333333336, ans=0.04949747468305833 2026-09-23 23:22:16,015 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.31 vs. limit=15.0 2026-09-23 23:22:19,187 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=38786.666666666664, ans=15.0 2026-09-23 23:22:22,060 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.23 vs. limit=15.0 2026-09-23 23:22:22,461 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=38820.0, ans=0.1 2026-09-23 23:22:22,957 INFO [train.py:1192] (1/2) Epoch 13, batch 150, loss[loss=0.3022, simple_loss=0.3826, pruned_loss=0.1109, over 24242.00 frames. ], tot_loss[loss=0.3483, simple_loss=0.4371, pruned_loss=0.1298, over 2559983.56 frames. ], batch size: 125, lr: 1.56e-02, grad_scale: 32.0 2026-09-23 23:22:31,687 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=38853.333333333336, ans=0.0 2026-09-23 23:22:36,062 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=38886.666666666664, ans=0.125 2026-09-23 23:22:42,543 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:22:42,553 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=38920.0, ans=0.125 2026-09-23 23:22:48,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=38953.333333333336, ans=0.1 2026-09-23 23:22:49,032 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=38986.666666666664, ans=0.125 2026-09-23 23:22:49,425 INFO [train.py:1192] (1/2) Epoch 13, batch 200, loss[loss=0.4114, simple_loss=0.497, pruned_loss=0.1629, over 24203.00 frames. ], tot_loss[loss=0.3488, simple_loss=0.4371, pruned_loss=0.1303, over 3058604.20 frames. ], batch size: 257, lr: 1.56e-02, grad_scale: 32.0 2026-09-23 23:22:58,839 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.62 vs. limit=15.0 2026-09-23 23:23:00,612 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.461e+02 3.338e+02 3.856e+02 4.462e+02 9.088e+02, threshold=7.711e+02, percent-clipped=1.0 2026-09-23 23:23:06,004 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=39086.666666666664, ans=0.0 2026-09-23 23:23:09,904 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=39120.0, ans=0.125 2026-09-23 23:23:10,398 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=39120.0, ans=0.0 2026-09-23 23:23:16,061 INFO [train.py:1192] (1/2) Epoch 13, batch 250, loss[loss=0.3854, simple_loss=0.479, pruned_loss=0.1459, over 24364.00 frames. ], tot_loss[loss=0.349, simple_loss=0.4367, pruned_loss=0.1306, over 3442379.89 frames. ], batch size: 225, lr: 1.55e-02, grad_scale: 32.0 2026-09-23 23:23:36,279 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.99 vs. limit=15.0 2026-09-23 23:23:38,157 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=39286.666666666664, ans=0.1 2026-09-23 23:23:41,850 INFO [train.py:1192] (1/2) Epoch 13, batch 300, loss[loss=0.3416, simple_loss=0.4507, pruned_loss=0.1163, over 24551.00 frames. ], tot_loss[loss=0.348, simple_loss=0.4357, pruned_loss=0.1302, over 3756391.05 frames. ], batch size: 204, lr: 1.55e-02, grad_scale: 32.0 2026-09-23 23:23:52,604 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.522e+02 3.399e+02 3.954e+02 4.648e+02 7.202e+02, threshold=7.908e+02, percent-clipped=0.0 2026-09-23 23:23:57,548 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=39420.0, ans=0.2 2026-09-23 23:24:01,547 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=39420.0, ans=10.0 2026-09-23 23:24:07,904 INFO [train.py:1192] (1/2) Epoch 13, batch 350, loss[loss=0.2799, simple_loss=0.375, pruned_loss=0.09237, over 24552.00 frames. ], tot_loss[loss=0.3474, simple_loss=0.4358, pruned_loss=0.1295, over 3993613.30 frames. ], batch size: 137, lr: 1.55e-02, grad_scale: 32.0 2026-09-23 23:24:09,697 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=39486.666666666664, ans=0.1 2026-09-23 23:24:14,112 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.61 vs. limit=15.0 2026-09-23 23:24:28,823 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=39620.0, ans=0.0 2026-09-23 23:24:33,816 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=39653.333333333336, ans=0.125 2026-09-23 23:24:34,311 INFO [train.py:1192] (1/2) Epoch 13, batch 400, loss[loss=0.3037, simple_loss=0.4126, pruned_loss=0.09737, over 24575.00 frames. ], tot_loss[loss=0.3453, simple_loss=0.4342, pruned_loss=0.1282, over 4176632.48 frames. ], batch size: 170, lr: 1.55e-02, grad_scale: 32.0 2026-09-23 23:24:34,403 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=39653.333333333336, ans=0.025 2026-09-23 23:24:43,838 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=39686.666666666664, ans=0.1 2026-09-23 23:24:45,094 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.476e+02 3.301e+02 3.730e+02 4.213e+02 6.408e+02, threshold=7.460e+02, percent-clipped=0.0 2026-09-23 23:24:46,751 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=39720.0, ans=0.1 2026-09-23 23:24:59,857 INFO [train.py:1192] (1/2) Epoch 13, batch 450, loss[loss=0.3418, simple_loss=0.4349, pruned_loss=0.1244, over 24625.00 frames. ], tot_loss[loss=0.3447, simple_loss=0.4337, pruned_loss=0.1279, over 4318494.23 frames. ], batch size: 175, lr: 1.54e-02, grad_scale: 32.0 2026-09-23 23:25:01,925 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.80 vs. limit=10.0 2026-09-23 23:25:12,167 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=39886.666666666664, ans=0.125 2026-09-23 23:25:14,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=39886.666666666664, ans=0.125 2026-09-23 23:25:17,973 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=39920.0, ans=0.125 2026-09-23 23:25:25,632 INFO [train.py:1192] (1/2) Epoch 13, batch 500, loss[loss=0.3588, simple_loss=0.4589, pruned_loss=0.1294, over 24512.00 frames. ], tot_loss[loss=0.3435, simple_loss=0.4322, pruned_loss=0.1274, over 4436804.85 frames. ], batch size: 218, lr: 1.54e-02, grad_scale: 32.0 2026-09-23 23:25:32,143 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=40020.0, ans=0.125 2026-09-23 23:25:37,295 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.487e+02 3.141e+02 3.810e+02 4.692e+02 7.001e+02, threshold=7.621e+02, percent-clipped=0.0 2026-09-23 23:25:43,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=40086.666666666664, ans=0.0 2026-09-23 23:25:49,113 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=40120.0, ans=0.2 2026-09-23 23:25:50,735 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=40120.0, ans=0.0 2026-09-23 23:25:51,571 INFO [train.py:1192] (1/2) Epoch 13, batch 550, loss[loss=0.3758, simple_loss=0.4699, pruned_loss=0.1409, over 24240.00 frames. ], tot_loss[loss=0.3439, simple_loss=0.4328, pruned_loss=0.1275, over 4521900.51 frames. ], batch size: 257, lr: 1.54e-02, grad_scale: 16.0 2026-09-23 23:26:07,781 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=40253.333333333336, ans=0.95 2026-09-23 23:26:12,255 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=40253.333333333336, ans=0.0 2026-09-23 23:26:12,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=40286.666666666664, ans=0.125 2026-09-23 23:26:15,215 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=40286.666666666664, ans=10.0 2026-09-23 23:26:18,338 INFO [train.py:1192] (1/2) Epoch 13, batch 600, loss[loss=0.3488, simple_loss=0.4485, pruned_loss=0.1245, over 24355.00 frames. ], tot_loss[loss=0.3442, simple_loss=0.4332, pruned_loss=0.1276, over 4588328.35 frames. ], batch size: 234, lr: 1.54e-02, grad_scale: 16.0 2026-09-23 23:26:26,215 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=40353.333333333336, ans=0.125 2026-09-23 23:26:29,963 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.559e+02 3.184e+02 3.728e+02 4.522e+02 8.525e+02, threshold=7.455e+02, percent-clipped=1.0 2026-09-23 23:26:44,153 INFO [train.py:1192] (1/2) Epoch 13, batch 650, loss[loss=0.3247, simple_loss=0.4096, pruned_loss=0.1199, over 24617.00 frames. ], tot_loss[loss=0.3422, simple_loss=0.4319, pruned_loss=0.1263, over 4653524.08 frames. ], batch size: 154, lr: 1.53e-02, grad_scale: 16.0 2026-09-23 23:26:45,280 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=40486.666666666664, ans=0.0 2026-09-23 23:26:59,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=40586.666666666664, ans=0.125 2026-09-23 23:27:00,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=40586.666666666664, ans=22.5 2026-09-23 23:27:01,107 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=40586.666666666664, ans=0.125 2026-09-23 23:27:04,387 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:27:07,661 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.28 vs. limit=12.0 2026-09-23 23:27:09,839 INFO [train.py:1192] (1/2) Epoch 13, batch 700, loss[loss=0.3456, simple_loss=0.4327, pruned_loss=0.1293, over 24553.00 frames. ], tot_loss[loss=0.3424, simple_loss=0.4327, pruned_loss=0.1261, over 4689647.33 frames. ], batch size: 158, lr: 1.53e-02, grad_scale: 16.0 2026-09-23 23:27:14,072 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=40653.333333333336, ans=0.0 2026-09-23 23:27:14,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=40686.666666666664, ans=0.125 2026-09-23 23:27:17,839 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=40686.666666666664, ans=0.2 2026-09-23 23:27:21,089 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.487e+02 3.495e+02 4.223e+02 5.427e+02 8.388e+02, threshold=8.446e+02, percent-clipped=3.0 2026-09-23 23:27:35,112 INFO [train.py:1192] (1/2) Epoch 13, batch 750, loss[loss=0.3601, simple_loss=0.4522, pruned_loss=0.134, over 24641.00 frames. ], tot_loss[loss=0.3412, simple_loss=0.4314, pruned_loss=0.1255, over 4722040.26 frames. ], batch size: 175, lr: 1.53e-02, grad_scale: 16.0 2026-09-23 23:27:37,138 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=40820.0, ans=0.125 2026-09-23 23:27:42,680 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=40853.333333333336, ans=0.0 2026-09-23 23:27:43,185 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:27:46,077 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=40886.666666666664, ans=0.025 2026-09-23 23:27:55,938 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.93 vs. limit=15.0 2026-09-23 23:27:56,298 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=40953.333333333336, ans=0.125 2026-09-23 23:27:58,215 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=40953.333333333336, ans=0.0 2026-09-23 23:27:59,126 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=40953.333333333336, ans=0.125 2026-09-23 23:28:00,446 INFO [train.py:1192] (1/2) Epoch 13, batch 800, loss[loss=0.321, simple_loss=0.4071, pruned_loss=0.1174, over 24560.00 frames. ], tot_loss[loss=0.3412, simple_loss=0.4313, pruned_loss=0.1255, over 4748351.66 frames. ], batch size: 137, lr: 1.53e-02, grad_scale: 32.0 2026-09-23 23:28:04,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=40986.666666666664, ans=0.0 2026-09-23 23:28:10,814 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=41053.333333333336, ans=0.0 2026-09-23 23:28:12,117 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.613e+02 3.339e+02 3.897e+02 4.682e+02 7.943e+02, threshold=7.793e+02, percent-clipped=0.0 2026-09-23 23:28:17,776 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=41086.666666666664, ans=0.125 2026-09-23 23:28:19,100 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=41086.666666666664, ans=0.0019376811594202896 2026-09-23 23:28:24,776 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=41120.0, ans=0.125 2026-09-23 23:28:26,626 INFO [train.py:1192] (1/2) Epoch 13, batch 850, loss[loss=0.3507, simple_loss=0.4511, pruned_loss=0.1251, over 24560.00 frames. ], tot_loss[loss=0.3396, simple_loss=0.4302, pruned_loss=0.1246, over 4768205.63 frames. ], batch size: 204, lr: 1.53e-02, grad_scale: 32.0 2026-09-23 23:28:30,281 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.36 vs. limit=12.0 2026-09-23 23:28:30,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=41153.333333333336, ans=0.125 2026-09-23 23:28:52,438 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=41320.0, ans=0.125 2026-09-23 23:28:52,873 INFO [train.py:1192] (1/2) Epoch 13, batch 900, loss[loss=0.2934, simple_loss=0.3908, pruned_loss=0.09804, over 24537.00 frames. ], tot_loss[loss=0.3409, simple_loss=0.431, pruned_loss=0.1254, over 4779268.89 frames. ], batch size: 137, lr: 1.52e-02, grad_scale: 32.0 2026-09-23 23:29:04,176 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.460e+02 3.273e+02 3.594e+02 4.174e+02 7.421e+02, threshold=7.188e+02, percent-clipped=0.0 2026-09-23 23:29:14,636 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=41453.333333333336, ans=0.04949747468305833 2026-09-23 23:29:18,118 INFO [train.py:1192] (1/2) Epoch 13, batch 950, loss[loss=0.4507, simple_loss=0.4763, pruned_loss=0.2126, over 11394.00 frames. ], tot_loss[loss=0.3425, simple_loss=0.4304, pruned_loss=0.1273, over 4708380.33 frames. ], batch size: 333, lr: 1.52e-02, grad_scale: 32.0 2026-09-23 23:29:18,672 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=41486.666666666664, ans=0.0 2026-09-23 23:29:19,237 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=16.44 vs. limit=15.0 2026-09-23 23:29:29,338 INFO [train.py:1192] (1/2) Epoch 14, batch 0, loss[loss=0.265, simple_loss=0.372, pruned_loss=0.07902, over 24583.00 frames. ], tot_loss[loss=0.265, simple_loss=0.372, pruned_loss=0.07902, over 24583.00 frames. ], batch size: 137, lr: 1.46e-02, grad_scale: 32.0 2026-09-23 23:29:29,338 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 23:29:33,348 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.0758, 3.5509, 3.9442, 3.6603], device='cuda:1') 2026-09-23 23:29:35,923 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.9601, 2.7250, 2.4726, 2.1890, 2.5555, 2.7734, 2.6819, 2.3182], device='cuda:1') 2026-09-23 23:29:38,956 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.8485, 2.2913, 1.7324, 2.6780], device='cuda:1') 2026-09-23 23:29:40,949 INFO [train.py:1224] (1/2) Epoch 14, validation: loss=0.2077, simple_loss=0.3229, pruned_loss=0.04624, over 2564189.00 frames. 2026-09-23 23:29:40,949 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 23:30:02,375 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.25 vs. limit=5.0 2026-09-23 23:30:05,622 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=41646.666666666664, ans=0.125 2026-09-23 23:30:06,381 INFO [train.py:1192] (1/2) Epoch 14, batch 50, loss[loss=0.276, simple_loss=0.3696, pruned_loss=0.09124, over 24250.00 frames. ], tot_loss[loss=0.3505, simple_loss=0.4394, pruned_loss=0.1308, over 1082621.62 frames. ], batch size: 125, lr: 1.46e-02, grad_scale: 32.0 2026-09-23 23:30:13,341 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.567e+02 3.394e+02 3.972e+02 4.555e+02 8.560e+02, threshold=7.945e+02, percent-clipped=1.0 2026-09-23 23:30:17,179 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=41746.666666666664, ans=0.1 2026-09-23 23:30:25,178 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=41780.0, ans=0.0 2026-09-23 23:30:31,892 INFO [train.py:1192] (1/2) Epoch 14, batch 100, loss[loss=0.2878, simple_loss=0.387, pruned_loss=0.09428, over 24582.00 frames. ], tot_loss[loss=0.3488, simple_loss=0.44, pruned_loss=0.1288, over 1916884.88 frames. ], batch size: 154, lr: 1.46e-02, grad_scale: 32.0 2026-09-23 23:30:40,347 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=41880.0, ans=0.2 2026-09-23 23:30:42,511 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.30 vs. limit=15.0 2026-09-23 23:30:52,355 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=41980.0, ans=0.125 2026-09-23 23:30:57,600 INFO [train.py:1192] (1/2) Epoch 14, batch 150, loss[loss=0.2669, simple_loss=0.3592, pruned_loss=0.0873, over 24243.00 frames. ], tot_loss[loss=0.3459, simple_loss=0.4361, pruned_loss=0.1278, over 2561733.21 frames. ], batch size: 125, lr: 1.46e-02, grad_scale: 32.0 2026-09-23 23:31:04,512 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.486e+02 3.227e+02 3.665e+02 4.456e+02 7.037e+02, threshold=7.330e+02, percent-clipped=0.0 2026-09-23 23:31:11,016 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=42080.0, ans=0.125 2026-09-23 23:31:12,045 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=42113.333333333336, ans=0.125 2026-09-23 23:31:19,014 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=42146.666666666664, ans=0.125 2026-09-23 23:31:21,792 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=42146.666666666664, ans=0.07 2026-09-23 23:31:23,168 INFO [train.py:1192] (1/2) Epoch 14, batch 200, loss[loss=0.3773, simple_loss=0.4733, pruned_loss=0.1406, over 24221.00 frames. ], tot_loss[loss=0.3426, simple_loss=0.4331, pruned_loss=0.1261, over 3060267.58 frames. ], batch size: 257, lr: 1.46e-02, grad_scale: 32.0 2026-09-23 23:31:32,095 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.03 vs. limit=15.0 2026-09-23 23:31:33,413 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=42246.666666666664, ans=0.0 2026-09-23 23:31:40,905 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=42280.0, ans=0.05 2026-09-23 23:31:43,985 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=42313.333333333336, ans=0.05 2026-09-23 23:31:46,512 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=6.69 vs. limit=15.0 2026-09-23 23:31:48,780 INFO [train.py:1192] (1/2) Epoch 14, batch 250, loss[loss=0.3784, simple_loss=0.4696, pruned_loss=0.1436, over 24381.00 frames. ], tot_loss[loss=0.342, simple_loss=0.4322, pruned_loss=0.1259, over 3444018.84 frames. ], batch size: 225, lr: 1.45e-02, grad_scale: 32.0 2026-09-23 23:31:48,870 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=42346.666666666664, ans=0.0 2026-09-23 23:31:55,688 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.358e+02 3.519e+02 3.925e+02 4.523e+02 7.075e+02, threshold=7.850e+02, percent-clipped=0.0 2026-09-23 23:32:00,099 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=42413.333333333336, ans=0.125 2026-09-23 23:32:14,258 INFO [train.py:1192] (1/2) Epoch 14, batch 300, loss[loss=0.4004, simple_loss=0.4876, pruned_loss=0.1566, over 24561.00 frames. ], tot_loss[loss=0.3401, simple_loss=0.4308, pruned_loss=0.1247, over 3758139.00 frames. ], batch size: 204, lr: 1.45e-02, grad_scale: 32.0 2026-09-23 23:32:16,278 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=42513.333333333336, ans=0.125 2026-09-23 23:32:23,400 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=42546.666666666664, ans=0.025 2026-09-23 23:32:25,262 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.28 vs. limit=15.0 2026-09-23 23:32:36,390 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.09 vs. limit=15.0 2026-09-23 23:32:40,074 INFO [train.py:1192] (1/2) Epoch 14, batch 350, loss[loss=0.2914, simple_loss=0.3798, pruned_loss=0.1015, over 24582.00 frames. ], tot_loss[loss=0.3403, simple_loss=0.4312, pruned_loss=0.1247, over 3998358.55 frames. ], batch size: 137, lr: 1.45e-02, grad_scale: 32.0 2026-09-23 23:32:43,818 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.20 vs. limit=15.0 2026-09-23 23:32:47,135 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.578e+02 3.219e+02 3.708e+02 4.100e+02 5.899e+02, threshold=7.416e+02, percent-clipped=0.0 2026-09-23 23:32:54,887 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.57 vs. limit=15.0 2026-09-23 23:33:06,221 INFO [train.py:1192] (1/2) Epoch 14, batch 400, loss[loss=0.3589, simple_loss=0.4466, pruned_loss=0.1356, over 24598.00 frames. ], tot_loss[loss=0.3402, simple_loss=0.4311, pruned_loss=0.1247, over 4180725.33 frames. ], batch size: 170, lr: 1.45e-02, grad_scale: 32.0 2026-09-23 23:33:07,986 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.57 vs. limit=6.0 2026-09-23 23:33:13,804 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.96 vs. limit=22.5 2026-09-23 23:33:14,758 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=42880.0, ans=0.0 2026-09-23 23:33:16,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=42913.333333333336, ans=0.0 2026-09-23 23:33:17,062 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=42913.333333333336, ans=0.125 2026-09-23 23:33:28,559 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=42980.0, ans=0.1 2026-09-23 23:33:32,733 INFO [train.py:1192] (1/2) Epoch 14, batch 450, loss[loss=0.3209, simple_loss=0.4224, pruned_loss=0.1096, over 24631.00 frames. ], tot_loss[loss=0.3413, simple_loss=0.4319, pruned_loss=0.1254, over 4321179.15 frames. ], batch size: 175, lr: 1.45e-02, grad_scale: 32.0 2026-09-23 23:33:32,842 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=43013.333333333336, ans=0.0 2026-09-23 23:33:33,386 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.84 vs. limit=15.0 2026-09-23 23:33:39,875 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.447e+02 3.281e+02 3.675e+02 4.156e+02 6.613e+02, threshold=7.350e+02, percent-clipped=0.0 2026-09-23 23:33:49,691 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.44 vs. limit=15.0 2026-09-23 23:33:54,810 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=43146.666666666664, ans=0.125 2026-09-23 23:33:58,587 INFO [train.py:1192] (1/2) Epoch 14, batch 500, loss[loss=0.3762, simple_loss=0.4756, pruned_loss=0.1384, over 24504.00 frames. ], tot_loss[loss=0.3384, simple_loss=0.4293, pruned_loss=0.1238, over 4438949.08 frames. ], batch size: 218, lr: 1.44e-02, grad_scale: 32.0 2026-09-23 23:33:59,431 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=43180.0, ans=0.0014826086956521746 2026-09-23 23:34:18,157 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=43280.0, ans=0.0 2026-09-23 23:34:21,313 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=43313.333333333336, ans=0.125 2026-09-23 23:34:23,904 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=43313.333333333336, ans=0.0014536231884057955 2026-09-23 23:34:24,334 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=43346.666666666664, ans=0.125 2026-09-23 23:34:24,666 INFO [train.py:1192] (1/2) Epoch 14, batch 550, loss[loss=0.3698, simple_loss=0.4675, pruned_loss=0.136, over 24257.00 frames. ], tot_loss[loss=0.3376, simple_loss=0.429, pruned_loss=0.1231, over 4523727.58 frames. ], batch size: 257, lr: 1.44e-02, grad_scale: 32.0 2026-09-23 23:34:32,374 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.484e+02 3.155e+02 3.570e+02 4.011e+02 7.207e+02, threshold=7.140e+02, percent-clipped=0.0 2026-09-23 23:34:34,932 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=43413.333333333336, ans=0.0 2026-09-23 23:34:47,805 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.23 vs. limit=10.0 2026-09-23 23:34:50,849 INFO [train.py:1192] (1/2) Epoch 14, batch 600, loss[loss=0.4022, simple_loss=0.4946, pruned_loss=0.1549, over 24299.00 frames. ], tot_loss[loss=0.3377, simple_loss=0.4296, pruned_loss=0.1229, over 4590763.40 frames. ], batch size: 234, lr: 1.44e-02, grad_scale: 32.0 2026-09-23 23:35:17,058 INFO [train.py:1192] (1/2) Epoch 14, batch 650, loss[loss=0.2857, simple_loss=0.3928, pruned_loss=0.08926, over 24602.00 frames. ], tot_loss[loss=0.3359, simple_loss=0.4279, pruned_loss=0.1219, over 4654900.76 frames. ], batch size: 154, lr: 1.44e-02, grad_scale: 32.0 2026-09-23 23:35:23,868 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.23 vs. limit=15.0 2026-09-23 23:35:24,144 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.655e+02 3.233e+02 3.641e+02 4.223e+02 7.455e+02, threshold=7.283e+02, percent-clipped=1.0 2026-09-23 23:35:31,606 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=43746.666666666664, ans=0.0 2026-09-23 23:35:36,934 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=43780.0, ans=0.1 2026-09-23 23:35:42,908 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=43846.666666666664, ans=0.0 2026-09-23 23:35:43,297 INFO [train.py:1192] (1/2) Epoch 14, batch 700, loss[loss=0.3353, simple_loss=0.427, pruned_loss=0.1218, over 24565.00 frames. ], tot_loss[loss=0.3369, simple_loss=0.4292, pruned_loss=0.1224, over 4690240.14 frames. ], batch size: 158, lr: 1.44e-02, grad_scale: 32.0 2026-09-23 23:35:45,372 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=43846.666666666664, ans=0.0 2026-09-23 23:35:52,445 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=43880.0, ans=0.025 2026-09-23 23:35:52,968 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=43880.0, ans=0.125 2026-09-23 23:35:55,576 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.30 vs. limit=22.5 2026-09-23 23:35:59,080 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=43946.666666666664, ans=0.125 2026-09-23 23:35:59,096 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=43946.666666666664, ans=0.025 2026-09-23 23:36:09,620 INFO [train.py:1192] (1/2) Epoch 14, batch 750, loss[loss=0.3446, simple_loss=0.4371, pruned_loss=0.126, over 24660.00 frames. ], tot_loss[loss=0.337, simple_loss=0.4287, pruned_loss=0.1226, over 4726735.53 frames. ], batch size: 175, lr: 1.43e-02, grad_scale: 32.0 2026-09-23 23:36:16,552 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.458e+02 3.410e+02 3.887e+02 4.924e+02 8.810e+02, threshold=7.774e+02, percent-clipped=3.0 2026-09-23 23:36:18,973 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=44046.666666666664, ans=0.125 2026-09-23 23:36:20,029 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=44080.0, ans=0.1 2026-09-23 23:36:20,056 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=44080.0, ans=0.5 2026-09-23 23:36:23,775 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=44080.0, ans=0.025 2026-09-23 23:36:25,723 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=44113.333333333336, ans=0.125 2026-09-23 23:36:31,557 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=44146.666666666664, ans=0.2 2026-09-23 23:36:32,731 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.79 vs. limit=22.5 2026-09-23 23:36:34,547 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=44146.666666666664, ans=0.05 2026-09-23 23:36:35,425 INFO [train.py:1192] (1/2) Epoch 14, batch 800, loss[loss=0.2557, simple_loss=0.3618, pruned_loss=0.07477, over 24562.00 frames. ], tot_loss[loss=0.3366, simple_loss=0.4284, pruned_loss=0.1224, over 4752189.12 frames. ], batch size: 137, lr: 1.43e-02, grad_scale: 32.0 2026-09-23 23:37:01,134 INFO [train.py:1192] (1/2) Epoch 14, batch 850, loss[loss=0.3411, simple_loss=0.4487, pruned_loss=0.1168, over 24534.00 frames. ], tot_loss[loss=0.3355, simple_loss=0.4275, pruned_loss=0.1217, over 4770480.10 frames. ], batch size: 204, lr: 1.43e-02, grad_scale: 32.0 2026-09-23 23:37:01,236 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=44346.666666666664, ans=0.1 2026-09-23 23:37:08,655 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.320e+02 3.426e+02 4.029e+02 4.509e+02 7.575e+02, threshold=8.058e+02, percent-clipped=0.0 2026-09-23 23:37:09,163 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=44380.0, ans=0.0012217391304347822 2026-09-23 23:37:10,087 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=44380.0, ans=0.1 2026-09-23 23:37:17,383 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.87 vs. limit=12.0 2026-09-23 23:37:23,324 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=44480.0, ans=0.125 2026-09-23 23:37:25,292 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=44480.0, ans=0.025 2026-09-23 23:37:27,244 INFO [train.py:1192] (1/2) Epoch 14, batch 900, loss[loss=0.3013, simple_loss=0.3918, pruned_loss=0.1055, over 24559.00 frames. ], tot_loss[loss=0.3364, simple_loss=0.428, pruned_loss=0.1224, over 4781843.12 frames. ], batch size: 137, lr: 1.43e-02, grad_scale: 32.0 2026-09-23 23:37:27,370 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=44513.333333333336, ans=0.125 2026-09-23 23:37:35,674 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=44546.666666666664, ans=0.1 2026-09-23 23:37:43,227 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=44613.333333333336, ans=0.2 2026-09-23 23:37:48,303 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=44646.666666666664, ans=0.125 2026-09-23 23:37:52,366 INFO [train.py:1192] (1/2) Epoch 14, batch 950, loss[loss=0.4769, simple_loss=0.486, pruned_loss=0.2338, over 10945.00 frames. ], tot_loss[loss=0.338, simple_loss=0.4276, pruned_loss=0.1242, over 4711376.43 frames. ], batch size: 333, lr: 1.43e-02, grad_scale: 32.0 2026-09-23 23:38:04,065 INFO [train.py:1192] (1/2) Epoch 15, batch 0, loss[loss=0.2922, simple_loss=0.3875, pruned_loss=0.09843, over 24558.00 frames. ], tot_loss[loss=0.2922, simple_loss=0.3875, pruned_loss=0.09843, over 24558.00 frames. ], batch size: 137, lr: 1.38e-02, grad_scale: 32.0 2026-09-23 23:38:04,065 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 23:38:06,031 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2767, 2.4196, 2.7041, 2.5139, 2.1751, 2.6189, 1.5157, 2.1382], device='cuda:1') 2026-09-23 23:38:07,557 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8627, 2.5706, 3.4902, 1.6634], device='cuda:1') 2026-09-23 23:38:15,992 INFO [train.py:1224] (1/2) Epoch 15, validation: loss=0.2061, simple_loss=0.3207, pruned_loss=0.04577, over 2564189.00 frames. 2026-09-23 23:38:15,992 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 23:38:16,520 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=44706.666666666664, ans=0.125 2026-09-23 23:38:17,761 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=44706.666666666664, ans=0.125 2026-09-23 23:38:19,721 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.501e+02 3.380e+02 3.957e+02 4.664e+02 1.051e+03, threshold=7.915e+02, percent-clipped=1.0 2026-09-23 23:38:22,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=44740.0, ans=0.1 2026-09-23 23:38:24,556 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=44740.0, ans=0.035 2026-09-23 23:38:25,040 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=44740.0, ans=0.125 2026-09-23 23:38:30,508 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=44773.333333333336, ans=0.125 2026-09-23 23:38:31,341 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=44806.666666666664, ans=0.125 2026-09-23 23:38:32,437 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=44806.666666666664, ans=0.125 2026-09-23 23:38:41,708 INFO [train.py:1192] (1/2) Epoch 15, batch 50, loss[loss=0.2722, simple_loss=0.3623, pruned_loss=0.09104, over 24234.00 frames. ], tot_loss[loss=0.3443, simple_loss=0.4342, pruned_loss=0.1272, over 1081727.01 frames. ], batch size: 125, lr: 1.37e-02, grad_scale: 32.0 2026-09-23 23:38:47,225 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=44906.666666666664, ans=0.125 2026-09-23 23:39:07,645 INFO [train.py:1192] (1/2) Epoch 15, batch 100, loss[loss=0.3313, simple_loss=0.4148, pruned_loss=0.1239, over 24584.00 frames. ], tot_loss[loss=0.3458, simple_loss=0.4375, pruned_loss=0.1271, over 1915321.60 frames. ], batch size: 154, lr: 1.37e-02, grad_scale: 32.0 2026-09-23 23:39:10,964 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.413e+02 3.189e+02 3.581e+02 3.932e+02 5.918e+02, threshold=7.163e+02, percent-clipped=0.0 2026-09-23 23:39:12,527 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=45073.333333333336, ans=0.125 2026-09-23 23:39:15,652 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=45073.333333333336, ans=0.1 2026-09-23 23:39:16,147 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=45073.333333333336, ans=0.2 2026-09-23 23:39:27,590 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=45140.0, ans=0.5 2026-09-23 23:39:34,012 INFO [train.py:1192] (1/2) Epoch 15, batch 150, loss[loss=0.279, simple_loss=0.3665, pruned_loss=0.09579, over 24228.00 frames. ], tot_loss[loss=0.3408, simple_loss=0.4321, pruned_loss=0.1247, over 2560930.67 frames. ], batch size: 125, lr: 1.37e-02, grad_scale: 32.0 2026-09-23 23:39:45,817 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=45273.333333333336, ans=0.125 2026-09-23 23:39:56,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=45340.0, ans=0.0 2026-09-23 23:39:59,901 INFO [train.py:1192] (1/2) Epoch 15, batch 200, loss[loss=0.3717, simple_loss=0.4697, pruned_loss=0.1369, over 24175.00 frames. ], tot_loss[loss=0.3375, simple_loss=0.4295, pruned_loss=0.1227, over 3060454.34 frames. ], batch size: 257, lr: 1.37e-02, grad_scale: 32.0 2026-09-23 23:40:03,854 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.661e+02 3.367e+02 3.964e+02 4.660e+02 6.467e+02, threshold=7.928e+02, percent-clipped=0.0 2026-09-23 23:40:20,263 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=45506.666666666664, ans=0.125 2026-09-23 23:40:23,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=45506.666666666664, ans=0.1 2026-09-23 23:40:24,636 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.15 vs. limit=10.0 2026-09-23 23:40:25,871 INFO [train.py:1192] (1/2) Epoch 15, batch 250, loss[loss=0.3609, simple_loss=0.4582, pruned_loss=0.1318, over 24371.00 frames. ], tot_loss[loss=0.3371, simple_loss=0.4286, pruned_loss=0.1228, over 3445583.26 frames. ], batch size: 225, lr: 1.37e-02, grad_scale: 32.0 2026-09-23 23:40:37,809 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.14 vs. limit=6.0 2026-09-23 23:40:48,643 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.95 vs. limit=15.0 2026-09-23 23:40:51,266 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=45673.333333333336, ans=0.0 2026-09-23 23:40:52,021 INFO [train.py:1192] (1/2) Epoch 15, batch 300, loss[loss=0.3356, simple_loss=0.4381, pruned_loss=0.1166, over 24541.00 frames. ], tot_loss[loss=0.3355, simple_loss=0.4274, pruned_loss=0.1218, over 3758133.68 frames. ], batch size: 204, lr: 1.36e-02, grad_scale: 32.0 2026-09-23 23:40:55,402 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=45706.666666666664, ans=0.1 2026-09-23 23:40:55,733 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.700e+02 3.209e+02 3.542e+02 4.096e+02 7.328e+02, threshold=7.083e+02, percent-clipped=0.0 2026-09-23 23:40:57,381 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=45740.0, ans=0.125 2026-09-23 23:41:07,528 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=45806.666666666664, ans=0.0 2026-09-23 23:41:08,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=45806.666666666664, ans=0.1 2026-09-23 23:41:18,350 INFO [train.py:1192] (1/2) Epoch 15, batch 350, loss[loss=0.2804, simple_loss=0.378, pruned_loss=0.09134, over 24586.00 frames. ], tot_loss[loss=0.3361, simple_loss=0.4282, pruned_loss=0.122, over 3998440.07 frames. ], batch size: 137, lr: 1.36e-02, grad_scale: 32.0 2026-09-23 23:41:18,426 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=45873.333333333336, ans=0.0 2026-09-23 23:41:26,740 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys.whitening_limit, batch_count=45906.666666666664, ans=6.0 2026-09-23 23:41:27,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=45906.666666666664, ans=0.125 2026-09-23 23:41:32,761 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=5.37 vs. limit=15.0 2026-09-23 23:41:42,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=46006.666666666664, ans=0.0 2026-09-23 23:41:44,561 INFO [train.py:1192] (1/2) Epoch 15, batch 400, loss[loss=0.3422, simple_loss=0.432, pruned_loss=0.1263, over 24557.00 frames. ], tot_loss[loss=0.3349, simple_loss=0.4272, pruned_loss=0.1212, over 4179594.02 frames. ], batch size: 170, lr: 1.36e-02, grad_scale: 32.0 2026-09-23 23:41:46,168 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=46040.0, ans=0.125 2026-09-23 23:41:47,914 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.656e+02 3.547e+02 3.870e+02 4.473e+02 8.428e+02, threshold=7.740e+02, percent-clipped=1.0 2026-09-23 23:41:53,334 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=46073.333333333336, ans=0.125 2026-09-23 23:42:01,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=46140.0, ans=0.0008391304347826079 2026-09-23 23:42:10,753 INFO [train.py:1192] (1/2) Epoch 15, batch 450, loss[loss=0.345, simple_loss=0.4407, pruned_loss=0.1247, over 24612.00 frames. ], tot_loss[loss=0.3352, simple_loss=0.4274, pruned_loss=0.1215, over 4319431.78 frames. ], batch size: 175, lr: 1.36e-02, grad_scale: 32.0 2026-09-23 23:42:10,846 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=46206.666666666664, ans=0.0 2026-09-23 23:42:12,181 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=46206.666666666664, ans=0.025 2026-09-23 23:42:13,308 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=46206.666666666664, ans=0.125 2026-09-23 23:42:36,256 INFO [train.py:1192] (1/2) Epoch 15, batch 500, loss[loss=0.3872, simple_loss=0.4757, pruned_loss=0.1494, over 24514.00 frames. ], tot_loss[loss=0.3331, simple_loss=0.4254, pruned_loss=0.1204, over 4437168.11 frames. ], batch size: 218, lr: 1.36e-02, grad_scale: 32.0 2026-09-23 23:42:38,480 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=46373.333333333336, ans=0.04949747468305833 2026-09-23 23:42:39,842 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.477e+02 3.140e+02 3.597e+02 4.164e+02 8.514e+02, threshold=7.194e+02, percent-clipped=1.0 2026-09-23 23:42:48,410 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=46440.0, ans=0.0 2026-09-23 23:42:58,295 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.57 vs. limit=12.0 2026-09-23 23:43:02,401 INFO [train.py:1192] (1/2) Epoch 15, batch 550, loss[loss=0.3506, simple_loss=0.4472, pruned_loss=0.127, over 24251.00 frames. ], tot_loss[loss=0.3336, simple_loss=0.426, pruned_loss=0.1206, over 4522400.63 frames. ], batch size: 257, lr: 1.36e-02, grad_scale: 32.0 2026-09-23 23:43:22,137 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=46640.0, ans=0.125 2026-09-23 23:43:25,032 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=46673.333333333336, ans=10.0 2026-09-23 23:43:28,121 INFO [train.py:1192] (1/2) Epoch 15, batch 600, loss[loss=0.3328, simple_loss=0.4386, pruned_loss=0.1134, over 24323.00 frames. ], tot_loss[loss=0.3334, simple_loss=0.4262, pruned_loss=0.1203, over 4588551.27 frames. ], batch size: 234, lr: 1.35e-02, grad_scale: 32.0 2026-09-23 23:43:29,079 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:43:31,839 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.550e+02 3.078e+02 3.546e+02 4.123e+02 7.010e+02, threshold=7.092e+02, percent-clipped=0.0 2026-09-23 23:43:32,830 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=46740.0, ans=0.0 2026-09-23 23:43:38,596 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=46773.333333333336, ans=0.125 2026-09-23 23:43:42,479 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.10 vs. limit=15.0 2026-09-23 23:43:44,594 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=46806.666666666664, ans=0.1 2026-09-23 23:43:53,374 INFO [train.py:1192] (1/2) Epoch 15, batch 650, loss[loss=0.3119, simple_loss=0.4033, pruned_loss=0.1103, over 24595.00 frames. ], tot_loss[loss=0.3306, simple_loss=0.4241, pruned_loss=0.1185, over 4653632.99 frames. ], batch size: 154, lr: 1.35e-02, grad_scale: 32.0 2026-09-23 23:43:56,888 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=46873.333333333336, ans=0.0006797101449275353 2026-09-23 23:43:59,321 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=46906.666666666664, ans=0.125 2026-09-23 23:44:14,425 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=47006.666666666664, ans=0.025 2026-09-23 23:44:19,589 INFO [train.py:1192] (1/2) Epoch 15, batch 700, loss[loss=0.3058, simple_loss=0.4066, pruned_loss=0.1025, over 24557.00 frames. ], tot_loss[loss=0.3321, simple_loss=0.4256, pruned_loss=0.1193, over 4689471.23 frames. ], batch size: 158, lr: 1.35e-02, grad_scale: 32.0 2026-09-23 23:44:20,516 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=47040.0, ans=0.2 2026-09-23 23:44:23,067 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.64 vs. limit=22.5 2026-09-23 23:44:23,381 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.653e+02 3.383e+02 3.864e+02 4.776e+02 7.601e+02, threshold=7.728e+02, percent-clipped=2.0 2026-09-23 23:44:28,389 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=47073.333333333336, ans=0.125 2026-09-23 23:44:31,697 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=47106.666666666664, ans=0.035 2026-09-23 23:44:36,721 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=47140.0, ans=6.0 2026-09-23 23:44:37,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=47140.0, ans=0.1 2026-09-23 23:44:42,018 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.51 vs. limit=15.0 2026-09-23 23:44:45,982 INFO [train.py:1192] (1/2) Epoch 15, batch 750, loss[loss=0.3201, simple_loss=0.4223, pruned_loss=0.109, over 24640.00 frames. ], tot_loss[loss=0.3319, simple_loss=0.425, pruned_loss=0.1194, over 4725923.14 frames. ], batch size: 175, lr: 1.35e-02, grad_scale: 32.0 2026-09-23 23:44:46,986 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=47206.666666666664, ans=0.04949747468305833 2026-09-23 23:44:49,494 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=5.53 vs. limit=15.0 2026-09-23 23:44:51,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=47240.0, ans=0.125 2026-09-23 23:44:51,610 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=47240.0, ans=0.125 2026-09-23 23:44:59,976 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=11.86 vs. limit=15.0 2026-09-23 23:45:07,681 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=47340.0, ans=0.1 2026-09-23 23:45:11,018 INFO [train.py:1192] (1/2) Epoch 15, batch 800, loss[loss=0.3025, simple_loss=0.3917, pruned_loss=0.1066, over 24551.00 frames. ], tot_loss[loss=0.3311, simple_loss=0.4243, pruned_loss=0.119, over 4752950.52 frames. ], batch size: 137, lr: 1.35e-02, grad_scale: 32.0 2026-09-23 23:45:14,710 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.721e+02 3.390e+02 3.796e+02 4.598e+02 6.702e+02, threshold=7.592e+02, percent-clipped=0.0 2026-09-23 23:45:17,990 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=47406.666666666664, ans=0.0 2026-09-23 23:45:22,800 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=47440.0, ans=0.125 2026-09-23 23:45:24,861 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.26 vs. limit=15.0 2026-09-23 23:45:29,597 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.27 vs. limit=10.0 2026-09-23 23:45:32,213 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=47506.666666666664, ans=0.04949747468305833 2026-09-23 23:45:37,138 INFO [train.py:1192] (1/2) Epoch 15, batch 850, loss[loss=0.3797, simple_loss=0.4728, pruned_loss=0.1433, over 24557.00 frames. ], tot_loss[loss=0.3303, simple_loss=0.4237, pruned_loss=0.1185, over 4771274.23 frames. ], batch size: 204, lr: 1.34e-02, grad_scale: 32.0 2026-09-23 23:45:42,478 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=47573.333333333336, ans=0.1 2026-09-23 23:45:45,564 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=47573.333333333336, ans=0.2 2026-09-23 23:45:55,551 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=47640.0, ans=0.1 2026-09-23 23:45:57,227 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.11 vs. limit=15.0 2026-09-23 23:45:58,726 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=47673.333333333336, ans=0.5 2026-09-23 23:45:59,908 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.55 vs. limit=6.0 2026-09-23 23:46:03,087 INFO [train.py:1192] (1/2) Epoch 15, batch 900, loss[loss=0.2993, simple_loss=0.39, pruned_loss=0.1043, over 24567.00 frames. ], tot_loss[loss=0.3319, simple_loss=0.4247, pruned_loss=0.1196, over 4782121.46 frames. ], batch size: 137, lr: 1.34e-02, grad_scale: 32.0 2026-09-23 23:46:06,710 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.630e+02 3.321e+02 3.769e+02 4.391e+02 7.411e+02, threshold=7.539e+02, percent-clipped=0.0 2026-09-23 23:46:16,536 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=47773.333333333336, ans=0.125 2026-09-23 23:46:22,781 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=47806.666666666664, ans=0.0004768115942028993 2026-09-23 23:46:28,313 INFO [train.py:1192] (1/2) Epoch 15, batch 950, loss[loss=0.4928, simple_loss=0.5133, pruned_loss=0.2362, over 10869.00 frames. ], tot_loss[loss=0.3321, simple_loss=0.4232, pruned_loss=0.1205, over 4711372.36 frames. ], batch size: 333, lr: 1.34e-02, grad_scale: 32.0 2026-09-23 23:46:28,443 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=47873.333333333336, ans=0.125 2026-09-23 23:46:40,276 INFO [train.py:1192] (1/2) Epoch 16, batch 0, loss[loss=0.2794, simple_loss=0.3881, pruned_loss=0.08531, over 24584.00 frames. ], tot_loss[loss=0.2794, simple_loss=0.3881, pruned_loss=0.08531, over 24584.00 frames. ], batch size: 137, lr: 1.30e-02, grad_scale: 32.0 2026-09-23 23:46:40,276 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 23:46:51,011 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.9992, 3.6218, 3.9068, 3.6013], device='cuda:1') 2026-09-23 23:46:52,127 INFO [train.py:1224] (1/2) Epoch 16, validation: loss=0.2069, simple_loss=0.3218, pruned_loss=0.04601, over 2564189.00 frames. 2026-09-23 23:46:52,127 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 23:46:56,924 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=47933.333333333336, ans=0.125 2026-09-23 23:47:04,774 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=47966.666666666664, ans=0.1 2026-09-23 23:47:09,690 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=48000.0, ans=0.0004347826086956528 2026-09-23 23:47:17,138 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.88 vs. limit=15.0 2026-09-23 23:47:17,777 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.506e+02 3.512e+02 4.009e+02 4.590e+02 7.436e+02, threshold=8.017e+02, percent-clipped=0.0 2026-09-23 23:47:17,864 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=48066.666666666664, ans=0.000420289855072465 2026-09-23 23:47:18,323 INFO [train.py:1192] (1/2) Epoch 16, batch 50, loss[loss=0.2541, simple_loss=0.3484, pruned_loss=0.07994, over 24252.00 frames. ], tot_loss[loss=0.3371, simple_loss=0.43, pruned_loss=0.1221, over 1080373.32 frames. ], batch size: 125, lr: 1.30e-02, grad_scale: 32.0 2026-09-23 23:47:23,381 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=48100.0, ans=0.0 2026-09-23 23:47:23,888 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=48100.0, ans=0.125 2026-09-23 23:47:28,728 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=48133.333333333336, ans=0.2 2026-09-23 23:47:41,392 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=48200.0, ans=0.1 2026-09-23 23:47:44,711 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=48233.333333333336, ans=0.1 2026-09-23 23:47:45,059 INFO [train.py:1192] (1/2) Epoch 16, batch 100, loss[loss=0.3257, simple_loss=0.4105, pruned_loss=0.1205, over 24622.00 frames. ], tot_loss[loss=0.3409, simple_loss=0.4348, pruned_loss=0.1235, over 1914314.84 frames. ], batch size: 154, lr: 1.29e-02, grad_scale: 32.0 2026-09-23 23:47:54,270 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=48266.666666666664, ans=0.125 2026-09-23 23:48:10,500 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.527e+02 3.264e+02 3.645e+02 3.998e+02 6.612e+02, threshold=7.291e+02, percent-clipped=0.0 2026-09-23 23:48:10,889 INFO [train.py:1192] (1/2) Epoch 16, batch 150, loss[loss=0.259, simple_loss=0.3511, pruned_loss=0.08348, over 24315.00 frames. ], tot_loss[loss=0.3351, simple_loss=0.4289, pruned_loss=0.1206, over 2559499.22 frames. ], batch size: 125, lr: 1.29e-02, grad_scale: 32.0 2026-09-23 23:48:22,668 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=48466.666666666664, ans=0.125 2026-09-23 23:48:33,598 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=48533.333333333336, ans=0.0003188405797101435 2026-09-23 23:48:36,699 INFO [train.py:1192] (1/2) Epoch 16, batch 200, loss[loss=0.3604, simple_loss=0.4615, pruned_loss=0.1297, over 24186.00 frames. ], tot_loss[loss=0.3316, simple_loss=0.4259, pruned_loss=0.1186, over 3058789.35 frames. ], batch size: 257, lr: 1.29e-02, grad_scale: 32.0 2026-09-23 23:48:52,948 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=48666.666666666664, ans=0.1 2026-09-23 23:49:02,677 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.320e+02 3.341e+02 3.785e+02 4.483e+02 7.624e+02, threshold=7.571e+02, percent-clipped=1.0 2026-09-23 23:49:03,195 INFO [train.py:1192] (1/2) Epoch 16, batch 250, loss[loss=0.3627, simple_loss=0.4597, pruned_loss=0.1329, over 24381.00 frames. ], tot_loss[loss=0.3316, simple_loss=0.4253, pruned_loss=0.119, over 3442701.22 frames. ], batch size: 225, lr: 1.29e-02, grad_scale: 32.0 2026-09-23 23:49:12,119 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=48766.666666666664, ans=0.0 2026-09-23 23:49:15,044 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=48800.0, ans=0.125 2026-09-23 23:49:15,470 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=48800.0, ans=0.2 2026-09-23 23:49:17,502 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.34 vs. limit=22.5 2026-09-23 23:49:24,209 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.37 vs. limit=15.0 2026-09-23 23:49:28,663 INFO [train.py:1192] (1/2) Epoch 16, batch 300, loss[loss=0.3706, simple_loss=0.4732, pruned_loss=0.134, over 24519.00 frames. ], tot_loss[loss=0.3304, simple_loss=0.4244, pruned_loss=0.1182, over 3756039.28 frames. ], batch size: 204, lr: 1.29e-02, grad_scale: 32.0 2026-09-23 23:49:36,635 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.48 vs. limit=15.0 2026-09-23 23:49:46,356 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=49000.0, ans=0.07 2026-09-23 23:49:53,859 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.703e+02 3.205e+02 3.601e+02 4.111e+02 5.860e+02, threshold=7.201e+02, percent-clipped=0.0 2026-09-23 23:49:54,410 INFO [train.py:1192] (1/2) Epoch 16, batch 350, loss[loss=0.3108, simple_loss=0.3929, pruned_loss=0.1143, over 24579.00 frames. ], tot_loss[loss=0.3303, simple_loss=0.4247, pruned_loss=0.118, over 3997024.07 frames. ], batch size: 137, lr: 1.28e-02, grad_scale: 32.0 2026-09-23 23:49:58,125 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=49066.666666666664, ans=0.125 2026-09-23 23:49:58,440 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.57 vs. limit=15.0 2026-09-23 23:50:01,253 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=49100.0, ans=0.125 2026-09-23 23:50:03,915 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=49100.0, ans=0.125 2026-09-23 23:50:11,196 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=49166.666666666664, ans=0.05 2026-09-23 23:50:13,100 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=49166.666666666664, ans=0.125 2026-09-23 23:50:19,957 INFO [train.py:1192] (1/2) Epoch 16, batch 400, loss[loss=0.3273, simple_loss=0.4181, pruned_loss=0.1183, over 24578.00 frames. ], tot_loss[loss=0.3289, simple_loss=0.4234, pruned_loss=0.1173, over 4181654.23 frames. ], batch size: 170, lr: 1.28e-02, grad_scale: 32.0 2026-09-23 23:50:22,717 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=49233.333333333336, ans=0.125 2026-09-23 23:50:23,273 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.19 vs. limit=22.5 2026-09-23 23:50:37,017 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=49333.333333333336, ans=0.07 2026-09-23 23:50:37,160 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.11 vs. limit=12.0 2026-09-23 23:50:40,909 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=49366.666666666664, ans=0.125 2026-09-23 23:50:45,306 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.507e+02 3.240e+02 3.670e+02 4.199e+02 6.759e+02, threshold=7.340e+02, percent-clipped=0.0 2026-09-23 23:50:45,829 INFO [train.py:1192] (1/2) Epoch 16, batch 450, loss[loss=0.3439, simple_loss=0.4429, pruned_loss=0.1224, over 24629.00 frames. ], tot_loss[loss=0.3299, simple_loss=0.4242, pruned_loss=0.1178, over 4321923.60 frames. ], batch size: 175, lr: 1.28e-02, grad_scale: 32.0 2026-09-23 23:50:49,938 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=49400.0, ans=0.125 2026-09-23 23:51:11,446 INFO [train.py:1192] (1/2) Epoch 16, batch 500, loss[loss=0.3781, simple_loss=0.4693, pruned_loss=0.1435, over 24522.00 frames. ], tot_loss[loss=0.327, simple_loss=0.4217, pruned_loss=0.1161, over 4439128.88 frames. ], batch size: 218, lr: 1.28e-02, grad_scale: 32.0 2026-09-23 23:51:13,066 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=49566.666666666664, ans=0.0 2026-09-23 23:51:21,883 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:51:22,461 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=49633.333333333336, ans=0.125 2026-09-23 23:51:23,311 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:51:26,459 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.52 vs. limit=15.0 2026-09-23 23:51:29,725 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=49666.666666666664, ans=0.1 2026-09-23 23:51:36,977 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.627e+02 3.089e+02 3.648e+02 4.070e+02 6.215e+02, threshold=7.297e+02, percent-clipped=0.0 2026-09-23 23:51:37,496 INFO [train.py:1192] (1/2) Epoch 16, batch 550, loss[loss=0.3747, simple_loss=0.4576, pruned_loss=0.1459, over 24263.00 frames. ], tot_loss[loss=0.3274, simple_loss=0.4221, pruned_loss=0.1163, over 4523969.21 frames. ], batch size: 257, lr: 1.28e-02, grad_scale: 32.0 2026-09-23 23:51:41,449 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=49733.333333333336, ans=0.2 2026-09-23 23:51:48,510 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=49800.0, ans=0.125 2026-09-23 23:51:53,745 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.51 vs. limit=15.0 2026-09-23 23:51:56,962 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=49833.333333333336, ans=0.125 2026-09-23 23:52:03,873 INFO [train.py:1192] (1/2) Epoch 16, batch 600, loss[loss=0.3549, simple_loss=0.4517, pruned_loss=0.129, over 24336.00 frames. ], tot_loss[loss=0.3282, simple_loss=0.4229, pruned_loss=0.1168, over 4589288.41 frames. ], batch size: 234, lr: 1.28e-02, grad_scale: 32.0 2026-09-23 23:52:16,951 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=49966.666666666664, ans=0.125 2026-09-23 23:52:26,490 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=50033.333333333336, ans=0.1 2026-09-23 23:52:29,268 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.504e+02 3.407e+02 4.001e+02 4.729e+02 7.317e+02, threshold=8.001e+02, percent-clipped=1.0 2026-09-23 23:52:29,790 INFO [train.py:1192] (1/2) Epoch 16, batch 650, loss[loss=0.3165, simple_loss=0.4151, pruned_loss=0.109, over 24615.00 frames. ], tot_loss[loss=0.3259, simple_loss=0.4213, pruned_loss=0.1153, over 4654026.92 frames. ], batch size: 154, lr: 1.27e-02, grad_scale: 32.0 2026-09-23 23:52:34,870 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=50100.0, ans=0.125 2026-09-23 23:52:55,355 INFO [train.py:1192] (1/2) Epoch 16, batch 700, loss[loss=0.3485, simple_loss=0.4305, pruned_loss=0.1332, over 24559.00 frames. ], tot_loss[loss=0.3273, simple_loss=0.4226, pruned_loss=0.116, over 4690123.34 frames. ], batch size: 158, lr: 1.27e-02, grad_scale: 32.0 2026-09-23 23:52:58,033 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=50233.333333333336, ans=0.1 2026-09-23 23:53:04,481 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=50266.666666666664, ans=0.125 2026-09-23 23:53:05,535 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=50300.0, ans=0.0 2026-09-23 23:53:07,075 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=50300.0, ans=0.1 2026-09-23 23:53:18,839 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=50366.666666666664, ans=0.0 2026-09-23 23:53:19,911 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=50366.666666666664, ans=0.125 2026-09-23 23:53:21,226 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.364e+02 3.437e+02 3.892e+02 4.404e+02 6.968e+02, threshold=7.784e+02, percent-clipped=0.0 2026-09-23 23:53:21,674 INFO [train.py:1192] (1/2) Epoch 16, batch 750, loss[loss=0.3312, simple_loss=0.4263, pruned_loss=0.1181, over 24635.00 frames. ], tot_loss[loss=0.3278, simple_loss=0.4226, pruned_loss=0.1165, over 4725930.80 frames. ], batch size: 175, lr: 1.27e-02, grad_scale: 32.0 2026-09-23 23:53:28,526 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=50433.333333333336, ans=0.0 2026-09-23 23:53:33,324 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.39 vs. limit=15.0 2026-09-23 23:53:34,373 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=50466.666666666664, ans=0.125 2026-09-23 23:53:39,328 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=50500.0, ans=0.125 2026-09-23 23:53:40,051 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.11 vs. limit=15.0 2026-09-23 23:53:45,027 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=50533.333333333336, ans=0.0 2026-09-23 23:53:48,137 INFO [train.py:1192] (1/2) Epoch 16, batch 800, loss[loss=0.2502, simple_loss=0.3624, pruned_loss=0.06901, over 24563.00 frames. ], tot_loss[loss=0.3275, simple_loss=0.4223, pruned_loss=0.1163, over 4752138.66 frames. ], batch size: 137, lr: 1.27e-02, grad_scale: 32.0 2026-09-23 23:53:53,150 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.66 vs. limit=15.0 2026-09-23 23:53:54,000 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=50600.0, ans=0.07 2026-09-23 23:54:13,382 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.330e+02 3.262e+02 3.651e+02 4.190e+02 5.928e+02, threshold=7.301e+02, percent-clipped=0.0 2026-09-23 23:54:13,924 INFO [train.py:1192] (1/2) Epoch 16, batch 850, loss[loss=0.3806, simple_loss=0.4725, pruned_loss=0.1443, over 24543.00 frames. ], tot_loss[loss=0.3273, simple_loss=0.4218, pruned_loss=0.1164, over 4771042.58 frames. ], batch size: 204, lr: 1.27e-02, grad_scale: 32.0 2026-09-23 23:54:18,819 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=50766.666666666664, ans=0.125 2026-09-23 23:54:24,496 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=50800.0, ans=0.0 2026-09-23 23:54:25,962 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=50800.0, ans=0.125 2026-09-23 23:54:32,951 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=50833.333333333336, ans=0.1 2026-09-23 23:54:33,427 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=50833.333333333336, ans=0.1 2026-09-23 23:54:36,917 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=50866.666666666664, ans=0.025 2026-09-23 23:54:39,972 INFO [train.py:1192] (1/2) Epoch 16, batch 900, loss[loss=0.2875, simple_loss=0.3818, pruned_loss=0.09665, over 24547.00 frames. ], tot_loss[loss=0.3285, simple_loss=0.4226, pruned_loss=0.1172, over 4781726.14 frames. ], batch size: 137, lr: 1.27e-02, grad_scale: 32.0 2026-09-23 23:54:41,382 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=50900.0, ans=0.0 2026-09-23 23:54:51,130 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=50966.666666666664, ans=0.125 2026-09-23 23:54:51,175 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=50966.666666666664, ans=0.0 2026-09-23 23:54:57,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=51000.0, ans=0.035 2026-09-23 23:54:59,884 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=51033.333333333336, ans=0.0 2026-09-23 23:55:00,421 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=51033.333333333336, ans=0.125 2026-09-23 23:55:04,778 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.543e+02 3.475e+02 3.921e+02 4.492e+02 7.001e+02, threshold=7.841e+02, percent-clipped=0.0 2026-09-23 23:55:05,291 INFO [train.py:1192] (1/2) Epoch 16, batch 950, loss[loss=0.4392, simple_loss=0.4669, pruned_loss=0.2058, over 10953.00 frames. ], tot_loss[loss=0.3289, simple_loss=0.4212, pruned_loss=0.1183, over 4713771.70 frames. ], batch size: 333, lr: 1.26e-02, grad_scale: 32.0 2026-09-23 23:55:05,913 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=7.46 vs. limit=6.0 2026-09-23 23:55:15,796 INFO [train.py:1192] (1/2) Epoch 17, batch 0, loss[loss=0.3187, simple_loss=0.4101, pruned_loss=0.1137, over 24549.00 frames. ], tot_loss[loss=0.3187, simple_loss=0.4101, pruned_loss=0.1137, over 24549.00 frames. ], batch size: 137, lr: 1.23e-02, grad_scale: 32.0 2026-09-23 23:55:15,796 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-23 23:55:17,215 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1252, 2.0723, 2.9336, 1.8131], device='cuda:1') 2026-09-23 23:55:21,984 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.4307, 2.7133, 2.9954, 2.6949, 2.2917, 2.7878, 1.4431, 2.4619], device='cuda:1') 2026-09-23 23:55:22,357 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8755, 2.4924, 3.4793, 1.5287], device='cuda:1') 2026-09-23 23:55:27,424 INFO [train.py:1224] (1/2) Epoch 17, validation: loss=0.2007, simple_loss=0.3174, pruned_loss=0.04202, over 2564189.00 frames. 2026-09-23 23:55:27,424 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-23 23:55:33,128 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=51126.666666666664, ans=0.0 2026-09-23 23:55:35,506 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=51126.666666666664, ans=0.125 2026-09-23 23:55:38,173 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=51160.0, ans=0.0 2026-09-23 23:55:40,268 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.91 vs. limit=15.0 2026-09-23 23:55:41,628 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=51160.0, ans=0.125 2026-09-23 23:55:50,336 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=51226.666666666664, ans=0.1 2026-09-23 23:55:52,507 INFO [train.py:1192] (1/2) Epoch 17, batch 50, loss[loss=0.2621, simple_loss=0.3601, pruned_loss=0.08211, over 24232.00 frames. ], tot_loss[loss=0.3375, simple_loss=0.4311, pruned_loss=0.1219, over 1081261.85 frames. ], batch size: 125, lr: 1.22e-02, grad_scale: 32.0 2026-09-23 23:55:52,586 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=51260.0, ans=0.2 2026-09-23 23:55:56,979 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=51293.333333333336, ans=0.2 2026-09-23 23:56:00,169 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=51293.333333333336, ans=0.125 2026-09-23 23:56:13,384 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.772e+02 3.266e+02 3.742e+02 4.253e+02 6.443e+02, threshold=7.483e+02, percent-clipped=0.0 2026-09-23 23:56:16,623 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=51393.333333333336, ans=0.025 2026-09-23 23:56:17,138 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=51426.666666666664, ans=0.1 2026-09-23 23:56:17,526 INFO [train.py:1192] (1/2) Epoch 17, batch 100, loss[loss=0.3139, simple_loss=0.4063, pruned_loss=0.1108, over 24625.00 frames. ], tot_loss[loss=0.3351, simple_loss=0.4315, pruned_loss=0.1194, over 1915429.57 frames. ], batch size: 154, lr: 1.22e-02, grad_scale: 64.0 2026-09-23 23:56:22,928 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=51460.0, ans=0.0 2026-09-23 23:56:30,211 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=51493.333333333336, ans=0.025 2026-09-23 23:56:40,104 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=51560.0, ans=0.025 2026-09-23 23:56:42,663 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=51593.333333333336, ans=0.125 2026-09-23 23:56:43,128 INFO [train.py:1192] (1/2) Epoch 17, batch 150, loss[loss=0.2491, simple_loss=0.3469, pruned_loss=0.0756, over 24282.00 frames. ], tot_loss[loss=0.3273, simple_loss=0.424, pruned_loss=0.1153, over 2561063.92 frames. ], batch size: 125, lr: 1.22e-02, grad_scale: 64.0 2026-09-23 23:56:46,537 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=51593.333333333336, ans=0.125 2026-09-23 23:56:47,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=51593.333333333336, ans=0.2 2026-09-23 23:56:47,981 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=51626.666666666664, ans=0.125 2026-09-23 23:56:54,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=51660.0, ans=0.5 2026-09-23 23:56:57,380 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=51660.0, ans=0.125 2026-09-23 23:57:04,108 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.461e+02 3.307e+02 3.706e+02 4.440e+02 7.679e+02, threshold=7.412e+02, percent-clipped=1.0 2026-09-23 23:57:06,420 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-23 23:57:09,035 INFO [train.py:1192] (1/2) Epoch 17, batch 200, loss[loss=0.3799, simple_loss=0.4741, pruned_loss=0.1428, over 24217.00 frames. ], tot_loss[loss=0.326, simple_loss=0.4224, pruned_loss=0.1148, over 3059547.09 frames. ], batch size: 257, lr: 1.22e-02, grad_scale: 32.0 2026-09-23 23:57:27,381 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=51860.0, ans=0.2 2026-09-23 23:57:27,399 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=51860.0, ans=0.125 2026-09-23 23:57:28,088 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.36 vs. limit=8.0 2026-09-23 23:57:28,270 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=51860.0, ans=0.125 2026-09-23 23:57:32,773 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=51893.333333333336, ans=0.125 2026-09-23 23:57:34,610 INFO [train.py:1192] (1/2) Epoch 17, batch 250, loss[loss=0.3349, simple_loss=0.4406, pruned_loss=0.1146, over 24372.00 frames. ], tot_loss[loss=0.3261, simple_loss=0.422, pruned_loss=0.1151, over 3444130.89 frames. ], batch size: 225, lr: 1.22e-02, grad_scale: 32.0 2026-09-23 23:57:46,436 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=51993.333333333336, ans=0.1 2026-09-23 23:57:49,868 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=52026.666666666664, ans=0.0 2026-09-23 23:57:53,096 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.98 vs. limit=6.0 2026-09-23 23:57:56,534 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.479e+02 3.375e+02 3.794e+02 4.631e+02 7.260e+02, threshold=7.588e+02, percent-clipped=0.0 2026-09-23 23:58:00,329 INFO [train.py:1192] (1/2) Epoch 17, batch 300, loss[loss=0.3834, simple_loss=0.4735, pruned_loss=0.1466, over 24522.00 frames. ], tot_loss[loss=0.3255, simple_loss=0.4214, pruned_loss=0.1148, over 3757121.43 frames. ], batch size: 204, lr: 1.22e-02, grad_scale: 16.0 2026-09-23 23:58:10,219 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=52160.0, ans=0.025 2026-09-23 23:58:11,068 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=52160.0, ans=0.2 2026-09-23 23:58:13,174 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=52160.0, ans=0.0 2026-09-23 23:58:13,287 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.96 vs. limit=12.0 2026-09-23 23:58:25,921 INFO [train.py:1192] (1/2) Epoch 17, batch 350, loss[loss=0.2787, simple_loss=0.3713, pruned_loss=0.09305, over 24554.00 frames. ], tot_loss[loss=0.3263, simple_loss=0.4219, pruned_loss=0.1154, over 3997767.67 frames. ], batch size: 137, lr: 1.21e-02, grad_scale: 16.0 2026-09-23 23:58:26,270 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.31 vs. limit=22.5 2026-09-23 23:58:28,686 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.20 vs. limit=22.5 2026-09-23 23:58:28,920 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=52260.0, ans=0.125 2026-09-23 23:58:28,927 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=52260.0, ans=0.2 2026-09-23 23:58:37,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=52326.666666666664, ans=0.125 2026-09-23 23:58:47,175 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=52393.333333333336, ans=0.5 2026-09-23 23:58:47,571 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.561e+02 3.171e+02 3.579e+02 4.200e+02 7.417e+02, threshold=7.158e+02, percent-clipped=0.0 2026-09-23 23:58:51,345 INFO [train.py:1192] (1/2) Epoch 17, batch 400, loss[loss=0.3118, simple_loss=0.412, pruned_loss=0.1058, over 24565.00 frames. ], tot_loss[loss=0.3254, simple_loss=0.421, pruned_loss=0.1149, over 4182677.61 frames. ], batch size: 170, lr: 1.21e-02, grad_scale: 32.0 2026-09-23 23:59:07,109 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=52526.666666666664, ans=15.0 2026-09-23 23:59:16,581 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=52593.333333333336, ans=0.125 2026-09-23 23:59:16,998 INFO [train.py:1192] (1/2) Epoch 17, batch 450, loss[loss=0.3272, simple_loss=0.4316, pruned_loss=0.1114, over 24654.00 frames. ], tot_loss[loss=0.3259, simple_loss=0.4216, pruned_loss=0.1152, over 4319758.29 frames. ], batch size: 175, lr: 1.21e-02, grad_scale: 32.0 2026-09-23 23:59:21,054 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=52593.333333333336, ans=0.125 2026-09-23 23:59:21,963 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=52626.666666666664, ans=0.1 2026-09-23 23:59:26,321 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=52660.0, ans=0.0 2026-09-23 23:59:27,214 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=52660.0, ans=0.1 2026-09-23 23:59:38,265 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.585e+02 3.216e+02 3.919e+02 4.471e+02 7.457e+02, threshold=7.837e+02, percent-clipped=1.0 2026-09-23 23:59:38,489 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.56 vs. limit=15.0 2026-09-23 23:59:41,435 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=52760.0, ans=0.125 2026-09-23 23:59:41,862 INFO [train.py:1192] (1/2) Epoch 17, batch 500, loss[loss=0.3937, simple_loss=0.4809, pruned_loss=0.1532, over 24489.00 frames. ], tot_loss[loss=0.3228, simple_loss=0.4189, pruned_loss=0.1134, over 4437600.31 frames. ], batch size: 218, lr: 1.21e-02, grad_scale: 32.0 2026-09-23 23:59:46,651 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=52793.333333333336, ans=0.1 2026-09-23 23:59:47,907 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=13.74 vs. limit=15.0 2026-09-23 23:59:50,943 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=52793.333333333336, ans=0.1 2026-09-23 23:59:58,948 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=52860.0, ans=0.1 2026-09-24 00:00:00,434 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=52860.0, ans=0.125 2026-09-24 00:00:07,637 INFO [train.py:1192] (1/2) Epoch 17, batch 550, loss[loss=0.3874, simple_loss=0.4725, pruned_loss=0.1512, over 24303.00 frames. ], tot_loss[loss=0.3234, simple_loss=0.4194, pruned_loss=0.1137, over 4523684.76 frames. ], batch size: 257, lr: 1.21e-02, grad_scale: 32.0 2026-09-24 00:00:15,364 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.26 vs. limit=15.0 2026-09-24 00:00:16,129 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=52960.0, ans=0.125 2026-09-24 00:00:21,618 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=52993.333333333336, ans=0.0 2026-09-24 00:00:25,912 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=53026.666666666664, ans=0.125 2026-09-24 00:00:27,718 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=53060.0, ans=0.0 2026-09-24 00:00:29,244 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=53060.0, ans=0.125 2026-09-24 00:00:29,552 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.579e+02 3.286e+02 3.817e+02 4.388e+02 6.403e+02, threshold=7.633e+02, percent-clipped=0.0 2026-09-24 00:00:33,275 INFO [train.py:1192] (1/2) Epoch 17, batch 600, loss[loss=0.3583, simple_loss=0.4573, pruned_loss=0.1297, over 24313.00 frames. ], tot_loss[loss=0.3244, simple_loss=0.4203, pruned_loss=0.1142, over 4591597.84 frames. ], batch size: 234, lr: 1.21e-02, grad_scale: 32.0 2026-09-24 00:00:58,146 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=53260.0, ans=0.1 2026-09-24 00:00:58,508 INFO [train.py:1192] (1/2) Epoch 17, batch 650, loss[loss=0.2827, simple_loss=0.3922, pruned_loss=0.08666, over 24587.00 frames. ], tot_loss[loss=0.3237, simple_loss=0.4196, pruned_loss=0.1139, over 4655673.93 frames. ], batch size: 154, lr: 1.21e-02, grad_scale: 32.0 2026-09-24 00:01:00,324 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=53260.0, ans=0.1 2026-09-24 00:01:05,257 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=53293.333333333336, ans=0.025 2026-09-24 00:01:05,280 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=53293.333333333336, ans=0.1 2026-09-24 00:01:11,477 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=53326.666666666664, ans=0.125 2026-09-24 00:01:11,485 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=53326.666666666664, ans=0.0 2026-09-24 00:01:20,177 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.620e+02 3.299e+02 3.782e+02 4.517e+02 7.737e+02, threshold=7.565e+02, percent-clipped=1.0 2026-09-24 00:01:23,592 INFO [train.py:1192] (1/2) Epoch 17, batch 700, loss[loss=0.3211, simple_loss=0.4119, pruned_loss=0.1152, over 24552.00 frames. ], tot_loss[loss=0.3241, simple_loss=0.4205, pruned_loss=0.1139, over 4690517.31 frames. ], batch size: 158, lr: 1.20e-02, grad_scale: 32.0 2026-09-24 00:01:29,446 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=53460.0, ans=0.125 2026-09-24 00:01:44,507 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=53560.0, ans=0.125 2026-09-24 00:01:49,880 INFO [train.py:1192] (1/2) Epoch 17, batch 750, loss[loss=0.3086, simple_loss=0.4159, pruned_loss=0.1007, over 24622.00 frames. ], tot_loss[loss=0.323, simple_loss=0.4192, pruned_loss=0.1133, over 4726968.68 frames. ], batch size: 175, lr: 1.20e-02, grad_scale: 32.0 2026-09-24 00:01:52,051 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=53593.333333333336, ans=0.2 2026-09-24 00:02:00,824 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=53660.0, ans=0.125 2026-09-24 00:02:01,453 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=11.07 vs. limit=15.0 2026-09-24 00:02:06,305 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.69 vs. limit=15.0 2026-09-24 00:02:11,792 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.751e+02 3.366e+02 3.846e+02 4.642e+02 7.031e+02, threshold=7.693e+02, percent-clipped=0.0 2026-09-24 00:02:12,301 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=53726.666666666664, ans=0.2 2026-09-24 00:02:15,307 INFO [train.py:1192] (1/2) Epoch 17, batch 800, loss[loss=0.2911, simple_loss=0.3838, pruned_loss=0.09915, over 24544.00 frames. ], tot_loss[loss=0.3216, simple_loss=0.4183, pruned_loss=0.1125, over 4753410.51 frames. ], batch size: 137, lr: 1.20e-02, grad_scale: 32.0 2026-09-24 00:02:15,939 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=53760.0, ans=0.2 2026-09-24 00:02:32,564 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=53860.0, ans=0.125 2026-09-24 00:02:36,066 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=53893.333333333336, ans=0.125 2026-09-24 00:02:36,539 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=53893.333333333336, ans=0.0 2026-09-24 00:02:39,147 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.65 vs. limit=6.0 2026-09-24 00:02:41,006 INFO [train.py:1192] (1/2) Epoch 17, batch 850, loss[loss=0.3357, simple_loss=0.4398, pruned_loss=0.1158, over 24556.00 frames. ], tot_loss[loss=0.3211, simple_loss=0.4177, pruned_loss=0.1122, over 4771245.60 frames. ], batch size: 204, lr: 1.20e-02, grad_scale: 32.0 2026-09-24 00:02:45,839 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.75 vs. limit=15.0 2026-09-24 00:02:48,956 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=53960.0, ans=0.125 2026-09-24 00:03:03,503 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.516e+02 3.169e+02 3.598e+02 4.008e+02 6.242e+02, threshold=7.196e+02, percent-clipped=0.0 2026-09-24 00:03:07,090 INFO [train.py:1192] (1/2) Epoch 17, batch 900, loss[loss=0.2608, simple_loss=0.3644, pruned_loss=0.07866, over 24547.00 frames. ], tot_loss[loss=0.3215, simple_loss=0.4181, pruned_loss=0.1125, over 4781697.85 frames. ], batch size: 137, lr: 1.20e-02, grad_scale: 32.0 2026-09-24 00:03:08,831 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.34 vs. limit=15.0 2026-09-24 00:03:15,451 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=54126.666666666664, ans=0.1 2026-09-24 00:03:20,480 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=54160.0, ans=0.125 2026-09-24 00:03:32,563 INFO [train.py:1192] (1/2) Epoch 17, batch 950, loss[loss=0.4933, simple_loss=0.4928, pruned_loss=0.247, over 11512.00 frames. ], tot_loss[loss=0.3243, simple_loss=0.4182, pruned_loss=0.1153, over 4715484.49 frames. ], batch size: 334, lr: 1.20e-02, grad_scale: 32.0 2026-09-24 00:03:34,297 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=54260.0, ans=0.125 2026-09-24 00:03:43,754 INFO [train.py:1192] (1/2) Epoch 18, batch 0, loss[loss=0.2795, simple_loss=0.3813, pruned_loss=0.08882, over 24561.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3813, pruned_loss=0.08882, over 24561.00 frames. ], batch size: 137, lr: 1.16e-02, grad_scale: 32.0 2026-09-24 00:03:43,754 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 00:03:48,847 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9180, 2.5500, 2.3346, 1.9824], device='cuda:1') 2026-09-24 00:03:55,531 INFO [train.py:1224] (1/2) Epoch 18, validation: loss=0.1978, simple_loss=0.314, pruned_loss=0.04077, over 2564189.00 frames. 2026-09-24 00:03:55,531 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-24 00:03:58,093 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=54286.666666666664, ans=0.025 2026-09-24 00:03:59,635 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=54286.666666666664, ans=0.125 2026-09-24 00:04:01,205 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.19 vs. limit=15.0 2026-09-24 00:04:13,550 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.645e+02 3.595e+02 4.226e+02 5.155e+02 1.027e+03, threshold=8.453e+02, percent-clipped=2.0 2026-09-24 00:04:14,111 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=54386.666666666664, ans=0.1 2026-09-24 00:04:21,178 INFO [train.py:1192] (1/2) Epoch 18, batch 50, loss[loss=0.301, simple_loss=0.3867, pruned_loss=0.1076, over 24298.00 frames. ], tot_loss[loss=0.3317, simple_loss=0.427, pruned_loss=0.1182, over 1082030.93 frames. ], batch size: 125, lr: 1.16e-02, grad_scale: 32.0 2026-09-24 00:04:24,557 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=54453.333333333336, ans=0.125 2026-09-24 00:04:27,033 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=54486.666666666664, ans=0.125 2026-09-24 00:04:31,933 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=54520.0, ans=0.1 2026-09-24 00:04:34,436 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=54520.0, ans=0.125 2026-09-24 00:04:38,227 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=54553.333333333336, ans=0.0 2026-09-24 00:04:40,453 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=54553.333333333336, ans=0.0 2026-09-24 00:04:41,335 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=54586.666666666664, ans=0.2 2026-09-24 00:04:42,779 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=54586.666666666664, ans=0.125 2026-09-24 00:04:46,231 INFO [train.py:1192] (1/2) Epoch 18, batch 100, loss[loss=0.3006, simple_loss=0.4009, pruned_loss=0.1002, over 24610.00 frames. ], tot_loss[loss=0.3303, simple_loss=0.428, pruned_loss=0.1163, over 1914523.89 frames. ], batch size: 154, lr: 1.16e-02, grad_scale: 32.0 2026-09-24 00:05:04,427 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.666e+02 3.279e+02 3.662e+02 4.041e+02 5.656e+02, threshold=7.323e+02, percent-clipped=0.0 2026-09-24 00:05:11,796 INFO [train.py:1192] (1/2) Epoch 18, batch 150, loss[loss=0.2529, simple_loss=0.3457, pruned_loss=0.08001, over 24280.00 frames. ], tot_loss[loss=0.3262, simple_loss=0.4229, pruned_loss=0.1147, over 2560903.35 frames. ], batch size: 125, lr: 1.16e-02, grad_scale: 32.0 2026-09-24 00:05:12,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=54786.666666666664, ans=0.125 2026-09-24 00:05:19,301 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=54820.0, ans=0.1 2026-09-24 00:05:32,630 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=5.46 vs. limit=15.0 2026-09-24 00:05:36,505 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=54920.0, ans=0.0 2026-09-24 00:05:37,759 INFO [train.py:1192] (1/2) Epoch 18, batch 200, loss[loss=0.3809, simple_loss=0.4791, pruned_loss=0.1413, over 24204.00 frames. ], tot_loss[loss=0.3238, simple_loss=0.4207, pruned_loss=0.1134, over 3059504.17 frames. ], batch size: 257, lr: 1.16e-02, grad_scale: 32.0 2026-09-24 00:05:44,740 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=54986.666666666664, ans=10.0 2026-09-24 00:05:53,006 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.06 vs. limit=15.0 2026-09-24 00:05:53,938 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=55053.333333333336, ans=0.125 2026-09-24 00:05:54,622 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=55053.333333333336, ans=10.0 2026-09-24 00:05:56,031 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.514e+02 3.426e+02 3.972e+02 4.847e+02 1.093e+03, threshold=7.944e+02, percent-clipped=3.0 2026-09-24 00:06:00,726 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=14.20 vs. limit=22.5 2026-09-24 00:06:01,446 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=55086.666666666664, ans=0.0 2026-09-24 00:06:03,498 INFO [train.py:1192] (1/2) Epoch 18, batch 250, loss[loss=0.353, simple_loss=0.4548, pruned_loss=0.1255, over 24360.00 frames. ], tot_loss[loss=0.3226, simple_loss=0.4196, pruned_loss=0.1128, over 3443504.66 frames. ], batch size: 225, lr: 1.15e-02, grad_scale: 32.0 2026-09-24 00:06:09,658 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=7.54 vs. limit=15.0 2026-09-24 00:06:20,602 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=55220.0, ans=0.0 2026-09-24 00:06:20,623 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=55220.0, ans=0.0 2026-09-24 00:06:27,594 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:06:28,931 INFO [train.py:1192] (1/2) Epoch 18, batch 300, loss[loss=0.3071, simple_loss=0.4175, pruned_loss=0.0983, over 24532.00 frames. ], tot_loss[loss=0.3205, simple_loss=0.4178, pruned_loss=0.1116, over 3757803.22 frames. ], batch size: 204, lr: 1.15e-02, grad_scale: 32.0 2026-09-24 00:06:30,620 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=55286.666666666664, ans=0.0 2026-09-24 00:06:34,032 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=55320.0, ans=0.125 2026-09-24 00:06:47,192 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.468e+02 3.228e+02 3.688e+02 4.089e+02 5.893e+02, threshold=7.377e+02, percent-clipped=0.0 2026-09-24 00:06:50,691 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=55420.0, ans=0.125 2026-09-24 00:06:52,111 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=55420.0, ans=0.0 2026-09-24 00:06:54,406 INFO [train.py:1192] (1/2) Epoch 18, batch 350, loss[loss=0.2669, simple_loss=0.3666, pruned_loss=0.08366, over 24600.00 frames. ], tot_loss[loss=0.3212, simple_loss=0.4187, pruned_loss=0.1119, over 3997959.71 frames. ], batch size: 137, lr: 1.15e-02, grad_scale: 32.0 2026-09-24 00:07:19,801 INFO [train.py:1192] (1/2) Epoch 18, batch 400, loss[loss=0.3704, simple_loss=0.4515, pruned_loss=0.1446, over 24559.00 frames. ], tot_loss[loss=0.3216, simple_loss=0.4186, pruned_loss=0.1123, over 4183470.21 frames. ], batch size: 170, lr: 1.15e-02, grad_scale: 32.0 2026-09-24 00:07:27,577 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=55653.333333333336, ans=0.0 2026-09-24 00:07:38,706 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.576e+02 3.216e+02 3.635e+02 4.179e+02 6.381e+02, threshold=7.269e+02, percent-clipped=0.0 2026-09-24 00:07:45,999 INFO [train.py:1192] (1/2) Epoch 18, batch 450, loss[loss=0.3469, simple_loss=0.4385, pruned_loss=0.1277, over 24630.00 frames. ], tot_loss[loss=0.3227, simple_loss=0.4194, pruned_loss=0.113, over 4320494.42 frames. ], batch size: 175, lr: 1.15e-02, grad_scale: 32.0 2026-09-24 00:07:56,628 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=2.525e-03 2026-09-24 00:07:59,441 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=55853.333333333336, ans=0.0 2026-09-24 00:08:05,265 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.67 vs. limit=15.0 2026-09-24 00:08:10,197 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=55920.0, ans=0.125 2026-09-24 00:08:11,418 INFO [train.py:1192] (1/2) Epoch 18, batch 500, loss[loss=0.3233, simple_loss=0.4309, pruned_loss=0.1078, over 24497.00 frames. ], tot_loss[loss=0.3202, simple_loss=0.4171, pruned_loss=0.1117, over 4437764.09 frames. ], batch size: 218, lr: 1.15e-02, grad_scale: 32.0 2026-09-24 00:08:14,487 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=55953.333333333336, ans=0.04949747468305833 2026-09-24 00:08:20,317 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=55986.666666666664, ans=0.0 2026-09-24 00:08:23,140 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=56020.0, ans=0.125 2026-09-24 00:08:23,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=56020.0, ans=0.0 2026-09-24 00:08:29,842 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.316e+02 3.128e+02 3.590e+02 4.213e+02 6.818e+02, threshold=7.180e+02, percent-clipped=0.0 2026-09-24 00:08:36,296 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.52 vs. limit=22.5 2026-09-24 00:08:37,075 INFO [train.py:1192] (1/2) Epoch 18, batch 550, loss[loss=0.3459, simple_loss=0.4481, pruned_loss=0.1219, over 24251.00 frames. ], tot_loss[loss=0.3201, simple_loss=0.4172, pruned_loss=0.1115, over 4521903.51 frames. ], batch size: 257, lr: 1.15e-02, grad_scale: 32.0 2026-09-24 00:08:51,437 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:09:00,273 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=56253.333333333336, ans=0.09899494936611666 2026-09-24 00:09:02,616 INFO [train.py:1192] (1/2) Epoch 18, batch 600, loss[loss=0.372, simple_loss=0.4712, pruned_loss=0.1364, over 24337.00 frames. ], tot_loss[loss=0.3203, simple_loss=0.4177, pruned_loss=0.1114, over 4589466.88 frames. ], batch size: 234, lr: 1.14e-02, grad_scale: 32.0 2026-09-24 00:09:12,463 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=56353.333333333336, ans=0.125 2026-09-24 00:09:14,450 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.88 vs. limit=15.0 2026-09-24 00:09:17,806 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=56386.666666666664, ans=0.2 2026-09-24 00:09:20,507 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.559e+02 3.331e+02 3.759e+02 4.324e+02 6.098e+02, threshold=7.519e+02, percent-clipped=0.0 2026-09-24 00:09:23,097 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=56420.0, ans=0.0 2026-09-24 00:09:23,124 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=56420.0, ans=0.125 2026-09-24 00:09:28,007 INFO [train.py:1192] (1/2) Epoch 18, batch 650, loss[loss=0.3345, simple_loss=0.4178, pruned_loss=0.1256, over 24631.00 frames. ], tot_loss[loss=0.3188, simple_loss=0.4165, pruned_loss=0.1106, over 4654339.22 frames. ], batch size: 154, lr: 1.14e-02, grad_scale: 32.0 2026-09-24 00:09:33,906 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=56486.666666666664, ans=0.2 2026-09-24 00:09:35,952 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.35 vs. limit=15.0 2026-09-24 00:09:40,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=56520.0, ans=0.125 2026-09-24 00:09:47,529 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=56586.666666666664, ans=0.125 2026-09-24 00:09:53,162 INFO [train.py:1192] (1/2) Epoch 18, batch 700, loss[loss=0.3188, simple_loss=0.4146, pruned_loss=0.1115, over 24560.00 frames. ], tot_loss[loss=0.3197, simple_loss=0.4177, pruned_loss=0.1109, over 4689234.69 frames. ], batch size: 158, lr: 1.14e-02, grad_scale: 32.0 2026-09-24 00:10:04,734 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=56686.666666666664, ans=0.2 2026-09-24 00:10:10,794 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=56720.0, ans=0.125 2026-09-24 00:10:11,061 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.563e+02 3.294e+02 3.823e+02 4.482e+02 7.746e+02, threshold=7.647e+02, percent-clipped=1.0 2026-09-24 00:10:18,467 INFO [train.py:1192] (1/2) Epoch 18, batch 750, loss[loss=0.3074, simple_loss=0.4169, pruned_loss=0.0989, over 24634.00 frames. ], tot_loss[loss=0.3187, simple_loss=0.4166, pruned_loss=0.1104, over 4725928.02 frames. ], batch size: 175, lr: 1.14e-02, grad_scale: 32.0 2026-09-24 00:10:23,458 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.59 vs. limit=15.0 2026-09-24 00:10:25,336 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=56820.0, ans=0.1 2026-09-24 00:10:27,249 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.44 vs. limit=10.0 2026-09-24 00:10:36,581 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=56886.666666666664, ans=0.125 2026-09-24 00:10:40,162 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=56920.0, ans=0.025 2026-09-24 00:10:44,025 INFO [train.py:1192] (1/2) Epoch 18, batch 800, loss[loss=0.2692, simple_loss=0.3687, pruned_loss=0.08487, over 24550.00 frames. ], tot_loss[loss=0.3186, simple_loss=0.4164, pruned_loss=0.1104, over 4751837.45 frames. ], batch size: 137, lr: 1.14e-02, grad_scale: 32.0 2026-09-24 00:10:55,166 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=57020.0, ans=0.125 2026-09-24 00:10:55,689 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=57020.0, ans=0.04949747468305833 2026-09-24 00:10:58,732 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.40 vs. limit=12.0 2026-09-24 00:11:03,091 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.619e+02 3.257e+02 3.850e+02 4.421e+02 6.931e+02, threshold=7.700e+02, percent-clipped=0.0 2026-09-24 00:11:07,635 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=57086.666666666664, ans=0.125 2026-09-24 00:11:10,034 INFO [train.py:1192] (1/2) Epoch 18, batch 850, loss[loss=0.3912, simple_loss=0.4742, pruned_loss=0.1541, over 24516.00 frames. ], tot_loss[loss=0.3184, simple_loss=0.4161, pruned_loss=0.1104, over 4771276.68 frames. ], batch size: 204, lr: 1.14e-02, grad_scale: 32.0 2026-09-24 00:11:19,133 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=57153.333333333336, ans=0.125 2026-09-24 00:11:19,135 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=57153.333333333336, ans=0.0 2026-09-24 00:11:21,706 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=57186.666666666664, ans=0.2 2026-09-24 00:11:28,366 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=57220.0, ans=0.0 2026-09-24 00:11:32,767 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=57253.333333333336, ans=0.125 2026-09-24 00:11:34,693 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=57253.333333333336, ans=0.1 2026-09-24 00:11:35,699 INFO [train.py:1192] (1/2) Epoch 18, batch 900, loss[loss=0.2723, simple_loss=0.375, pruned_loss=0.08483, over 24575.00 frames. ], tot_loss[loss=0.3184, simple_loss=0.4161, pruned_loss=0.1103, over 4782058.01 frames. ], batch size: 137, lr: 1.14e-02, grad_scale: 32.0 2026-09-24 00:11:36,624 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=57286.666666666664, ans=0.125 2026-09-24 00:11:42,130 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=57320.0, ans=0.2 2026-09-24 00:11:45,870 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=57353.333333333336, ans=0.2 2026-09-24 00:11:46,380 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:11:54,002 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.493e+02 3.195e+02 3.716e+02 4.460e+02 7.730e+02, threshold=7.431e+02, percent-clipped=0.0 2026-09-24 00:11:54,602 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=57386.666666666664, ans=0.125 2026-09-24 00:12:01,094 INFO [train.py:1192] (1/2) Epoch 18, batch 950, loss[loss=0.4451, simple_loss=0.4731, pruned_loss=0.2086, over 11683.00 frames. ], tot_loss[loss=0.3198, simple_loss=0.4157, pruned_loss=0.112, over 4713903.58 frames. ], batch size: 333, lr: 1.13e-02, grad_scale: 32.0 2026-09-24 00:12:12,683 INFO [train.py:1192] (1/2) Epoch 19, batch 0, loss[loss=0.285, simple_loss=0.3885, pruned_loss=0.0908, over 24575.00 frames. ], tot_loss[loss=0.285, simple_loss=0.3885, pruned_loss=0.0908, over 24575.00 frames. ], batch size: 137, lr: 1.10e-02, grad_scale: 32.0 2026-09-24 00:12:12,683 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 00:12:24,364 INFO [train.py:1224] (1/2) Epoch 19, validation: loss=0.1973, simple_loss=0.3132, pruned_loss=0.04069, over 2564189.00 frames. 2026-09-24 00:12:24,364 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-24 00:12:31,011 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=57513.333333333336, ans=0.04949747468305833 2026-09-24 00:12:32,112 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.87 vs. limit=15.0 2026-09-24 00:12:34,500 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=57546.666666666664, ans=0.2 2026-09-24 00:12:35,506 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=57546.666666666664, ans=0.125 2026-09-24 00:12:47,169 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=57613.333333333336, ans=0.0 2026-09-24 00:12:49,018 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=57613.333333333336, ans=0.1 2026-09-24 00:12:50,323 INFO [train.py:1192] (1/2) Epoch 19, batch 50, loss[loss=0.2835, simple_loss=0.3756, pruned_loss=0.09563, over 24245.00 frames. ], tot_loss[loss=0.3289, simple_loss=0.4249, pruned_loss=0.1165, over 1082159.21 frames. ], batch size: 125, lr: 1.10e-02, grad_scale: 32.0 2026-09-24 00:12:58,694 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=57680.0, ans=0.1 2026-09-24 00:13:02,661 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=57713.333333333336, ans=0.1 2026-09-24 00:13:04,398 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.526e+02 3.413e+02 3.881e+02 4.761e+02 8.226e+02, threshold=7.762e+02, percent-clipped=3.0 2026-09-24 00:13:08,083 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=57746.666666666664, ans=0.125 2026-09-24 00:13:16,074 INFO [train.py:1192] (1/2) Epoch 19, batch 100, loss[loss=0.297, simple_loss=0.3991, pruned_loss=0.09744, over 24600.00 frames. ], tot_loss[loss=0.3293, simple_loss=0.4273, pruned_loss=0.1156, over 1915886.83 frames. ], batch size: 154, lr: 1.10e-02, grad_scale: 32.0 2026-09-24 00:13:16,167 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=57813.333333333336, ans=0.2 2026-09-24 00:13:19,090 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=57813.333333333336, ans=0.0 2026-09-24 00:13:19,485 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=57813.333333333336, ans=0.125 2026-09-24 00:13:28,417 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.19 vs. limit=15.0 2026-09-24 00:13:32,794 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=57913.333333333336, ans=0.1 2026-09-24 00:13:38,085 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=57946.666666666664, ans=0.0 2026-09-24 00:13:41,319 INFO [train.py:1192] (1/2) Epoch 19, batch 150, loss[loss=0.2769, simple_loss=0.3656, pruned_loss=0.09409, over 24300.00 frames. ], tot_loss[loss=0.3218, simple_loss=0.42, pruned_loss=0.1118, over 2561115.63 frames. ], batch size: 125, lr: 1.10e-02, grad_scale: 32.0 2026-09-24 00:13:43,495 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.44 vs. limit=12.0 2026-09-24 00:13:43,800 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=57980.0, ans=0.0 2026-09-24 00:13:55,669 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.658e+02 3.215e+02 3.678e+02 4.342e+02 6.275e+02, threshold=7.355e+02, percent-clipped=0.0 2026-09-24 00:13:57,834 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=58080.0, ans=0.125 2026-09-24 00:14:06,875 INFO [train.py:1192] (1/2) Epoch 19, batch 200, loss[loss=0.3645, simple_loss=0.468, pruned_loss=0.1305, over 24219.00 frames. ], tot_loss[loss=0.3183, simple_loss=0.417, pruned_loss=0.1098, over 3059608.38 frames. ], batch size: 257, lr: 1.10e-02, grad_scale: 32.0 2026-09-24 00:14:12,781 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=3.81 vs. limit=12.0 2026-09-24 00:14:14,950 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=58180.0, ans=0.0 2026-09-24 00:14:19,157 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=58213.333333333336, ans=0.025 2026-09-24 00:14:20,697 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.01 vs. limit=15.0 2026-09-24 00:14:21,638 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.08 vs. limit=12.0 2026-09-24 00:14:28,824 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=58280.0, ans=0.1 2026-09-24 00:14:31,773 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=58313.333333333336, ans=0.125 2026-09-24 00:14:32,156 INFO [train.py:1192] (1/2) Epoch 19, batch 250, loss[loss=0.3501, simple_loss=0.4562, pruned_loss=0.122, over 24368.00 frames. ], tot_loss[loss=0.3181, simple_loss=0.4165, pruned_loss=0.1098, over 3443798.56 frames. ], batch size: 225, lr: 1.10e-02, grad_scale: 32.0 2026-09-24 00:14:33,147 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=58313.333333333336, ans=0.0 2026-09-24 00:14:36,715 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=58346.666666666664, ans=0.125 2026-09-24 00:14:41,448 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=58346.666666666664, ans=0.2 2026-09-24 00:14:45,512 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=58380.0, ans=0.125 2026-09-24 00:14:46,446 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.462e+02 3.293e+02 3.643e+02 4.390e+02 7.831e+02, threshold=7.286e+02, percent-clipped=2.0 2026-09-24 00:14:57,671 INFO [train.py:1192] (1/2) Epoch 19, batch 300, loss[loss=0.3374, simple_loss=0.4477, pruned_loss=0.1135, over 24525.00 frames. ], tot_loss[loss=0.3164, simple_loss=0.4151, pruned_loss=0.1088, over 3756945.20 frames. ], batch size: 204, lr: 1.10e-02, grad_scale: 32.0 2026-09-24 00:15:00,152 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=58480.0, ans=0.125 2026-09-24 00:15:12,707 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.01 vs. limit=5.0 2026-09-24 00:15:16,791 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=58580.0, ans=0.1 2026-09-24 00:15:22,281 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.74 vs. limit=6.0 2026-09-24 00:15:23,420 INFO [train.py:1192] (1/2) Epoch 19, batch 350, loss[loss=0.2822, simple_loss=0.3793, pruned_loss=0.09254, over 24559.00 frames. ], tot_loss[loss=0.3168, simple_loss=0.4158, pruned_loss=0.1089, over 3997626.82 frames. ], batch size: 137, lr: 1.09e-02, grad_scale: 32.0 2026-09-24 00:15:36,051 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=58713.333333333336, ans=0.1 2026-09-24 00:15:37,277 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.461e+02 3.163e+02 3.527e+02 4.072e+02 5.266e+02, threshold=7.054e+02, percent-clipped=0.0 2026-09-24 00:15:48,730 INFO [train.py:1192] (1/2) Epoch 19, batch 400, loss[loss=0.32, simple_loss=0.4131, pruned_loss=0.1134, over 24574.00 frames. ], tot_loss[loss=0.3165, simple_loss=0.415, pruned_loss=0.109, over 4183334.98 frames. ], batch size: 170, lr: 1.09e-02, grad_scale: 32.0 2026-09-24 00:15:53,857 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=58846.666666666664, ans=0.025 2026-09-24 00:15:56,074 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=58846.666666666664, ans=0.125 2026-09-24 00:16:09,207 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=58946.666666666664, ans=0.125 2026-09-24 00:16:09,686 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=58946.666666666664, ans=0.0 2026-09-24 00:16:13,772 INFO [train.py:1192] (1/2) Epoch 19, batch 450, loss[loss=0.3581, simple_loss=0.449, pruned_loss=0.1336, over 24634.00 frames. ], tot_loss[loss=0.3163, simple_loss=0.4149, pruned_loss=0.1088, over 4323591.43 frames. ], batch size: 175, lr: 1.09e-02, grad_scale: 32.0 2026-09-24 00:16:28,057 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.424e+02 3.210e+02 3.696e+02 4.183e+02 6.561e+02, threshold=7.393e+02, percent-clipped=0.0 2026-09-24 00:16:28,668 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=59080.0, ans=0.0 2026-09-24 00:16:39,371 INFO [train.py:1192] (1/2) Epoch 19, batch 500, loss[loss=0.3191, simple_loss=0.4349, pruned_loss=0.1017, over 24531.00 frames. ], tot_loss[loss=0.3155, simple_loss=0.4139, pruned_loss=0.1085, over 4440515.64 frames. ], batch size: 218, lr: 1.09e-02, grad_scale: 32.0 2026-09-24 00:16:57,269 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.54 vs. limit=15.0 2026-09-24 00:17:05,371 INFO [train.py:1192] (1/2) Epoch 19, batch 550, loss[loss=0.3287, simple_loss=0.4403, pruned_loss=0.1086, over 24271.00 frames. ], tot_loss[loss=0.3173, simple_loss=0.4153, pruned_loss=0.1096, over 4525548.64 frames. ], batch size: 257, lr: 1.09e-02, grad_scale: 32.0 2026-09-24 00:17:06,406 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=59313.333333333336, ans=0.0 2026-09-24 00:17:09,766 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=59313.333333333336, ans=0.1 2026-09-24 00:17:13,628 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=59346.666666666664, ans=0.2 2026-09-24 00:17:13,640 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=59346.666666666664, ans=0.125 2026-09-24 00:17:16,436 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=59380.0, ans=0.0 2026-09-24 00:17:17,113 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=59380.0, ans=0.0 2026-09-24 00:17:19,411 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.650e+02 3.204e+02 3.511e+02 3.985e+02 7.550e+02, threshold=7.022e+02, percent-clipped=1.0 2026-09-24 00:17:19,522 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=59380.0, ans=0.0 2026-09-24 00:17:22,531 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:17:25,356 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=59446.666666666664, ans=0.2 2026-09-24 00:17:31,097 INFO [train.py:1192] (1/2) Epoch 19, batch 600, loss[loss=0.3261, simple_loss=0.4406, pruned_loss=0.1058, over 24308.00 frames. ], tot_loss[loss=0.3184, simple_loss=0.4166, pruned_loss=0.1101, over 4591662.23 frames. ], batch size: 234, lr: 1.09e-02, grad_scale: 32.0 2026-09-24 00:17:34,483 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=59480.0, ans=0.0 2026-09-24 00:17:40,289 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=59513.333333333336, ans=0.1 2026-09-24 00:17:43,736 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=59546.666666666664, ans=0.025 2026-09-24 00:17:46,936 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.59 vs. limit=15.0 2026-09-24 00:17:47,832 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.77 vs. limit=15.0 2026-09-24 00:17:55,683 INFO [train.py:1192] (1/2) Epoch 19, batch 650, loss[loss=0.27, simple_loss=0.3788, pruned_loss=0.08061, over 24598.00 frames. ], tot_loss[loss=0.316, simple_loss=0.4146, pruned_loss=0.1087, over 4656138.51 frames. ], batch size: 154, lr: 1.09e-02, grad_scale: 32.0 2026-09-24 00:17:59,356 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.00 vs. limit=15.0 2026-09-24 00:18:11,013 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.719e+02 3.292e+02 3.623e+02 4.647e+02 7.311e+02, threshold=7.246e+02, percent-clipped=1.0 2026-09-24 00:18:11,905 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=59746.666666666664, ans=0.0 2026-09-24 00:18:13,472 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=59746.666666666664, ans=0.0 2026-09-24 00:18:17,455 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=59780.0, ans=0.0 2026-09-24 00:18:22,468 INFO [train.py:1192] (1/2) Epoch 19, batch 700, loss[loss=0.3213, simple_loss=0.4164, pruned_loss=0.1131, over 24560.00 frames. ], tot_loss[loss=0.317, simple_loss=0.416, pruned_loss=0.109, over 4690860.51 frames. ], batch size: 158, lr: 1.09e-02, grad_scale: 32.0 2026-09-24 00:18:22,997 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=59813.333333333336, ans=0.2 2026-09-24 00:18:29,889 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=59846.666666666664, ans=0.1 2026-09-24 00:18:36,395 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.10 vs. limit=8.0 2026-09-24 00:18:43,272 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=59946.666666666664, ans=0.0 2026-09-24 00:18:48,879 INFO [train.py:1192] (1/2) Epoch 19, batch 750, loss[loss=0.3096, simple_loss=0.4182, pruned_loss=0.1005, over 24632.00 frames. ], tot_loss[loss=0.3168, simple_loss=0.4153, pruned_loss=0.1092, over 4727216.39 frames. ], batch size: 175, lr: 1.08e-02, grad_scale: 32.0 2026-09-24 00:18:49,585 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.76 vs. limit=6.0 2026-09-24 00:18:55,514 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.99 vs. limit=6.0 2026-09-24 00:19:03,165 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.802e+02 3.406e+02 3.960e+02 4.663e+02 7.283e+02, threshold=7.919e+02, percent-clipped=1.0 2026-09-24 00:19:08,611 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=60080.0, ans=0.1 2026-09-24 00:19:14,822 INFO [train.py:1192] (1/2) Epoch 19, batch 800, loss[loss=0.2449, simple_loss=0.3552, pruned_loss=0.06725, over 24546.00 frames. ], tot_loss[loss=0.3165, simple_loss=0.4149, pruned_loss=0.109, over 4753063.16 frames. ], batch size: 137, lr: 1.08e-02, grad_scale: 32.0 2026-09-24 00:19:21,399 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.58 vs. limit=15.0 2026-09-24 00:19:35,465 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.65 vs. limit=15.0 2026-09-24 00:19:39,705 INFO [train.py:1192] (1/2) Epoch 19, batch 850, loss[loss=0.3264, simple_loss=0.4322, pruned_loss=0.1103, over 24531.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.4137, pruned_loss=0.1078, over 4771691.55 frames. ], batch size: 204, lr: 1.08e-02, grad_scale: 32.0 2026-09-24 00:19:39,831 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=60313.333333333336, ans=0.2 2026-09-24 00:19:54,001 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.449e+02 3.136e+02 3.636e+02 4.203e+02 7.119e+02, threshold=7.271e+02, percent-clipped=0.0 2026-09-24 00:20:02,042 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=60446.666666666664, ans=0.2 2026-09-24 00:20:04,740 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=60480.0, ans=0.0 2026-09-24 00:20:05,070 INFO [train.py:1192] (1/2) Epoch 19, batch 900, loss[loss=0.2439, simple_loss=0.3528, pruned_loss=0.06751, over 24585.00 frames. ], tot_loss[loss=0.3154, simple_loss=0.4143, pruned_loss=0.1083, over 4782212.47 frames. ], batch size: 137, lr: 1.08e-02, grad_scale: 32.0 2026-09-24 00:20:07,700 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=60480.0, ans=0.125 2026-09-24 00:20:17,197 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=60546.666666666664, ans=0.0 2026-09-24 00:20:24,947 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=60613.333333333336, ans=0.95 2026-09-24 00:20:27,774 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=60613.333333333336, ans=0.0 2026-09-24 00:20:29,905 INFO [train.py:1192] (1/2) Epoch 19, batch 950, loss[loss=0.4477, simple_loss=0.4752, pruned_loss=0.2101, over 11239.00 frames. ], tot_loss[loss=0.3153, simple_loss=0.4128, pruned_loss=0.1089, over 4715421.01 frames. ], batch size: 333, lr: 1.08e-02, grad_scale: 16.0 2026-09-24 00:20:41,046 INFO [train.py:1192] (1/2) Epoch 20, batch 0, loss[loss=0.2899, simple_loss=0.39, pruned_loss=0.09488, over 24534.00 frames. ], tot_loss[loss=0.2899, simple_loss=0.39, pruned_loss=0.09488, over 24534.00 frames. ], batch size: 137, lr: 1.05e-02, grad_scale: 32.0 2026-09-24 00:20:41,046 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 00:20:51,509 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.8582, 4.3436, 4.6378, 4.3149], device='cuda:1') 2026-09-24 00:20:52,690 INFO [train.py:1224] (1/2) Epoch 20, validation: loss=0.1968, simple_loss=0.3133, pruned_loss=0.04012, over 2564189.00 frames. 2026-09-24 00:20:52,690 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 12909MB 2026-09-24 00:20:56,690 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=60673.333333333336, ans=0.0 2026-09-24 00:21:03,573 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.422e+02 3.285e+02 3.850e+02 4.927e+02 9.160e+02, threshold=7.699e+02, percent-clipped=3.0 2026-09-24 00:21:05,128 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=60740.0, ans=0.2 2026-09-24 00:21:05,140 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=60740.0, ans=0.0 2026-09-24 00:21:18,285 INFO [train.py:1192] (1/2) Epoch 20, batch 50, loss[loss=0.2572, simple_loss=0.3536, pruned_loss=0.08039, over 24235.00 frames. ], tot_loss[loss=0.3234, simple_loss=0.4205, pruned_loss=0.1131, over 1081995.46 frames. ], batch size: 125, lr: 1.05e-02, grad_scale: 32.0 2026-09-24 00:21:22,476 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=60840.0, ans=0.125 2026-09-24 00:21:24,228 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.32 vs. limit=10.0 2026-09-24 00:21:33,511 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=60940.0, ans=0.125 2026-09-24 00:21:44,211 INFO [train.py:1192] (1/2) Epoch 20, batch 100, loss[loss=0.3266, simple_loss=0.4117, pruned_loss=0.1207, over 24614.00 frames. ], tot_loss[loss=0.3235, simple_loss=0.4231, pruned_loss=0.112, over 1915703.49 frames. ], batch size: 154, lr: 1.05e-02, grad_scale: 32.0 2026-09-24 00:21:54,566 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.627e+02 3.176e+02 3.578e+02 4.126e+02 5.402e+02, threshold=7.156e+02, percent-clipped=0.0 2026-09-24 00:21:57,561 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.58 vs. limit=12.0 2026-09-24 00:22:09,580 INFO [train.py:1192] (1/2) Epoch 20, batch 150, loss[loss=0.2603, simple_loss=0.3583, pruned_loss=0.08114, over 24244.00 frames. ], tot_loss[loss=0.3184, simple_loss=0.4177, pruned_loss=0.1095, over 2560416.15 frames. ], batch size: 125, lr: 1.05e-02, grad_scale: 32.0 2026-09-24 00:22:18,691 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=61206.666666666664, ans=0.125 2026-09-24 00:22:35,274 INFO [train.py:1192] (1/2) Epoch 20, batch 200, loss[loss=0.3659, simple_loss=0.4628, pruned_loss=0.1345, over 24195.00 frames. ], tot_loss[loss=0.3143, simple_loss=0.4147, pruned_loss=0.1069, over 3059427.74 frames. ], batch size: 257, lr: 1.05e-02, grad_scale: 32.0 2026-09-24 00:22:35,866 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=61340.0, ans=0.1 2026-09-24 00:22:45,857 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.whiten.whitening_limit, batch_count=61406.666666666664, ans=12.0 2026-09-24 00:22:46,211 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.384e+02 3.179e+02 3.621e+02 4.117e+02 8.279e+02, threshold=7.242e+02, percent-clipped=1.0 2026-09-24 00:22:47,355 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=61406.666666666664, ans=0.0 2026-09-24 00:22:47,919 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=61406.666666666664, ans=0.1 2026-09-24 00:23:00,779 INFO [train.py:1192] (1/2) Epoch 20, batch 250, loss[loss=0.3497, simple_loss=0.4591, pruned_loss=0.1202, over 24366.00 frames. ], tot_loss[loss=0.3149, simple_loss=0.4146, pruned_loss=0.1077, over 3444567.15 frames. ], batch size: 225, lr: 1.04e-02, grad_scale: 32.0 2026-09-24 00:23:00,880 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=61506.666666666664, ans=0.0 2026-09-24 00:23:09,272 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=61540.0, ans=0.125 2026-09-24 00:23:12,574 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=61573.333333333336, ans=0.125 2026-09-24 00:23:16,822 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=61606.666666666664, ans=0.125 2026-09-24 00:23:20,696 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.39 vs. limit=6.0 2026-09-24 00:23:26,129 INFO [train.py:1192] (1/2) Epoch 20, batch 300, loss[loss=0.3318, simple_loss=0.4414, pruned_loss=0.1111, over 24540.00 frames. ], tot_loss[loss=0.3138, simple_loss=0.4137, pruned_loss=0.107, over 3757264.85 frames. ], batch size: 204, lr: 1.04e-02, grad_scale: 32.0 2026-09-24 00:23:30,506 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=61706.666666666664, ans=0.125 2026-09-24 00:23:36,738 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.703e+02 3.287e+02 3.773e+02 4.449e+02 6.240e+02, threshold=7.545e+02, percent-clipped=0.0 2026-09-24 00:23:50,274 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=61806.666666666664, ans=0.0 2026-09-24 00:23:51,737 INFO [train.py:1192] (1/2) Epoch 20, batch 350, loss[loss=0.2837, simple_loss=0.3732, pruned_loss=0.09707, over 24571.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.4145, pruned_loss=0.1074, over 3994293.90 frames. ], batch size: 137, lr: 1.04e-02, grad_scale: 16.0 2026-09-24 00:23:52,829 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=61840.0, ans=0.0 2026-09-24 00:23:55,465 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=61840.0, ans=0.1 2026-09-24 00:23:57,550 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=61873.333333333336, ans=0.2 2026-09-24 00:24:02,456 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=61906.666666666664, ans=0.125 2026-09-24 00:24:09,294 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=61940.0, ans=0.125 2026-09-24 00:24:15,382 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=61973.333333333336, ans=0.1 2026-09-24 00:24:17,834 INFO [train.py:1192] (1/2) Epoch 20, batch 400, loss[loss=0.3266, simple_loss=0.4274, pruned_loss=0.1129, over 24552.00 frames. ], tot_loss[loss=0.3132, simple_loss=0.413, pruned_loss=0.1067, over 4177669.38 frames. ], batch size: 170, lr: 1.04e-02, grad_scale: 32.0 2026-09-24 00:24:23,454 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.39 vs. limit=10.0 2026-09-24 00:24:26,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=62040.0, ans=0.125 2026-09-24 00:24:26,486 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=3.99 vs. limit=12.0 2026-09-24 00:24:29,298 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.592e+02 3.221e+02 3.696e+02 4.247e+02 5.702e+02, threshold=7.392e+02, percent-clipped=0.0 2026-09-24 00:24:40,331 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=62140.0, ans=0.125 2026-09-24 00:24:43,914 INFO [train.py:1192] (1/2) Epoch 20, batch 450, loss[loss=0.2971, simple_loss=0.4022, pruned_loss=0.09598, over 24618.00 frames. ], tot_loss[loss=0.3141, simple_loss=0.4138, pruned_loss=0.1072, over 4317469.49 frames. ], batch size: 175, lr: 1.04e-02, grad_scale: 32.0 2026-09-24 00:24:48,395 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.07 vs. limit=15.0 2026-09-24 00:24:54,279 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=62240.0, ans=0.05 2026-09-24 00:24:55,066 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.16 vs. limit=12.0 2026-09-24 00:25:00,204 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.79 vs. limit=15.0 2026-09-24 00:25:04,145 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=62306.666666666664, ans=0.0 2026-09-24 00:25:10,149 INFO [train.py:1192] (1/2) Epoch 20, batch 500, loss[loss=0.344, simple_loss=0.4505, pruned_loss=0.1187, over 24522.00 frames. ], tot_loss[loss=0.3128, simple_loss=0.4123, pruned_loss=0.1066, over 4435842.22 frames. ], batch size: 218, lr: 1.04e-02, grad_scale: 32.0 2026-09-24 00:25:11,185 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=62340.0, ans=0.2 2026-09-24 00:25:11,680 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=62340.0, ans=0.125 2026-09-24 00:25:21,097 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.458e+02 3.194e+02 3.580e+02 4.057e+02 7.296e+02, threshold=7.161e+02, percent-clipped=0.0 2026-09-24 00:25:31,830 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=62473.333333333336, ans=0.125 2026-09-24 00:25:34,073 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=62473.333333333336, ans=0.125 2026-09-24 00:25:34,073 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=62473.333333333336, ans=10.0 2026-09-24 00:25:35,316 INFO [train.py:1192] (1/2) Epoch 20, batch 550, loss[loss=0.35, simple_loss=0.4512, pruned_loss=0.1244, over 24291.00 frames. ], tot_loss[loss=0.3123, simple_loss=0.4123, pruned_loss=0.1062, over 4521117.26 frames. ], batch size: 257, lr: 1.04e-02, grad_scale: 32.0 2026-09-24 00:25:40,933 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=62540.0, ans=0.0 2026-09-24 00:25:46,515 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=62573.333333333336, ans=0.0 2026-09-24 00:26:01,525 INFO [train.py:1192] (1/2) Epoch 20, batch 600, loss[loss=0.348, simple_loss=0.4442, pruned_loss=0.1259, over 24329.00 frames. ], tot_loss[loss=0.3136, simple_loss=0.4132, pruned_loss=0.107, over 4587192.23 frames. ], batch size: 234, lr: 1.04e-02, grad_scale: 32.0 2026-09-24 00:26:03,823 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=2.84 vs. limit=15.0 2026-09-24 00:26:05,605 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=62673.333333333336, ans=0.125 2026-09-24 00:26:09,762 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=62706.666666666664, ans=0.1 2026-09-24 00:26:11,818 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=62740.0, ans=0.125 2026-09-24 00:26:12,607 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.425e+02 3.205e+02 3.582e+02 4.002e+02 6.555e+02, threshold=7.163e+02, percent-clipped=0.0 2026-09-24 00:26:26,806 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=62840.0, ans=0.0 2026-09-24 00:26:27,156 INFO [train.py:1192] (1/2) Epoch 20, batch 650, loss[loss=0.3057, simple_loss=0.4033, pruned_loss=0.104, over 24555.00 frames. ], tot_loss[loss=0.3127, simple_loss=0.4123, pruned_loss=0.1066, over 4652234.48 frames. ], batch size: 154, lr: 1.04e-02, grad_scale: 32.0 2026-09-24 00:26:52,039 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=62973.333333333336, ans=0.125 2026-09-24 00:26:53,214 INFO [train.py:1192] (1/2) Epoch 20, batch 700, loss[loss=0.2894, simple_loss=0.3938, pruned_loss=0.09255, over 24556.00 frames. ], tot_loss[loss=0.3133, simple_loss=0.4132, pruned_loss=0.1067, over 4687488.29 frames. ], batch size: 158, lr: 1.03e-02, grad_scale: 32.0 2026-09-24 00:27:04,397 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.341e+02 3.283e+02 3.811e+02 4.565e+02 8.019e+02, threshold=7.622e+02, percent-clipped=1.0 2026-09-24 00:27:18,634 INFO [train.py:1192] (1/2) Epoch 20, batch 750, loss[loss=0.3285, simple_loss=0.4308, pruned_loss=0.113, over 24662.00 frames. ], tot_loss[loss=0.3131, simple_loss=0.4128, pruned_loss=0.1067, over 4721347.91 frames. ], batch size: 175, lr: 1.03e-02, grad_scale: 32.0 2026-09-24 00:27:23,979 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.21 vs. limit=15.0 2026-09-24 00:27:27,963 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=63206.666666666664, ans=0.1 2026-09-24 00:27:28,877 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=63240.0, ans=0.125 2026-09-24 00:27:31,521 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=63240.0, ans=0.0 2026-09-24 00:27:42,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=63306.666666666664, ans=0.0 2026-09-24 00:27:44,855 INFO [train.py:1192] (1/2) Epoch 20, batch 800, loss[loss=0.2734, simple_loss=0.3656, pruned_loss=0.09064, over 24540.00 frames. ], tot_loss[loss=0.3131, simple_loss=0.4126, pruned_loss=0.1068, over 4748021.09 frames. ], batch size: 137, lr: 1.03e-02, grad_scale: 32.0 2026-09-24 00:27:44,934 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=63340.0, ans=0.0 2026-09-24 00:27:49,988 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=63373.333333333336, ans=0.125 2026-09-24 00:27:56,443 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.500e+02 3.155e+02 3.540e+02 4.118e+02 5.592e+02, threshold=7.080e+02, percent-clipped=0.0 2026-09-24 00:28:04,102 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=63440.0, ans=0.04949747468305833 2026-09-24 00:28:11,224 INFO [train.py:1192] (1/2) Epoch 20, batch 850, loss[loss=0.3211, simple_loss=0.4275, pruned_loss=0.1074, over 24549.00 frames. ], tot_loss[loss=0.3118, simple_loss=0.4116, pruned_loss=0.1061, over 4767042.85 frames. ], batch size: 204, lr: 1.03e-02, grad_scale: 32.0 2026-09-24 00:28:12,195 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=63506.666666666664, ans=0.07 2026-09-24 00:28:24,985 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=11.86 vs. limit=15.0 2026-09-24 00:28:34,644 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.86 vs. limit=15.0 2026-09-24 00:28:35,891 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=63640.0, ans=0.0 2026-09-24 00:28:37,217 INFO [train.py:1192] (1/2) Epoch 20, batch 900, loss[loss=0.2876, simple_loss=0.3873, pruned_loss=0.09394, over 24575.00 frames. ], tot_loss[loss=0.3116, simple_loss=0.4115, pruned_loss=0.1059, over 4778939.37 frames. ], batch size: 137, lr: 1.03e-02, grad_scale: 32.0 2026-09-24 00:28:44,391 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=63706.666666666664, ans=0.125 2026-09-24 00:28:48,704 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.595e+02 3.360e+02 3.782e+02 4.282e+02 5.847e+02, threshold=7.564e+02, percent-clipped=0.0 2026-09-24 00:29:01,405 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=63806.666666666664, ans=0.0 2026-09-24 00:29:01,408 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=63806.666666666664, ans=0.125 2026-09-24 00:29:02,177 INFO [train.py:1192] (1/2) Epoch 20, batch 950, loss[loss=0.4415, simple_loss=0.4748, pruned_loss=0.2041, over 10920.00 frames. ], tot_loss[loss=0.3121, simple_loss=0.4101, pruned_loss=0.107, over 4712665.10 frames. ], batch size: 333, lr: 1.03e-02, grad_scale: 32.0 2026-09-24 00:29:03,715 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.79 vs. limit=15.0 2026-09-24 00:29:13,683 INFO [train.py:1192] (1/2) Epoch 21, batch 0, loss[loss=0.2498, simple_loss=0.3616, pruned_loss=0.06896, over 24583.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3616, pruned_loss=0.06896, over 24583.00 frames. ], batch size: 137, lr: 1.00e-02, grad_scale: 32.0 2026-09-24 00:29:13,683 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 00:29:25,320 INFO [train.py:1224] (1/2) Epoch 21, validation: loss=0.195, simple_loss=0.3111, pruned_loss=0.03952, over 2564189.00 frames. 2026-09-24 00:29:25,320 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 00:29:32,775 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=63900.0, ans=0.0 2026-09-24 00:29:39,435 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=7.11 vs. limit=15.0 2026-09-24 00:29:40,268 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=63966.666666666664, ans=0.0 2026-09-24 00:29:48,852 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=64000.0, ans=0.125 2026-09-24 00:29:51,390 INFO [train.py:1192] (1/2) Epoch 21, batch 50, loss[loss=0.2453, simple_loss=0.3451, pruned_loss=0.07277, over 24290.00 frames. ], tot_loss[loss=0.3223, simple_loss=0.4196, pruned_loss=0.1125, over 1080542.69 frames. ], batch size: 125, lr: 1.00e-02, grad_scale: 32.0 2026-09-24 00:29:58,675 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.549e+02 3.392e+02 4.011e+02 4.620e+02 8.651e+02, threshold=8.021e+02, percent-clipped=2.0 2026-09-24 00:30:07,743 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=64133.333333333336, ans=0.0 2026-09-24 00:30:15,045 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=64166.666666666664, ans=0.2 2026-09-24 00:30:15,577 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=64166.666666666664, ans=0.125 2026-09-24 00:30:16,381 INFO [train.py:1192] (1/2) Epoch 21, batch 100, loss[loss=0.2988, simple_loss=0.3979, pruned_loss=0.09978, over 24608.00 frames. ], tot_loss[loss=0.3194, simple_loss=0.4199, pruned_loss=0.1094, over 1915304.21 frames. ], batch size: 154, lr: 1.00e-02, grad_scale: 32.0 2026-09-24 00:30:29,324 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=2.96 vs. limit=15.0 2026-09-24 00:30:41,964 INFO [train.py:1192] (1/2) Epoch 21, batch 150, loss[loss=0.2703, simple_loss=0.3642, pruned_loss=0.08819, over 24264.00 frames. ], tot_loss[loss=0.313, simple_loss=0.4135, pruned_loss=0.1063, over 2560804.78 frames. ], batch size: 125, lr: 9.99e-03, grad_scale: 32.0 2026-09-24 00:30:45,164 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=64366.666666666664, ans=0.125 2026-09-24 00:30:49,823 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.497e+02 3.235e+02 3.700e+02 4.139e+02 6.411e+02, threshold=7.399e+02, percent-clipped=0.0 2026-09-24 00:30:55,375 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=64433.333333333336, ans=0.0 2026-09-24 00:31:07,603 INFO [train.py:1192] (1/2) Epoch 21, batch 200, loss[loss=0.3868, simple_loss=0.4791, pruned_loss=0.1472, over 24240.00 frames. ], tot_loss[loss=0.3128, simple_loss=0.4132, pruned_loss=0.1062, over 3059789.80 frames. ], batch size: 257, lr: 9.98e-03, grad_scale: 32.0 2026-09-24 00:31:10,345 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.56 vs. limit=15.0 2026-09-24 00:31:17,053 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=64566.666666666664, ans=0.0 2026-09-24 00:31:17,068 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=64566.666666666664, ans=0.2 2026-09-24 00:31:28,141 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=64666.666666666664, ans=0.0 2026-09-24 00:31:32,805 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=64700.0, ans=0.125 2026-09-24 00:31:33,205 INFO [train.py:1192] (1/2) Epoch 21, batch 250, loss[loss=0.3598, simple_loss=0.4648, pruned_loss=0.1274, over 24368.00 frames. ], tot_loss[loss=0.3119, simple_loss=0.4122, pruned_loss=0.1058, over 3443103.65 frames. ], batch size: 225, lr: 9.97e-03, grad_scale: 32.0 2026-09-24 00:31:38,244 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=64733.333333333336, ans=0.07 2026-09-24 00:31:38,637 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:31:40,178 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.26 vs. limit=15.0 2026-09-24 00:31:40,886 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.329e+02 3.379e+02 3.844e+02 4.387e+02 9.015e+02, threshold=7.688e+02, percent-clipped=1.0 2026-09-24 00:31:46,624 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=64766.666666666664, ans=0.125 2026-09-24 00:31:47,472 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=64766.666666666664, ans=0.125 2026-09-24 00:31:49,811 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_na.min_abs, batch_count=64800.0, ans=0.02 2026-09-24 00:31:58,373 INFO [train.py:1192] (1/2) Epoch 21, batch 300, loss[loss=0.3172, simple_loss=0.4281, pruned_loss=0.1031, over 24532.00 frames. ], tot_loss[loss=0.3096, simple_loss=0.4104, pruned_loss=0.1044, over 3756323.83 frames. ], batch size: 204, lr: 9.96e-03, grad_scale: 32.0 2026-09-24 00:32:06,470 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=64900.0, ans=0.0 2026-09-24 00:32:07,558 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.34 vs. limit=15.0 2026-09-24 00:32:22,547 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=65000.0, ans=0.125 2026-09-24 00:32:23,988 INFO [train.py:1192] (1/2) Epoch 21, batch 350, loss[loss=0.273, simple_loss=0.3692, pruned_loss=0.08844, over 24546.00 frames. ], tot_loss[loss=0.3115, simple_loss=0.4121, pruned_loss=0.1055, over 3997335.74 frames. ], batch size: 137, lr: 9.95e-03, grad_scale: 32.0 2026-09-24 00:32:31,862 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.496e+02 3.120e+02 3.532e+02 3.996e+02 6.407e+02, threshold=7.064e+02, percent-clipped=0.0 2026-09-24 00:32:34,216 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.80 vs. limit=10.0 2026-09-24 00:32:49,199 INFO [train.py:1192] (1/2) Epoch 21, batch 400, loss[loss=0.3281, simple_loss=0.4246, pruned_loss=0.1158, over 24555.00 frames. ], tot_loss[loss=0.3106, simple_loss=0.4112, pruned_loss=0.105, over 4183511.94 frames. ], batch size: 170, lr: 9.94e-03, grad_scale: 32.0 2026-09-24 00:32:50,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=65200.0, ans=0.1 2026-09-24 00:32:50,947 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=2.884e-03 2026-09-24 00:32:57,705 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=65233.333333333336, ans=0.125 2026-09-24 00:33:03,340 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=8.56 vs. limit=15.0 2026-09-24 00:33:10,159 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.61 vs. limit=22.5 2026-09-24 00:33:10,576 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=65333.333333333336, ans=0.025 2026-09-24 00:33:11,571 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=65333.333333333336, ans=0.125 2026-09-24 00:33:14,388 INFO [train.py:1192] (1/2) Epoch 21, batch 450, loss[loss=0.3364, simple_loss=0.4358, pruned_loss=0.1185, over 24630.00 frames. ], tot_loss[loss=0.3116, simple_loss=0.4118, pruned_loss=0.1057, over 4322759.69 frames. ], batch size: 175, lr: 9.93e-03, grad_scale: 32.0 2026-09-24 00:33:15,015 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=65366.666666666664, ans=0.1 2026-09-24 00:33:21,116 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=65400.0, ans=0.125 2026-09-24 00:33:22,298 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.589e+02 3.354e+02 3.803e+02 4.582e+02 8.143e+02, threshold=7.607e+02, percent-clipped=1.0 2026-09-24 00:33:37,286 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=65500.0, ans=0.2 2026-09-24 00:33:39,969 INFO [train.py:1192] (1/2) Epoch 21, batch 500, loss[loss=0.3489, simple_loss=0.4481, pruned_loss=0.1248, over 24503.00 frames. ], tot_loss[loss=0.3097, simple_loss=0.4099, pruned_loss=0.1048, over 4439528.38 frames. ], batch size: 218, lr: 9.91e-03, grad_scale: 32.0 2026-09-24 00:33:53,962 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=65600.0, ans=0.0 2026-09-24 00:34:06,300 INFO [train.py:1192] (1/2) Epoch 21, batch 550, loss[loss=0.3795, simple_loss=0.4778, pruned_loss=0.1406, over 24267.00 frames. ], tot_loss[loss=0.3117, simple_loss=0.4116, pruned_loss=0.1059, over 4523596.92 frames. ], batch size: 257, lr: 9.90e-03, grad_scale: 32.0 2026-09-24 00:34:13,336 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.501e+02 3.059e+02 3.593e+02 4.039e+02 5.830e+02, threshold=7.187e+02, percent-clipped=0.0 2026-09-24 00:34:31,576 INFO [train.py:1192] (1/2) Epoch 21, batch 600, loss[loss=0.2974, simple_loss=0.416, pruned_loss=0.08941, over 24403.00 frames. ], tot_loss[loss=0.3118, simple_loss=0.4121, pruned_loss=0.1057, over 4590821.45 frames. ], batch size: 235, lr: 9.89e-03, grad_scale: 32.0 2026-09-24 00:34:37,222 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.72 vs. limit=15.0 2026-09-24 00:34:37,747 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.08 vs. limit=15.0 2026-09-24 00:34:54,370 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=66000.0, ans=0.125 2026-09-24 00:34:55,808 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=66000.0, ans=0.125 2026-09-24 00:34:57,471 INFO [train.py:1192] (1/2) Epoch 21, batch 650, loss[loss=0.3009, simple_loss=0.3985, pruned_loss=0.1016, over 24603.00 frames. ], tot_loss[loss=0.3106, simple_loss=0.4111, pruned_loss=0.105, over 4655541.60 frames. ], batch size: 154, lr: 9.88e-03, grad_scale: 32.0 2026-09-24 00:34:58,093 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=66033.33333333333, ans=0.125 2026-09-24 00:35:04,493 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.662e+02 3.257e+02 3.614e+02 4.275e+02 7.702e+02, threshold=7.227e+02, percent-clipped=2.0 2026-09-24 00:35:06,922 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=66100.0, ans=0.0 2026-09-24 00:35:23,290 INFO [train.py:1192] (1/2) Epoch 21, batch 700, loss[loss=0.2901, simple_loss=0.3944, pruned_loss=0.09293, over 24556.00 frames. ], tot_loss[loss=0.3113, simple_loss=0.4123, pruned_loss=0.1052, over 4690696.93 frames. ], batch size: 158, lr: 9.87e-03, grad_scale: 32.0 2026-09-24 00:35:26,851 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=66200.0, ans=0.125 2026-09-24 00:35:35,197 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=66266.66666666667, ans=0.125 2026-09-24 00:35:38,120 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=66300.0, ans=0.0 2026-09-24 00:35:39,066 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:35:46,908 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.36 vs. limit=15.0 2026-09-24 00:35:48,274 INFO [train.py:1192] (1/2) Epoch 21, batch 750, loss[loss=0.3361, simple_loss=0.4379, pruned_loss=0.1171, over 24649.00 frames. ], tot_loss[loss=0.3101, simple_loss=0.4111, pruned_loss=0.1046, over 4726823.75 frames. ], batch size: 175, lr: 9.86e-03, grad_scale: 32.0 2026-09-24 00:35:50,311 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.93 vs. limit=6.0 2026-09-24 00:35:53,490 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=66400.0, ans=0.125 2026-09-24 00:35:54,828 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=66400.0, ans=0.125 2026-09-24 00:35:56,350 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.596e+02 3.197e+02 3.654e+02 4.498e+02 6.742e+02, threshold=7.308e+02, percent-clipped=0.0 2026-09-24 00:35:56,455 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=66400.0, ans=0.1 2026-09-24 00:35:59,384 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:35:59,846 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:36:09,247 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.01 vs. limit=15.0 2026-09-24 00:36:14,114 INFO [train.py:1192] (1/2) Epoch 21, batch 800, loss[loss=0.2332, simple_loss=0.3397, pruned_loss=0.06335, over 24603.00 frames. ], tot_loss[loss=0.3095, simple_loss=0.4104, pruned_loss=0.1043, over 4752467.31 frames. ], batch size: 137, lr: 9.85e-03, grad_scale: 32.0 2026-09-24 00:36:23,583 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=66566.66666666667, ans=0.0 2026-09-24 00:36:29,939 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:36:37,045 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=66666.66666666667, ans=0.125 2026-09-24 00:36:39,843 INFO [train.py:1192] (1/2) Epoch 21, batch 850, loss[loss=0.3408, simple_loss=0.4391, pruned_loss=0.1212, over 24548.00 frames. ], tot_loss[loss=0.309, simple_loss=0.4098, pruned_loss=0.1041, over 4770669.44 frames. ], batch size: 204, lr: 9.84e-03, grad_scale: 32.0 2026-09-24 00:36:41,608 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=66700.0, ans=0.2 2026-09-24 00:36:42,196 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=66700.0, ans=0.125 2026-09-24 00:36:47,761 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.669e+02 3.302e+02 3.726e+02 4.097e+02 5.783e+02, threshold=7.453e+02, percent-clipped=0.0 2026-09-24 00:36:54,511 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=66766.66666666667, ans=0.0 2026-09-24 00:37:05,598 INFO [train.py:1192] (1/2) Epoch 21, batch 900, loss[loss=0.2609, simple_loss=0.3643, pruned_loss=0.0788, over 24554.00 frames. ], tot_loss[loss=0.3099, simple_loss=0.4104, pruned_loss=0.1047, over 4781495.90 frames. ], batch size: 137, lr: 9.83e-03, grad_scale: 16.0 2026-09-24 00:37:12,503 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=66900.0, ans=0.1 2026-09-24 00:37:14,412 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=66900.0, ans=0.0 2026-09-24 00:37:30,100 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=67000.0, ans=0.1 2026-09-24 00:37:30,990 INFO [train.py:1192] (1/2) Epoch 21, batch 950, loss[loss=0.4013, simple_loss=0.4415, pruned_loss=0.1805, over 11700.00 frames. ], tot_loss[loss=0.3112, simple_loss=0.4096, pruned_loss=0.1064, over 4712574.70 frames. ], batch size: 334, lr: 9.82e-03, grad_scale: 16.0 2026-09-24 00:37:43,086 INFO [train.py:1192] (1/2) Epoch 22, batch 0, loss[loss=0.2629, simple_loss=0.3698, pruned_loss=0.07802, over 24531.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3698, pruned_loss=0.07802, over 24531.00 frames. ], batch size: 137, lr: 9.59e-03, grad_scale: 32.0 2026-09-24 00:37:43,086 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 00:37:48,315 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.4919, 1.8855, 1.8809, 1.6070, 1.9510, 1.6996, 1.7805, 1.6113], device='cuda:1') 2026-09-24 00:37:54,393 INFO [train.py:1224] (1/2) Epoch 22, validation: loss=0.1913, simple_loss=0.3083, pruned_loss=0.03717, over 2564189.00 frames. 2026-09-24 00:37:54,393 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 00:37:54,493 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=67060.0, ans=0.0 2026-09-24 00:37:56,551 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=67060.0, ans=0.025 2026-09-24 00:37:58,356 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.465e+02 4.030e+02 4.667e+02 7.455e+02, threshold=8.059e+02, percent-clipped=1.0 2026-09-24 00:38:08,872 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=67126.66666666667, ans=0.1 2026-09-24 00:38:15,907 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=67193.33333333333, ans=0.125 2026-09-24 00:38:19,926 INFO [train.py:1192] (1/2) Epoch 22, batch 50, loss[loss=0.24, simple_loss=0.3431, pruned_loss=0.06851, over 24254.00 frames. ], tot_loss[loss=0.315, simple_loss=0.416, pruned_loss=0.107, over 1081199.72 frames. ], batch size: 125, lr: 9.57e-03, grad_scale: 32.0 2026-09-24 00:38:20,040 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=67226.66666666667, ans=0.125 2026-09-24 00:38:26,950 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=10.43 vs. limit=15.0 2026-09-24 00:38:30,802 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=67293.33333333333, ans=0.1 2026-09-24 00:38:33,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=67293.33333333333, ans=0.2 2026-09-24 00:38:43,307 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=67360.0, ans=0.0 2026-09-24 00:38:45,037 INFO [train.py:1192] (1/2) Epoch 22, batch 100, loss[loss=0.2867, simple_loss=0.39, pruned_loss=0.09168, over 24595.00 frames. ], tot_loss[loss=0.3142, simple_loss=0.4169, pruned_loss=0.1057, over 1914489.73 frames. ], batch size: 154, lr: 9.56e-03, grad_scale: 32.0 2026-09-24 00:38:49,283 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.513e+02 3.095e+02 3.490e+02 4.082e+02 5.584e+02, threshold=6.980e+02, percent-clipped=0.0 2026-09-24 00:38:50,990 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=67426.66666666667, ans=0.025 2026-09-24 00:38:51,305 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.80 vs. limit=15.0 2026-09-24 00:39:06,088 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:39:08,611 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=67526.66666666667, ans=0.125 2026-09-24 00:39:11,403 INFO [train.py:1192] (1/2) Epoch 22, batch 150, loss[loss=0.2795, simple_loss=0.3693, pruned_loss=0.09488, over 24283.00 frames. ], tot_loss[loss=0.3115, simple_loss=0.4134, pruned_loss=0.1048, over 2559802.91 frames. ], batch size: 125, lr: 9.55e-03, grad_scale: 32.0 2026-09-24 00:39:13,773 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=67560.0, ans=0.125 2026-09-24 00:39:26,540 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=67660.0, ans=0.125 2026-09-24 00:39:27,106 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=67660.0, ans=0.125 2026-09-24 00:39:28,291 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=67660.0, ans=0.2 2026-09-24 00:39:37,253 INFO [train.py:1192] (1/2) Epoch 22, batch 200, loss[loss=0.354, simple_loss=0.4581, pruned_loss=0.1249, over 24194.00 frames. ], tot_loss[loss=0.3099, simple_loss=0.4113, pruned_loss=0.1042, over 3058999.71 frames. ], batch size: 257, lr: 9.54e-03, grad_scale: 32.0 2026-09-24 00:39:38,263 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=67726.66666666667, ans=0.125 2026-09-24 00:39:41,517 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.529e+02 3.283e+02 3.694e+02 4.375e+02 6.321e+02, threshold=7.388e+02, percent-clipped=0.0 2026-09-24 00:39:43,109 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=67760.0, ans=0.125 2026-09-24 00:39:51,657 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=67793.33333333333, ans=0.125 2026-09-24 00:39:56,116 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=21.30 vs. limit=22.5 2026-09-24 00:40:03,095 INFO [train.py:1192] (1/2) Epoch 22, batch 250, loss[loss=0.3345, simple_loss=0.4449, pruned_loss=0.112, over 24360.00 frames. ], tot_loss[loss=0.309, simple_loss=0.4105, pruned_loss=0.1037, over 3443927.41 frames. ], batch size: 225, lr: 9.53e-03, grad_scale: 16.0 2026-09-24 00:40:08,317 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=67926.66666666667, ans=0.2 2026-09-24 00:40:19,263 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=67993.33333333333, ans=0.05 2026-09-24 00:40:23,005 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.87 vs. limit=22.5 2026-09-24 00:40:23,704 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=68026.66666666667, ans=0.025 2026-09-24 00:40:25,997 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.66 vs. limit=15.0 2026-09-24 00:40:26,195 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=68026.66666666667, ans=0.125 2026-09-24 00:40:28,798 INFO [train.py:1192] (1/2) Epoch 22, batch 300, loss[loss=0.3269, simple_loss=0.4341, pruned_loss=0.1099, over 24572.00 frames. ], tot_loss[loss=0.3072, simple_loss=0.4092, pruned_loss=0.1026, over 3757992.72 frames. ], batch size: 204, lr: 9.52e-03, grad_scale: 16.0 2026-09-24 00:40:30,339 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.50 vs. limit=10.0 2026-09-24 00:40:33,113 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.694e+02 3.361e+02 3.840e+02 4.584e+02 9.476e+02, threshold=7.681e+02, percent-clipped=4.0 2026-09-24 00:40:50,574 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=68193.33333333333, ans=0.0 2026-09-24 00:40:54,231 INFO [train.py:1192] (1/2) Epoch 22, batch 350, loss[loss=0.2368, simple_loss=0.3451, pruned_loss=0.06421, over 24549.00 frames. ], tot_loss[loss=0.3071, simple_loss=0.4095, pruned_loss=0.1024, over 3998468.59 frames. ], batch size: 137, lr: 9.51e-03, grad_scale: 16.0 2026-09-24 00:40:57,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=68226.66666666667, ans=0.125 2026-09-24 00:40:59,753 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=68260.0, ans=0.125 2026-09-24 00:41:02,726 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=68260.0, ans=0.0 2026-09-24 00:41:19,454 INFO [train.py:1192] (1/2) Epoch 22, batch 400, loss[loss=0.3378, simple_loss=0.4364, pruned_loss=0.1196, over 24557.00 frames. ], tot_loss[loss=0.3058, simple_loss=0.4081, pruned_loss=0.1018, over 4184213.14 frames. ], batch size: 170, lr: 9.50e-03, grad_scale: 32.0 2026-09-24 00:41:23,734 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.589e+02 3.105e+02 3.438e+02 4.062e+02 5.782e+02, threshold=6.875e+02, percent-clipped=0.0 2026-09-24 00:41:26,879 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=68426.66666666667, ans=0.5 2026-09-24 00:41:30,110 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=68460.0, ans=0.1 2026-09-24 00:41:42,245 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=68526.66666666667, ans=0.125 2026-09-24 00:41:43,760 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=68526.66666666667, ans=0.09899494936611666 2026-09-24 00:41:45,175 INFO [train.py:1192] (1/2) Epoch 22, batch 450, loss[loss=0.349, simple_loss=0.4444, pruned_loss=0.1268, over 24620.00 frames. ], tot_loss[loss=0.3064, simple_loss=0.4085, pruned_loss=0.1021, over 4322854.85 frames. ], batch size: 175, lr: 9.49e-03, grad_scale: 32.0 2026-09-24 00:41:46,675 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=68560.0, ans=0.125 2026-09-24 00:41:54,140 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=68593.33333333333, ans=0.2 2026-09-24 00:42:01,402 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=68660.0, ans=0.1 2026-09-24 00:42:06,702 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=68693.33333333333, ans=0.1 2026-09-24 00:42:10,648 INFO [train.py:1192] (1/2) Epoch 22, batch 500, loss[loss=0.3126, simple_loss=0.4336, pruned_loss=0.09581, over 24511.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.4069, pruned_loss=0.1012, over 4440225.89 frames. ], batch size: 218, lr: 9.48e-03, grad_scale: 32.0 2026-09-24 00:42:15,217 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.545e+02 3.417e+02 3.981e+02 4.780e+02 8.506e+02, threshold=7.962e+02, percent-clipped=3.0 2026-09-24 00:42:15,675 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=68760.0, ans=0.0 2026-09-24 00:42:16,364 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.25 vs. limit=22.5 2026-09-24 00:42:32,849 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=68860.0, ans=0.2 2026-09-24 00:42:36,249 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=68893.33333333333, ans=0.125 2026-09-24 00:42:36,642 INFO [train.py:1192] (1/2) Epoch 22, batch 550, loss[loss=0.35, simple_loss=0.4545, pruned_loss=0.1228, over 24286.00 frames. ], tot_loss[loss=0.3063, simple_loss=0.4082, pruned_loss=0.1022, over 4524272.66 frames. ], batch size: 257, lr: 9.47e-03, grad_scale: 32.0 2026-09-24 00:42:41,034 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=68893.33333333333, ans=0.125 2026-09-24 00:42:49,716 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=68960.0, ans=0.1 2026-09-24 00:43:02,559 INFO [train.py:1192] (1/2) Epoch 22, batch 600, loss[loss=0.3303, simple_loss=0.4399, pruned_loss=0.1104, over 24397.00 frames. ], tot_loss[loss=0.3079, simple_loss=0.4095, pruned_loss=0.1031, over 4591484.46 frames. ], batch size: 235, lr: 9.46e-03, grad_scale: 32.0 2026-09-24 00:43:07,409 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.420e+02 3.284e+02 3.687e+02 4.180e+02 5.343e+02, threshold=7.374e+02, percent-clipped=0.0 2026-09-24 00:43:17,672 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=69160.0, ans=0.0 2026-09-24 00:43:26,286 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=69193.33333333333, ans=0.0 2026-09-24 00:43:26,829 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=69193.33333333333, ans=0.125 2026-09-24 00:43:28,116 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=69226.66666666667, ans=0.1 2026-09-24 00:43:28,439 INFO [train.py:1192] (1/2) Epoch 22, batch 650, loss[loss=0.3195, simple_loss=0.4107, pruned_loss=0.1142, over 24598.00 frames. ], tot_loss[loss=0.3076, simple_loss=0.409, pruned_loss=0.1031, over 4656029.16 frames. ], batch size: 154, lr: 9.45e-03, grad_scale: 32.0 2026-09-24 00:43:36,070 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=69260.0, ans=10.0 2026-09-24 00:43:39,026 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=69293.33333333333, ans=0.1 2026-09-24 00:43:43,097 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=69326.66666666667, ans=0.125 2026-09-24 00:43:54,194 INFO [train.py:1192] (1/2) Epoch 22, batch 700, loss[loss=0.2895, simple_loss=0.3915, pruned_loss=0.09372, over 24551.00 frames. ], tot_loss[loss=0.3087, simple_loss=0.4102, pruned_loss=0.1036, over 4690703.04 frames. ], batch size: 158, lr: 9.44e-03, grad_scale: 32.0 2026-09-24 00:43:58,621 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.790e+02 3.448e+02 3.947e+02 4.620e+02 7.877e+02, threshold=7.894e+02, percent-clipped=2.0 2026-09-24 00:43:58,889 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.25 vs. limit=22.5 2026-09-24 00:44:13,068 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.45 vs. limit=6.0 2026-09-24 00:44:16,873 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=69526.66666666667, ans=0.07 2026-09-24 00:44:19,070 INFO [train.py:1192] (1/2) Epoch 22, batch 750, loss[loss=0.3016, simple_loss=0.4134, pruned_loss=0.09494, over 24609.00 frames. ], tot_loss[loss=0.3068, simple_loss=0.4086, pruned_loss=0.1025, over 4723323.71 frames. ], batch size: 175, lr: 9.43e-03, grad_scale: 32.0 2026-09-24 00:44:21,895 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.86 vs. limit=15.0 2026-09-24 00:44:22,149 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=69560.0, ans=0.125 2026-09-24 00:44:24,966 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=69593.33333333333, ans=0.125 2026-09-24 00:44:26,452 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=69593.33333333333, ans=0.0 2026-09-24 00:44:30,836 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=69626.66666666667, ans=0.125 2026-09-24 00:44:40,163 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=69693.33333333333, ans=0.125 2026-09-24 00:44:42,137 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=69693.33333333333, ans=0.1 2026-09-24 00:44:42,254 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.15 vs. limit=15.0 2026-09-24 00:44:44,679 INFO [train.py:1192] (1/2) Epoch 22, batch 800, loss[loss=0.2679, simple_loss=0.3698, pruned_loss=0.08298, over 24587.00 frames. ], tot_loss[loss=0.3063, simple_loss=0.4081, pruned_loss=0.1022, over 4749137.08 frames. ], batch size: 137, lr: 9.42e-03, grad_scale: 32.0 2026-09-24 00:44:48,944 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.402e+02 3.206e+02 3.625e+02 4.112e+02 5.774e+02, threshold=7.249e+02, percent-clipped=0.0 2026-09-24 00:44:55,941 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=69793.33333333333, ans=0.125 2026-09-24 00:45:02,000 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=69826.66666666667, ans=0.125 2026-09-24 00:45:02,536 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=69826.66666666667, ans=0.125 2026-09-24 00:45:03,935 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.33 vs. limit=15.0 2026-09-24 00:45:10,112 INFO [train.py:1192] (1/2) Epoch 22, batch 850, loss[loss=0.3518, simple_loss=0.4492, pruned_loss=0.1273, over 24550.00 frames. ], tot_loss[loss=0.3057, simple_loss=0.4076, pruned_loss=0.1019, over 4769577.47 frames. ], batch size: 204, lr: 9.41e-03, grad_scale: 32.0 2026-09-24 00:45:11,272 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.07 vs. limit=6.0 2026-09-24 00:45:16,269 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.92 vs. limit=15.0 2026-09-24 00:45:26,979 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=69993.33333333333, ans=0.125 2026-09-24 00:45:33,754 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.29 vs. limit=12.0 2026-09-24 00:45:36,327 INFO [train.py:1192] (1/2) Epoch 22, batch 900, loss[loss=0.2582, simple_loss=0.3647, pruned_loss=0.07587, over 24547.00 frames. ], tot_loss[loss=0.3062, simple_loss=0.4078, pruned_loss=0.1023, over 4780404.89 frames. ], batch size: 137, lr: 9.40e-03, grad_scale: 32.0 2026-09-24 00:45:40,947 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.552e+02 3.184e+02 3.761e+02 4.280e+02 6.226e+02, threshold=7.523e+02, percent-clipped=0.0 2026-09-24 00:45:43,796 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=70093.33333333333, ans=0.0 2026-09-24 00:45:49,067 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=70126.66666666667, ans=0.0 2026-09-24 00:46:01,429 INFO [train.py:1192] (1/2) Epoch 22, batch 950, loss[loss=0.396, simple_loss=0.441, pruned_loss=0.1756, over 11766.00 frames. ], tot_loss[loss=0.3068, simple_loss=0.4067, pruned_loss=0.1034, over 4715991.05 frames. ], batch size: 333, lr: 9.39e-03, grad_scale: 32.0 2026-09-24 00:46:01,509 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=70226.66666666667, ans=0.0 2026-09-24 00:46:02,431 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=70226.66666666667, ans=0.125 2026-09-24 00:46:04,374 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=70226.66666666667, ans=0.0 2026-09-24 00:46:12,763 INFO [train.py:1192] (1/2) Epoch 23, batch 0, loss[loss=0.2425, simple_loss=0.3573, pruned_loss=0.06388, over 24603.00 frames. ], tot_loss[loss=0.2425, simple_loss=0.3573, pruned_loss=0.06388, over 24603.00 frames. ], batch size: 137, lr: 9.18e-03, grad_scale: 32.0 2026-09-24 00:46:12,763 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 00:46:24,381 INFO [train.py:1224] (1/2) Epoch 23, validation: loss=0.1918, simple_loss=0.3086, pruned_loss=0.0375, over 2564189.00 frames. 2026-09-24 00:46:24,382 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 00:46:35,140 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=70320.0, ans=0.2 2026-09-24 00:46:35,665 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=70320.0, ans=0.2 2026-09-24 00:46:38,028 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=70320.0, ans=0.125 2026-09-24 00:46:40,371 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.37 vs. limit=22.5 2026-09-24 00:46:43,019 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=70353.33333333333, ans=0.125 2026-09-24 00:46:49,880 INFO [train.py:1192] (1/2) Epoch 23, batch 50, loss[loss=0.2441, simple_loss=0.3492, pruned_loss=0.06948, over 24261.00 frames. ], tot_loss[loss=0.3146, simple_loss=0.4163, pruned_loss=0.1065, over 1081348.67 frames. ], batch size: 125, lr: 9.17e-03, grad_scale: 32.0 2026-09-24 00:46:50,292 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.535e+02 3.301e+02 3.835e+02 4.680e+02 7.694e+02, threshold=7.671e+02, percent-clipped=1.0 2026-09-24 00:47:10,681 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=70553.33333333333, ans=0.1 2026-09-24 00:47:15,829 INFO [train.py:1192] (1/2) Epoch 23, batch 100, loss[loss=0.311, simple_loss=0.4062, pruned_loss=0.1079, over 24581.00 frames. ], tot_loss[loss=0.3137, simple_loss=0.4171, pruned_loss=0.1052, over 1914369.14 frames. ], batch size: 154, lr: 9.16e-03, grad_scale: 32.0 2026-09-24 00:47:17,472 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.80 vs. limit=6.0 2026-09-24 00:47:20,561 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.59 vs. limit=15.0 2026-09-24 00:47:27,775 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=70653.33333333333, ans=0.125 2026-09-24 00:47:34,828 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=70686.66666666667, ans=0.05 2026-09-24 00:47:41,017 INFO [train.py:1192] (1/2) Epoch 23, batch 150, loss[loss=0.2644, simple_loss=0.3583, pruned_loss=0.08529, over 24275.00 frames. ], tot_loss[loss=0.3073, simple_loss=0.4104, pruned_loss=0.1021, over 2560715.31 frames. ], batch size: 125, lr: 9.15e-03, grad_scale: 32.0 2026-09-24 00:47:41,480 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.365e+02 3.149e+02 3.599e+02 4.194e+02 5.146e+02, threshold=7.197e+02, percent-clipped=0.0 2026-09-24 00:47:57,326 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=70853.33333333333, ans=0.125 2026-09-24 00:47:57,830 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=70853.33333333333, ans=0.125 2026-09-24 00:48:04,038 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=70886.66666666667, ans=0.0 2026-09-24 00:48:06,368 INFO [train.py:1192] (1/2) Epoch 23, batch 200, loss[loss=0.3746, simple_loss=0.4686, pruned_loss=0.1403, over 24216.00 frames. ], tot_loss[loss=0.3037, simple_loss=0.4072, pruned_loss=0.1001, over 3060331.10 frames. ], batch size: 257, lr: 9.14e-03, grad_scale: 32.0 2026-09-24 00:48:10,382 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.95 vs. limit=15.0 2026-09-24 00:48:31,769 INFO [train.py:1192] (1/2) Epoch 23, batch 250, loss[loss=0.3495, simple_loss=0.4542, pruned_loss=0.1224, over 24365.00 frames. ], tot_loss[loss=0.3047, simple_loss=0.4075, pruned_loss=0.1009, over 3444315.72 frames. ], batch size: 225, lr: 9.13e-03, grad_scale: 32.0 2026-09-24 00:48:32,325 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.652e+02 3.248e+02 3.854e+02 4.393e+02 1.028e+03, threshold=7.707e+02, percent-clipped=3.0 2026-09-24 00:48:34,653 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.41 vs. limit=12.0 2026-09-24 00:48:36,959 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=71120.0, ans=0.2 2026-09-24 00:48:39,849 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=71120.0, ans=0.125 2026-09-24 00:48:50,584 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=71186.66666666667, ans=0.0 2026-09-24 00:48:57,976 INFO [train.py:1192] (1/2) Epoch 23, batch 300, loss[loss=0.3235, simple_loss=0.4298, pruned_loss=0.1086, over 24531.00 frames. ], tot_loss[loss=0.3049, simple_loss=0.4076, pruned_loss=0.1011, over 3753122.63 frames. ], batch size: 204, lr: 9.12e-03, grad_scale: 32.0 2026-09-24 00:49:11,370 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=71320.0, ans=0.1 2026-09-24 00:49:18,941 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=71386.66666666667, ans=0.1 2026-09-24 00:49:23,871 INFO [train.py:1192] (1/2) Epoch 23, batch 350, loss[loss=0.2517, simple_loss=0.3534, pruned_loss=0.07503, over 24593.00 frames. ], tot_loss[loss=0.3051, simple_loss=0.408, pruned_loss=0.1011, over 3991537.93 frames. ], batch size: 137, lr: 9.11e-03, grad_scale: 32.0 2026-09-24 00:49:24,399 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.415e+02 3.149e+02 3.538e+02 4.133e+02 6.571e+02, threshold=7.075e+02, percent-clipped=0.0 2026-09-24 00:49:26,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=71420.0, ans=0.125 2026-09-24 00:49:45,088 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.46 vs. limit=15.0 2026-09-24 00:49:47,355 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.45 vs. limit=15.0 2026-09-24 00:49:49,539 INFO [train.py:1192] (1/2) Epoch 23, batch 400, loss[loss=0.3208, simple_loss=0.4181, pruned_loss=0.1117, over 24568.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.4073, pruned_loss=0.1009, over 4177939.81 frames. ], batch size: 170, lr: 9.10e-03, grad_scale: 32.0 2026-09-24 00:49:51,514 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=71586.66666666667, ans=0.0 2026-09-24 00:49:54,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=71620.0, ans=0.125 2026-09-24 00:49:58,999 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.06 vs. limit=15.0 2026-09-24 00:50:00,197 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=71653.33333333333, ans=10.0 2026-09-24 00:50:08,645 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=71686.66666666667, ans=0.1 2026-09-24 00:50:09,090 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=71686.66666666667, ans=0.95 2026-09-24 00:50:13,501 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=71720.0, ans=0.0 2026-09-24 00:50:14,917 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=71753.33333333333, ans=0.125 2026-09-24 00:50:15,351 INFO [train.py:1192] (1/2) Epoch 23, batch 450, loss[loss=0.3115, simple_loss=0.4152, pruned_loss=0.1039, over 24611.00 frames. ], tot_loss[loss=0.3065, simple_loss=0.4085, pruned_loss=0.1023, over 4317636.80 frames. ], batch size: 175, lr: 9.09e-03, grad_scale: 32.0 2026-09-24 00:50:15,793 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.435e+02 3.309e+02 3.663e+02 4.121e+02 6.849e+02, threshold=7.325e+02, percent-clipped=0.0 2026-09-24 00:50:20,610 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.62 vs. limit=15.0 2026-09-24 00:50:25,932 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=71820.0, ans=0.125 2026-09-24 00:50:37,856 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:50:40,266 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer_na.min_abs, batch_count=71886.66666666667, ans=0.02 2026-09-24 00:50:41,172 INFO [train.py:1192] (1/2) Epoch 23, batch 500, loss[loss=0.3587, simple_loss=0.4622, pruned_loss=0.1276, over 24524.00 frames. ], tot_loss[loss=0.305, simple_loss=0.4069, pruned_loss=0.1016, over 4435745.00 frames. ], batch size: 218, lr: 9.08e-03, grad_scale: 32.0 2026-09-24 00:50:48,269 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=71953.33333333333, ans=0.025 2026-09-24 00:50:59,603 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=72020.0, ans=0.07 2026-09-24 00:51:07,064 INFO [train.py:1192] (1/2) Epoch 23, batch 550, loss[loss=0.3039, simple_loss=0.4217, pruned_loss=0.09307, over 24249.00 frames. ], tot_loss[loss=0.3055, simple_loss=0.4075, pruned_loss=0.1018, over 4520674.86 frames. ], batch size: 257, lr: 9.08e-03, grad_scale: 32.0 2026-09-24 00:51:07,605 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.625e+02 3.353e+02 3.894e+02 4.521e+02 6.252e+02, threshold=7.789e+02, percent-clipped=0.0 2026-09-24 00:51:19,655 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=72153.33333333333, ans=0.2 2026-09-24 00:51:31,254 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=72220.0, ans=0.0 2026-09-24 00:51:32,633 INFO [train.py:1192] (1/2) Epoch 23, batch 600, loss[loss=0.3234, simple_loss=0.4411, pruned_loss=0.1029, over 24308.00 frames. ], tot_loss[loss=0.3049, simple_loss=0.4075, pruned_loss=0.1012, over 4587503.90 frames. ], batch size: 234, lr: 9.07e-03, grad_scale: 32.0 2026-09-24 00:51:36,070 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=72253.33333333333, ans=0.125 2026-09-24 00:51:38,812 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.10 vs. limit=22.5 2026-09-24 00:51:40,845 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=72286.66666666667, ans=0.035 2026-09-24 00:51:57,110 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=72386.66666666667, ans=0.2 2026-09-24 00:51:57,954 INFO [train.py:1192] (1/2) Epoch 23, batch 650, loss[loss=0.2871, simple_loss=0.3897, pruned_loss=0.09221, over 24621.00 frames. ], tot_loss[loss=0.3036, simple_loss=0.4064, pruned_loss=0.1004, over 4653092.04 frames. ], batch size: 154, lr: 9.06e-03, grad_scale: 32.0 2026-09-24 00:51:58,344 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.562e+02 3.127e+02 3.494e+02 4.174e+02 5.653e+02, threshold=6.988e+02, percent-clipped=0.0 2026-09-24 00:52:05,351 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=72453.33333333333, ans=0.125 2026-09-24 00:52:10,054 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=72486.66666666667, ans=0.125 2026-09-24 00:52:12,169 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=72486.66666666667, ans=0.125 2026-09-24 00:52:19,040 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=72553.33333333333, ans=0.0 2026-09-24 00:52:20,404 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.81 vs. limit=10.0 2026-09-24 00:52:22,248 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=72553.33333333333, ans=0.0 2026-09-24 00:52:24,009 INFO [train.py:1192] (1/2) Epoch 23, batch 700, loss[loss=0.3034, simple_loss=0.4059, pruned_loss=0.1004, over 24558.00 frames. ], tot_loss[loss=0.305, simple_loss=0.4078, pruned_loss=0.1011, over 4689152.23 frames. ], batch size: 158, lr: 9.05e-03, grad_scale: 32.0 2026-09-24 00:52:41,626 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=72686.66666666667, ans=0.2 2026-09-24 00:52:43,028 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=72686.66666666667, ans=0.1 2026-09-24 00:52:45,971 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=72720.0, ans=0.125 2026-09-24 00:52:49,652 INFO [train.py:1192] (1/2) Epoch 23, batch 750, loss[loss=0.2781, simple_loss=0.3954, pruned_loss=0.08037, over 24631.00 frames. ], tot_loss[loss=0.3042, simple_loss=0.4067, pruned_loss=0.1008, over 4725492.97 frames. ], batch size: 175, lr: 9.04e-03, grad_scale: 32.0 2026-09-24 00:52:50,106 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.747e+02 3.315e+02 3.683e+02 4.146e+02 6.702e+02, threshold=7.366e+02, percent-clipped=0.0 2026-09-24 00:52:57,540 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=72786.66666666667, ans=0.0 2026-09-24 00:53:05,103 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=72853.33333333333, ans=0.125 2026-09-24 00:53:07,282 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.49 vs. limit=15.0 2026-09-24 00:53:11,505 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=72886.66666666667, ans=0.1 2026-09-24 00:53:15,296 INFO [train.py:1192] (1/2) Epoch 23, batch 800, loss[loss=0.2776, simple_loss=0.374, pruned_loss=0.09055, over 24551.00 frames. ], tot_loss[loss=0.3034, simple_loss=0.406, pruned_loss=0.1004, over 4751394.25 frames. ], batch size: 137, lr: 9.03e-03, grad_scale: 32.0 2026-09-24 00:53:16,482 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.28 vs. limit=15.0 2026-09-24 00:53:19,246 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=11.05 vs. limit=15.0 2026-09-24 00:53:25,660 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=72986.66666666667, ans=0.125 2026-09-24 00:53:40,873 INFO [train.py:1192] (1/2) Epoch 23, batch 850, loss[loss=0.3381, simple_loss=0.4428, pruned_loss=0.1167, over 24543.00 frames. ], tot_loss[loss=0.3023, simple_loss=0.405, pruned_loss=0.09978, over 4769749.89 frames. ], batch size: 204, lr: 9.02e-03, grad_scale: 32.0 2026-09-24 00:53:41,370 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.638e+02 3.291e+02 3.719e+02 4.187e+02 5.970e+02, threshold=7.437e+02, percent-clipped=0.0 2026-09-24 00:53:45,628 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 00:53:52,970 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=73153.33333333333, ans=0.0 2026-09-24 00:54:07,476 INFO [train.py:1192] (1/2) Epoch 23, batch 900, loss[loss=0.2402, simple_loss=0.3474, pruned_loss=0.06651, over 24566.00 frames. ], tot_loss[loss=0.3031, simple_loss=0.4057, pruned_loss=0.1002, over 4780524.37 frames. ], batch size: 137, lr: 9.01e-03, grad_scale: 32.0 2026-09-24 00:54:08,502 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=73253.33333333333, ans=0.0 2026-09-24 00:54:23,829 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=18.53 vs. limit=22.5 2026-09-24 00:54:29,192 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=73386.66666666667, ans=0.2 2026-09-24 00:54:32,535 INFO [train.py:1192] (1/2) Epoch 23, batch 950, loss[loss=0.4178, simple_loss=0.4577, pruned_loss=0.189, over 11813.00 frames. ], tot_loss[loss=0.3044, simple_loss=0.405, pruned_loss=0.1019, over 4710833.58 frames. ], batch size: 334, lr: 9.00e-03, grad_scale: 32.0 2026-09-24 00:54:33,096 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.255e+02 3.474e+02 3.945e+02 4.433e+02 7.088e+02, threshold=7.889e+02, percent-clipped=0.0 2026-09-24 00:54:43,064 INFO [train.py:1192] (1/2) Epoch 24, batch 0, loss[loss=0.2812, simple_loss=0.3814, pruned_loss=0.09053, over 24597.00 frames. ], tot_loss[loss=0.2812, simple_loss=0.3814, pruned_loss=0.09053, over 24597.00 frames. ], batch size: 137, lr: 8.81e-03, grad_scale: 32.0 2026-09-24 00:54:43,064 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 00:54:51,367 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7021, 4.6998, 4.4918, 4.3003], device='cuda:1') 2026-09-24 00:54:54,494 INFO [train.py:1224] (1/2) Epoch 24, validation: loss=0.1877, simple_loss=0.3051, pruned_loss=0.03517, over 2564189.00 frames. 2026-09-24 00:54:54,494 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 00:55:18,181 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=73580.0, ans=0.125 2026-09-24 00:55:20,090 INFO [train.py:1192] (1/2) Epoch 24, batch 50, loss[loss=0.2336, simple_loss=0.3367, pruned_loss=0.06527, over 24231.00 frames. ], tot_loss[loss=0.3122, simple_loss=0.4142, pruned_loss=0.1051, over 1080432.08 frames. ], batch size: 125, lr: 8.80e-03, grad_scale: 32.0 2026-09-24 00:55:25,571 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=73646.66666666667, ans=0.2 2026-09-24 00:55:42,124 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.601e+02 3.263e+02 3.527e+02 3.981e+02 7.523e+02, threshold=7.053e+02, percent-clipped=0.0 2026-09-24 00:55:45,662 INFO [train.py:1192] (1/2) Epoch 24, batch 100, loss[loss=0.3094, simple_loss=0.4043, pruned_loss=0.1073, over 24619.00 frames. ], tot_loss[loss=0.3122, simple_loss=0.416, pruned_loss=0.1042, over 1915688.42 frames. ], batch size: 154, lr: 8.79e-03, grad_scale: 32.0 2026-09-24 00:55:48,936 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=73780.0, ans=0.125 2026-09-24 00:56:04,788 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=73880.0, ans=0.125 2026-09-24 00:56:09,903 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=73913.33333333333, ans=0.125 2026-09-24 00:56:11,245 INFO [train.py:1192] (1/2) Epoch 24, batch 150, loss[loss=0.2417, simple_loss=0.3419, pruned_loss=0.07073, over 24241.00 frames. ], tot_loss[loss=0.307, simple_loss=0.4102, pruned_loss=0.1019, over 2560893.52 frames. ], batch size: 125, lr: 8.78e-03, grad_scale: 32.0 2026-09-24 00:56:25,427 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=74013.33333333333, ans=0.125 2026-09-24 00:56:25,922 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=74046.66666666667, ans=0.0 2026-09-24 00:56:29,529 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=74046.66666666667, ans=0.0 2026-09-24 00:56:31,128 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=74080.0, ans=0.025 2026-09-24 00:56:33,203 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.517e+02 3.199e+02 3.618e+02 4.148e+02 6.081e+02, threshold=7.236e+02, percent-clipped=0.0 2026-09-24 00:56:36,649 INFO [train.py:1192] (1/2) Epoch 24, batch 200, loss[loss=0.3543, simple_loss=0.4562, pruned_loss=0.1262, over 24216.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.408, pruned_loss=0.1006, over 3059749.34 frames. ], batch size: 257, lr: 8.77e-03, grad_scale: 32.0 2026-09-24 00:56:43,458 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=74146.66666666667, ans=0.125 2026-09-24 00:56:45,372 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.22 vs. limit=15.0 2026-09-24 00:56:54,188 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=74213.33333333333, ans=0.2 2026-09-24 00:56:57,513 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=74246.66666666667, ans=0.125 2026-09-24 00:57:02,135 INFO [train.py:1192] (1/2) Epoch 24, batch 250, loss[loss=0.323, simple_loss=0.4331, pruned_loss=0.1065, over 24360.00 frames. ], tot_loss[loss=0.3046, simple_loss=0.4076, pruned_loss=0.1008, over 3443425.21 frames. ], batch size: 225, lr: 8.76e-03, grad_scale: 32.0 2026-09-24 00:57:04,823 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.16 vs. limit=15.0 2026-09-24 00:57:05,340 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.min_positive, batch_count=74280.0, ans=0.025 2026-09-24 00:57:09,577 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=74313.33333333333, ans=0.0 2026-09-24 00:57:19,414 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=74380.0, ans=0.125 2026-09-24 00:57:23,987 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.559e+02 3.355e+02 3.739e+02 4.396e+02 1.004e+03, threshold=7.479e+02, percent-clipped=1.0 2026-09-24 00:57:27,460 INFO [train.py:1192] (1/2) Epoch 24, batch 300, loss[loss=0.312, simple_loss=0.4237, pruned_loss=0.1001, over 24511.00 frames. ], tot_loss[loss=0.3023, simple_loss=0.4058, pruned_loss=0.09945, over 3756776.50 frames. ], batch size: 204, lr: 8.75e-03, grad_scale: 64.0 2026-09-24 00:57:47,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=74546.66666666667, ans=0.1 2026-09-24 00:57:51,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=74580.0, ans=0.125 2026-09-24 00:57:52,825 INFO [train.py:1192] (1/2) Epoch 24, batch 350, loss[loss=0.2473, simple_loss=0.3546, pruned_loss=0.07004, over 24581.00 frames. ], tot_loss[loss=0.3035, simple_loss=0.4071, pruned_loss=0.09999, over 3997743.11 frames. ], batch size: 137, lr: 8.74e-03, grad_scale: 32.0 2026-09-24 00:58:12,413 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=74713.33333333333, ans=0.125 2026-09-24 00:58:15,649 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.641e+02 3.311e+02 3.647e+02 4.189e+02 5.156e+02, threshold=7.294e+02, percent-clipped=0.0 2026-09-24 00:58:16,256 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=74746.66666666667, ans=0.125 2026-09-24 00:58:16,470 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.50 vs. limit=22.5 2026-09-24 00:58:17,658 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=74746.66666666667, ans=0.2 2026-09-24 00:58:18,607 INFO [train.py:1192] (1/2) Epoch 24, batch 400, loss[loss=0.3118, simple_loss=0.4113, pruned_loss=0.1061, over 24563.00 frames. ], tot_loss[loss=0.3027, simple_loss=0.4061, pruned_loss=0.09967, over 4182571.67 frames. ], batch size: 170, lr: 8.74e-03, grad_scale: 32.0 2026-09-24 00:58:18,712 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=74780.0, ans=0.125 2026-09-24 00:58:21,601 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=74780.0, ans=0.0 2026-09-24 00:58:27,085 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=74813.33333333333, ans=0.125 2026-09-24 00:58:30,796 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=74846.66666666667, ans=0.125 2026-09-24 00:58:33,461 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=74846.66666666667, ans=0.1 2026-09-24 00:58:37,454 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.14 vs. limit=15.0 2026-09-24 00:58:42,155 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_na.min_abs, batch_count=74913.33333333333, ans=0.02 2026-09-24 00:58:44,654 INFO [train.py:1192] (1/2) Epoch 24, batch 450, loss[loss=0.3033, simple_loss=0.4093, pruned_loss=0.09868, over 24609.00 frames. ], tot_loss[loss=0.3036, simple_loss=0.407, pruned_loss=0.1001, over 4319018.23 frames. ], batch size: 175, lr: 8.73e-03, grad_scale: 32.0 2026-09-24 00:58:52,725 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=74980.0, ans=0.2 2026-09-24 00:58:55,588 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=75013.33333333333, ans=0.125 2026-09-24 00:58:58,273 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=75013.33333333333, ans=0.125 2026-09-24 00:58:58,782 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=75013.33333333333, ans=0.125 2026-09-24 00:59:01,063 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=75046.66666666667, ans=0.2 2026-09-24 00:59:05,726 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.56 vs. limit=6.0 2026-09-24 00:59:06,523 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.377e+02 3.242e+02 3.715e+02 4.285e+02 7.156e+02, threshold=7.431e+02, percent-clipped=0.0 2026-09-24 00:59:09,839 INFO [train.py:1192] (1/2) Epoch 24, batch 500, loss[loss=0.3442, simple_loss=0.448, pruned_loss=0.1202, over 24508.00 frames. ], tot_loss[loss=0.3025, simple_loss=0.4057, pruned_loss=0.09967, over 4437365.92 frames. ], batch size: 218, lr: 8.72e-03, grad_scale: 32.0 2026-09-24 00:59:35,613 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.87 vs. limit=15.0 2026-09-24 00:59:35,797 INFO [train.py:1192] (1/2) Epoch 24, batch 550, loss[loss=0.3128, simple_loss=0.4269, pruned_loss=0.09928, over 24328.00 frames. ], tot_loss[loss=0.3038, simple_loss=0.4066, pruned_loss=0.1005, over 4522460.90 frames. ], batch size: 257, lr: 8.71e-03, grad_scale: 32.0 2026-09-24 00:59:38,031 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=75280.0, ans=0.125 2026-09-24 00:59:58,732 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.456e+02 3.257e+02 3.612e+02 4.275e+02 6.072e+02, threshold=7.224e+02, percent-clipped=0.0 2026-09-24 01:00:01,510 INFO [train.py:1192] (1/2) Epoch 24, batch 600, loss[loss=0.3127, simple_loss=0.4279, pruned_loss=0.09869, over 24339.00 frames. ], tot_loss[loss=0.3044, simple_loss=0.4073, pruned_loss=0.1007, over 4589579.55 frames. ], batch size: 234, lr: 8.70e-03, grad_scale: 32.0 2026-09-24 01:00:27,024 INFO [train.py:1192] (1/2) Epoch 24, batch 650, loss[loss=0.2822, simple_loss=0.3809, pruned_loss=0.09174, over 24605.00 frames. ], tot_loss[loss=0.3017, simple_loss=0.4054, pruned_loss=0.09902, over 4654294.78 frames. ], batch size: 154, lr: 8.69e-03, grad_scale: 32.0 2026-09-24 01:00:34,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=75646.66666666667, ans=0.2 2026-09-24 01:00:37,772 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=75680.0, ans=0.1 2026-09-24 01:00:40,136 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=75680.0, ans=0.125 2026-09-24 01:00:50,074 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.734e+02 3.308e+02 3.642e+02 4.382e+02 5.994e+02, threshold=7.284e+02, percent-clipped=0.0 2026-09-24 01:00:50,211 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=75746.66666666667, ans=0.125 2026-09-24 01:00:53,193 INFO [train.py:1192] (1/2) Epoch 24, batch 700, loss[loss=0.2916, simple_loss=0.3952, pruned_loss=0.09396, over 24554.00 frames. ], tot_loss[loss=0.3031, simple_loss=0.4067, pruned_loss=0.09973, over 4690004.17 frames. ], batch size: 158, lr: 8.68e-03, grad_scale: 32.0 2026-09-24 01:00:59,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=75813.33333333333, ans=0.125 2026-09-24 01:01:01,278 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=75813.33333333333, ans=0.09899494936611666 2026-09-24 01:01:05,308 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.14 vs. limit=15.0 2026-09-24 01:01:07,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=75846.66666666667, ans=0.1 2026-09-24 01:01:13,417 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=75880.0, ans=0.125 2026-09-24 01:01:16,783 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=75913.33333333333, ans=10.0 2026-09-24 01:01:17,534 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=75913.33333333333, ans=0.125 2026-09-24 01:01:18,179 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.26 vs. limit=15.0 2026-09-24 01:01:19,355 INFO [train.py:1192] (1/2) Epoch 24, batch 750, loss[loss=0.2822, simple_loss=0.3942, pruned_loss=0.08512, over 24620.00 frames. ], tot_loss[loss=0.3021, simple_loss=0.4055, pruned_loss=0.0993, over 4726505.58 frames. ], batch size: 175, lr: 8.68e-03, grad_scale: 32.0 2026-09-24 01:01:24,848 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=75980.0, ans=0.125 2026-09-24 01:01:28,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=75980.0, ans=0.0 2026-09-24 01:01:29,668 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=76013.33333333333, ans=0.0 2026-09-24 01:01:36,975 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=76046.66666666667, ans=0.125 2026-09-24 01:01:42,462 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.581e+02 3.355e+02 3.755e+02 4.221e+02 7.670e+02, threshold=7.510e+02, percent-clipped=2.0 2026-09-24 01:01:43,041 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=76080.0, ans=10.0 2026-09-24 01:01:45,207 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=76080.0, ans=0.09899494936611666 2026-09-24 01:01:45,983 INFO [train.py:1192] (1/2) Epoch 24, batch 800, loss[loss=0.2573, simple_loss=0.3637, pruned_loss=0.07549, over 24571.00 frames. ], tot_loss[loss=0.3018, simple_loss=0.4052, pruned_loss=0.09917, over 4753422.08 frames. ], batch size: 137, lr: 8.67e-03, grad_scale: 32.0 2026-09-24 01:01:53,212 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=76146.66666666667, ans=0.125 2026-09-24 01:01:57,977 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=76180.0, ans=0.0 2026-09-24 01:01:59,880 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=76180.0, ans=0.0 2026-09-24 01:02:11,027 INFO [train.py:1192] (1/2) Epoch 24, batch 850, loss[loss=0.3175, simple_loss=0.4249, pruned_loss=0.105, over 24529.00 frames. ], tot_loss[loss=0.301, simple_loss=0.4046, pruned_loss=0.09869, over 4772141.23 frames. ], batch size: 204, lr: 8.66e-03, grad_scale: 32.0 2026-09-24 01:02:14,684 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=76280.0, ans=0.125 2026-09-24 01:02:20,502 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=76313.33333333333, ans=0.125 2026-09-24 01:02:27,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=76380.0, ans=0.2 2026-09-24 01:02:32,719 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=6.23 vs. limit=15.0 2026-09-24 01:02:33,952 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.514e+02 3.189e+02 3.492e+02 4.274e+02 6.534e+02, threshold=6.983e+02, percent-clipped=0.0 2026-09-24 01:02:35,523 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=76413.33333333333, ans=0.125 2026-09-24 01:02:37,265 INFO [train.py:1192] (1/2) Epoch 24, batch 900, loss[loss=0.2617, simple_loss=0.3614, pruned_loss=0.08101, over 24554.00 frames. ], tot_loss[loss=0.3018, simple_loss=0.4051, pruned_loss=0.09921, over 4782651.35 frames. ], batch size: 137, lr: 8.65e-03, grad_scale: 32.0 2026-09-24 01:02:40,291 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=76446.66666666667, ans=0.0 2026-09-24 01:02:52,510 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=76546.66666666667, ans=0.125 2026-09-24 01:02:56,543 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=76546.66666666667, ans=0.125 2026-09-24 01:02:56,683 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.93 vs. limit=12.0 2026-09-24 01:02:58,998 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=76580.0, ans=0.1 2026-09-24 01:03:02,344 INFO [train.py:1192] (1/2) Epoch 24, batch 950, loss[loss=0.3855, simple_loss=0.439, pruned_loss=0.166, over 10817.00 frames. ], tot_loss[loss=0.3019, simple_loss=0.4037, pruned_loss=0.1001, over 4720068.38 frames. ], batch size: 333, lr: 8.64e-03, grad_scale: 32.0 2026-09-24 01:03:14,039 INFO [train.py:1192] (1/2) Epoch 25, batch 0, loss[loss=0.2682, simple_loss=0.3773, pruned_loss=0.07951, over 24568.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3773, pruned_loss=0.07951, over 24568.00 frames. ], batch size: 137, lr: 8.46e-03, grad_scale: 32.0 2026-09-24 01:03:14,040 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 01:03:25,461 INFO [train.py:1224] (1/2) Epoch 25, validation: loss=0.1888, simple_loss=0.3056, pruned_loss=0.03594, over 2564189.00 frames. 2026-09-24 01:03:25,461 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 01:03:26,057 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=76640.0, ans=0.2 2026-09-24 01:03:43,896 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.648e+02 3.558e+02 3.928e+02 4.511e+02 6.708e+02, threshold=7.857e+02, percent-clipped=0.0 2026-09-24 01:03:43,996 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=76740.0, ans=0.025 2026-09-24 01:03:45,239 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=76740.0, ans=0.1 2026-09-24 01:03:49,546 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=76773.33333333333, ans=0.1 2026-09-24 01:03:50,899 INFO [train.py:1192] (1/2) Epoch 25, batch 50, loss[loss=0.2511, simple_loss=0.3515, pruned_loss=0.07538, over 24279.00 frames. ], tot_loss[loss=0.3079, simple_loss=0.4115, pruned_loss=0.1021, over 1081204.55 frames. ], batch size: 125, lr: 8.45e-03, grad_scale: 32.0 2026-09-24 01:03:56,747 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.67 vs. limit=15.0 2026-09-24 01:04:03,289 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=76873.33333333333, ans=0.0 2026-09-24 01:04:06,922 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=76906.66666666667, ans=0.5 2026-09-24 01:04:08,173 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=76906.66666666667, ans=0.125 2026-09-24 01:04:16,385 INFO [train.py:1192] (1/2) Epoch 25, batch 100, loss[loss=0.2679, simple_loss=0.3756, pruned_loss=0.08012, over 24630.00 frames. ], tot_loss[loss=0.3102, simple_loss=0.4147, pruned_loss=0.1029, over 1915300.92 frames. ], batch size: 154, lr: 8.45e-03, grad_scale: 32.0 2026-09-24 01:04:17,286 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=7.73 vs. limit=10.0 2026-09-24 01:04:32,123 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=77073.33333333333, ans=0.1 2026-09-24 01:04:34,990 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.604e+02 3.288e+02 3.807e+02 4.554e+02 6.437e+02, threshold=7.614e+02, percent-clipped=0.0 2026-09-24 01:04:41,943 INFO [train.py:1192] (1/2) Epoch 25, batch 150, loss[loss=0.2495, simple_loss=0.3469, pruned_loss=0.07599, over 24313.00 frames. ], tot_loss[loss=0.3037, simple_loss=0.4081, pruned_loss=0.0997, over 2560499.25 frames. ], batch size: 125, lr: 8.44e-03, grad_scale: 32.0 2026-09-24 01:04:42,704 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.29 vs. limit=15.0 2026-09-24 01:04:45,172 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=77140.0, ans=0.125 2026-09-24 01:04:45,631 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=77140.0, ans=0.125 2026-09-24 01:04:49,038 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=77173.33333333333, ans=0.125 2026-09-24 01:04:50,815 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=77173.33333333333, ans=0.0 2026-09-24 01:04:59,182 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=77240.0, ans=0.125 2026-09-24 01:05:06,812 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=77273.33333333333, ans=0.125 2026-09-24 01:05:07,704 INFO [train.py:1192] (1/2) Epoch 25, batch 200, loss[loss=0.3325, simple_loss=0.447, pruned_loss=0.109, over 24232.00 frames. ], tot_loss[loss=0.3005, simple_loss=0.4051, pruned_loss=0.09797, over 3060292.83 frames. ], batch size: 257, lr: 8.43e-03, grad_scale: 32.0 2026-09-24 01:05:16,244 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=77340.0, ans=0.2 2026-09-24 01:05:26,376 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.524e+02 3.418e+02 3.833e+02 4.710e+02 1.132e+03, threshold=7.665e+02, percent-clipped=3.0 2026-09-24 01:05:33,652 INFO [train.py:1192] (1/2) Epoch 25, batch 250, loss[loss=0.3281, simple_loss=0.4413, pruned_loss=0.1075, over 24374.00 frames. ], tot_loss[loss=0.3016, simple_loss=0.4055, pruned_loss=0.09879, over 3443945.83 frames. ], batch size: 225, lr: 8.42e-03, grad_scale: 32.0 2026-09-24 01:05:40,574 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.28 vs. limit=10.0 2026-09-24 01:05:42,582 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=77506.66666666667, ans=0.125 2026-09-24 01:05:50,450 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=77573.33333333333, ans=0.125 2026-09-24 01:05:52,909 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=77573.33333333333, ans=0.125 2026-09-24 01:05:55,581 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.74 vs. limit=22.5 2026-09-24 01:05:56,405 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=77606.66666666667, ans=0.1 2026-09-24 01:05:56,869 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=77606.66666666667, ans=0.04949747468305833 2026-09-24 01:05:56,880 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=77606.66666666667, ans=0.0 2026-09-24 01:05:57,597 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.51 vs. limit=15.0 2026-09-24 01:05:58,780 INFO [train.py:1192] (1/2) Epoch 25, batch 300, loss[loss=0.3077, simple_loss=0.4232, pruned_loss=0.09605, over 24540.00 frames. ], tot_loss[loss=0.3, simple_loss=0.4042, pruned_loss=0.09788, over 3757702.40 frames. ], batch size: 204, lr: 8.41e-03, grad_scale: 32.0 2026-09-24 01:05:58,863 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=77640.0, ans=0.0 2026-09-24 01:06:01,237 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=77640.0, ans=0.125 2026-09-24 01:06:06,950 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=77673.33333333333, ans=0.125 2026-09-24 01:06:09,980 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=77706.66666666667, ans=0.0 2026-09-24 01:06:12,034 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=77706.66666666667, ans=0.1 2026-09-24 01:06:14,919 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=77740.0, ans=0.125 2026-09-24 01:06:16,578 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=77740.0, ans=0.2 2026-09-24 01:06:17,485 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.405e+02 3.316e+02 3.736e+02 4.252e+02 6.211e+02, threshold=7.471e+02, percent-clipped=0.0 2026-09-24 01:06:21,774 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=77773.33333333333, ans=0.125 2026-09-24 01:06:22,337 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=77773.33333333333, ans=0.125 2026-09-24 01:06:24,330 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=77806.66666666667, ans=0.125 2026-09-24 01:06:24,744 INFO [train.py:1192] (1/2) Epoch 25, batch 350, loss[loss=0.2346, simple_loss=0.3438, pruned_loss=0.06276, over 24570.00 frames. ], tot_loss[loss=0.3006, simple_loss=0.4051, pruned_loss=0.09803, over 3994608.45 frames. ], batch size: 137, lr: 8.40e-03, grad_scale: 32.0 2026-09-24 01:06:32,414 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=77840.0, ans=0.125 2026-09-24 01:06:45,568 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=77940.0, ans=0.125 2026-09-24 01:06:49,602 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=77940.0, ans=0.025 2026-09-24 01:06:50,457 INFO [train.py:1192] (1/2) Epoch 25, batch 400, loss[loss=0.2962, simple_loss=0.4073, pruned_loss=0.09259, over 24569.00 frames. ], tot_loss[loss=0.2996, simple_loss=0.4042, pruned_loss=0.09755, over 4180355.89 frames. ], batch size: 170, lr: 8.40e-03, grad_scale: 32.0 2026-09-24 01:06:51,951 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=77973.33333333333, ans=0.05 2026-09-24 01:07:02,319 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.15 vs. limit=15.0 2026-09-24 01:07:06,522 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=78073.33333333333, ans=0.0 2026-09-24 01:07:09,270 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.561e+02 3.215e+02 3.447e+02 4.158e+02 5.856e+02, threshold=6.895e+02, percent-clipped=0.0 2026-09-24 01:07:16,275 INFO [train.py:1192] (1/2) Epoch 25, batch 450, loss[loss=0.2836, simple_loss=0.3945, pruned_loss=0.08632, over 24640.00 frames. ], tot_loss[loss=0.3004, simple_loss=0.4048, pruned_loss=0.09803, over 4320146.60 frames. ], batch size: 175, lr: 8.39e-03, grad_scale: 32.0 2026-09-24 01:07:18,279 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=78140.0, ans=0.125 2026-09-24 01:07:24,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=78173.33333333333, ans=0.125 2026-09-24 01:07:32,812 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:07:34,692 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:07:34,696 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=78240.0, ans=0.0 2026-09-24 01:07:36,072 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=78240.0, ans=0.0 2026-09-24 01:07:38,848 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=78273.33333333333, ans=0.125 2026-09-24 01:07:39,261 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=78273.33333333333, ans=0.125 2026-09-24 01:07:42,169 INFO [train.py:1192] (1/2) Epoch 25, batch 500, loss[loss=0.3059, simple_loss=0.4168, pruned_loss=0.09752, over 24510.00 frames. ], tot_loss[loss=0.2992, simple_loss=0.4033, pruned_loss=0.09755, over 4438241.25 frames. ], batch size: 218, lr: 8.38e-03, grad_scale: 32.0 2026-09-24 01:07:59,110 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=78406.66666666667, ans=0.1 2026-09-24 01:08:00,941 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.588e+02 3.246e+02 3.675e+02 4.099e+02 7.002e+02, threshold=7.349e+02, percent-clipped=1.0 2026-09-24 01:08:04,471 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=78440.0, ans=0.125 2026-09-24 01:08:06,137 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=78440.0, ans=0.04949747468305833 2026-09-24 01:08:08,370 INFO [train.py:1192] (1/2) Epoch 25, batch 550, loss[loss=0.3253, simple_loss=0.4385, pruned_loss=0.106, over 24268.00 frames. ], tot_loss[loss=0.2998, simple_loss=0.4041, pruned_loss=0.09777, over 4522623.54 frames. ], batch size: 257, lr: 8.37e-03, grad_scale: 32.0 2026-09-24 01:08:08,496 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=78473.33333333333, ans=0.04949747468305833 2026-09-24 01:08:09,940 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=78473.33333333333, ans=0.025 2026-09-24 01:08:12,014 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=78473.33333333333, ans=0.125 2026-09-24 01:08:15,666 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=78506.66666666667, ans=0.2 2026-09-24 01:08:17,505 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=78506.66666666667, ans=0.2 2026-09-24 01:08:23,162 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=78573.33333333333, ans=0.125 2026-09-24 01:08:31,194 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=78606.66666666667, ans=0.0 2026-09-24 01:08:34,153 INFO [train.py:1192] (1/2) Epoch 25, batch 600, loss[loss=0.3307, simple_loss=0.4391, pruned_loss=0.1112, over 24335.00 frames. ], tot_loss[loss=0.3007, simple_loss=0.4047, pruned_loss=0.09829, over 4588993.73 frames. ], batch size: 234, lr: 8.36e-03, grad_scale: 32.0 2026-09-24 01:08:41,845 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=78673.33333333333, ans=0.0 2026-09-24 01:08:51,869 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=78740.0, ans=10.0 2026-09-24 01:08:52,198 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.725e+02 3.335e+02 3.829e+02 4.379e+02 5.837e+02, threshold=7.659e+02, percent-clipped=0.0 2026-09-24 01:08:54,208 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=78773.33333333333, ans=0.2 2026-09-24 01:08:54,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=78773.33333333333, ans=0.0 2026-09-24 01:08:57,559 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=78773.33333333333, ans=0.125 2026-09-24 01:08:59,359 INFO [train.py:1192] (1/2) Epoch 25, batch 650, loss[loss=0.2653, simple_loss=0.3761, pruned_loss=0.07725, over 24584.00 frames. ], tot_loss[loss=0.2986, simple_loss=0.4031, pruned_loss=0.09703, over 4653909.14 frames. ], batch size: 154, lr: 8.36e-03, grad_scale: 32.0 2026-09-24 01:09:02,013 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=78806.66666666667, ans=0.125 2026-09-24 01:09:05,884 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=78840.0, ans=0.125 2026-09-24 01:09:16,973 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.82 vs. limit=15.0 2026-09-24 01:09:19,557 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=78940.0, ans=0.125 2026-09-24 01:09:24,577 INFO [train.py:1192] (1/2) Epoch 25, batch 700, loss[loss=0.2981, simple_loss=0.4015, pruned_loss=0.09739, over 24554.00 frames. ], tot_loss[loss=0.2994, simple_loss=0.4042, pruned_loss=0.09726, over 4689059.93 frames. ], batch size: 158, lr: 8.35e-03, grad_scale: 32.0 2026-09-24 01:09:43,174 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.541e+02 3.236e+02 3.745e+02 4.278e+02 6.318e+02, threshold=7.490e+02, percent-clipped=0.0 2026-09-24 01:09:50,974 INFO [train.py:1192] (1/2) Epoch 25, batch 750, loss[loss=0.3128, simple_loss=0.4192, pruned_loss=0.1032, over 24638.00 frames. ], tot_loss[loss=0.299, simple_loss=0.4035, pruned_loss=0.09726, over 4725713.03 frames. ], batch size: 175, lr: 8.34e-03, grad_scale: 32.0 2026-09-24 01:09:56,085 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=79173.33333333333, ans=0.05 2026-09-24 01:10:02,566 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=79206.66666666667, ans=0.125 2026-09-24 01:10:13,743 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=79273.33333333333, ans=0.0 2026-09-24 01:10:16,673 INFO [train.py:1192] (1/2) Epoch 25, batch 800, loss[loss=0.2937, simple_loss=0.3841, pruned_loss=0.1016, over 24526.00 frames. ], tot_loss[loss=0.2988, simple_loss=0.4031, pruned_loss=0.0972, over 4751916.30 frames. ], batch size: 137, lr: 8.33e-03, grad_scale: 32.0 2026-09-24 01:10:18,182 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=79306.66666666667, ans=0.0 2026-09-24 01:10:18,592 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=79306.66666666667, ans=0.0 2026-09-24 01:10:20,668 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.15 vs. limit=10.0 2026-09-24 01:10:35,666 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.734e+02 3.368e+02 3.828e+02 4.294e+02 6.886e+02, threshold=7.657e+02, percent-clipped=0.0 2026-09-24 01:10:42,446 INFO [train.py:1192] (1/2) Epoch 25, batch 850, loss[loss=0.3237, simple_loss=0.4355, pruned_loss=0.1059, over 24567.00 frames. ], tot_loss[loss=0.298, simple_loss=0.4025, pruned_loss=0.0968, over 4770594.04 frames. ], batch size: 204, lr: 8.32e-03, grad_scale: 32.0 2026-09-24 01:10:44,762 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=79473.33333333333, ans=0.0 2026-09-24 01:10:48,056 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=79506.66666666667, ans=0.1 2026-09-24 01:10:51,465 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=79506.66666666667, ans=0.1 2026-09-24 01:10:51,469 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=79506.66666666667, ans=0.125 2026-09-24 01:10:55,929 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:11:02,391 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=79606.66666666667, ans=0.0 2026-09-24 01:11:04,317 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=79606.66666666667, ans=0.125 2026-09-24 01:11:05,177 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=79606.66666666667, ans=0.125 2026-09-24 01:11:07,803 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=79640.0, ans=0.125 2026-09-24 01:11:08,248 INFO [train.py:1192] (1/2) Epoch 25, batch 900, loss[loss=0.2778, simple_loss=0.3781, pruned_loss=0.08876, over 24552.00 frames. ], tot_loss[loss=0.2984, simple_loss=0.4028, pruned_loss=0.09697, over 4782047.68 frames. ], batch size: 137, lr: 8.32e-03, grad_scale: 32.0 2026-09-24 01:11:14,310 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=79673.33333333333, ans=0.125 2026-09-24 01:11:16,355 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=79673.33333333333, ans=0.125 2026-09-24 01:11:16,684 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=4.92 vs. limit=15.0 2026-09-24 01:11:22,488 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=79740.0, ans=0.0 2026-09-24 01:11:25,607 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.551e+02 3.146e+02 3.608e+02 4.225e+02 6.295e+02, threshold=7.217e+02, percent-clipped=0.0 2026-09-24 01:11:28,039 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=79773.33333333333, ans=0.125 2026-09-24 01:11:29,904 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=79773.33333333333, ans=0.0 2026-09-24 01:11:31,708 INFO [train.py:1192] (1/2) Epoch 25, batch 950, loss[loss=0.4272, simple_loss=0.4649, pruned_loss=0.1947, over 10767.00 frames. ], tot_loss[loss=0.2995, simple_loss=0.402, pruned_loss=0.09854, over 4711876.79 frames. ], batch size: 333, lr: 8.31e-03, grad_scale: 32.0 2026-09-24 01:11:40,576 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:11:42,184 INFO [train.py:1192] (1/2) Epoch 26, batch 0, loss[loss=0.2823, simple_loss=0.3865, pruned_loss=0.089, over 24574.00 frames. ], tot_loss[loss=0.2823, simple_loss=0.3865, pruned_loss=0.089, over 24574.00 frames. ], batch size: 137, lr: 8.14e-03, grad_scale: 32.0 2026-09-24 01:11:42,184 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 01:11:53,554 INFO [train.py:1224] (1/2) Epoch 26, validation: loss=0.186, simple_loss=0.3043, pruned_loss=0.03381, over 2564189.00 frames. 2026-09-24 01:11:53,554 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 01:11:53,661 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=79833.33333333333, ans=0.125 2026-09-24 01:11:55,780 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=79833.33333333333, ans=0.1 2026-09-24 01:12:00,651 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=79866.66666666667, ans=0.0 2026-09-24 01:12:01,190 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:12:05,843 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=79900.0, ans=0.125 2026-09-24 01:12:17,159 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.72 vs. limit=22.5 2026-09-24 01:12:19,839 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=80000.0, ans=0.125 2026-09-24 01:12:20,453 INFO [train.py:1192] (1/2) Epoch 26, batch 50, loss[loss=0.2274, simple_loss=0.3343, pruned_loss=0.06024, over 24276.00 frames. ], tot_loss[loss=0.3073, simple_loss=0.4108, pruned_loss=0.1019, over 1082235.17 frames. ], batch size: 125, lr: 8.13e-03, grad_scale: 32.0 2026-09-24 01:12:23,660 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.06 vs. limit=15.0 2026-09-24 01:12:27,953 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=80033.33333333333, ans=0.0 2026-09-24 01:12:31,269 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=80066.66666666667, ans=0.0 2026-09-24 01:12:35,342 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.648e+02 3.559e+02 3.997e+02 5.046e+02 7.744e+02, threshold=7.995e+02, percent-clipped=3.0 2026-09-24 01:12:39,407 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=80100.0, ans=0.125 2026-09-24 01:12:41,712 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=80133.33333333333, ans=0.0 2026-09-24 01:12:45,731 INFO [train.py:1192] (1/2) Epoch 26, batch 100, loss[loss=0.2983, simple_loss=0.3942, pruned_loss=0.1012, over 24610.00 frames. ], tot_loss[loss=0.307, simple_loss=0.4123, pruned_loss=0.1008, over 1917102.98 frames. ], batch size: 154, lr: 8.13e-03, grad_scale: 32.0 2026-09-24 01:12:53,889 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.94 vs. limit=15.0 2026-09-24 01:12:55,069 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=80200.0, ans=0.0 2026-09-24 01:13:09,847 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=80300.0, ans=0.2 2026-09-24 01:13:10,391 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.04 vs. limit=22.5 2026-09-24 01:13:11,258 INFO [train.py:1192] (1/2) Epoch 26, batch 150, loss[loss=0.2718, simple_loss=0.3607, pruned_loss=0.09143, over 24264.00 frames. ], tot_loss[loss=0.3012, simple_loss=0.4063, pruned_loss=0.09807, over 2561275.31 frames. ], batch size: 125, lr: 8.12e-03, grad_scale: 32.0 2026-09-24 01:13:12,691 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=80333.33333333333, ans=0.125 2026-09-24 01:13:12,694 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=80333.33333333333, ans=0.035 2026-09-24 01:13:15,427 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=80333.33333333333, ans=0.0 2026-09-24 01:13:21,214 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.22 vs. limit=6.0 2026-09-24 01:13:24,844 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=80400.0, ans=0.0 2026-09-24 01:13:25,726 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.558e+02 3.240e+02 3.503e+02 3.960e+02 5.414e+02, threshold=7.006e+02, percent-clipped=0.0 2026-09-24 01:13:34,305 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=80466.66666666667, ans=0.0 2026-09-24 01:13:36,621 INFO [train.py:1192] (1/2) Epoch 26, batch 200, loss[loss=0.3428, simple_loss=0.4521, pruned_loss=0.1167, over 24240.00 frames. ], tot_loss[loss=0.299, simple_loss=0.4041, pruned_loss=0.09692, over 3059732.88 frames. ], batch size: 257, lr: 8.11e-03, grad_scale: 32.0 2026-09-24 01:13:38,037 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=80500.0, ans=0.0 2026-09-24 01:13:54,670 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=80600.0, ans=0.125 2026-09-24 01:14:01,691 INFO [train.py:1192] (1/2) Epoch 26, batch 250, loss[loss=0.3032, simple_loss=0.4225, pruned_loss=0.09193, over 24393.00 frames. ], tot_loss[loss=0.2996, simple_loss=0.4044, pruned_loss=0.09736, over 3443911.68 frames. ], batch size: 225, lr: 8.10e-03, grad_scale: 32.0 2026-09-24 01:14:03,485 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=80666.66666666667, ans=0.125 2026-09-24 01:14:05,853 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten.whitening_limit, batch_count=80666.66666666667, ans=15.0 2026-09-24 01:14:16,436 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.260e+02 3.537e+02 4.089e+02 5.149e+02 1.030e+03, threshold=8.177e+02, percent-clipped=7.0 2026-09-24 01:14:26,835 INFO [train.py:1192] (1/2) Epoch 26, batch 300, loss[loss=0.3166, simple_loss=0.43, pruned_loss=0.1016, over 24538.00 frames. ], tot_loss[loss=0.2974, simple_loss=0.4024, pruned_loss=0.09622, over 3758022.93 frames. ], batch size: 204, lr: 8.10e-03, grad_scale: 32.0 2026-09-24 01:14:34,219 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.02 vs. limit=15.0 2026-09-24 01:14:35,963 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=80866.66666666667, ans=0.125 2026-09-24 01:14:52,679 INFO [train.py:1192] (1/2) Epoch 26, batch 350, loss[loss=0.2674, simple_loss=0.3668, pruned_loss=0.08401, over 24576.00 frames. ], tot_loss[loss=0.2973, simple_loss=0.4024, pruned_loss=0.09608, over 3998207.41 frames. ], batch size: 137, lr: 8.09e-03, grad_scale: 32.0 2026-09-24 01:14:55,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=81000.0, ans=0.1 2026-09-24 01:14:59,155 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=81033.33333333333, ans=0.1 2026-09-24 01:15:00,618 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=81033.33333333333, ans=0.0 2026-09-24 01:15:07,604 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.513e+02 3.230e+02 3.645e+02 3.996e+02 6.167e+02, threshold=7.290e+02, percent-clipped=0.0 2026-09-24 01:15:07,705 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=81100.0, ans=0.0 2026-09-24 01:15:14,184 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=81133.33333333333, ans=0.0 2026-09-24 01:15:16,877 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=81133.33333333333, ans=0.0 2026-09-24 01:15:18,629 INFO [train.py:1192] (1/2) Epoch 26, batch 400, loss[loss=0.2884, simple_loss=0.3963, pruned_loss=0.09024, over 24576.00 frames. ], tot_loss[loss=0.2976, simple_loss=0.4025, pruned_loss=0.09638, over 4182090.70 frames. ], batch size: 170, lr: 8.08e-03, grad_scale: 32.0 2026-09-24 01:15:29,759 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=81233.33333333333, ans=0.0 2026-09-24 01:15:31,346 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=81233.33333333333, ans=0.2 2026-09-24 01:15:32,826 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=81233.33333333333, ans=0.0 2026-09-24 01:15:34,459 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=81266.66666666667, ans=0.2 2026-09-24 01:15:44,951 INFO [train.py:1192] (1/2) Epoch 26, batch 450, loss[loss=0.3003, simple_loss=0.4068, pruned_loss=0.09687, over 24639.00 frames. ], tot_loss[loss=0.2986, simple_loss=0.4032, pruned_loss=0.097, over 4321519.57 frames. ], batch size: 175, lr: 8.07e-03, grad_scale: 32.0 2026-09-24 01:15:59,654 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.632e+02 3.397e+02 3.913e+02 4.554e+02 6.749e+02, threshold=7.825e+02, percent-clipped=0.0 2026-09-24 01:16:00,301 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=81433.33333333333, ans=0.0 2026-09-24 01:16:10,897 INFO [train.py:1192] (1/2) Epoch 26, batch 500, loss[loss=0.3443, simple_loss=0.4487, pruned_loss=0.1199, over 24528.00 frames. ], tot_loss[loss=0.2977, simple_loss=0.4021, pruned_loss=0.09665, over 4438588.86 frames. ], batch size: 218, lr: 8.07e-03, grad_scale: 32.0 2026-09-24 01:16:16,155 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=81533.33333333333, ans=0.125 2026-09-24 01:16:27,321 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=81600.0, ans=0.2 2026-09-24 01:16:32,326 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.07 vs. limit=15.0 2026-09-24 01:16:32,635 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=81633.33333333333, ans=0.125 2026-09-24 01:16:36,117 INFO [train.py:1192] (1/2) Epoch 26, batch 550, loss[loss=0.3492, simple_loss=0.4612, pruned_loss=0.1186, over 24251.00 frames. ], tot_loss[loss=0.2979, simple_loss=0.4026, pruned_loss=0.09662, over 4523464.76 frames. ], batch size: 257, lr: 8.06e-03, grad_scale: 32.0 2026-09-24 01:16:37,425 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=3.78 vs. limit=12.0 2026-09-24 01:16:45,649 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=81700.0, ans=0.125 2026-09-24 01:16:51,113 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.714e+02 3.189e+02 3.611e+02 4.230e+02 5.792e+02, threshold=7.223e+02, percent-clipped=0.0 2026-09-24 01:16:52,646 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=81766.66666666667, ans=0.0 2026-09-24 01:17:01,246 INFO [train.py:1192] (1/2) Epoch 26, batch 600, loss[loss=0.3321, simple_loss=0.4464, pruned_loss=0.1089, over 24340.00 frames. ], tot_loss[loss=0.2982, simple_loss=0.4032, pruned_loss=0.09659, over 4589554.76 frames. ], batch size: 234, lr: 8.05e-03, grad_scale: 32.0 2026-09-24 01:17:06,753 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=81866.66666666667, ans=0.1 2026-09-24 01:17:17,297 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.55 vs. limit=22.5 2026-09-24 01:17:26,423 INFO [train.py:1192] (1/2) Epoch 26, batch 650, loss[loss=0.2826, simple_loss=0.3838, pruned_loss=0.09072, over 24607.00 frames. ], tot_loss[loss=0.2968, simple_loss=0.402, pruned_loss=0.09575, over 4654183.01 frames. ], batch size: 154, lr: 8.04e-03, grad_scale: 32.0 2026-09-24 01:17:36,972 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.71 vs. limit=15.0 2026-09-24 01:17:37,814 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=82066.66666666667, ans=0.0 2026-09-24 01:17:41,515 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.485e+02 3.234e+02 3.746e+02 4.393e+02 6.257e+02, threshold=7.492e+02, percent-clipped=0.0 2026-09-24 01:17:47,867 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.27 vs. limit=15.0 2026-09-24 01:17:52,308 INFO [train.py:1192] (1/2) Epoch 26, batch 700, loss[loss=0.2608, simple_loss=0.3733, pruned_loss=0.07414, over 24556.00 frames. ], tot_loss[loss=0.2973, simple_loss=0.4029, pruned_loss=0.09588, over 4689555.01 frames. ], batch size: 158, lr: 8.04e-03, grad_scale: 32.0 2026-09-24 01:18:00,409 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.72 vs. limit=12.0 2026-09-24 01:18:00,882 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=14.35 vs. limit=15.0 2026-09-24 01:18:07,822 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=82266.66666666667, ans=0.2 2026-09-24 01:18:17,428 INFO [train.py:1192] (1/2) Epoch 26, batch 750, loss[loss=0.3158, simple_loss=0.4196, pruned_loss=0.1059, over 24633.00 frames. ], tot_loss[loss=0.2966, simple_loss=0.4021, pruned_loss=0.09558, over 4725932.35 frames. ], batch size: 175, lr: 8.03e-03, grad_scale: 32.0 2026-09-24 01:18:20,557 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=82333.33333333333, ans=0.125 2026-09-24 01:18:22,574 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=82366.66666666667, ans=0.0 2026-09-24 01:18:22,671 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.49 vs. limit=15.0 2026-09-24 01:18:31,437 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=82400.0, ans=0.0 2026-09-24 01:18:31,985 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.78 vs. limit=15.0 2026-09-24 01:18:32,209 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.741e+02 3.352e+02 3.754e+02 4.234e+02 6.090e+02, threshold=7.507e+02, percent-clipped=0.0 2026-09-24 01:18:35,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=82433.33333333333, ans=0.0 2026-09-24 01:18:39,598 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=82466.66666666667, ans=0.125 2026-09-24 01:18:42,790 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=82500.0, ans=0.125 2026-09-24 01:18:43,222 INFO [train.py:1192] (1/2) Epoch 26, batch 800, loss[loss=0.2613, simple_loss=0.3647, pruned_loss=0.07895, over 24535.00 frames. ], tot_loss[loss=0.2961, simple_loss=0.4016, pruned_loss=0.09532, over 4752593.95 frames. ], batch size: 137, lr: 8.02e-03, grad_scale: 32.0 2026-09-24 01:18:58,224 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=82600.0, ans=0.2 2026-09-24 01:19:00,122 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=82600.0, ans=0.1 2026-09-24 01:19:05,918 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=82633.33333333333, ans=0.125 2026-09-24 01:19:09,073 INFO [train.py:1192] (1/2) Epoch 26, batch 850, loss[loss=0.3282, simple_loss=0.4323, pruned_loss=0.112, over 24550.00 frames. ], tot_loss[loss=0.2955, simple_loss=0.4009, pruned_loss=0.09507, over 4770348.99 frames. ], batch size: 204, lr: 8.01e-03, grad_scale: 32.0 2026-09-24 01:19:18,265 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=82733.33333333333, ans=0.0 2026-09-24 01:19:23,299 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.687e+02 3.286e+02 3.795e+02 4.177e+02 6.657e+02, threshold=7.591e+02, percent-clipped=0.0 2026-09-24 01:19:31,278 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.42 vs. limit=15.0 2026-09-24 01:19:34,167 INFO [train.py:1192] (1/2) Epoch 26, batch 900, loss[loss=0.2561, simple_loss=0.3597, pruned_loss=0.07625, over 24546.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.4013, pruned_loss=0.0956, over 4781711.99 frames. ], batch size: 137, lr: 8.01e-03, grad_scale: 32.0 2026-09-24 01:19:39,472 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=82866.66666666667, ans=0.1 2026-09-24 01:19:59,488 INFO [train.py:1192] (1/2) Epoch 26, batch 950, loss[loss=0.4447, simple_loss=0.4726, pruned_loss=0.2084, over 11299.00 frames. ], tot_loss[loss=0.2971, simple_loss=0.4003, pruned_loss=0.09694, over 4714682.36 frames. ], batch size: 334, lr: 8.00e-03, grad_scale: 32.0 2026-09-24 01:20:09,649 INFO [train.py:1192] (1/2) Epoch 27, batch 0, loss[loss=0.2741, simple_loss=0.3789, pruned_loss=0.08464, over 24575.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.3789, pruned_loss=0.08464, over 24575.00 frames. ], batch size: 137, lr: 7.84e-03, grad_scale: 32.0 2026-09-24 01:20:09,649 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 01:20:12,761 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.2416, 1.6191, 1.1071, 1.3296, 1.6923, 1.6062, 1.6496, 1.4643], device='cuda:1') 2026-09-24 01:20:20,475 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.2971, 1.9685, 2.9768, 2.1230], device='cuda:1') 2026-09-24 01:20:21,080 INFO [train.py:1224] (1/2) Epoch 27, validation: loss=0.1849, simple_loss=0.3023, pruned_loss=0.03371, over 2564189.00 frames. 2026-09-24 01:20:21,080 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 01:20:22,642 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=83026.66666666667, ans=0.025 2026-09-24 01:20:31,627 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.571e+02 3.432e+02 4.011e+02 4.668e+02 7.874e+02, threshold=8.023e+02, percent-clipped=1.0 2026-09-24 01:20:39,152 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=83126.66666666667, ans=0.125 2026-09-24 01:20:45,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=83160.0, ans=0.125 2026-09-24 01:20:46,495 INFO [train.py:1192] (1/2) Epoch 27, batch 50, loss[loss=0.2297, simple_loss=0.3332, pruned_loss=0.06307, over 24259.00 frames. ], tot_loss[loss=0.3031, simple_loss=0.4082, pruned_loss=0.09906, over 1081503.76 frames. ], batch size: 125, lr: 7.84e-03, grad_scale: 32.0 2026-09-24 01:20:52,864 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=83226.66666666667, ans=0.0 2026-09-24 01:20:56,375 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=83260.0, ans=0.0 2026-09-24 01:21:10,972 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=83326.66666666667, ans=0.0 2026-09-24 01:21:12,260 INFO [train.py:1192] (1/2) Epoch 27, batch 100, loss[loss=0.3026, simple_loss=0.4004, pruned_loss=0.1024, over 24635.00 frames. ], tot_loss[loss=0.306, simple_loss=0.4123, pruned_loss=0.0998, over 1915680.88 frames. ], batch size: 154, lr: 7.83e-03, grad_scale: 32.0 2026-09-24 01:21:22,899 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.627e+02 3.379e+02 3.839e+02 4.300e+02 6.857e+02, threshold=7.678e+02, percent-clipped=0.0 2026-09-24 01:21:30,454 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=83460.0, ans=0.0 2026-09-24 01:21:37,903 INFO [train.py:1192] (1/2) Epoch 27, batch 150, loss[loss=0.2467, simple_loss=0.3498, pruned_loss=0.07187, over 24257.00 frames. ], tot_loss[loss=0.2996, simple_loss=0.4055, pruned_loss=0.09684, over 2559876.66 frames. ], batch size: 125, lr: 7.82e-03, grad_scale: 32.0 2026-09-24 01:21:44,829 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.27 vs. limit=15.0 2026-09-24 01:21:45,970 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=83560.0, ans=0.1 2026-09-24 01:21:51,178 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=83593.33333333333, ans=0.0 2026-09-24 01:21:54,008 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=83626.66666666667, ans=0.0 2026-09-24 01:21:58,180 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=83660.0, ans=0.07 2026-09-24 01:22:02,087 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.10 vs. limit=15.0 2026-09-24 01:22:03,903 INFO [train.py:1192] (1/2) Epoch 27, batch 200, loss[loss=0.3375, simple_loss=0.4535, pruned_loss=0.1108, over 24211.00 frames. ], tot_loss[loss=0.2979, simple_loss=0.4037, pruned_loss=0.09605, over 3056938.90 frames. ], batch size: 257, lr: 7.82e-03, grad_scale: 32.0 2026-09-24 01:22:14,488 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.221e+02 3.661e+02 4.138e+02 9.713e+02, threshold=7.323e+02, percent-clipped=3.0 2026-09-24 01:22:19,932 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=83793.33333333333, ans=0.1 2026-09-24 01:22:23,761 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=83826.66666666667, ans=0.5 2026-09-24 01:22:26,266 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=83826.66666666667, ans=0.025 2026-09-24 01:22:28,416 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.79 vs. limit=15.0 2026-09-24 01:22:29,061 INFO [train.py:1192] (1/2) Epoch 27, batch 250, loss[loss=0.3357, simple_loss=0.4449, pruned_loss=0.1133, over 24374.00 frames. ], tot_loss[loss=0.2979, simple_loss=0.4032, pruned_loss=0.09635, over 3441005.09 frames. ], batch size: 225, lr: 7.81e-03, grad_scale: 32.0 2026-09-24 01:22:31,652 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=83860.0, ans=0.0 2026-09-24 01:22:42,780 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=4.88 vs. limit=15.0 2026-09-24 01:22:43,498 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=83960.0, ans=0.2 2026-09-24 01:22:49,164 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.79 vs. limit=6.0 2026-09-24 01:22:54,052 INFO [train.py:1192] (1/2) Epoch 27, batch 300, loss[loss=0.316, simple_loss=0.4277, pruned_loss=0.1022, over 24540.00 frames. ], tot_loss[loss=0.2965, simple_loss=0.4017, pruned_loss=0.0956, over 3755433.46 frames. ], batch size: 204, lr: 7.80e-03, grad_scale: 32.0 2026-09-24 01:23:02,181 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.50 vs. limit=15.0 2026-09-24 01:23:04,241 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.635e+02 3.238e+02 3.753e+02 4.253e+02 6.492e+02, threshold=7.505e+02, percent-clipped=0.0 2026-09-24 01:23:08,888 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=84126.66666666667, ans=0.025 2026-09-24 01:23:18,844 INFO [train.py:1192] (1/2) Epoch 27, batch 350, loss[loss=0.2796, simple_loss=0.3748, pruned_loss=0.09222, over 24517.00 frames. ], tot_loss[loss=0.2968, simple_loss=0.4024, pruned_loss=0.09562, over 3992697.81 frames. ], batch size: 137, lr: 7.79e-03, grad_scale: 32.0 2026-09-24 01:23:19,350 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=84193.33333333333, ans=0.125 2026-09-24 01:23:23,809 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=84226.66666666667, ans=0.125 2026-09-24 01:23:23,819 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=84226.66666666667, ans=0.0 2026-09-24 01:23:32,888 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=84260.0, ans=0.1 2026-09-24 01:23:44,383 INFO [train.py:1192] (1/2) Epoch 27, batch 400, loss[loss=0.3025, simple_loss=0.4032, pruned_loss=0.1009, over 24571.00 frames. ], tot_loss[loss=0.2948, simple_loss=0.4007, pruned_loss=0.09447, over 4178775.53 frames. ], batch size: 170, lr: 7.79e-03, grad_scale: 32.0 2026-09-24 01:23:51,882 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=84393.33333333333, ans=0.1 2026-09-24 01:23:54,242 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:23:55,152 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.594e+02 3.209e+02 3.534e+02 4.019e+02 5.305e+02, threshold=7.068e+02, percent-clipped=0.0 2026-09-24 01:24:01,038 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=84460.0, ans=0.2 2026-09-24 01:24:10,083 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=84526.66666666667, ans=0.0 2026-09-24 01:24:10,373 INFO [train.py:1192] (1/2) Epoch 27, batch 450, loss[loss=0.2946, simple_loss=0.4093, pruned_loss=0.08989, over 24623.00 frames. ], tot_loss[loss=0.2957, simple_loss=0.4015, pruned_loss=0.09499, over 4317840.32 frames. ], batch size: 175, lr: 7.78e-03, grad_scale: 32.0 2026-09-24 01:24:11,037 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=84526.66666666667, ans=0.07 2026-09-24 01:24:15,677 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:24:17,876 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.47 vs. limit=6.0 2026-09-24 01:24:23,412 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=6.65 vs. limit=10.0 2026-09-24 01:24:25,756 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=84626.66666666667, ans=15.0 2026-09-24 01:24:33,663 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=84660.0, ans=0.0 2026-09-24 01:24:35,855 INFO [train.py:1192] (1/2) Epoch 27, batch 500, loss[loss=0.3498, simple_loss=0.4437, pruned_loss=0.128, over 24484.00 frames. ], tot_loss[loss=0.2945, simple_loss=0.4001, pruned_loss=0.09441, over 4435824.97 frames. ], batch size: 218, lr: 7.77e-03, grad_scale: 32.0 2026-09-24 01:24:37,392 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=84693.33333333333, ans=0.2 2026-09-24 01:24:42,300 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.45 vs. limit=6.0 2026-09-24 01:24:46,630 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.511e+02 3.213e+02 3.588e+02 4.289e+02 6.663e+02, threshold=7.177e+02, percent-clipped=0.0 2026-09-24 01:24:47,201 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=84760.0, ans=0.1 2026-09-24 01:24:47,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=84760.0, ans=0.07 2026-09-24 01:24:52,178 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=84793.33333333333, ans=0.0 2026-09-24 01:25:01,612 INFO [train.py:1192] (1/2) Epoch 27, batch 550, loss[loss=0.319, simple_loss=0.4373, pruned_loss=0.1004, over 24271.00 frames. ], tot_loss[loss=0.2949, simple_loss=0.4006, pruned_loss=0.09456, over 4521665.98 frames. ], batch size: 257, lr: 7.77e-03, grad_scale: 32.0 2026-09-24 01:25:15,676 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=84926.66666666667, ans=0.1 2026-09-24 01:25:22,024 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=84993.33333333333, ans=0.0 2026-09-24 01:25:23,703 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=84993.33333333333, ans=0.0 2026-09-24 01:25:27,066 INFO [train.py:1192] (1/2) Epoch 27, batch 600, loss[loss=0.3296, simple_loss=0.4402, pruned_loss=0.1095, over 24432.00 frames. ], tot_loss[loss=0.2962, simple_loss=0.4019, pruned_loss=0.09524, over 4588972.56 frames. ], batch size: 235, lr: 7.76e-03, grad_scale: 32.0 2026-09-24 01:25:36,891 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=85060.0, ans=0.2 2026-09-24 01:25:38,198 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.522e+02 3.192e+02 3.627e+02 4.252e+02 5.650e+02, threshold=7.253e+02, percent-clipped=0.0 2026-09-24 01:25:41,936 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=85093.33333333333, ans=0.0 2026-09-24 01:25:53,051 INFO [train.py:1192] (1/2) Epoch 27, batch 650, loss[loss=0.2729, simple_loss=0.3833, pruned_loss=0.08125, over 24588.00 frames. ], tot_loss[loss=0.2942, simple_loss=0.4002, pruned_loss=0.09415, over 4653824.36 frames. ], batch size: 154, lr: 7.75e-03, grad_scale: 32.0 2026-09-24 01:26:02,348 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.70 vs. limit=15.0 2026-09-24 01:26:03,113 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=85260.0, ans=10.0 2026-09-24 01:26:03,576 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=85260.0, ans=0.125 2026-09-24 01:26:06,930 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=85260.0, ans=0.1 2026-09-24 01:26:16,485 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.52 vs. limit=12.0 2026-09-24 01:26:18,801 INFO [train.py:1192] (1/2) Epoch 27, batch 700, loss[loss=0.2799, simple_loss=0.389, pruned_loss=0.08541, over 24555.00 frames. ], tot_loss[loss=0.2955, simple_loss=0.4016, pruned_loss=0.09472, over 4689675.70 frames. ], batch size: 158, lr: 7.75e-03, grad_scale: 32.0 2026-09-24 01:26:20,589 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=85360.0, ans=0.125 2026-09-24 01:26:24,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=85393.33333333333, ans=0.09899494936611666 2026-09-24 01:26:25,642 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=85393.33333333333, ans=0.125 2026-09-24 01:26:29,687 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.715e+02 3.445e+02 3.970e+02 4.680e+02 6.665e+02, threshold=7.940e+02, percent-clipped=0.0 2026-09-24 01:26:36,507 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=85460.0, ans=0.125 2026-09-24 01:26:41,088 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.32 vs. limit=15.0 2026-09-24 01:26:44,380 INFO [train.py:1192] (1/2) Epoch 27, batch 750, loss[loss=0.2977, simple_loss=0.4132, pruned_loss=0.09104, over 24620.00 frames. ], tot_loss[loss=0.2943, simple_loss=0.4004, pruned_loss=0.09406, over 4726130.22 frames. ], batch size: 175, lr: 7.74e-03, grad_scale: 32.0 2026-09-24 01:27:09,815 INFO [train.py:1192] (1/2) Epoch 27, batch 800, loss[loss=0.2445, simple_loss=0.3516, pruned_loss=0.06876, over 24558.00 frames. ], tot_loss[loss=0.2935, simple_loss=0.3997, pruned_loss=0.09364, over 4752075.26 frames. ], batch size: 137, lr: 7.73e-03, grad_scale: 32.0 2026-09-24 01:27:11,339 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=85693.33333333333, ans=0.125 2026-09-24 01:27:16,378 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=85726.66666666667, ans=0.2 2026-09-24 01:27:20,251 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.589e+02 3.155e+02 3.540e+02 3.989e+02 5.498e+02, threshold=7.080e+02, percent-clipped=0.0 2026-09-24 01:27:22,137 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=85760.0, ans=0.125 2026-09-24 01:27:22,699 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=85760.0, ans=0.025 2026-09-24 01:27:35,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=85860.0, ans=0.1 2026-09-24 01:27:35,541 INFO [train.py:1192] (1/2) Epoch 27, batch 850, loss[loss=0.3028, simple_loss=0.4249, pruned_loss=0.09035, over 24565.00 frames. ], tot_loss[loss=0.294, simple_loss=0.4, pruned_loss=0.09405, over 4770487.81 frames. ], batch size: 204, lr: 7.72e-03, grad_scale: 32.0 2026-09-24 01:27:41,665 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=85893.33333333333, ans=0.1 2026-09-24 01:27:44,205 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=10.78 vs. limit=22.5 2026-09-24 01:27:49,095 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.88 vs. limit=15.0 2026-09-24 01:27:53,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=85960.0, ans=0.125 2026-09-24 01:27:55,049 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=85993.33333333333, ans=0.125 2026-09-24 01:27:56,479 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=85993.33333333333, ans=0.125 2026-09-24 01:28:01,253 INFO [train.py:1192] (1/2) Epoch 27, batch 900, loss[loss=0.2659, simple_loss=0.3738, pruned_loss=0.07898, over 24571.00 frames. ], tot_loss[loss=0.295, simple_loss=0.4007, pruned_loss=0.09461, over 4781364.70 frames. ], batch size: 137, lr: 7.72e-03, grad_scale: 64.0 2026-09-24 01:28:11,879 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.467e+02 3.251e+02 3.662e+02 4.258e+02 5.909e+02, threshold=7.324e+02, percent-clipped=0.0 2026-09-24 01:28:15,303 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.32 vs. limit=22.5 2026-09-24 01:28:21,090 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=86160.0, ans=0.025 2026-09-24 01:28:25,958 INFO [train.py:1192] (1/2) Epoch 27, batch 950, loss[loss=0.3922, simple_loss=0.4434, pruned_loss=0.1705, over 11565.00 frames. ], tot_loss[loss=0.2955, simple_loss=0.3997, pruned_loss=0.09562, over 4717359.70 frames. ], batch size: 333, lr: 7.71e-03, grad_scale: 32.0 2026-09-24 01:28:37,269 INFO [train.py:1192] (1/2) Epoch 28, batch 0, loss[loss=0.2283, simple_loss=0.338, pruned_loss=0.05926, over 24576.00 frames. ], tot_loss[loss=0.2283, simple_loss=0.338, pruned_loss=0.05926, over 24576.00 frames. ], batch size: 137, lr: 7.57e-03, grad_scale: 32.0 2026-09-24 01:28:37,270 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 01:28:49,019 INFO [train.py:1224] (1/2) Epoch 28, validation: loss=0.1841, simple_loss=0.3017, pruned_loss=0.03321, over 2564189.00 frames. 2026-09-24 01:28:49,019 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 01:28:59,806 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=86286.66666666667, ans=0.0 2026-09-24 01:29:04,078 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=86320.0, ans=0.2 2026-09-24 01:29:04,086 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=86320.0, ans=0.125 2026-09-24 01:29:14,149 INFO [train.py:1192] (1/2) Epoch 28, batch 50, loss[loss=0.254, simple_loss=0.3534, pruned_loss=0.07727, over 24213.00 frames. ], tot_loss[loss=0.2987, simple_loss=0.4042, pruned_loss=0.09662, over 1081284.28 frames. ], batch size: 125, lr: 7.56e-03, grad_scale: 32.0 2026-09-24 01:29:21,927 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.593e+02 3.506e+02 4.112e+02 4.821e+02 1.243e+03, threshold=8.224e+02, percent-clipped=5.0 2026-09-24 01:29:22,514 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=7.887e-02 2026-09-24 01:29:24,988 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=86453.33333333333, ans=0.1 2026-09-24 01:29:30,734 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.62 vs. limit=15.0 2026-09-24 01:29:33,401 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=86486.66666666667, ans=0.0 2026-09-24 01:29:39,781 INFO [train.py:1192] (1/2) Epoch 28, batch 100, loss[loss=0.2887, simple_loss=0.3905, pruned_loss=0.09342, over 24603.00 frames. ], tot_loss[loss=0.3028, simple_loss=0.4092, pruned_loss=0.09823, over 1916701.25 frames. ], batch size: 154, lr: 7.55e-03, grad_scale: 32.0 2026-09-24 01:29:53,456 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.03 vs. limit=22.5 2026-09-24 01:30:05,322 INFO [train.py:1192] (1/2) Epoch 28, batch 150, loss[loss=0.2296, simple_loss=0.3346, pruned_loss=0.06224, over 24280.00 frames. ], tot_loss[loss=0.2975, simple_loss=0.4036, pruned_loss=0.09567, over 2561666.91 frames. ], batch size: 125, lr: 7.55e-03, grad_scale: 32.0 2026-09-24 01:30:12,823 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=86753.33333333333, ans=0.0 2026-09-24 01:30:13,162 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.392e+02 3.154e+02 3.543e+02 4.109e+02 5.817e+02, threshold=7.086e+02, percent-clipped=0.0 2026-09-24 01:30:17,907 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=86786.66666666667, ans=0.125 2026-09-24 01:30:17,915 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=86786.66666666667, ans=0.125 2026-09-24 01:30:26,321 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=86853.33333333333, ans=0.125 2026-09-24 01:30:29,645 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=86853.33333333333, ans=0.04949747468305833 2026-09-24 01:30:31,512 INFO [train.py:1192] (1/2) Epoch 28, batch 200, loss[loss=0.3477, simple_loss=0.4574, pruned_loss=0.119, over 24216.00 frames. ], tot_loss[loss=0.2963, simple_loss=0.4022, pruned_loss=0.09521, over 3060263.70 frames. ], batch size: 257, lr: 7.54e-03, grad_scale: 32.0 2026-09-24 01:30:42,390 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=86953.33333333333, ans=0.04949747468305833 2026-09-24 01:30:53,632 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.73 vs. limit=15.0 2026-09-24 01:30:57,021 INFO [train.py:1192] (1/2) Epoch 28, batch 250, loss[loss=0.3578, simple_loss=0.457, pruned_loss=0.1293, over 24397.00 frames. ], tot_loss[loss=0.2959, simple_loss=0.4017, pruned_loss=0.09506, over 3443738.08 frames. ], batch size: 225, lr: 7.53e-03, grad_scale: 32.0 2026-09-24 01:30:57,629 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:31:02,922 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.27 vs. limit=15.0 2026-09-24 01:31:03,793 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=87086.66666666667, ans=0.125 2026-09-24 01:31:04,851 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.536e+02 3.459e+02 4.032e+02 4.652e+02 9.855e+02, threshold=8.064e+02, percent-clipped=4.0 2026-09-24 01:31:05,909 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=87086.66666666667, ans=0.025 2026-09-24 01:31:07,923 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.56 vs. limit=15.0 2026-09-24 01:31:10,412 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=11.35 vs. limit=15.0 2026-09-24 01:31:11,347 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=87120.0, ans=0.2 2026-09-24 01:31:22,328 INFO [train.py:1192] (1/2) Epoch 28, batch 300, loss[loss=0.3465, simple_loss=0.4476, pruned_loss=0.1228, over 24513.00 frames. ], tot_loss[loss=0.2954, simple_loss=0.4011, pruned_loss=0.09487, over 3756666.61 frames. ], batch size: 204, lr: 7.53e-03, grad_scale: 32.0 2026-09-24 01:31:26,907 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=87253.33333333333, ans=0.2 2026-09-24 01:31:41,808 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=87320.0, ans=0.125 2026-09-24 01:31:48,339 INFO [train.py:1192] (1/2) Epoch 28, batch 350, loss[loss=0.2544, simple_loss=0.3558, pruned_loss=0.07655, over 24579.00 frames. ], tot_loss[loss=0.296, simple_loss=0.4017, pruned_loss=0.09516, over 3998381.44 frames. ], batch size: 137, lr: 7.52e-03, grad_scale: 32.0 2026-09-24 01:31:56,462 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.584e+02 3.212e+02 3.605e+02 4.264e+02 7.770e+02, threshold=7.210e+02, percent-clipped=0.0 2026-09-24 01:31:58,615 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.27 vs. limit=22.5 2026-09-24 01:31:58,889 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=87453.33333333333, ans=0.125 2026-09-24 01:32:09,670 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=87520.0, ans=0.0 2026-09-24 01:32:14,293 INFO [train.py:1192] (1/2) Epoch 28, batch 400, loss[loss=0.3014, simple_loss=0.4056, pruned_loss=0.0986, over 24572.00 frames. ], tot_loss[loss=0.2954, simple_loss=0.401, pruned_loss=0.09488, over 4181253.55 frames. ], batch size: 170, lr: 7.51e-03, grad_scale: 32.0 2026-09-24 01:32:18,550 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.30 vs. limit=12.0 2026-09-24 01:32:19,138 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=87586.66666666667, ans=0.125 2026-09-24 01:32:29,241 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.26 vs. limit=22.5 2026-09-24 01:32:39,727 INFO [train.py:1192] (1/2) Epoch 28, batch 450, loss[loss=0.3087, simple_loss=0.4169, pruned_loss=0.1003, over 24641.00 frames. ], tot_loss[loss=0.295, simple_loss=0.4009, pruned_loss=0.09457, over 4320551.83 frames. ], batch size: 175, lr: 7.51e-03, grad_scale: 32.0 2026-09-24 01:32:45,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=87753.33333333333, ans=0.1 2026-09-24 01:32:45,271 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.50 vs. limit=15.0 2026-09-24 01:32:47,431 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.492e+02 3.188e+02 3.687e+02 4.228e+02 7.096e+02, threshold=7.375e+02, percent-clipped=0.0 2026-09-24 01:32:56,946 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=87820.0, ans=0.125 2026-09-24 01:33:03,339 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=87853.33333333333, ans=0.0 2026-09-24 01:33:04,835 INFO [train.py:1192] (1/2) Epoch 28, batch 500, loss[loss=0.3062, simple_loss=0.425, pruned_loss=0.09375, over 24523.00 frames. ], tot_loss[loss=0.2927, simple_loss=0.3987, pruned_loss=0.09334, over 4438090.36 frames. ], batch size: 218, lr: 7.50e-03, grad_scale: 32.0 2026-09-24 01:33:17,746 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=87953.33333333333, ans=0.09899494936611666 2026-09-24 01:33:25,486 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=88020.0, ans=0.0 2026-09-24 01:33:27,898 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=88020.0, ans=0.125 2026-09-24 01:33:30,701 INFO [train.py:1192] (1/2) Epoch 28, batch 550, loss[loss=0.3305, simple_loss=0.4479, pruned_loss=0.1065, over 24285.00 frames. ], tot_loss[loss=0.2939, simple_loss=0.3999, pruned_loss=0.09399, over 4524154.64 frames. ], batch size: 257, lr: 7.50e-03, grad_scale: 32.0 2026-09-24 01:33:32,997 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=88053.33333333333, ans=22.5 2026-09-24 01:33:38,380 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.413e+02 3.323e+02 3.632e+02 4.082e+02 5.460e+02, threshold=7.264e+02, percent-clipped=0.0 2026-09-24 01:33:46,951 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=88153.33333333333, ans=0.1 2026-09-24 01:33:49,252 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=88153.33333333333, ans=0.1 2026-09-24 01:33:49,712 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=88153.33333333333, ans=0.1 2026-09-24 01:33:56,026 INFO [train.py:1192] (1/2) Epoch 28, batch 600, loss[loss=0.2966, simple_loss=0.419, pruned_loss=0.0871, over 24317.00 frames. ], tot_loss[loss=0.2942, simple_loss=0.4004, pruned_loss=0.09397, over 4590233.97 frames. ], batch size: 234, lr: 7.49e-03, grad_scale: 16.0 2026-09-24 01:33:57,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=88220.0, ans=0.1 2026-09-24 01:34:01,965 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=88253.33333333333, ans=0.125 2026-09-24 01:34:03,128 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=88253.33333333333, ans=0.0 2026-09-24 01:34:04,823 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=88253.33333333333, ans=0.125 2026-09-24 01:34:04,845 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=88253.33333333333, ans=0.025 2026-09-24 01:34:08,816 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=88286.66666666667, ans=0.1 2026-09-24 01:34:20,661 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=88386.66666666667, ans=0.125 2026-09-24 01:34:21,078 INFO [train.py:1192] (1/2) Epoch 28, batch 650, loss[loss=0.291, simple_loss=0.3912, pruned_loss=0.09541, over 24590.00 frames. ], tot_loss[loss=0.2931, simple_loss=0.3994, pruned_loss=0.09339, over 4654837.74 frames. ], batch size: 154, lr: 7.48e-03, grad_scale: 16.0 2026-09-24 01:34:28,981 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=88420.0, ans=0.0 2026-09-24 01:34:29,709 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.498e+02 3.177e+02 3.682e+02 4.351e+02 7.098e+02, threshold=7.363e+02, percent-clipped=0.0 2026-09-24 01:34:30,758 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=88420.0, ans=0.0 2026-09-24 01:34:34,907 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=88453.33333333333, ans=0.125 2026-09-24 01:34:40,258 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=88486.66666666667, ans=0.05 2026-09-24 01:34:41,232 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=88486.66666666667, ans=0.0 2026-09-24 01:34:47,652 INFO [train.py:1192] (1/2) Epoch 28, batch 700, loss[loss=0.3005, simple_loss=0.3995, pruned_loss=0.1007, over 24558.00 frames. ], tot_loss[loss=0.294, simple_loss=0.4004, pruned_loss=0.0938, over 4689958.08 frames. ], batch size: 158, lr: 7.48e-03, grad_scale: 16.0 2026-09-24 01:34:58,969 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=88620.0, ans=0.125 2026-09-24 01:34:59,513 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.67 vs. limit=12.0 2026-09-24 01:35:00,908 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=88620.0, ans=0.0 2026-09-24 01:35:13,519 INFO [train.py:1192] (1/2) Epoch 28, batch 750, loss[loss=0.3075, simple_loss=0.4159, pruned_loss=0.09958, over 24621.00 frames. ], tot_loss[loss=0.2931, simple_loss=0.3994, pruned_loss=0.09335, over 4726205.54 frames. ], batch size: 175, lr: 7.47e-03, grad_scale: 16.0 2026-09-24 01:35:16,024 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=88720.0, ans=0.0 2026-09-24 01:35:21,670 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.551e+02 3.294e+02 3.656e+02 4.145e+02 6.425e+02, threshold=7.312e+02, percent-clipped=0.0 2026-09-24 01:35:30,302 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.22 vs. limit=15.0 2026-09-24 01:35:30,377 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.87 vs. limit=15.0 2026-09-24 01:35:33,626 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=88853.33333333333, ans=0.125 2026-09-24 01:35:35,287 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=88853.33333333333, ans=0.025 2026-09-24 01:35:39,244 INFO [train.py:1192] (1/2) Epoch 28, batch 800, loss[loss=0.2373, simple_loss=0.3471, pruned_loss=0.06375, over 24511.00 frames. ], tot_loss[loss=0.2932, simple_loss=0.3995, pruned_loss=0.09346, over 4752898.87 frames. ], batch size: 137, lr: 7.46e-03, grad_scale: 32.0 2026-09-24 01:35:43,557 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.55 vs. limit=6.0 2026-09-24 01:36:04,634 INFO [train.py:1192] (1/2) Epoch 28, batch 850, loss[loss=0.3185, simple_loss=0.4283, pruned_loss=0.1043, over 24557.00 frames. ], tot_loss[loss=0.2921, simple_loss=0.3984, pruned_loss=0.09294, over 4771198.43 frames. ], batch size: 204, lr: 7.46e-03, grad_scale: 32.0 2026-09-24 01:36:12,821 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.573e+02 3.192e+02 3.623e+02 4.155e+02 5.810e+02, threshold=7.245e+02, percent-clipped=0.0 2026-09-24 01:36:18,326 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=89120.0, ans=0.0 2026-09-24 01:36:29,532 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=6.60 vs. limit=15.0 2026-09-24 01:36:30,326 INFO [train.py:1192] (1/2) Epoch 28, batch 900, loss[loss=0.2657, simple_loss=0.3648, pruned_loss=0.08331, over 24560.00 frames. ], tot_loss[loss=0.2925, simple_loss=0.3988, pruned_loss=0.09315, over 4781476.41 frames. ], batch size: 137, lr: 7.45e-03, grad_scale: 32.0 2026-09-24 01:36:31,848 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=89220.0, ans=0.0 2026-09-24 01:36:45,141 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.16 vs. limit=15.0 2026-09-24 01:36:54,899 INFO [train.py:1192] (1/2) Epoch 28, batch 950, loss[loss=0.4311, simple_loss=0.4668, pruned_loss=0.1977, over 11570.00 frames. ], tot_loss[loss=0.293, simple_loss=0.3977, pruned_loss=0.09419, over 4714161.35 frames. ], batch size: 333, lr: 7.44e-03, grad_scale: 32.0 2026-09-24 01:37:06,576 INFO [train.py:1192] (1/2) Epoch 29, batch 0, loss[loss=0.24, simple_loss=0.3559, pruned_loss=0.06203, over 24566.00 frames. ], tot_loss[loss=0.24, simple_loss=0.3559, pruned_loss=0.06203, over 24566.00 frames. ], batch size: 137, lr: 7.31e-03, grad_scale: 32.0 2026-09-24 01:37:06,576 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 01:37:18,073 INFO [train.py:1224] (1/2) Epoch 29, validation: loss=0.1827, simple_loss=0.3014, pruned_loss=0.03193, over 2564189.00 frames. 2026-09-24 01:37:18,073 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 01:37:22,093 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.384e+02 3.217e+02 3.666e+02 4.169e+02 7.339e+02, threshold=7.333e+02, percent-clipped=0.0 2026-09-24 01:37:22,715 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=89446.66666666667, ans=0.1 2026-09-24 01:37:43,814 INFO [train.py:1192] (1/2) Epoch 29, batch 50, loss[loss=0.2396, simple_loss=0.3458, pruned_loss=0.06667, over 24274.00 frames. ], tot_loss[loss=0.296, simple_loss=0.4035, pruned_loss=0.09423, over 1079929.53 frames. ], batch size: 125, lr: 7.30e-03, grad_scale: 32.0 2026-09-24 01:37:51,676 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer_ff3.min_abs, batch_count=89613.33333333333, ans=0.2 2026-09-24 01:38:00,999 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=89680.0, ans=0.125 2026-09-24 01:38:06,365 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=89713.33333333333, ans=0.125 2026-09-24 01:38:09,296 INFO [train.py:1192] (1/2) Epoch 29, batch 100, loss[loss=0.2723, simple_loss=0.3764, pruned_loss=0.08408, over 24597.00 frames. ], tot_loss[loss=0.2999, simple_loss=0.4079, pruned_loss=0.09593, over 1914575.19 frames. ], batch size: 154, lr: 7.30e-03, grad_scale: 32.0 2026-09-24 01:38:10,876 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=89746.66666666667, ans=0.0 2026-09-24 01:38:13,479 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.584e+02 3.381e+02 3.673e+02 4.125e+02 5.808e+02, threshold=7.347e+02, percent-clipped=1.0 2026-09-24 01:38:20,229 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=89813.33333333333, ans=0.125 2026-09-24 01:38:33,492 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=89880.0, ans=0.125 2026-09-24 01:38:34,729 INFO [train.py:1192] (1/2) Epoch 29, batch 150, loss[loss=0.2237, simple_loss=0.3327, pruned_loss=0.05731, over 24268.00 frames. ], tot_loss[loss=0.2955, simple_loss=0.4026, pruned_loss=0.09418, over 2560193.15 frames. ], batch size: 125, lr: 7.29e-03, grad_scale: 32.0 2026-09-24 01:38:41,700 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.68 vs. limit=15.0 2026-09-24 01:38:45,318 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:38:48,457 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=89980.0, ans=0.125 2026-09-24 01:38:58,474 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=90046.66666666667, ans=0.125 2026-09-24 01:38:59,672 INFO [train.py:1192] (1/2) Epoch 29, batch 200, loss[loss=0.3113, simple_loss=0.433, pruned_loss=0.09478, over 24208.00 frames. ], tot_loss[loss=0.2919, simple_loss=0.3995, pruned_loss=0.09214, over 3059308.36 frames. ], batch size: 257, lr: 7.28e-03, grad_scale: 32.0 2026-09-24 01:39:03,728 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.419e+02 3.215e+02 3.663e+02 4.520e+02 7.059e+02, threshold=7.326e+02, percent-clipped=0.0 2026-09-24 01:39:21,083 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=90213.33333333333, ans=0.025 2026-09-24 01:39:23,086 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=90213.33333333333, ans=0.125 2026-09-24 01:39:25,694 INFO [train.py:1192] (1/2) Epoch 29, batch 250, loss[loss=0.3097, simple_loss=0.4257, pruned_loss=0.0969, over 24395.00 frames. ], tot_loss[loss=0.2929, simple_loss=0.3999, pruned_loss=0.09298, over 3444463.07 frames. ], batch size: 225, lr: 7.28e-03, grad_scale: 32.0 2026-09-24 01:39:40,169 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=90313.33333333333, ans=0.2 2026-09-24 01:39:50,963 INFO [train.py:1192] (1/2) Epoch 29, batch 300, loss[loss=0.2913, simple_loss=0.409, pruned_loss=0.08681, over 24525.00 frames. ], tot_loss[loss=0.2919, simple_loss=0.3987, pruned_loss=0.09252, over 3758393.55 frames. ], batch size: 204, lr: 7.27e-03, grad_scale: 32.0 2026-09-24 01:39:53,921 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=90413.33333333333, ans=0.0 2026-09-24 01:39:54,469 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=90413.33333333333, ans=0.025 2026-09-24 01:39:55,378 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.798e+02 3.448e+02 3.952e+02 4.554e+02 9.567e+02, threshold=7.905e+02, percent-clipped=3.0 2026-09-24 01:40:02,916 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=90480.0, ans=0.1 2026-09-24 01:40:07,053 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.28 vs. limit=15.0 2026-09-24 01:40:08,046 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.47 vs. limit=15.0 2026-09-24 01:40:16,977 INFO [train.py:1192] (1/2) Epoch 29, batch 350, loss[loss=0.2548, simple_loss=0.3613, pruned_loss=0.07419, over 24567.00 frames. ], tot_loss[loss=0.2923, simple_loss=0.399, pruned_loss=0.09281, over 3998816.40 frames. ], batch size: 137, lr: 7.27e-03, grad_scale: 32.0 2026-09-24 01:40:17,602 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=90580.0, ans=0.2 2026-09-24 01:40:18,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=90580.0, ans=0.0 2026-09-24 01:40:20,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=90580.0, ans=0.125 2026-09-24 01:40:27,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=90646.66666666667, ans=0.0 2026-09-24 01:40:28,985 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=90646.66666666667, ans=0.125 2026-09-24 01:40:33,505 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.72 vs. limit=15.0 2026-09-24 01:40:37,394 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=90713.33333333333, ans=0.125 2026-09-24 01:40:38,341 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=90713.33333333333, ans=0.2 2026-09-24 01:40:41,357 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.41 vs. limit=15.0 2026-09-24 01:40:42,510 INFO [train.py:1192] (1/2) Epoch 29, batch 400, loss[loss=0.2776, simple_loss=0.3897, pruned_loss=0.08273, over 24552.00 frames. ], tot_loss[loss=0.2907, simple_loss=0.3976, pruned_loss=0.09192, over 4182317.28 frames. ], batch size: 170, lr: 7.26e-03, grad_scale: 32.0 2026-09-24 01:40:44,897 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=90746.66666666667, ans=0.2 2026-09-24 01:40:44,906 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=90746.66666666667, ans=0.025 2026-09-24 01:40:46,578 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.611e+02 3.076e+02 3.524e+02 4.096e+02 7.653e+02, threshold=7.049e+02, percent-clipped=0.0 2026-09-24 01:40:47,511 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=90780.0, ans=0.2 2026-09-24 01:40:47,927 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=90780.0, ans=0.125 2026-09-24 01:41:07,796 INFO [train.py:1192] (1/2) Epoch 29, batch 450, loss[loss=0.2949, simple_loss=0.4047, pruned_loss=0.09257, over 24636.00 frames. ], tot_loss[loss=0.2911, simple_loss=0.398, pruned_loss=0.09208, over 4320927.96 frames. ], batch size: 175, lr: 7.25e-03, grad_scale: 32.0 2026-09-24 01:41:20,119 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=90980.0, ans=0.125 2026-09-24 01:41:21,184 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=90980.0, ans=0.2 2026-09-24 01:41:22,064 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=90980.0, ans=0.125 2026-09-24 01:41:26,214 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=91013.33333333333, ans=0.0 2026-09-24 01:41:33,392 INFO [train.py:1192] (1/2) Epoch 29, batch 500, loss[loss=0.3182, simple_loss=0.4298, pruned_loss=0.1032, over 24511.00 frames. ], tot_loss[loss=0.2902, simple_loss=0.3971, pruned_loss=0.09168, over 4438897.42 frames. ], batch size: 218, lr: 7.25e-03, grad_scale: 32.0 2026-09-24 01:41:37,533 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.610e+02 3.257e+02 3.691e+02 4.343e+02 9.379e+02, threshold=7.382e+02, percent-clipped=2.0 2026-09-24 01:41:38,518 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.30 vs. limit=22.5 2026-09-24 01:41:42,846 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=91113.33333333333, ans=0.125 2026-09-24 01:41:56,894 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=91213.33333333333, ans=0.125 2026-09-24 01:41:59,581 INFO [train.py:1192] (1/2) Epoch 29, batch 550, loss[loss=0.327, simple_loss=0.4367, pruned_loss=0.1087, over 24256.00 frames. ], tot_loss[loss=0.2917, simple_loss=0.3984, pruned_loss=0.09257, over 4523745.23 frames. ], batch size: 257, lr: 7.24e-03, grad_scale: 32.0 2026-09-24 01:42:04,571 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=91280.0, ans=0.0 2026-09-24 01:42:13,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=91313.33333333333, ans=0.125 2026-09-24 01:42:14,794 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=91346.66666666667, ans=0.2 2026-09-24 01:42:25,276 INFO [train.py:1192] (1/2) Epoch 29, batch 600, loss[loss=0.3312, simple_loss=0.4403, pruned_loss=0.1111, over 24295.00 frames. ], tot_loss[loss=0.2922, simple_loss=0.399, pruned_loss=0.09269, over 4590254.94 frames. ], batch size: 234, lr: 7.24e-03, grad_scale: 32.0 2026-09-24 01:42:26,945 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.12 vs. limit=22.5 2026-09-24 01:42:27,840 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.09 vs. limit=15.0 2026-09-24 01:42:29,067 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.662e+02 3.191e+02 3.540e+02 4.132e+02 5.235e+02, threshold=7.080e+02, percent-clipped=0.0 2026-09-24 01:42:31,648 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.81 vs. limit=15.0 2026-09-24 01:42:33,938 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=91446.66666666667, ans=0.0 2026-09-24 01:42:46,329 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=91546.66666666667, ans=0.1 2026-09-24 01:42:50,896 INFO [train.py:1192] (1/2) Epoch 29, batch 650, loss[loss=0.2959, simple_loss=0.3947, pruned_loss=0.09851, over 24584.00 frames. ], tot_loss[loss=0.2904, simple_loss=0.3976, pruned_loss=0.09162, over 4655081.66 frames. ], batch size: 154, lr: 7.23e-03, grad_scale: 32.0 2026-09-24 01:43:13,197 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.26 vs. limit=6.0 2026-09-24 01:43:16,998 INFO [train.py:1192] (1/2) Epoch 29, batch 700, loss[loss=0.2759, simple_loss=0.3845, pruned_loss=0.08368, over 24539.00 frames. ], tot_loss[loss=0.292, simple_loss=0.3992, pruned_loss=0.0924, over 4690266.59 frames. ], batch size: 158, lr: 7.22e-03, grad_scale: 32.0 2026-09-24 01:43:17,582 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=91746.66666666667, ans=0.0 2026-09-24 01:43:20,797 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.643e+02 3.317e+02 3.742e+02 4.248e+02 7.075e+02, threshold=7.485e+02, percent-clipped=0.0 2026-09-24 01:43:20,890 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=91746.66666666667, ans=10.0 2026-09-24 01:43:36,879 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:43:42,384 INFO [train.py:1192] (1/2) Epoch 29, batch 750, loss[loss=0.3043, simple_loss=0.4118, pruned_loss=0.09839, over 24649.00 frames. ], tot_loss[loss=0.2912, simple_loss=0.3982, pruned_loss=0.09209, over 4723028.48 frames. ], batch size: 175, lr: 7.22e-03, grad_scale: 32.0 2026-09-24 01:44:00,019 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=92013.33333333333, ans=0.125 2026-09-24 01:44:04,932 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=92046.66666666667, ans=0.125 2026-09-24 01:44:07,232 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=92080.0, ans=0.125 2026-09-24 01:44:07,589 INFO [train.py:1192] (1/2) Epoch 29, batch 800, loss[loss=0.2571, simple_loss=0.3566, pruned_loss=0.07879, over 24546.00 frames. ], tot_loss[loss=0.2904, simple_loss=0.3977, pruned_loss=0.09156, over 4750777.48 frames. ], batch size: 137, lr: 7.21e-03, grad_scale: 32.0 2026-09-24 01:44:11,513 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.222e+02 3.268e+02 3.638e+02 4.164e+02 6.382e+02, threshold=7.275e+02, percent-clipped=0.0 2026-09-24 01:44:28,795 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=92213.33333333333, ans=0.2 2026-09-24 01:44:33,102 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=92246.66666666667, ans=0.2 2026-09-24 01:44:33,107 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=92246.66666666667, ans=0.025 2026-09-24 01:44:33,418 INFO [train.py:1192] (1/2) Epoch 29, batch 850, loss[loss=0.3094, simple_loss=0.4216, pruned_loss=0.09854, over 24552.00 frames. ], tot_loss[loss=0.2904, simple_loss=0.3976, pruned_loss=0.09162, over 4769910.35 frames. ], batch size: 204, lr: 7.20e-03, grad_scale: 32.0 2026-09-24 01:44:42,899 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=92280.0, ans=0.125 2026-09-24 01:44:58,928 INFO [train.py:1192] (1/2) Epoch 29, batch 900, loss[loss=0.2364, simple_loss=0.3518, pruned_loss=0.06054, over 24568.00 frames. ], tot_loss[loss=0.2905, simple_loss=0.3978, pruned_loss=0.09161, over 4781160.13 frames. ], batch size: 137, lr: 7.20e-03, grad_scale: 32.0 2026-09-24 01:45:01,652 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=92413.33333333333, ans=0.0 2026-09-24 01:45:02,994 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.552e+02 3.260e+02 3.615e+02 4.126e+02 6.303e+02, threshold=7.230e+02, percent-clipped=0.0 2026-09-24 01:45:03,097 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=92413.33333333333, ans=0.125 2026-09-24 01:45:03,113 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=92413.33333333333, ans=0.125 2026-09-24 01:45:09,983 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=92480.0, ans=0.125 2026-09-24 01:45:18,021 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=92546.66666666667, ans=0.0 2026-09-24 01:45:23,359 INFO [train.py:1192] (1/2) Epoch 29, batch 950, loss[loss=0.399, simple_loss=0.4534, pruned_loss=0.1722, over 12098.00 frames. ], tot_loss[loss=0.2901, simple_loss=0.396, pruned_loss=0.09211, over 4711099.94 frames. ], batch size: 333, lr: 7.19e-03, grad_scale: 32.0 2026-09-24 01:45:31,956 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=92606.66666666667, ans=0.2 2026-09-24 01:45:34,452 INFO [train.py:1192] (1/2) Epoch 30, batch 0, loss[loss=0.228, simple_loss=0.3444, pruned_loss=0.05578, over 24556.00 frames. ], tot_loss[loss=0.228, simple_loss=0.3444, pruned_loss=0.05578, over 24556.00 frames. ], batch size: 137, lr: 7.07e-03, grad_scale: 32.0 2026-09-24 01:45:34,452 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 01:45:45,790 INFO [train.py:1224] (1/2) Epoch 30, validation: loss=0.1842, simple_loss=0.3023, pruned_loss=0.03305, over 2564189.00 frames. 2026-09-24 01:45:45,790 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 01:45:49,503 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.62 vs. limit=15.0 2026-09-24 01:45:52,198 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=92640.0, ans=0.125 2026-09-24 01:45:56,681 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.05 vs. limit=15.0 2026-09-24 01:46:04,243 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.18 vs. limit=15.0 2026-09-24 01:46:08,697 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.19 vs. limit=22.5 2026-09-24 01:46:11,798 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.221e+02 3.410e+02 3.907e+02 4.808e+02 7.268e+02, threshold=7.815e+02, percent-clipped=1.0 2026-09-24 01:46:11,804 INFO [train.py:1192] (1/2) Epoch 30, batch 50, loss[loss=0.2221, simple_loss=0.3276, pruned_loss=0.05835, over 24254.00 frames. ], tot_loss[loss=0.2998, simple_loss=0.4053, pruned_loss=0.09715, over 1082863.78 frames. ], batch size: 125, lr: 7.06e-03, grad_scale: 32.0 2026-09-24 01:46:15,216 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=92773.33333333333, ans=0.0 2026-09-24 01:46:16,241 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.03 vs. limit=15.0 2026-09-24 01:46:27,357 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=92873.33333333333, ans=0.125 2026-09-24 01:46:37,969 INFO [train.py:1192] (1/2) Epoch 30, batch 100, loss[loss=0.2897, simple_loss=0.3928, pruned_loss=0.09332, over 24614.00 frames. ], tot_loss[loss=0.3009, simple_loss=0.4087, pruned_loss=0.09656, over 1915857.43 frames. ], batch size: 154, lr: 7.06e-03, grad_scale: 32.0 2026-09-24 01:46:39,848 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:46:42,241 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=92973.33333333333, ans=0.1 2026-09-24 01:46:57,072 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:46:57,085 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:46:57,096 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=93040.0, ans=0.95 2026-09-24 01:47:02,898 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.616e+02 3.261e+02 3.575e+02 4.013e+02 5.707e+02, threshold=7.151e+02, percent-clipped=0.0 2026-09-24 01:47:02,904 INFO [train.py:1192] (1/2) Epoch 30, batch 150, loss[loss=0.2666, simple_loss=0.3596, pruned_loss=0.08679, over 24283.00 frames. ], tot_loss[loss=0.2943, simple_loss=0.402, pruned_loss=0.09334, over 2561110.96 frames. ], batch size: 125, lr: 7.05e-03, grad_scale: 32.0 2026-09-24 01:47:15,015 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=93173.33333333333, ans=0.125 2026-09-24 01:47:20,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=93206.66666666667, ans=0.0 2026-09-24 01:47:22,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=93206.66666666667, ans=0.0 2026-09-24 01:47:27,194 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=93240.0, ans=0.1 2026-09-24 01:47:28,970 INFO [train.py:1192] (1/2) Epoch 30, batch 200, loss[loss=0.3272, simple_loss=0.4381, pruned_loss=0.1081, over 24211.00 frames. ], tot_loss[loss=0.2926, simple_loss=0.4002, pruned_loss=0.09255, over 3059923.41 frames. ], batch size: 257, lr: 7.04e-03, grad_scale: 32.0 2026-09-24 01:47:29,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=93273.33333333333, ans=0.2 2026-09-24 01:47:30,549 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=93273.33333333333, ans=0.125 2026-09-24 01:47:47,533 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.01 vs. limit=5.0 2026-09-24 01:47:48,222 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=93373.33333333333, ans=0.125 2026-09-24 01:47:48,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=93373.33333333333, ans=0.125 2026-09-24 01:47:49,226 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=93406.66666666667, ans=0.125 2026-09-24 01:47:53,043 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.92 vs. limit=6.0 2026-09-24 01:47:54,228 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.683e+02 3.373e+02 3.806e+02 4.796e+02 1.018e+03, threshold=7.612e+02, percent-clipped=3.0 2026-09-24 01:47:54,248 INFO [train.py:1192] (1/2) Epoch 30, batch 250, loss[loss=0.3423, simple_loss=0.4472, pruned_loss=0.1187, over 24385.00 frames. ], tot_loss[loss=0.2915, simple_loss=0.399, pruned_loss=0.092, over 3443710.25 frames. ], batch size: 225, lr: 7.04e-03, grad_scale: 32.0 2026-09-24 01:47:55,646 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=93440.0, ans=0.0 2026-09-24 01:47:55,667 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=93440.0, ans=0.0 2026-09-24 01:48:04,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=93506.66666666667, ans=0.1 2026-09-24 01:48:06,434 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=93506.66666666667, ans=0.125 2026-09-24 01:48:06,464 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=93506.66666666667, ans=0.125 2026-09-24 01:48:19,620 INFO [train.py:1192] (1/2) Epoch 30, batch 300, loss[loss=0.3095, simple_loss=0.429, pruned_loss=0.09504, over 24554.00 frames. ], tot_loss[loss=0.2902, simple_loss=0.3979, pruned_loss=0.09123, over 3757861.14 frames. ], batch size: 204, lr: 7.03e-03, grad_scale: 32.0 2026-09-24 01:48:19,826 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.39 vs. limit=15.0 2026-09-24 01:48:23,875 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.28 vs. limit=15.0 2026-09-24 01:48:26,748 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=93640.0, ans=0.1 2026-09-24 01:48:28,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=93640.0, ans=0.125 2026-09-24 01:48:31,350 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=93673.33333333333, ans=0.125 2026-09-24 01:48:45,146 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.431e+02 3.204e+02 3.555e+02 3.928e+02 5.395e+02, threshold=7.109e+02, percent-clipped=0.0 2026-09-24 01:48:45,152 INFO [train.py:1192] (1/2) Epoch 30, batch 350, loss[loss=0.2321, simple_loss=0.3399, pruned_loss=0.0622, over 24589.00 frames. ], tot_loss[loss=0.2909, simple_loss=0.3986, pruned_loss=0.09161, over 3998245.84 frames. ], batch size: 137, lr: 7.03e-03, grad_scale: 32.0 2026-09-24 01:48:46,945 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=93773.33333333333, ans=0.1 2026-09-24 01:49:08,393 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=93906.66666666667, ans=0.0 2026-09-24 01:49:10,447 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=93906.66666666667, ans=0.1 2026-09-24 01:49:11,431 INFO [train.py:1192] (1/2) Epoch 30, batch 400, loss[loss=0.2681, simple_loss=0.3909, pruned_loss=0.0727, over 24547.00 frames. ], tot_loss[loss=0.2895, simple_loss=0.3973, pruned_loss=0.09092, over 4182862.42 frames. ], batch size: 170, lr: 7.02e-03, grad_scale: 32.0 2026-09-24 01:49:20,687 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=93973.33333333333, ans=0.1 2026-09-24 01:49:28,143 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=94040.0, ans=0.2 2026-09-24 01:49:30,521 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:49:31,311 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=94040.0, ans=0.0 2026-09-24 01:49:37,088 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.395e+02 3.288e+02 3.765e+02 4.465e+02 8.248e+02, threshold=7.529e+02, percent-clipped=1.0 2026-09-24 01:49:37,094 INFO [train.py:1192] (1/2) Epoch 30, batch 450, loss[loss=0.3031, simple_loss=0.4105, pruned_loss=0.09784, over 24625.00 frames. ], tot_loss[loss=0.2905, simple_loss=0.398, pruned_loss=0.09148, over 4322604.66 frames. ], batch size: 175, lr: 7.02e-03, grad_scale: 32.0 2026-09-24 01:49:41,452 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.41 vs. limit=22.5 2026-09-24 01:49:46,504 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.17 vs. limit=15.0 2026-09-24 01:49:46,693 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=94140.0, ans=0.1 2026-09-24 01:49:51,684 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=94173.33333333333, ans=0.1 2026-09-24 01:49:57,790 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=94240.0, ans=0.05 2026-09-24 01:49:59,620 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=94240.0, ans=0.0 2026-09-24 01:50:02,834 INFO [train.py:1192] (1/2) Epoch 30, batch 500, loss[loss=0.3173, simple_loss=0.4291, pruned_loss=0.1028, over 24485.00 frames. ], tot_loss[loss=0.2892, simple_loss=0.3965, pruned_loss=0.09099, over 4439559.73 frames. ], batch size: 218, lr: 7.01e-03, grad_scale: 32.0 2026-09-24 01:50:02,921 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=94273.33333333333, ans=0.125 2026-09-24 01:50:05,162 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=94273.33333333333, ans=0.125 2026-09-24 01:50:11,439 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=94306.66666666667, ans=0.0 2026-09-24 01:50:13,791 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=94340.0, ans=0.0 2026-09-24 01:50:17,001 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=94373.33333333333, ans=0.2 2026-09-24 01:50:20,153 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=94373.33333333333, ans=0.125 2026-09-24 01:50:27,286 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.484e+02 3.186e+02 3.398e+02 3.899e+02 5.797e+02, threshold=6.796e+02, percent-clipped=0.0 2026-09-24 01:50:27,292 INFO [train.py:1192] (1/2) Epoch 30, batch 550, loss[loss=0.317, simple_loss=0.4326, pruned_loss=0.1007, over 24263.00 frames. ], tot_loss[loss=0.2892, simple_loss=0.3968, pruned_loss=0.09085, over 4525011.61 frames. ], batch size: 257, lr: 7.00e-03, grad_scale: 32.0 2026-09-24 01:50:50,504 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=94573.33333333333, ans=0.125 2026-09-24 01:50:53,390 INFO [train.py:1192] (1/2) Epoch 30, batch 600, loss[loss=0.2798, simple_loss=0.4008, pruned_loss=0.07934, over 24401.00 frames. ], tot_loss[loss=0.2905, simple_loss=0.3979, pruned_loss=0.09157, over 4590609.30 frames. ], batch size: 235, lr: 7.00e-03, grad_scale: 32.0 2026-09-24 01:51:16,249 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=94740.0, ans=0.0 2026-09-24 01:51:19,301 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.444e+02 3.185e+02 3.671e+02 4.261e+02 6.024e+02, threshold=7.341e+02, percent-clipped=0.0 2026-09-24 01:51:19,307 INFO [train.py:1192] (1/2) Epoch 30, batch 650, loss[loss=0.2817, simple_loss=0.3843, pruned_loss=0.0896, over 24602.00 frames. ], tot_loss[loss=0.289, simple_loss=0.3968, pruned_loss=0.09062, over 4655065.44 frames. ], batch size: 154, lr: 6.99e-03, grad_scale: 32.0 2026-09-24 01:51:28,110 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=10.19 vs. limit=15.0 2026-09-24 01:51:28,949 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=94840.0, ans=0.125 2026-09-24 01:51:34,181 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=94873.33333333333, ans=0.0 2026-09-24 01:51:34,668 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=94873.33333333333, ans=0.125 2026-09-24 01:51:38,175 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=94873.33333333333, ans=0.2 2026-09-24 01:51:41,928 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=94906.66666666667, ans=0.1 2026-09-24 01:51:45,002 INFO [train.py:1192] (1/2) Epoch 30, batch 700, loss[loss=0.2742, simple_loss=0.3859, pruned_loss=0.08121, over 24563.00 frames. ], tot_loss[loss=0.2896, simple_loss=0.3978, pruned_loss=0.09069, over 4692071.97 frames. ], batch size: 158, lr: 6.99e-03, grad_scale: 64.0 2026-09-24 01:52:05,261 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=95073.33333333333, ans=0.125 2026-09-24 01:52:11,072 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.585e+02 3.195e+02 3.649e+02 4.277e+02 7.083e+02, threshold=7.297e+02, percent-clipped=0.0 2026-09-24 01:52:11,079 INFO [train.py:1192] (1/2) Epoch 30, batch 750, loss[loss=0.2958, simple_loss=0.4079, pruned_loss=0.09188, over 24607.00 frames. ], tot_loss[loss=0.2889, simple_loss=0.3969, pruned_loss=0.09043, over 4727983.24 frames. ], batch size: 175, lr: 6.98e-03, grad_scale: 64.0 2026-09-24 01:52:14,258 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=95106.66666666667, ans=0.025 2026-09-24 01:52:17,746 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.51 vs. limit=15.0 2026-09-24 01:52:30,683 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=95206.66666666667, ans=0.0 2026-09-24 01:52:36,686 INFO [train.py:1192] (1/2) Epoch 30, batch 800, loss[loss=0.2795, simple_loss=0.3769, pruned_loss=0.09103, over 24556.00 frames. ], tot_loss[loss=0.2886, simple_loss=0.3965, pruned_loss=0.09034, over 4753690.82 frames. ], batch size: 137, lr: 6.98e-03, grad_scale: 64.0 2026-09-24 01:52:56,193 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=95373.33333333333, ans=0.2 2026-09-24 01:52:59,410 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=95406.66666666667, ans=0.2 2026-09-24 01:53:02,362 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.841e+02 3.325e+02 3.701e+02 4.282e+02 6.167e+02, threshold=7.403e+02, percent-clipped=0.0 2026-09-24 01:53:02,368 INFO [train.py:1192] (1/2) Epoch 30, batch 850, loss[loss=0.2981, simple_loss=0.4144, pruned_loss=0.09091, over 24561.00 frames. ], tot_loss[loss=0.2877, simple_loss=0.3958, pruned_loss=0.08985, over 4772707.22 frames. ], batch size: 204, lr: 6.97e-03, grad_scale: 64.0 2026-09-24 01:53:05,453 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=95440.0, ans=0.125 2026-09-24 01:53:16,777 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=95506.66666666667, ans=0.025 2026-09-24 01:53:19,335 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=95540.0, ans=0.1 2026-09-24 01:53:19,380 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=95540.0, ans=0.5 2026-09-24 01:53:27,624 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=95606.66666666667, ans=0.125 2026-09-24 01:53:27,964 INFO [train.py:1192] (1/2) Epoch 30, batch 900, loss[loss=0.2628, simple_loss=0.3656, pruned_loss=0.08, over 24554.00 frames. ], tot_loss[loss=0.2878, simple_loss=0.3957, pruned_loss=0.08996, over 4782530.33 frames. ], batch size: 137, lr: 6.96e-03, grad_scale: 64.0 2026-09-24 01:53:29,036 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=95606.66666666667, ans=0.125 2026-09-24 01:53:30,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=95606.66666666667, ans=0.2 2026-09-24 01:53:32,636 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=95640.0, ans=0.125 2026-09-24 01:53:35,171 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=95640.0, ans=22.5 2026-09-24 01:53:48,194 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=95740.0, ans=0.125 2026-09-24 01:53:52,830 INFO [train.py:1192] (1/2) Epoch 30, batch 950, loss[loss=0.4138, simple_loss=0.4588, pruned_loss=0.1844, over 11105.00 frames. ], tot_loss[loss=0.2896, simple_loss=0.3955, pruned_loss=0.09184, over 4711835.92 frames. ], batch size: 333, lr: 6.96e-03, grad_scale: 32.0 2026-09-24 01:53:53,319 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.530e+02 3.392e+02 3.907e+02 4.492e+02 6.002e+02, threshold=7.815e+02, percent-clipped=0.0 2026-09-24 01:54:02,964 INFO [train.py:1192] (1/2) Epoch 31, batch 0, loss[loss=0.2569, simple_loss=0.3665, pruned_loss=0.0736, over 24555.00 frames. ], tot_loss[loss=0.2569, simple_loss=0.3665, pruned_loss=0.0736, over 24555.00 frames. ], batch size: 137, lr: 6.84e-03, grad_scale: 32.0 2026-09-24 01:54:02,965 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 01:54:11,706 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([1.6848, 3.1431, 3.0464, 1.6584], device='cuda:1') 2026-09-24 01:54:14,452 INFO [train.py:1224] (1/2) Epoch 31, validation: loss=0.1821, simple_loss=0.3003, pruned_loss=0.03191, over 2564189.00 frames. 2026-09-24 01:54:14,452 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 01:54:37,052 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:54:39,603 INFO [train.py:1192] (1/2) Epoch 31, batch 50, loss[loss=0.2385, simple_loss=0.3405, pruned_loss=0.06826, over 24253.00 frames. ], tot_loss[loss=0.2935, simple_loss=0.4024, pruned_loss=0.09235, over 1080966.35 frames. ], batch size: 125, lr: 6.84e-03, grad_scale: 32.0 2026-09-24 01:54:40,470 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=95966.66666666667, ans=0.125 2026-09-24 01:54:44,778 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=96000.0, ans=0.125 2026-09-24 01:54:50,781 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.77 vs. limit=15.0 2026-09-24 01:54:54,244 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.35 vs. limit=10.0 2026-09-24 01:55:01,758 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.596e+02 3.558e+02 3.996e+02 4.712e+02 6.980e+02, threshold=7.991e+02, percent-clipped=0.0 2026-09-24 01:55:05,867 INFO [train.py:1192] (1/2) Epoch 31, batch 100, loss[loss=0.2905, simple_loss=0.3889, pruned_loss=0.09604, over 24600.00 frames. ], tot_loss[loss=0.2949, simple_loss=0.4049, pruned_loss=0.09252, over 1914491.75 frames. ], batch size: 154, lr: 6.83e-03, grad_scale: 32.0 2026-09-24 01:55:07,388 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=96133.33333333333, ans=0.025 2026-09-24 01:55:09,712 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=96133.33333333333, ans=0.125 2026-09-24 01:55:11,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=96166.66666666667, ans=0.0 2026-09-24 01:55:14,591 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=96166.66666666667, ans=0.015 2026-09-24 01:55:27,431 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=96266.66666666667, ans=0.125 2026-09-24 01:55:29,482 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=96266.66666666667, ans=0.125 2026-09-24 01:55:31,291 INFO [train.py:1192] (1/2) Epoch 31, batch 150, loss[loss=0.2412, simple_loss=0.3454, pruned_loss=0.06852, over 24263.00 frames. ], tot_loss[loss=0.2902, simple_loss=0.3992, pruned_loss=0.09062, over 2560220.19 frames. ], batch size: 125, lr: 6.83e-03, grad_scale: 32.0 2026-09-24 01:55:43,395 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=96366.66666666667, ans=0.0 2026-09-24 01:55:50,003 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.07 vs. limit=15.0 2026-09-24 01:55:50,313 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=96400.0, ans=0.125 2026-09-24 01:55:50,375 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=96400.0, ans=0.5 2026-09-24 01:55:51,882 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=96433.33333333333, ans=0.025 2026-09-24 01:55:53,305 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.532e+02 3.348e+02 3.902e+02 4.462e+02 6.331e+02, threshold=7.805e+02, percent-clipped=0.0 2026-09-24 01:55:56,661 INFO [train.py:1192] (1/2) Epoch 31, batch 200, loss[loss=0.3151, simple_loss=0.4319, pruned_loss=0.09912, over 24195.00 frames. ], tot_loss[loss=0.2886, simple_loss=0.3976, pruned_loss=0.08976, over 3059077.59 frames. ], batch size: 257, lr: 6.82e-03, grad_scale: 32.0 2026-09-24 01:55:58,967 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=96466.66666666667, ans=0.0 2026-09-24 01:55:59,048 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.43 vs. limit=15.0 2026-09-24 01:56:11,045 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=96533.33333333333, ans=0.125 2026-09-24 01:56:22,026 INFO [train.py:1192] (1/2) Epoch 31, batch 250, loss[loss=0.3463, simple_loss=0.4505, pruned_loss=0.121, over 24361.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.3969, pruned_loss=0.08986, over 3442868.65 frames. ], batch size: 225, lr: 6.81e-03, grad_scale: 32.0 2026-09-24 01:56:23,359 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=96633.33333333333, ans=0.0 2026-09-24 01:56:34,963 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.23 vs. limit=12.0 2026-09-24 01:56:35,285 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=96700.0, ans=0.2 2026-09-24 01:56:44,174 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.501e+02 3.365e+02 3.961e+02 4.949e+02 8.830e+02, threshold=7.921e+02, percent-clipped=2.0 2026-09-24 01:56:45,856 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.57 vs. limit=15.0 2026-09-24 01:56:47,466 INFO [train.py:1192] (1/2) Epoch 31, batch 300, loss[loss=0.3002, simple_loss=0.4227, pruned_loss=0.08889, over 24541.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3965, pruned_loss=0.08978, over 3757426.80 frames. ], batch size: 204, lr: 6.81e-03, grad_scale: 32.0 2026-09-24 01:56:55,042 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=96833.33333333333, ans=0.125 2026-09-24 01:56:58,638 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=96866.66666666667, ans=0.125 2026-09-24 01:57:03,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=96900.0, ans=0.125 2026-09-24 01:57:03,876 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=96900.0, ans=0.0 2026-09-24 01:57:13,217 INFO [train.py:1192] (1/2) Epoch 31, batch 350, loss[loss=0.2597, simple_loss=0.3628, pruned_loss=0.0783, over 24566.00 frames. ], tot_loss[loss=0.2886, simple_loss=0.3971, pruned_loss=0.09005, over 3994978.87 frames. ], batch size: 137, lr: 6.80e-03, grad_scale: 32.0 2026-09-24 01:57:17,464 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=96966.66666666667, ans=0.0 2026-09-24 01:57:21,207 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=5.28 vs. limit=15.0 2026-09-24 01:57:35,377 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.524e+02 3.164e+02 3.531e+02 3.999e+02 7.327e+02, threshold=7.063e+02, percent-clipped=0.0 2026-09-24 01:57:39,408 INFO [train.py:1192] (1/2) Epoch 31, batch 400, loss[loss=0.2996, simple_loss=0.4018, pruned_loss=0.0987, over 24551.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3964, pruned_loss=0.08981, over 4179901.69 frames. ], batch size: 170, lr: 6.80e-03, grad_scale: 32.0 2026-09-24 01:57:47,261 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=97166.66666666667, ans=0.0 2026-09-24 01:57:48,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=97166.66666666667, ans=0.125 2026-09-24 01:57:49,332 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.71 vs. limit=6.0 2026-09-24 01:57:59,541 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=97266.66666666667, ans=0.1 2026-09-24 01:58:00,012 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.58 vs. limit=15.0 2026-09-24 01:58:05,082 INFO [train.py:1192] (1/2) Epoch 31, batch 450, loss[loss=0.2924, simple_loss=0.4054, pruned_loss=0.08975, over 24622.00 frames. ], tot_loss[loss=0.2887, simple_loss=0.3969, pruned_loss=0.09025, over 4319700.55 frames. ], batch size: 175, lr: 6.79e-03, grad_scale: 32.0 2026-09-24 01:58:05,709 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=97300.0, ans=0.0 2026-09-24 01:58:10,444 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=97333.33333333333, ans=0.2 2026-09-24 01:58:12,117 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.45 vs. limit=10.0 2026-09-24 01:58:18,265 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=97366.66666666667, ans=0.2 2026-09-24 01:58:26,153 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=97433.33333333333, ans=0.125 2026-09-24 01:58:27,872 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.438e+02 3.243e+02 3.714e+02 4.161e+02 6.137e+02, threshold=7.427e+02, percent-clipped=0.0 2026-09-24 01:58:31,105 INFO [train.py:1192] (1/2) Epoch 31, batch 500, loss[loss=0.3067, simple_loss=0.4266, pruned_loss=0.09335, over 24522.00 frames. ], tot_loss[loss=0.2879, simple_loss=0.3957, pruned_loss=0.09005, over 4437221.28 frames. ], batch size: 218, lr: 6.79e-03, grad_scale: 32.0 2026-09-24 01:58:36,011 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.81 vs. limit=15.0 2026-09-24 01:58:40,713 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.89 vs. limit=22.5 2026-09-24 01:58:42,777 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.19 vs. limit=6.0 2026-09-24 01:58:49,861 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=97566.66666666667, ans=0.025 2026-09-24 01:58:53,504 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=97600.0, ans=0.1 2026-09-24 01:58:56,340 INFO [train.py:1192] (1/2) Epoch 31, batch 550, loss[loss=0.312, simple_loss=0.4278, pruned_loss=0.09813, over 24288.00 frames. ], tot_loss[loss=0.2871, simple_loss=0.3955, pruned_loss=0.08938, over 4523904.41 frames. ], batch size: 257, lr: 6.78e-03, grad_scale: 32.0 2026-09-24 01:58:56,452 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 01:58:57,806 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=97633.33333333333, ans=0.125 2026-09-24 01:59:12,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=97733.33333333333, ans=0.125 2026-09-24 01:59:18,285 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.697e+02 3.182e+02 3.595e+02 4.025e+02 7.869e+02, threshold=7.190e+02, percent-clipped=1.0 2026-09-24 01:59:21,848 INFO [train.py:1192] (1/2) Epoch 31, batch 600, loss[loss=0.3235, simple_loss=0.4416, pruned_loss=0.1027, over 24331.00 frames. ], tot_loss[loss=0.2876, simple_loss=0.3958, pruned_loss=0.08968, over 4590318.40 frames. ], batch size: 234, lr: 6.78e-03, grad_scale: 32.0 2026-09-24 01:59:43,549 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.81 vs. limit=22.5 2026-09-24 01:59:47,755 INFO [train.py:1192] (1/2) Epoch 31, batch 650, loss[loss=0.2949, simple_loss=0.3974, pruned_loss=0.09616, over 24605.00 frames. ], tot_loss[loss=0.2872, simple_loss=0.3955, pruned_loss=0.08944, over 4654706.24 frames. ], batch size: 154, lr: 6.77e-03, grad_scale: 32.0 2026-09-24 02:00:06,144 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=98066.66666666667, ans=0.025 2026-09-24 02:00:09,325 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=98100.0, ans=0.125 2026-09-24 02:00:09,695 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.509e+02 3.300e+02 3.827e+02 4.639e+02 6.903e+02, threshold=7.654e+02, percent-clipped=0.0 2026-09-24 02:00:13,295 INFO [train.py:1192] (1/2) Epoch 31, batch 700, loss[loss=0.2959, simple_loss=0.398, pruned_loss=0.09685, over 24560.00 frames. ], tot_loss[loss=0.2876, simple_loss=0.3963, pruned_loss=0.08942, over 4691079.60 frames. ], batch size: 158, lr: 6.77e-03, grad_scale: 32.0 2026-09-24 02:00:23,499 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=98200.0, ans=0.125 2026-09-24 02:00:27,679 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=98200.0, ans=0.0 2026-09-24 02:00:30,153 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=98233.33333333333, ans=0.04949747468305833 2026-09-24 02:00:36,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=98266.66666666667, ans=0.025 2026-09-24 02:00:39,513 INFO [train.py:1192] (1/2) Epoch 31, batch 750, loss[loss=0.2992, simple_loss=0.4058, pruned_loss=0.0963, over 24628.00 frames. ], tot_loss[loss=0.2871, simple_loss=0.3956, pruned_loss=0.08932, over 4725391.58 frames. ], batch size: 175, lr: 6.76e-03, grad_scale: 32.0 2026-09-24 02:00:42,078 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.00 vs. limit=15.0 2026-09-24 02:00:43,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=98300.0, ans=0.0 2026-09-24 02:00:45,834 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=98333.33333333333, ans=10.0 2026-09-24 02:00:46,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=98333.33333333333, ans=0.0 2026-09-24 02:00:48,596 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=98333.33333333333, ans=0.0 2026-09-24 02:00:52,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=98366.66666666667, ans=0.025 2026-09-24 02:00:52,690 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=98366.66666666667, ans=0.1 2026-09-24 02:00:58,682 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=98400.0, ans=0.125 2026-09-24 02:01:01,955 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.732e+02 3.346e+02 3.709e+02 4.178e+02 5.646e+02, threshold=7.418e+02, percent-clipped=0.0 2026-09-24 02:01:05,569 INFO [train.py:1192] (1/2) Epoch 31, batch 800, loss[loss=0.2358, simple_loss=0.345, pruned_loss=0.06332, over 24536.00 frames. ], tot_loss[loss=0.2876, simple_loss=0.3959, pruned_loss=0.08966, over 4751058.97 frames. ], batch size: 137, lr: 6.75e-03, grad_scale: 32.0 2026-09-24 02:01:23,072 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=98566.66666666667, ans=0.0 2026-09-24 02:01:31,109 INFO [train.py:1192] (1/2) Epoch 31, batch 850, loss[loss=0.2995, simple_loss=0.4153, pruned_loss=0.09178, over 24533.00 frames. ], tot_loss[loss=0.2872, simple_loss=0.3956, pruned_loss=0.08937, over 4769206.39 frames. ], batch size: 204, lr: 6.75e-03, grad_scale: 32.0 2026-09-24 02:01:35,998 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=98666.66666666667, ans=0.2 2026-09-24 02:01:52,952 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:01:54,184 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.796e+02 3.307e+02 3.789e+02 4.588e+02 6.875e+02, threshold=7.578e+02, percent-clipped=0.0 2026-09-24 02:01:56,369 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=98766.66666666667, ans=0.125 2026-09-24 02:01:57,774 INFO [train.py:1192] (1/2) Epoch 31, batch 900, loss[loss=0.2518, simple_loss=0.3642, pruned_loss=0.06963, over 24558.00 frames. ], tot_loss[loss=0.2882, simple_loss=0.3964, pruned_loss=0.08997, over 4779803.88 frames. ], batch size: 137, lr: 6.74e-03, grad_scale: 32.0 2026-09-24 02:01:58,854 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=98800.0, ans=0.0 2026-09-24 02:01:59,056 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.11 vs. limit=6.0 2026-09-24 02:02:23,254 INFO [train.py:1192] (1/2) Epoch 31, batch 950, loss[loss=0.3607, simple_loss=0.4258, pruned_loss=0.1478, over 12292.00 frames. ], tot_loss[loss=0.288, simple_loss=0.3944, pruned_loss=0.09081, over 4710079.99 frames. ], batch size: 333, lr: 6.74e-03, grad_scale: 32.0 2026-09-24 02:02:34,771 INFO [train.py:1192] (1/2) Epoch 32, batch 0, loss[loss=0.2328, simple_loss=0.3485, pruned_loss=0.05853, over 24583.00 frames. ], tot_loss[loss=0.2328, simple_loss=0.3485, pruned_loss=0.05853, over 24583.00 frames. ], batch size: 137, lr: 6.63e-03, grad_scale: 32.0 2026-09-24 02:02:34,771 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 02:02:46,481 INFO [train.py:1224] (1/2) Epoch 32, validation: loss=0.1822, simple_loss=0.3006, pruned_loss=0.03189, over 2564189.00 frames. 2026-09-24 02:02:46,481 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 02:02:47,782 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.50 vs. limit=22.5 2026-09-24 02:02:53,425 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=99026.66666666667, ans=0.1 2026-09-24 02:02:53,923 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=99026.66666666667, ans=0.125 2026-09-24 02:03:04,530 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.44 vs. limit=12.0 2026-09-24 02:03:04,818 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.330e+02 3.362e+02 3.762e+02 4.535e+02 8.123e+02, threshold=7.524e+02, percent-clipped=1.0 2026-09-24 02:03:09,438 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=99126.66666666667, ans=0.0 2026-09-24 02:03:13,300 INFO [train.py:1192] (1/2) Epoch 32, batch 50, loss[loss=0.2132, simple_loss=0.3229, pruned_loss=0.05177, over 24272.00 frames. ], tot_loss[loss=0.2971, simple_loss=0.4038, pruned_loss=0.09516, over 1080671.07 frames. ], batch size: 125, lr: 6.62e-03, grad_scale: 32.0 2026-09-24 02:03:15,865 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=99160.0, ans=10.0 2026-09-24 02:03:16,878 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=99160.0, ans=0.125 2026-09-24 02:03:18,968 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=5.09 vs. limit=12.0 2026-09-24 02:03:39,436 INFO [train.py:1192] (1/2) Epoch 32, batch 100, loss[loss=0.2916, simple_loss=0.396, pruned_loss=0.09361, over 24627.00 frames. ], tot_loss[loss=0.297, simple_loss=0.4058, pruned_loss=0.09406, over 1914620.23 frames. ], batch size: 154, lr: 6.62e-03, grad_scale: 32.0 2026-09-24 02:03:41,319 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=99326.66666666667, ans=0.125 2026-09-24 02:03:47,803 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=99360.0, ans=0.125 2026-09-24 02:03:53,697 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=99393.33333333333, ans=0.0 2026-09-24 02:03:55,209 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.41 vs. limit=22.5 2026-09-24 02:03:57,495 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.503e+02 3.354e+02 3.724e+02 4.176e+02 6.331e+02, threshold=7.449e+02, percent-clipped=0.0 2026-09-24 02:03:59,507 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:04:05,057 INFO [train.py:1192] (1/2) Epoch 32, batch 150, loss[loss=0.2281, simple_loss=0.3339, pruned_loss=0.06112, over 24255.00 frames. ], tot_loss[loss=0.2901, simple_loss=0.3988, pruned_loss=0.09071, over 2560726.62 frames. ], batch size: 125, lr: 6.61e-03, grad_scale: 32.0 2026-09-24 02:04:06,925 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=99493.33333333333, ans=0.125 2026-09-24 02:04:19,842 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=99593.33333333333, ans=0.1 2026-09-24 02:04:23,799 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.38 vs. limit=15.0 2026-09-24 02:04:24,757 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=99626.66666666667, ans=0.025 2026-09-24 02:04:27,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=99626.66666666667, ans=0.0 2026-09-24 02:04:30,776 INFO [train.py:1192] (1/2) Epoch 32, batch 200, loss[loss=0.3101, simple_loss=0.4255, pruned_loss=0.09733, over 24235.00 frames. ], tot_loss[loss=0.2875, simple_loss=0.3964, pruned_loss=0.08931, over 3059631.52 frames. ], batch size: 257, lr: 6.61e-03, grad_scale: 32.0 2026-09-24 02:04:37,772 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=99693.33333333333, ans=0.125 2026-09-24 02:04:40,292 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=99726.66666666667, ans=0.125 2026-09-24 02:04:45,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=99760.0, ans=0.1 2026-09-24 02:04:48,286 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.312e+02 3.441e+02 4.026e+02 4.721e+02 9.759e+02, threshold=8.053e+02, percent-clipped=2.0 2026-09-24 02:04:49,559 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.67 vs. limit=15.0 2026-09-24 02:04:52,652 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=99793.33333333333, ans=0.1 2026-09-24 02:04:56,164 INFO [train.py:1192] (1/2) Epoch 32, batch 250, loss[loss=0.3354, simple_loss=0.4422, pruned_loss=0.1143, over 24393.00 frames. ], tot_loss[loss=0.2865, simple_loss=0.3953, pruned_loss=0.08888, over 3442829.79 frames. ], batch size: 225, lr: 6.60e-03, grad_scale: 32.0 2026-09-24 02:05:02,201 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=99860.0, ans=0.0 2026-09-24 02:05:04,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=99860.0, ans=0.0 2026-09-24 02:05:11,744 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=99926.66666666667, ans=0.0 2026-09-24 02:05:21,920 INFO [train.py:1192] (1/2) Epoch 32, batch 300, loss[loss=0.2817, simple_loss=0.4011, pruned_loss=0.0812, over 24559.00 frames. ], tot_loss[loss=0.2862, simple_loss=0.3947, pruned_loss=0.08884, over 3757195.62 frames. ], batch size: 204, lr: 6.60e-03, grad_scale: 32.0 2026-09-24 02:05:27,565 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=100026.66666666667, ans=0.0 2026-09-24 02:05:27,582 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=100026.66666666667, ans=0.1 2026-09-24 02:05:29,965 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=100026.66666666667, ans=0.0 2026-09-24 02:05:31,915 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=100060.0, ans=0.125 2026-09-24 02:05:35,126 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=100060.0, ans=0.0 2026-09-24 02:05:40,274 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.472e+02 3.268e+02 3.539e+02 4.255e+02 5.982e+02, threshold=7.078e+02, percent-clipped=0.0 2026-09-24 02:05:43,451 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=100126.66666666667, ans=0.2 2026-09-24 02:05:47,879 INFO [train.py:1192] (1/2) Epoch 32, batch 350, loss[loss=0.2408, simple_loss=0.3456, pruned_loss=0.06796, over 24594.00 frames. ], tot_loss[loss=0.2874, simple_loss=0.3958, pruned_loss=0.08953, over 3999344.69 frames. ], batch size: 137, lr: 6.59e-03, grad_scale: 32.0 2026-09-24 02:05:50,419 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=100160.0, ans=0.2 2026-09-24 02:05:52,228 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=100160.0, ans=0.125 2026-09-24 02:05:54,696 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=100193.33333333333, ans=0.125 2026-09-24 02:05:56,059 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=100193.33333333333, ans=0.04949747468305833 2026-09-24 02:06:06,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=100260.0, ans=0.125 2026-09-24 02:06:10,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=100293.33333333333, ans=0.2 2026-09-24 02:06:12,945 INFO [train.py:1192] (1/2) Epoch 32, batch 400, loss[loss=0.2714, simple_loss=0.3802, pruned_loss=0.08127, over 24573.00 frames. ], tot_loss[loss=0.2859, simple_loss=0.3945, pruned_loss=0.08866, over 4183382.45 frames. ], batch size: 170, lr: 6.59e-03, grad_scale: 32.0 2026-09-24 02:06:28,662 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=100426.66666666667, ans=0.125 2026-09-24 02:06:31,147 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=100426.66666666667, ans=0.0 2026-09-24 02:06:31,447 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.511e+02 3.151e+02 3.623e+02 4.376e+02 7.654e+02, threshold=7.247e+02, percent-clipped=1.0 2026-09-24 02:06:34,508 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=100460.0, ans=0.0 2026-09-24 02:06:38,554 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=100493.33333333333, ans=0.0 2026-09-24 02:06:38,940 INFO [train.py:1192] (1/2) Epoch 32, batch 450, loss[loss=0.2922, simple_loss=0.4042, pruned_loss=0.09007, over 24620.00 frames. ], tot_loss[loss=0.2867, simple_loss=0.3953, pruned_loss=0.0891, over 4322136.98 frames. ], batch size: 175, lr: 6.58e-03, grad_scale: 32.0 2026-09-24 02:06:39,762 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=100493.33333333333, ans=0.125 2026-09-24 02:06:42,467 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=100493.33333333333, ans=0.025 2026-09-24 02:06:47,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=100526.66666666667, ans=0.04949747468305833 2026-09-24 02:06:56,419 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.58 vs. limit=15.0 2026-09-24 02:07:04,781 INFO [train.py:1192] (1/2) Epoch 32, batch 500, loss[loss=0.3371, simple_loss=0.4429, pruned_loss=0.1156, over 24503.00 frames. ], tot_loss[loss=0.2855, simple_loss=0.3941, pruned_loss=0.08846, over 4439712.50 frames. ], batch size: 218, lr: 6.58e-03, grad_scale: 32.0 2026-09-24 02:07:07,691 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.76 vs. limit=22.5 2026-09-24 02:07:23,026 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.464e+02 3.154e+02 3.498e+02 3.927e+02 5.241e+02, threshold=6.996e+02, percent-clipped=0.0 2026-09-24 02:07:24,371 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.95 vs. limit=10.0 2026-09-24 02:07:29,410 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=100793.33333333333, ans=10.0 2026-09-24 02:07:30,716 INFO [train.py:1192] (1/2) Epoch 32, batch 550, loss[loss=0.3101, simple_loss=0.4288, pruned_loss=0.09569, over 24259.00 frames. ], tot_loss[loss=0.2863, simple_loss=0.3948, pruned_loss=0.08889, over 4524391.37 frames. ], batch size: 257, lr: 6.57e-03, grad_scale: 32.0 2026-09-24 02:07:32,818 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=100826.66666666667, ans=0.125 2026-09-24 02:07:40,352 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=100860.0, ans=0.125 2026-09-24 02:07:41,736 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=100893.33333333333, ans=0.1 2026-09-24 02:07:42,713 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=100893.33333333333, ans=0.07 2026-09-24 02:07:42,964 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.02 vs. limit=15.0 2026-09-24 02:07:42,977 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.00 vs. limit=6.0 2026-09-24 02:07:46,488 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=100926.66666666667, ans=0.0 2026-09-24 02:07:46,491 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=100926.66666666667, ans=0.1 2026-09-24 02:07:56,913 INFO [train.py:1192] (1/2) Epoch 32, batch 600, loss[loss=0.2708, simple_loss=0.3945, pruned_loss=0.07356, over 24319.00 frames. ], tot_loss[loss=0.2866, simple_loss=0.3953, pruned_loss=0.08901, over 4590513.21 frames. ], batch size: 234, lr: 6.57e-03, grad_scale: 32.0 2026-09-24 02:08:14,995 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.576e+02 3.196e+02 3.540e+02 4.133e+02 6.002e+02, threshold=7.080e+02, percent-clipped=0.0 2026-09-24 02:08:20,056 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=101126.66666666667, ans=0.125 2026-09-24 02:08:22,482 INFO [train.py:1192] (1/2) Epoch 32, batch 650, loss[loss=0.2558, simple_loss=0.3629, pruned_loss=0.07437, over 24633.00 frames. ], tot_loss[loss=0.2857, simple_loss=0.3943, pruned_loss=0.08851, over 4654851.55 frames. ], batch size: 154, lr: 6.56e-03, grad_scale: 32.0 2026-09-24 02:08:25,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=101160.0, ans=0.2 2026-09-24 02:08:25,557 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=101160.0, ans=0.0 2026-09-24 02:08:27,811 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.45 vs. limit=12.0 2026-09-24 02:08:33,553 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=101226.66666666667, ans=0.1 2026-09-24 02:08:41,201 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.86 vs. limit=15.0 2026-09-24 02:08:45,265 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=101293.33333333333, ans=0.125 2026-09-24 02:08:47,330 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=101293.33333333333, ans=0.125 2026-09-24 02:08:48,679 INFO [train.py:1192] (1/2) Epoch 32, batch 700, loss[loss=0.2758, simple_loss=0.3844, pruned_loss=0.08356, over 24554.00 frames. ], tot_loss[loss=0.2859, simple_loss=0.3949, pruned_loss=0.08844, over 4689776.52 frames. ], batch size: 158, lr: 6.56e-03, grad_scale: 32.0 2026-09-24 02:08:55,494 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=101360.0, ans=0.125 2026-09-24 02:09:01,482 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=101393.33333333333, ans=0.125 2026-09-24 02:09:05,424 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=101426.66666666667, ans=0.1 2026-09-24 02:09:07,204 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.596e+02 3.226e+02 3.648e+02 4.091e+02 6.979e+02, threshold=7.297e+02, percent-clipped=0.0 2026-09-24 02:09:10,137 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=101460.0, ans=0.1 2026-09-24 02:09:12,602 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=101460.0, ans=0.125 2026-09-24 02:09:14,935 INFO [train.py:1192] (1/2) Epoch 32, batch 750, loss[loss=0.2954, simple_loss=0.4051, pruned_loss=0.09288, over 24638.00 frames. ], tot_loss[loss=0.2849, simple_loss=0.394, pruned_loss=0.08794, over 4725869.66 frames. ], batch size: 175, lr: 6.55e-03, grad_scale: 32.0 2026-09-24 02:09:15,562 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=101493.33333333333, ans=0.0 2026-09-24 02:09:23,963 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.51 vs. limit=22.5 2026-09-24 02:09:25,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=101560.0, ans=0.125 2026-09-24 02:09:29,139 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.77 vs. limit=12.0 2026-09-24 02:09:31,417 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=101593.33333333333, ans=0.0 2026-09-24 02:09:32,747 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:09:37,440 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=101626.66666666667, ans=0.0 2026-09-24 02:09:40,624 INFO [train.py:1192] (1/2) Epoch 32, batch 800, loss[loss=0.25, simple_loss=0.3574, pruned_loss=0.07126, over 24546.00 frames. ], tot_loss[loss=0.2847, simple_loss=0.3937, pruned_loss=0.08785, over 4751998.88 frames. ], batch size: 137, lr: 6.55e-03, grad_scale: 32.0 2026-09-24 02:09:47,095 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=101693.33333333333, ans=0.0 2026-09-24 02:09:58,468 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.569e+02 3.280e+02 3.615e+02 4.129e+02 5.868e+02, threshold=7.230e+02, percent-clipped=0.0 2026-09-24 02:10:01,196 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=101793.33333333333, ans=0.125 2026-09-24 02:10:06,169 INFO [train.py:1192] (1/2) Epoch 32, batch 850, loss[loss=0.2871, simple_loss=0.4019, pruned_loss=0.08611, over 24543.00 frames. ], tot_loss[loss=0.2833, simple_loss=0.3926, pruned_loss=0.08697, over 4770063.75 frames. ], batch size: 204, lr: 6.54e-03, grad_scale: 32.0 2026-09-24 02:10:24,512 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=101926.66666666667, ans=0.0 2026-09-24 02:10:29,521 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=101960.0, ans=0.0 2026-09-24 02:10:32,280 INFO [train.py:1192] (1/2) Epoch 32, batch 900, loss[loss=0.2536, simple_loss=0.3581, pruned_loss=0.07461, over 24554.00 frames. ], tot_loss[loss=0.2838, simple_loss=0.393, pruned_loss=0.08728, over 4781922.83 frames. ], batch size: 137, lr: 6.54e-03, grad_scale: 32.0 2026-09-24 02:10:38,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=102026.66666666667, ans=0.0 2026-09-24 02:10:44,602 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=102060.0, ans=0.125 2026-09-24 02:10:44,617 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=102060.0, ans=0.125 2026-09-24 02:10:49,908 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.338e+02 3.264e+02 3.534e+02 4.046e+02 5.970e+02, threshold=7.068e+02, percent-clipped=0.0 2026-09-24 02:10:57,047 INFO [train.py:1192] (1/2) Epoch 32, batch 950, loss[loss=0.3607, simple_loss=0.4186, pruned_loss=0.1514, over 11101.00 frames. ], tot_loss[loss=0.2836, simple_loss=0.3913, pruned_loss=0.08796, over 4712381.83 frames. ], batch size: 333, lr: 6.53e-03, grad_scale: 32.0 2026-09-24 02:10:59,644 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=102160.0, ans=0.125 2026-09-24 02:11:08,241 INFO [train.py:1192] (1/2) Epoch 33, batch 0, loss[loss=0.2534, simple_loss=0.3631, pruned_loss=0.07187, over 24582.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3631, pruned_loss=0.07187, over 24582.00 frames. ], batch size: 137, lr: 6.43e-03, grad_scale: 32.0 2026-09-24 02:11:08,241 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 02:11:17,275 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.1359, 3.0614, 2.6694, 3.9405], device='cuda:1') 2026-09-24 02:11:19,600 INFO [train.py:1224] (1/2) Epoch 33, validation: loss=0.1799, simple_loss=0.2984, pruned_loss=0.03069, over 2564189.00 frames. 2026-09-24 02:11:19,600 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 02:11:33,338 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.59 vs. limit=15.0 2026-09-24 02:11:42,053 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.71 vs. limit=22.5 2026-09-24 02:11:42,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=102320.0, ans=0.2 2026-09-24 02:11:46,277 INFO [train.py:1192] (1/2) Epoch 33, batch 50, loss[loss=0.2378, simple_loss=0.3441, pruned_loss=0.0658, over 24249.00 frames. ], tot_loss[loss=0.2932, simple_loss=0.4022, pruned_loss=0.09212, over 1081610.81 frames. ], batch size: 125, lr: 6.42e-03, grad_scale: 32.0 2026-09-24 02:11:46,913 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=102353.33333333333, ans=0.125 2026-09-24 02:11:58,788 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:11:58,810 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=102420.0, ans=0.125 2026-09-24 02:12:00,101 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.782e+02 3.355e+02 3.885e+02 4.255e+02 6.960e+02, threshold=7.770e+02, percent-clipped=0.0 2026-09-24 02:12:03,148 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=102453.33333333333, ans=0.2 2026-09-24 02:12:03,283 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.97 vs. limit=15.0 2026-09-24 02:12:10,411 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.51 vs. limit=15.0 2026-09-24 02:12:12,167 INFO [train.py:1192] (1/2) Epoch 33, batch 100, loss[loss=0.2606, simple_loss=0.3746, pruned_loss=0.07329, over 24612.00 frames. ], tot_loss[loss=0.2943, simple_loss=0.4052, pruned_loss=0.09172, over 1916525.20 frames. ], batch size: 154, lr: 6.42e-03, grad_scale: 64.0 2026-09-24 02:12:23,007 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=102586.66666666667, ans=0.025 2026-09-24 02:12:27,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=102620.0, ans=0.0 2026-09-24 02:12:33,025 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=102653.33333333333, ans=0.125 2026-09-24 02:12:34,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_ff3.min_abs, batch_count=102653.33333333333, ans=0.2 2026-09-24 02:12:35,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.whiten.whitening_limit, batch_count=102653.33333333333, ans=15.0 2026-09-24 02:12:38,094 INFO [train.py:1192] (1/2) Epoch 33, batch 150, loss[loss=0.2196, simple_loss=0.3267, pruned_loss=0.05621, over 24242.00 frames. ], tot_loss[loss=0.2898, simple_loss=0.399, pruned_loss=0.09023, over 2561745.23 frames. ], batch size: 125, lr: 6.41e-03, grad_scale: 64.0 2026-09-24 02:12:49,890 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=102753.33333333333, ans=0.025 2026-09-24 02:12:51,631 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.186e+02 3.285e+02 3.723e+02 4.193e+02 6.149e+02, threshold=7.446e+02, percent-clipped=0.0 2026-09-24 02:13:03,347 INFO [train.py:1192] (1/2) Epoch 33, batch 200, loss[loss=0.3128, simple_loss=0.4362, pruned_loss=0.09468, over 24220.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.3956, pruned_loss=0.08805, over 3061327.72 frames. ], batch size: 257, lr: 6.41e-03, grad_scale: 64.0 2026-09-24 02:13:03,607 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.96 vs. limit=15.0 2026-09-24 02:13:04,831 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=102853.33333333333, ans=0.125 2026-09-24 02:13:12,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=102886.66666666667, ans=0.025 2026-09-24 02:13:17,775 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=102920.0, ans=0.0 2026-09-24 02:13:17,980 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.whiten.whitening_limit, batch_count=102920.0, ans=15.0 2026-09-24 02:13:25,148 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=102986.66666666667, ans=0.125 2026-09-24 02:13:29,430 INFO [train.py:1192] (1/2) Epoch 33, batch 250, loss[loss=0.3344, simple_loss=0.4456, pruned_loss=0.1116, over 24385.00 frames. ], tot_loss[loss=0.2856, simple_loss=0.3953, pruned_loss=0.08795, over 3444286.87 frames. ], batch size: 225, lr: 6.40e-03, grad_scale: 32.0 2026-09-24 02:13:41,241 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=103086.66666666667, ans=0.0 2026-09-24 02:13:42,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=103086.66666666667, ans=0.0 2026-09-24 02:13:44,468 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.787e+02 3.379e+02 3.953e+02 4.528e+02 8.699e+02, threshold=7.906e+02, percent-clipped=3.0 2026-09-24 02:13:46,915 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=103120.0, ans=0.1 2026-09-24 02:13:55,535 INFO [train.py:1192] (1/2) Epoch 33, batch 300, loss[loss=0.3094, simple_loss=0.4231, pruned_loss=0.09786, over 24557.00 frames. ], tot_loss[loss=0.286, simple_loss=0.3953, pruned_loss=0.08836, over 3758095.12 frames. ], batch size: 204, lr: 6.40e-03, grad_scale: 32.0 2026-09-24 02:13:56,070 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.78 vs. limit=22.5 2026-09-24 02:13:56,462 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=103186.66666666667, ans=0.0 2026-09-24 02:14:01,033 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=103220.0, ans=0.125 2026-09-24 02:14:03,480 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=103220.0, ans=0.125 2026-09-24 02:14:04,366 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=103220.0, ans=0.1 2026-09-24 02:14:09,694 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:14:09,928 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.35 vs. limit=22.5 2026-09-24 02:14:15,496 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=103320.0, ans=0.2 2026-09-24 02:14:21,254 INFO [train.py:1192] (1/2) Epoch 33, batch 350, loss[loss=0.2382, simple_loss=0.3482, pruned_loss=0.06412, over 24576.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.3953, pruned_loss=0.08818, over 3998707.22 frames. ], batch size: 137, lr: 6.40e-03, grad_scale: 32.0 2026-09-24 02:14:28,917 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=103386.66666666667, ans=0.2 2026-09-24 02:14:33,219 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=103420.0, ans=0.1 2026-09-24 02:14:35,895 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.614e+02 3.333e+02 3.657e+02 4.243e+02 7.835e+02, threshold=7.314e+02, percent-clipped=0.0 2026-09-24 02:14:44,541 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=103486.66666666667, ans=0.0 2026-09-24 02:14:46,000 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=103486.66666666667, ans=0.0 2026-09-24 02:14:47,370 INFO [train.py:1192] (1/2) Epoch 33, batch 400, loss[loss=0.3017, simple_loss=0.4089, pruned_loss=0.09723, over 24570.00 frames. ], tot_loss[loss=0.2857, simple_loss=0.3948, pruned_loss=0.08827, over 4184454.74 frames. ], batch size: 170, lr: 6.39e-03, grad_scale: 32.0 2026-09-24 02:14:52,491 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=103553.33333333333, ans=0.09899494936611666 2026-09-24 02:14:52,497 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=103553.33333333333, ans=10.0 2026-09-24 02:15:11,291 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.00 vs. limit=15.0 2026-09-24 02:15:13,092 INFO [train.py:1192] (1/2) Epoch 33, batch 450, loss[loss=0.2929, simple_loss=0.4069, pruned_loss=0.08943, over 24638.00 frames. ], tot_loss[loss=0.2861, simple_loss=0.3952, pruned_loss=0.08851, over 4323807.22 frames. ], batch size: 175, lr: 6.39e-03, grad_scale: 32.0 2026-09-24 02:15:21,147 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=16.70 vs. limit=15.0 2026-09-24 02:15:23,386 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=103753.33333333333, ans=0.025 2026-09-24 02:15:27,676 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.570e+02 3.135e+02 3.583e+02 4.105e+02 5.710e+02, threshold=7.165e+02, percent-clipped=0.0 2026-09-24 02:15:34,511 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=103820.0, ans=0.1 2026-09-24 02:15:39,003 INFO [train.py:1192] (1/2) Epoch 33, batch 500, loss[loss=0.2993, simple_loss=0.4192, pruned_loss=0.08966, over 24525.00 frames. ], tot_loss[loss=0.2842, simple_loss=0.3933, pruned_loss=0.08751, over 4440140.28 frames. ], batch size: 218, lr: 6.38e-03, grad_scale: 32.0 2026-09-24 02:15:49,012 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=103920.0, ans=0.125 2026-09-24 02:15:51,482 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=103920.0, ans=0.125 2026-09-24 02:15:52,523 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=103920.0, ans=0.1 2026-09-24 02:15:55,254 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=103953.33333333333, ans=0.0 2026-09-24 02:15:59,503 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=103986.66666666667, ans=0.2 2026-09-24 02:16:00,052 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=103986.66666666667, ans=0.2 2026-09-24 02:16:03,235 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.64 vs. limit=22.5 2026-09-24 02:16:04,928 INFO [train.py:1192] (1/2) Epoch 33, batch 550, loss[loss=0.3021, simple_loss=0.4211, pruned_loss=0.09157, over 24291.00 frames. ], tot_loss[loss=0.2859, simple_loss=0.3947, pruned_loss=0.0886, over 4524406.45 frames. ], batch size: 257, lr: 6.38e-03, grad_scale: 32.0 2026-09-24 02:16:09,528 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=104020.0, ans=0.0 2026-09-24 02:16:09,612 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=104020.0, ans=15.0 2026-09-24 02:16:19,434 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.554e+02 3.273e+02 3.583e+02 4.011e+02 7.398e+02, threshold=7.166e+02, percent-clipped=1.0 2026-09-24 02:16:19,567 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=104086.66666666667, ans=0.0 2026-09-24 02:16:20,048 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=104120.0, ans=0.0 2026-09-24 02:16:22,140 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.27 vs. limit=6.0 2026-09-24 02:16:23,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=104120.0, ans=0.1 2026-09-24 02:16:23,450 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=104120.0, ans=0.125 2026-09-24 02:16:30,479 INFO [train.py:1192] (1/2) Epoch 33, batch 600, loss[loss=0.3073, simple_loss=0.4293, pruned_loss=0.09263, over 24294.00 frames. ], tot_loss[loss=0.2861, simple_loss=0.3951, pruned_loss=0.0886, over 4591817.59 frames. ], batch size: 234, lr: 6.37e-03, grad_scale: 32.0 2026-09-24 02:16:56,633 INFO [train.py:1192] (1/2) Epoch 33, batch 650, loss[loss=0.2722, simple_loss=0.3789, pruned_loss=0.08271, over 24588.00 frames. ], tot_loss[loss=0.2844, simple_loss=0.3936, pruned_loss=0.08762, over 4655770.93 frames. ], batch size: 154, lr: 6.37e-03, grad_scale: 32.0 2026-09-24 02:17:11,055 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.600e+02 3.293e+02 3.637e+02 4.365e+02 6.291e+02, threshold=7.274e+02, percent-clipped=0.0 2026-09-24 02:17:22,478 INFO [train.py:1192] (1/2) Epoch 33, batch 700, loss[loss=0.2744, simple_loss=0.3833, pruned_loss=0.08273, over 24552.00 frames. ], tot_loss[loss=0.2851, simple_loss=0.3945, pruned_loss=0.08782, over 4692131.64 frames. ], batch size: 158, lr: 6.36e-03, grad_scale: 32.0 2026-09-24 02:17:23,175 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=104520.0, ans=0.1 2026-09-24 02:17:33,545 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.71 vs. limit=12.0 2026-09-24 02:17:38,879 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=104620.0, ans=0.0 2026-09-24 02:17:42,091 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:17:45,038 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=104653.33333333333, ans=0.125 2026-09-24 02:17:46,232 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=104653.33333333333, ans=0.125 2026-09-24 02:17:48,757 INFO [train.py:1192] (1/2) Epoch 33, batch 750, loss[loss=0.2804, simple_loss=0.3988, pruned_loss=0.08099, over 24611.00 frames. ], tot_loss[loss=0.2842, simple_loss=0.3934, pruned_loss=0.08752, over 4727976.92 frames. ], batch size: 175, lr: 6.36e-03, grad_scale: 32.0 2026-09-24 02:18:03,151 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.772e+02 3.276e+02 3.703e+02 4.111e+02 6.270e+02, threshold=7.406e+02, percent-clipped=0.0 2026-09-24 02:18:14,148 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=104853.33333333333, ans=0.0 2026-09-24 02:18:14,585 INFO [train.py:1192] (1/2) Epoch 33, batch 800, loss[loss=0.2499, simple_loss=0.3573, pruned_loss=0.0713, over 24544.00 frames. ], tot_loss[loss=0.2836, simple_loss=0.3929, pruned_loss=0.08714, over 4753485.99 frames. ], batch size: 137, lr: 6.35e-03, grad_scale: 32.0 2026-09-24 02:18:15,729 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=104853.33333333333, ans=0.0 2026-09-24 02:18:18,551 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=104853.33333333333, ans=0.125 2026-09-24 02:18:27,563 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.98 vs. limit=15.0 2026-09-24 02:18:28,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=104920.0, ans=0.0 2026-09-24 02:18:32,864 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=104953.33333333333, ans=0.0 2026-09-24 02:18:40,257 INFO [train.py:1192] (1/2) Epoch 33, batch 850, loss[loss=0.3041, simple_loss=0.4221, pruned_loss=0.09309, over 24551.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.3921, pruned_loss=0.08685, over 4772341.97 frames. ], batch size: 204, lr: 6.35e-03, grad_scale: 16.0 2026-09-24 02:18:49,336 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=105053.33333333333, ans=0.125 2026-09-24 02:18:55,062 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.682e+02 3.209e+02 3.625e+02 3.953e+02 5.601e+02, threshold=7.250e+02, percent-clipped=0.0 2026-09-24 02:18:59,974 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=105120.0, ans=0.0 2026-09-24 02:19:05,837 INFO [train.py:1192] (1/2) Epoch 33, batch 900, loss[loss=0.2224, simple_loss=0.3433, pruned_loss=0.0508, over 24563.00 frames. ], tot_loss[loss=0.2834, simple_loss=0.3925, pruned_loss=0.08713, over 4783149.07 frames. ], batch size: 137, lr: 6.34e-03, grad_scale: 16.0 2026-09-24 02:19:05,930 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=105186.66666666667, ans=0.0 2026-09-24 02:19:17,935 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=105253.33333333333, ans=0.0 2026-09-24 02:19:27,267 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=105320.0, ans=0.125 2026-09-24 02:19:27,267 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=105320.0, ans=0.125 2026-09-24 02:19:30,975 INFO [train.py:1192] (1/2) Epoch 33, batch 950, loss[loss=0.3635, simple_loss=0.422, pruned_loss=0.1526, over 11397.00 frames. ], tot_loss[loss=0.2844, simple_loss=0.3919, pruned_loss=0.08848, over 4709431.94 frames. ], batch size: 333, lr: 6.34e-03, grad_scale: 16.0 2026-09-24 02:19:33,622 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=105353.33333333333, ans=0.025 2026-09-24 02:20:14,633 INFO [train.py:1192] (1/2) Epoch 34, batch 0, loss[loss=0.2477, simple_loss=0.364, pruned_loss=0.0657, over 24556.00 frames. ], tot_loss[loss=0.2477, simple_loss=0.364, pruned_loss=0.0657, over 24556.00 frames. ], batch size: 137, lr: 6.24e-03, grad_scale: 32.0 2026-09-24 02:20:14,633 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 02:20:16,982 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.8768, 3.3592, 3.2661, 2.7727, 3.4012, 2.8951, 3.3223, 3.0140], device='cuda:1') 2026-09-24 02:20:26,323 INFO [train.py:1224] (1/2) Epoch 34, validation: loss=0.1791, simple_loss=0.2978, pruned_loss=0.03024, over 2564189.00 frames. 2026-09-24 02:20:26,323 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 02:20:37,071 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.525e+02 3.365e+02 3.705e+02 4.305e+02 8.296e+02, threshold=7.410e+02, percent-clipped=2.0 2026-09-24 02:20:45,964 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.37 vs. limit=22.5 2026-09-24 02:20:47,877 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=105513.33333333333, ans=0.0 2026-09-24 02:20:51,248 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=105513.33333333333, ans=0.125 2026-09-24 02:20:52,267 INFO [train.py:1192] (1/2) Epoch 34, batch 50, loss[loss=0.2294, simple_loss=0.3337, pruned_loss=0.0626, over 24318.00 frames. ], tot_loss[loss=0.2891, simple_loss=0.3985, pruned_loss=0.08992, over 1082041.19 frames. ], batch size: 125, lr: 6.24e-03, grad_scale: 32.0 2026-09-24 02:21:18,025 INFO [train.py:1192] (1/2) Epoch 34, batch 100, loss[loss=0.2856, simple_loss=0.3918, pruned_loss=0.08966, over 24612.00 frames. ], tot_loss[loss=0.2918, simple_loss=0.4022, pruned_loss=0.09069, over 1916053.84 frames. ], batch size: 154, lr: 6.23e-03, grad_scale: 32.0 2026-09-24 02:21:23,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=105746.66666666667, ans=0.0 2026-09-24 02:21:28,609 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.684e+02 3.438e+02 3.805e+02 4.609e+02 7.277e+02, threshold=7.610e+02, percent-clipped=0.0 2026-09-24 02:21:33,389 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=105813.33333333333, ans=0.025 2026-09-24 02:21:33,461 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.63 vs. limit=22.5 2026-09-24 02:21:34,785 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=105813.33333333333, ans=0.125 2026-09-24 02:21:36,217 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=105813.33333333333, ans=0.2 2026-09-24 02:21:36,787 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.91 vs. limit=22.5 2026-09-24 02:21:38,173 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=105846.66666666667, ans=0.125 2026-09-24 02:21:43,485 INFO [train.py:1192] (1/2) Epoch 34, batch 150, loss[loss=0.2286, simple_loss=0.3383, pruned_loss=0.05947, over 24209.00 frames. ], tot_loss[loss=0.2852, simple_loss=0.3956, pruned_loss=0.08742, over 2561105.90 frames. ], batch size: 125, lr: 6.23e-03, grad_scale: 32.0 2026-09-24 02:21:47,178 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=105880.0, ans=0.125 2026-09-24 02:21:48,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=105913.33333333333, ans=0.2 2026-09-24 02:21:55,143 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=105946.66666666667, ans=0.2 2026-09-24 02:22:09,522 INFO [train.py:1192] (1/2) Epoch 34, batch 200, loss[loss=0.286, simple_loss=0.4159, pruned_loss=0.07806, over 24190.00 frames. ], tot_loss[loss=0.2835, simple_loss=0.3938, pruned_loss=0.08656, over 3060402.74 frames. ], batch size: 257, lr: 6.22e-03, grad_scale: 32.0 2026-09-24 02:22:11,721 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:22:20,345 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.605e+02 3.582e+02 4.182e+02 5.101e+02 7.909e+02, threshold=8.363e+02, percent-clipped=1.0 2026-09-24 02:22:20,618 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.67 vs. limit=22.5 2026-09-24 02:22:21,491 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=106113.33333333333, ans=0.0 2026-09-24 02:22:35,453 INFO [train.py:1192] (1/2) Epoch 34, batch 250, loss[loss=0.3245, simple_loss=0.4403, pruned_loss=0.1044, over 24375.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3931, pruned_loss=0.08642, over 3443542.77 frames. ], batch size: 225, lr: 6.22e-03, grad_scale: 32.0 2026-09-24 02:22:41,282 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=106246.66666666667, ans=0.125 2026-09-24 02:22:41,709 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=106246.66666666667, ans=0.2 2026-09-24 02:22:48,692 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=106280.0, ans=0.025 2026-09-24 02:22:49,212 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.20 vs. limit=15.0 2026-09-24 02:22:51,565 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=106313.33333333333, ans=0.2 2026-09-24 02:22:55,035 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=106313.33333333333, ans=0.125 2026-09-24 02:23:01,170 INFO [train.py:1192] (1/2) Epoch 34, batch 300, loss[loss=0.2961, simple_loss=0.4122, pruned_loss=0.09001, over 24531.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3928, pruned_loss=0.0862, over 3756528.92 frames. ], batch size: 204, lr: 6.21e-03, grad_scale: 32.0 2026-09-24 02:23:07,347 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.64 vs. limit=15.0 2026-09-24 02:23:12,089 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.676e+02 3.333e+02 3.803e+02 4.108e+02 6.453e+02, threshold=7.607e+02, percent-clipped=0.0 2026-09-24 02:23:12,205 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=106446.66666666667, ans=0.125 2026-09-24 02:23:21,892 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:23:26,967 INFO [train.py:1192] (1/2) Epoch 34, batch 350, loss[loss=0.24, simple_loss=0.3484, pruned_loss=0.06576, over 24556.00 frames. ], tot_loss[loss=0.2834, simple_loss=0.3936, pruned_loss=0.0866, over 3997686.44 frames. ], batch size: 137, lr: 6.21e-03, grad_scale: 32.0 2026-09-24 02:23:31,092 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=106546.66666666667, ans=0.125 2026-09-24 02:23:33,697 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=106580.0, ans=0.2 2026-09-24 02:23:37,921 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=106613.33333333333, ans=0.0 2026-09-24 02:23:39,191 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=106613.33333333333, ans=0.0 2026-09-24 02:23:44,087 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=106646.66666666667, ans=0.125 2026-09-24 02:23:50,190 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=106680.0, ans=0.1 2026-09-24 02:23:53,629 INFO [train.py:1192] (1/2) Epoch 34, batch 400, loss[loss=0.3003, simple_loss=0.405, pruned_loss=0.09784, over 24547.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3931, pruned_loss=0.08639, over 4181573.08 frames. ], batch size: 170, lr: 6.20e-03, grad_scale: 32.0 2026-09-24 02:24:04,410 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.466e+02 3.230e+02 3.623e+02 4.018e+02 6.105e+02, threshold=7.247e+02, percent-clipped=0.0 2026-09-24 02:24:10,738 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=106813.33333333333, ans=0.2 2026-09-24 02:24:11,194 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=106813.33333333333, ans=0.125 2026-09-24 02:24:16,753 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=106846.66666666667, ans=0.0 2026-09-24 02:24:19,490 INFO [train.py:1192] (1/2) Epoch 34, batch 450, loss[loss=0.3166, simple_loss=0.4227, pruned_loss=0.1053, over 24612.00 frames. ], tot_loss[loss=0.2834, simple_loss=0.3935, pruned_loss=0.08664, over 4319045.61 frames. ], batch size: 175, lr: 6.20e-03, grad_scale: 32.0 2026-09-24 02:24:21,616 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.69 vs. limit=15.0 2026-09-24 02:24:38,037 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.13 vs. limit=15.0 2026-09-24 02:24:39,026 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=107013.33333333333, ans=0.1 2026-09-24 02:24:41,831 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=107013.33333333333, ans=0.0 2026-09-24 02:24:43,287 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=107013.33333333333, ans=10.0 2026-09-24 02:24:44,674 INFO [train.py:1192] (1/2) Epoch 34, batch 500, loss[loss=0.3089, simple_loss=0.4273, pruned_loss=0.09527, over 24521.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3924, pruned_loss=0.08643, over 4437817.36 frames. ], batch size: 218, lr: 6.19e-03, grad_scale: 32.0 2026-09-24 02:24:55,873 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.612e+02 3.167e+02 3.567e+02 4.304e+02 6.742e+02, threshold=7.134e+02, percent-clipped=0.0 2026-09-24 02:25:04,495 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=107146.66666666667, ans=0.0 2026-09-24 02:25:10,541 INFO [train.py:1192] (1/2) Epoch 34, batch 550, loss[loss=0.2832, simple_loss=0.4047, pruned_loss=0.0808, over 24265.00 frames. ], tot_loss[loss=0.2831, simple_loss=0.3928, pruned_loss=0.08672, over 4522554.91 frames. ], batch size: 257, lr: 6.19e-03, grad_scale: 32.0 2026-09-24 02:25:13,507 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=107213.33333333333, ans=0.95 2026-09-24 02:25:14,482 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.min_positive, batch_count=107213.33333333333, ans=0.05 2026-09-24 02:25:22,359 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.42 vs. limit=10.0 2026-09-24 02:25:28,226 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=107313.33333333333, ans=0.0 2026-09-24 02:25:32,484 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=107346.66666666667, ans=0.125 2026-09-24 02:25:33,491 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=107346.66666666667, ans=0.125 2026-09-24 02:25:36,126 INFO [train.py:1192] (1/2) Epoch 34, batch 600, loss[loss=0.2989, simple_loss=0.4245, pruned_loss=0.0867, over 24313.00 frames. ], tot_loss[loss=0.283, simple_loss=0.393, pruned_loss=0.0865, over 4589202.22 frames. ], batch size: 234, lr: 6.19e-03, grad_scale: 32.0 2026-09-24 02:25:46,841 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.437e+02 3.136e+02 3.565e+02 4.273e+02 7.446e+02, threshold=7.130e+02, percent-clipped=2.0 2026-09-24 02:26:01,687 INFO [train.py:1192] (1/2) Epoch 34, batch 650, loss[loss=0.2698, simple_loss=0.3769, pruned_loss=0.08131, over 24564.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3919, pruned_loss=0.08586, over 4653960.05 frames. ], batch size: 154, lr: 6.18e-03, grad_scale: 32.0 2026-09-24 02:26:08,046 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.13 vs. limit=15.0 2026-09-24 02:26:11,820 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.58 vs. limit=15.0 2026-09-24 02:26:19,837 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=7.10 vs. limit=10.0 2026-09-24 02:26:23,413 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.84 vs. limit=15.0 2026-09-24 02:26:25,551 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.96 vs. limit=15.0 2026-09-24 02:26:27,331 INFO [train.py:1192] (1/2) Epoch 34, batch 700, loss[loss=0.2792, simple_loss=0.3834, pruned_loss=0.08745, over 24554.00 frames. ], tot_loss[loss=0.2818, simple_loss=0.3922, pruned_loss=0.08568, over 4691524.00 frames. ], batch size: 158, lr: 6.18e-03, grad_scale: 32.0 2026-09-24 02:26:27,444 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=107713.33333333333, ans=0.1 2026-09-24 02:26:28,312 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=107713.33333333333, ans=0.125 2026-09-24 02:26:28,333 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=107713.33333333333, ans=0.125 2026-09-24 02:26:38,061 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=107780.0, ans=0.1 2026-09-24 02:26:38,487 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.549e+02 3.203e+02 3.581e+02 4.318e+02 6.316e+02, threshold=7.163e+02, percent-clipped=0.0 2026-09-24 02:26:43,539 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=107813.33333333333, ans=0.125 2026-09-24 02:26:51,461 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=107846.66666666667, ans=0.125 2026-09-24 02:26:53,936 INFO [train.py:1192] (1/2) Epoch 34, batch 750, loss[loss=0.2857, simple_loss=0.3998, pruned_loss=0.08578, over 24634.00 frames. ], tot_loss[loss=0.2821, simple_loss=0.392, pruned_loss=0.08614, over 4727553.55 frames. ], batch size: 175, lr: 6.17e-03, grad_scale: 32.0 2026-09-24 02:27:05,205 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=107946.66666666667, ans=0.125 2026-09-24 02:27:19,625 INFO [train.py:1192] (1/2) Epoch 34, batch 800, loss[loss=0.2684, simple_loss=0.3674, pruned_loss=0.08468, over 24533.00 frames. ], tot_loss[loss=0.2822, simple_loss=0.392, pruned_loss=0.08621, over 4753076.69 frames. ], batch size: 137, lr: 6.17e-03, grad_scale: 32.0 2026-09-24 02:27:26,125 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.54 vs. limit=15.0 2026-09-24 02:27:29,927 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.511e+02 3.141e+02 3.618e+02 4.122e+02 6.477e+02, threshold=7.235e+02, percent-clipped=0.0 2026-09-24 02:27:30,803 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=108113.33333333333, ans=0.125 2026-09-24 02:27:31,303 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=108113.33333333333, ans=0.125 2026-09-24 02:27:33,170 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff3.min_abs, batch_count=108113.33333333333, ans=0.2 2026-09-24 02:27:34,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=108146.66666666667, ans=0.2 2026-09-24 02:27:37,687 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=108146.66666666667, ans=0.1 2026-09-24 02:27:45,346 INFO [train.py:1192] (1/2) Epoch 34, batch 850, loss[loss=0.3091, simple_loss=0.4204, pruned_loss=0.09896, over 24539.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.3922, pruned_loss=0.08663, over 4771167.07 frames. ], batch size: 204, lr: 6.16e-03, grad_scale: 32.0 2026-09-24 02:27:47,048 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=108213.33333333333, ans=0.0 2026-09-24 02:27:52,330 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=108246.66666666667, ans=0.125 2026-09-24 02:27:57,588 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=108280.0, ans=0.125 2026-09-24 02:28:11,293 INFO [train.py:1192] (1/2) Epoch 34, batch 900, loss[loss=0.2428, simple_loss=0.3546, pruned_loss=0.06545, over 24578.00 frames. ], tot_loss[loss=0.2831, simple_loss=0.3927, pruned_loss=0.08682, over 4782600.86 frames. ], batch size: 137, lr: 6.16e-03, grad_scale: 32.0 2026-09-24 02:28:22,342 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.549e+02 3.205e+02 3.680e+02 4.328e+02 6.277e+02, threshold=7.359e+02, percent-clipped=0.0 2026-09-24 02:28:23,479 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=108446.66666666667, ans=0.125 2026-09-24 02:28:24,555 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=11.27 vs. limit=15.0 2026-09-24 02:28:30,094 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=108480.0, ans=0.2 2026-09-24 02:28:36,638 INFO [train.py:1192] (1/2) Epoch 34, batch 950, loss[loss=0.407, simple_loss=0.453, pruned_loss=0.1804, over 10563.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3909, pruned_loss=0.08719, over 4713529.01 frames. ], batch size: 333, lr: 6.15e-03, grad_scale: 32.0 2026-09-24 02:28:47,828 INFO [train.py:1192] (1/2) Epoch 35, batch 0, loss[loss=0.26, simple_loss=0.3621, pruned_loss=0.07893, over 24565.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3621, pruned_loss=0.07893, over 24565.00 frames. ], batch size: 137, lr: 6.06e-03, grad_scale: 32.0 2026-09-24 02:28:47,828 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 02:28:57,893 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0613, 3.7042, 3.5464, 3.1222], device='cuda:1') 2026-09-24 02:28:59,562 INFO [train.py:1224] (1/2) Epoch 35, validation: loss=0.1781, simple_loss=0.2962, pruned_loss=0.02996, over 2564189.00 frames. 2026-09-24 02:28:59,563 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 02:29:18,972 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff3.min_abs, batch_count=108673.33333333333, ans=0.2 2026-09-24 02:29:24,219 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.49 vs. limit=15.0 2026-09-24 02:29:24,973 INFO [train.py:1192] (1/2) Epoch 35, batch 50, loss[loss=0.2299, simple_loss=0.3305, pruned_loss=0.06464, over 24257.00 frames. ], tot_loss[loss=0.2883, simple_loss=0.3973, pruned_loss=0.08962, over 1080189.75 frames. ], batch size: 125, lr: 6.06e-03, grad_scale: 32.0 2026-09-24 02:29:25,051 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=108740.0, ans=0.1 2026-09-24 02:29:32,001 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.435e+02 3.745e+02 4.015e+02 4.535e+02 7.894e+02, threshold=8.030e+02, percent-clipped=1.0 2026-09-24 02:29:34,657 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=108806.66666666667, ans=0.1 2026-09-24 02:29:43,580 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=108840.0, ans=0.05 2026-09-24 02:29:49,724 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=108873.33333333333, ans=0.035 2026-09-24 02:29:50,622 INFO [train.py:1192] (1/2) Epoch 35, batch 100, loss[loss=0.2764, simple_loss=0.3835, pruned_loss=0.08467, over 24579.00 frames. ], tot_loss[loss=0.2892, simple_loss=0.4002, pruned_loss=0.08909, over 1914262.22 frames. ], batch size: 154, lr: 6.05e-03, grad_scale: 32.0 2026-09-24 02:29:50,949 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.42 vs. limit=15.0 2026-09-24 02:30:00,008 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=108940.0, ans=0.0 2026-09-24 02:30:04,725 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:30:13,817 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:30:16,606 INFO [train.py:1192] (1/2) Epoch 35, batch 150, loss[loss=0.2181, simple_loss=0.3282, pruned_loss=0.05397, over 24286.00 frames. ], tot_loss[loss=0.2834, simple_loss=0.394, pruned_loss=0.08636, over 2559980.20 frames. ], batch size: 125, lr: 6.05e-03, grad_scale: 32.0 2026-09-24 02:30:21,664 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=109106.66666666667, ans=0.035 2026-09-24 02:30:23,450 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.624e+02 3.181e+02 3.537e+02 3.997e+02 6.502e+02, threshold=7.075e+02, percent-clipped=0.0 2026-09-24 02:30:29,735 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=109140.0, ans=0.0 2026-09-24 02:30:37,592 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=109206.66666666667, ans=0.2 2026-09-24 02:30:41,710 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=109240.0, ans=0.0 2026-09-24 02:30:42,069 INFO [train.py:1192] (1/2) Epoch 35, batch 200, loss[loss=0.3235, simple_loss=0.4368, pruned_loss=0.1051, over 24198.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3924, pruned_loss=0.08535, over 3058537.49 frames. ], batch size: 257, lr: 6.05e-03, grad_scale: 32.0 2026-09-24 02:30:43,755 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=109240.0, ans=0.0 2026-09-24 02:30:48,329 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.54 vs. limit=22.5 2026-09-24 02:30:50,125 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.89 vs. limit=15.0 2026-09-24 02:30:58,879 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=109340.0, ans=0.125 2026-09-24 02:31:07,729 INFO [train.py:1192] (1/2) Epoch 35, batch 250, loss[loss=0.2944, simple_loss=0.4163, pruned_loss=0.0862, over 24357.00 frames. ], tot_loss[loss=0.2821, simple_loss=0.3923, pruned_loss=0.0859, over 3443817.24 frames. ], batch size: 225, lr: 6.04e-03, grad_scale: 32.0 2026-09-24 02:31:13,808 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=109440.0, ans=0.125 2026-09-24 02:31:14,647 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=109440.0, ans=0.125 2026-09-24 02:31:15,039 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.659e+02 3.448e+02 3.807e+02 4.569e+02 7.598e+02, threshold=7.614e+02, percent-clipped=2.0 2026-09-24 02:31:21,501 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=109473.33333333333, ans=0.04949747468305833 2026-09-24 02:31:22,062 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=109473.33333333333, ans=0.125 2026-09-24 02:31:24,568 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=109506.66666666667, ans=0.125 2026-09-24 02:31:29,636 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=109540.0, ans=0.125 2026-09-24 02:31:33,591 INFO [train.py:1192] (1/2) Epoch 35, batch 300, loss[loss=0.2961, simple_loss=0.4168, pruned_loss=0.08767, over 24506.00 frames. ], tot_loss[loss=0.2804, simple_loss=0.3911, pruned_loss=0.08487, over 3757145.44 frames. ], batch size: 204, lr: 6.04e-03, grad_scale: 32.0 2026-09-24 02:31:43,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=109640.0, ans=0.0 2026-09-24 02:31:44,182 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=109640.0, ans=0.125 2026-09-24 02:31:47,019 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=109640.0, ans=0.025 2026-09-24 02:31:53,824 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=109706.66666666667, ans=0.125 2026-09-24 02:31:59,373 INFO [train.py:1192] (1/2) Epoch 35, batch 350, loss[loss=0.24, simple_loss=0.3525, pruned_loss=0.06371, over 24588.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.392, pruned_loss=0.0856, over 3997369.45 frames. ], batch size: 137, lr: 6.03e-03, grad_scale: 32.0 2026-09-24 02:32:00,460 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=109740.0, ans=0.1 2026-09-24 02:32:05,128 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=109773.33333333333, ans=0.1 2026-09-24 02:32:06,441 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.601e+02 3.296e+02 3.647e+02 4.243e+02 6.887e+02, threshold=7.294e+02, percent-clipped=0.0 2026-09-24 02:32:08,659 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.76 vs. limit=15.0 2026-09-24 02:32:09,576 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=109806.66666666667, ans=0.0 2026-09-24 02:32:13,804 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=109806.66666666667, ans=0.5 2026-09-24 02:32:17,850 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=109840.0, ans=0.0 2026-09-24 02:32:24,378 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.76 vs. limit=15.0 2026-09-24 02:32:25,362 INFO [train.py:1192] (1/2) Epoch 35, batch 400, loss[loss=0.3008, simple_loss=0.4052, pruned_loss=0.09815, over 24560.00 frames. ], tot_loss[loss=0.2822, simple_loss=0.3923, pruned_loss=0.08603, over 4182755.67 frames. ], batch size: 170, lr: 6.03e-03, grad_scale: 32.0 2026-09-24 02:32:28,934 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.78 vs. limit=22.5 2026-09-24 02:32:49,746 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.53 vs. limit=15.0 2026-09-24 02:32:51,299 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.37 vs. limit=15.0 2026-09-24 02:32:51,504 INFO [train.py:1192] (1/2) Epoch 35, batch 450, loss[loss=0.3112, simple_loss=0.4127, pruned_loss=0.1049, over 24629.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.3928, pruned_loss=0.0865, over 4321646.86 frames. ], batch size: 175, lr: 6.02e-03, grad_scale: 32.0 2026-09-24 02:32:51,596 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=110073.33333333333, ans=0.125 2026-09-24 02:32:54,123 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=110073.33333333333, ans=0.2 2026-09-24 02:32:58,611 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.579e+02 3.247e+02 3.647e+02 4.122e+02 5.699e+02, threshold=7.294e+02, percent-clipped=0.0 2026-09-24 02:33:09,296 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=110173.33333333333, ans=0.1 2026-09-24 02:33:15,994 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=110206.66666666667, ans=0.125 2026-09-24 02:33:17,574 INFO [train.py:1192] (1/2) Epoch 35, batch 500, loss[loss=0.301, simple_loss=0.4157, pruned_loss=0.09314, over 24515.00 frames. ], tot_loss[loss=0.2823, simple_loss=0.392, pruned_loss=0.08628, over 4439162.22 frames. ], batch size: 218, lr: 6.02e-03, grad_scale: 32.0 2026-09-24 02:33:27,366 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.73 vs. limit=15.0 2026-09-24 02:33:34,513 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=110340.0, ans=0.0 2026-09-24 02:33:43,413 INFO [train.py:1192] (1/2) Epoch 35, batch 550, loss[loss=0.3183, simple_loss=0.433, pruned_loss=0.1018, over 24249.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.3923, pruned_loss=0.08665, over 4523433.73 frames. ], batch size: 257, lr: 6.02e-03, grad_scale: 32.0 2026-09-24 02:33:50,291 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=110440.0, ans=0.025 2026-09-24 02:33:50,687 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.628e+02 3.226e+02 3.467e+02 4.015e+02 5.376e+02, threshold=6.935e+02, percent-clipped=0.0 2026-09-24 02:33:52,900 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:33:53,988 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten.whitening_limit, batch_count=110473.33333333333, ans=22.5 2026-09-24 02:34:08,867 INFO [train.py:1192] (1/2) Epoch 35, batch 600, loss[loss=0.3221, simple_loss=0.4297, pruned_loss=0.1072, over 24352.00 frames. ], tot_loss[loss=0.2829, simple_loss=0.3927, pruned_loss=0.08651, over 4589442.04 frames. ], batch size: 234, lr: 6.01e-03, grad_scale: 32.0 2026-09-24 02:34:21,327 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=110640.0, ans=0.125 2026-09-24 02:34:35,030 INFO [train.py:1192] (1/2) Epoch 35, batch 650, loss[loss=0.2745, simple_loss=0.3785, pruned_loss=0.08521, over 24612.00 frames. ], tot_loss[loss=0.282, simple_loss=0.392, pruned_loss=0.08598, over 4654212.41 frames. ], batch size: 154, lr: 6.01e-03, grad_scale: 32.0 2026-09-24 02:34:35,129 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=110740.0, ans=0.1 2026-09-24 02:34:36,107 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=110740.0, ans=0.0 2026-09-24 02:34:39,489 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=110740.0, ans=0.0 2026-09-24 02:34:42,653 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.489e+02 3.190e+02 3.547e+02 4.027e+02 6.396e+02, threshold=7.094e+02, percent-clipped=0.0 2026-09-24 02:34:47,765 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=110806.66666666667, ans=0.1 2026-09-24 02:34:51,017 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.36 vs. limit=15.0 2026-09-24 02:34:54,005 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=110840.0, ans=0.1 2026-09-24 02:34:58,474 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.33 vs. limit=10.0 2026-09-24 02:35:01,168 INFO [train.py:1192] (1/2) Epoch 35, batch 700, loss[loss=0.2717, simple_loss=0.3806, pruned_loss=0.08143, over 24552.00 frames. ], tot_loss[loss=0.2826, simple_loss=0.3929, pruned_loss=0.08611, over 4689059.79 frames. ], batch size: 158, lr: 6.00e-03, grad_scale: 32.0 2026-09-24 02:35:05,534 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=110906.66666666667, ans=0.125 2026-09-24 02:35:10,111 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=110940.0, ans=0.125 2026-09-24 02:35:11,248 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=110973.33333333333, ans=0.0 2026-09-24 02:35:22,131 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=111040.0, ans=0.125 2026-09-24 02:35:26,926 INFO [train.py:1192] (1/2) Epoch 35, batch 750, loss[loss=0.2698, simple_loss=0.3905, pruned_loss=0.07451, over 24641.00 frames. ], tot_loss[loss=0.2817, simple_loss=0.392, pruned_loss=0.08573, over 4725538.98 frames. ], batch size: 175, lr: 6.00e-03, grad_scale: 32.0 2026-09-24 02:35:28,377 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=111073.33333333333, ans=0.0 2026-09-24 02:35:29,255 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=111073.33333333333, ans=0.0 2026-09-24 02:35:29,633 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=111073.33333333333, ans=0.025 2026-09-24 02:35:32,889 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=111106.66666666667, ans=0.0 2026-09-24 02:35:34,285 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.475e+02 3.230e+02 3.760e+02 4.436e+02 7.049e+02, threshold=7.521e+02, percent-clipped=0.0 2026-09-24 02:35:40,298 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.87 vs. limit=15.0 2026-09-24 02:35:42,097 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:35:43,562 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=111173.33333333333, ans=0.125 2026-09-24 02:35:52,726 INFO [train.py:1192] (1/2) Epoch 35, batch 800, loss[loss=0.2456, simple_loss=0.3561, pruned_loss=0.06749, over 24553.00 frames. ], tot_loss[loss=0.2804, simple_loss=0.391, pruned_loss=0.08495, over 4751843.63 frames. ], batch size: 137, lr: 5.99e-03, grad_scale: 32.0 2026-09-24 02:36:01,130 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.85 vs. limit=15.0 2026-09-24 02:36:01,717 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=111273.33333333333, ans=0.05 2026-09-24 02:36:09,819 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=111340.0, ans=0.04949747468305833 2026-09-24 02:36:16,554 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=111373.33333333333, ans=0.125 2026-09-24 02:36:19,260 INFO [train.py:1192] (1/2) Epoch 35, batch 850, loss[loss=0.3214, simple_loss=0.4351, pruned_loss=0.1038, over 24571.00 frames. ], tot_loss[loss=0.2798, simple_loss=0.3905, pruned_loss=0.08462, over 4770298.83 frames. ], batch size: 204, lr: 5.99e-03, grad_scale: 32.0 2026-09-24 02:36:25,837 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=111440.0, ans=0.0 2026-09-24 02:36:26,218 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.635e+02 3.255e+02 3.671e+02 4.302e+02 6.594e+02, threshold=7.341e+02, percent-clipped=0.0 2026-09-24 02:36:26,451 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.49 vs. limit=15.0 2026-09-24 02:36:30,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=111473.33333333333, ans=0.09899494936611666 2026-09-24 02:36:31,459 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.15 vs. limit=15.0 2026-09-24 02:36:39,323 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=111540.0, ans=0.1 2026-09-24 02:36:44,998 INFO [train.py:1192] (1/2) Epoch 35, batch 900, loss[loss=0.2366, simple_loss=0.3516, pruned_loss=0.06082, over 24539.00 frames. ], tot_loss[loss=0.2808, simple_loss=0.3912, pruned_loss=0.08519, over 4780696.57 frames. ], batch size: 137, lr: 5.99e-03, grad_scale: 32.0 2026-09-24 02:36:50,766 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=111606.66666666667, ans=0.1 2026-09-24 02:36:55,274 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=111640.0, ans=0.125 2026-09-24 02:36:56,468 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=12.68 vs. limit=15.0 2026-09-24 02:37:03,679 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=111673.33333333333, ans=0.125 2026-09-24 02:37:08,926 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=111706.66666666667, ans=0.125 2026-09-24 02:37:10,270 INFO [train.py:1192] (1/2) Epoch 35, batch 950, loss[loss=0.408, simple_loss=0.4562, pruned_loss=0.1799, over 11640.00 frames. ], tot_loss[loss=0.2814, simple_loss=0.3901, pruned_loss=0.08632, over 4712408.70 frames. ], batch size: 334, lr: 5.98e-03, grad_scale: 32.0 2026-09-24 02:37:22,234 INFO [train.py:1192] (1/2) Epoch 36, batch 0, loss[loss=0.2126, simple_loss=0.3299, pruned_loss=0.0476, over 24535.00 frames. ], tot_loss[loss=0.2126, simple_loss=0.3299, pruned_loss=0.0476, over 24535.00 frames. ], batch size: 137, lr: 5.90e-03, grad_scale: 32.0 2026-09-24 02:37:22,234 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 02:37:25,161 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.1402, 3.1271, 2.6540, 2.3351], device='cuda:1') 2026-09-24 02:37:27,124 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7173, 4.2585, 4.2024, 4.5528], device='cuda:1') 2026-09-24 02:37:34,100 INFO [train.py:1224] (1/2) Epoch 36, validation: loss=0.1793, simple_loss=0.2976, pruned_loss=0.03051, over 2564189.00 frames. 2026-09-24 02:37:34,100 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 02:37:36,686 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=111766.66666666667, ans=0.0 2026-09-24 02:37:37,073 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.624e+02 3.274e+02 3.763e+02 4.231e+02 6.255e+02, threshold=7.525e+02, percent-clipped=0.0 2026-09-24 02:37:43,625 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=111800.0, ans=0.1 2026-09-24 02:37:53,207 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=111866.66666666667, ans=10.0 2026-09-24 02:38:00,197 INFO [train.py:1192] (1/2) Epoch 36, batch 50, loss[loss=0.2485, simple_loss=0.351, pruned_loss=0.07299, over 24247.00 frames. ], tot_loss[loss=0.2876, simple_loss=0.3975, pruned_loss=0.08891, over 1081765.14 frames. ], batch size: 125, lr: 5.89e-03, grad_scale: 32.0 2026-09-24 02:38:03,297 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=111933.33333333333, ans=0.125 2026-09-24 02:38:10,539 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=112000.0, ans=0.0 2026-09-24 02:38:12,519 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=112000.0, ans=0.0 2026-09-24 02:38:14,457 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=112000.0, ans=0.2 2026-09-24 02:38:25,777 INFO [train.py:1192] (1/2) Epoch 36, batch 100, loss[loss=0.292, simple_loss=0.3944, pruned_loss=0.09481, over 24590.00 frames. ], tot_loss[loss=0.2888, simple_loss=0.4004, pruned_loss=0.08863, over 1915441.68 frames. ], batch size: 154, lr: 5.89e-03, grad_scale: 32.0 2026-09-24 02:38:28,553 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=112100.0, ans=0.125 2026-09-24 02:38:28,995 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.701e+02 3.326e+02 3.642e+02 4.179e+02 6.797e+02, threshold=7.283e+02, percent-clipped=0.0 2026-09-24 02:38:35,097 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=112133.33333333333, ans=0.0 2026-09-24 02:38:49,347 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=112233.33333333333, ans=0.0 2026-09-24 02:38:49,374 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=112233.33333333333, ans=0.1 2026-09-24 02:38:50,299 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=112233.33333333333, ans=0.125 2026-09-24 02:38:51,329 INFO [train.py:1192] (1/2) Epoch 36, batch 150, loss[loss=0.2457, simple_loss=0.3444, pruned_loss=0.07349, over 24291.00 frames. ], tot_loss[loss=0.2846, simple_loss=0.3953, pruned_loss=0.0869, over 2560711.12 frames. ], batch size: 125, lr: 5.88e-03, grad_scale: 32.0 2026-09-24 02:38:57,353 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.33 vs. limit=15.0 2026-09-24 02:39:04,177 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=112333.33333333333, ans=0.0 2026-09-24 02:39:05,040 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=112333.33333333333, ans=0.125 2026-09-24 02:39:11,551 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=4.79 vs. limit=15.0 2026-09-24 02:39:17,600 INFO [train.py:1192] (1/2) Epoch 36, batch 200, loss[loss=0.3072, simple_loss=0.4306, pruned_loss=0.09189, over 24227.00 frames. ], tot_loss[loss=0.283, simple_loss=0.3937, pruned_loss=0.08609, over 3059303.86 frames. ], batch size: 257, lr: 5.88e-03, grad_scale: 32.0 2026-09-24 02:39:18,733 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=112433.33333333333, ans=0.125 2026-09-24 02:39:20,635 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.387e+02 3.387e+02 3.908e+02 4.634e+02 6.934e+02, threshold=7.816e+02, percent-clipped=0.0 2026-09-24 02:39:20,721 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=112433.33333333333, ans=0.125 2026-09-24 02:39:26,101 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.33 vs. limit=12.0 2026-09-24 02:39:27,262 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=112500.0, ans=0.07 2026-09-24 02:39:32,449 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=112533.33333333333, ans=0.125 2026-09-24 02:39:39,780 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=112566.66666666667, ans=10.0 2026-09-24 02:39:39,785 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=112566.66666666667, ans=0.1 2026-09-24 02:39:43,149 INFO [train.py:1192] (1/2) Epoch 36, batch 250, loss[loss=0.3021, simple_loss=0.4226, pruned_loss=0.09084, over 24359.00 frames. ], tot_loss[loss=0.2828, simple_loss=0.3933, pruned_loss=0.08618, over 3442213.69 frames. ], batch size: 225, lr: 5.87e-03, grad_scale: 32.0 2026-09-24 02:39:44,721 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=112600.0, ans=0.125 2026-09-24 02:39:46,998 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=112600.0, ans=0.125 2026-09-24 02:39:49,609 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.94 vs. limit=15.0 2026-09-24 02:39:57,283 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=112666.66666666667, ans=0.04949747468305833 2026-09-24 02:39:58,187 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=112700.0, ans=0.125 2026-09-24 02:39:59,334 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=112700.0, ans=0.2 2026-09-24 02:40:03,516 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=112733.33333333333, ans=0.125 2026-09-24 02:40:08,811 INFO [train.py:1192] (1/2) Epoch 36, batch 300, loss[loss=0.3014, simple_loss=0.4184, pruned_loss=0.09222, over 24559.00 frames. ], tot_loss[loss=0.2812, simple_loss=0.3918, pruned_loss=0.08527, over 3755206.72 frames. ], batch size: 204, lr: 5.87e-03, grad_scale: 32.0 2026-09-24 02:40:11,840 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.729e+02 3.343e+02 3.734e+02 4.325e+02 7.519e+02, threshold=7.468e+02, percent-clipped=0.0 2026-09-24 02:40:21,503 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=112833.33333333333, ans=0.125 2026-09-24 02:40:27,864 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=112866.66666666667, ans=0.125 2026-09-24 02:40:34,542 INFO [train.py:1192] (1/2) Epoch 36, batch 350, loss[loss=0.2399, simple_loss=0.3455, pruned_loss=0.06718, over 24557.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3923, pruned_loss=0.08547, over 3997375.08 frames. ], batch size: 137, lr: 5.87e-03, grad_scale: 32.0 2026-09-24 02:40:39,603 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=112966.66666666667, ans=0.125 2026-09-24 02:40:59,917 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=113100.0, ans=0.2 2026-09-24 02:41:00,255 INFO [train.py:1192] (1/2) Epoch 36, batch 400, loss[loss=0.2609, simple_loss=0.3803, pruned_loss=0.07075, over 24585.00 frames. ], tot_loss[loss=0.2806, simple_loss=0.3915, pruned_loss=0.0848, over 4179055.87 frames. ], batch size: 170, lr: 5.86e-03, grad_scale: 32.0 2026-09-24 02:41:03,094 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.772e+02 3.358e+02 3.839e+02 4.514e+02 6.504e+02, threshold=7.677e+02, percent-clipped=0.0 2026-09-24 02:41:04,849 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=113133.33333333333, ans=0.125 2026-09-24 02:41:13,623 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=113166.66666666667, ans=0.2 2026-09-24 02:41:21,998 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=113233.33333333333, ans=0.125 2026-09-24 02:41:25,733 INFO [train.py:1192] (1/2) Epoch 36, batch 450, loss[loss=0.2864, simple_loss=0.4015, pruned_loss=0.08566, over 24633.00 frames. ], tot_loss[loss=0.2803, simple_loss=0.3912, pruned_loss=0.08471, over 4321080.44 frames. ], batch size: 175, lr: 5.86e-03, grad_scale: 32.0 2026-09-24 02:41:37,265 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=113333.33333333333, ans=0.1 2026-09-24 02:41:40,291 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.83 vs. limit=10.0 2026-09-24 02:41:44,684 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.32 vs. limit=15.0 2026-09-24 02:41:47,611 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=113400.0, ans=0.0 2026-09-24 02:41:51,180 INFO [train.py:1192] (1/2) Epoch 36, batch 500, loss[loss=0.3379, simple_loss=0.4458, pruned_loss=0.115, over 24509.00 frames. ], tot_loss[loss=0.2792, simple_loss=0.3899, pruned_loss=0.08426, over 4438849.91 frames. ], batch size: 218, lr: 5.85e-03, grad_scale: 32.0 2026-09-24 02:41:54,350 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.330e+02 3.193e+02 3.643e+02 4.133e+02 7.090e+02, threshold=7.286e+02, percent-clipped=0.0 2026-09-24 02:41:55,484 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=113433.33333333333, ans=0.0 2026-09-24 02:42:05,741 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=113500.0, ans=0.125 2026-09-24 02:42:09,509 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.63 vs. limit=8.0 2026-09-24 02:42:15,615 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=20.12 vs. limit=22.5 2026-09-24 02:42:17,050 INFO [train.py:1192] (1/2) Epoch 36, batch 550, loss[loss=0.3275, simple_loss=0.4371, pruned_loss=0.1089, over 24282.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3902, pruned_loss=0.08437, over 4523381.37 frames. ], batch size: 257, lr: 5.85e-03, grad_scale: 32.0 2026-09-24 02:42:19,552 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=113600.0, ans=0.0 2026-09-24 02:42:36,176 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=113700.0, ans=0.0 2026-09-24 02:42:37,840 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=113733.33333333333, ans=0.0 2026-09-24 02:42:41,030 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=113733.33333333333, ans=0.1 2026-09-24 02:42:43,195 INFO [train.py:1192] (1/2) Epoch 36, batch 600, loss[loss=0.3056, simple_loss=0.4205, pruned_loss=0.09535, over 24326.00 frames. ], tot_loss[loss=0.2807, simple_loss=0.3913, pruned_loss=0.08509, over 4589719.08 frames. ], batch size: 234, lr: 5.85e-03, grad_scale: 32.0 2026-09-24 02:42:46,142 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.574e+02 3.225e+02 3.585e+02 4.048e+02 6.274e+02, threshold=7.170e+02, percent-clipped=0.0 2026-09-24 02:42:48,735 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=113800.0, ans=0.015 2026-09-24 02:42:51,453 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=113800.0, ans=0.125 2026-09-24 02:43:07,302 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.74 vs. limit=12.0 2026-09-24 02:43:08,521 INFO [train.py:1192] (1/2) Epoch 36, batch 650, loss[loss=0.2693, simple_loss=0.3785, pruned_loss=0.08006, over 24606.00 frames. ], tot_loss[loss=0.2797, simple_loss=0.3905, pruned_loss=0.0845, over 4654555.58 frames. ], batch size: 154, lr: 5.84e-03, grad_scale: 32.0 2026-09-24 02:43:13,510 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=113966.66666666667, ans=0.0 2026-09-24 02:43:14,871 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=113966.66666666667, ans=0.0 2026-09-24 02:43:17,058 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=113966.66666666667, ans=0.125 2026-09-24 02:43:18,551 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.21 vs. limit=6.0 2026-09-24 02:43:22,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=114000.0, ans=0.1 2026-09-24 02:43:28,945 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.16 vs. limit=22.5 2026-09-24 02:43:33,366 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=114100.0, ans=0.2 2026-09-24 02:43:33,749 INFO [train.py:1192] (1/2) Epoch 36, batch 700, loss[loss=0.2638, simple_loss=0.376, pruned_loss=0.07586, over 24552.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3907, pruned_loss=0.08412, over 4689767.52 frames. ], batch size: 158, lr: 5.84e-03, grad_scale: 32.0 2026-09-24 02:43:36,916 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.532e+02 3.172e+02 3.783e+02 4.286e+02 7.223e+02, threshold=7.566e+02, percent-clipped=1.0 2026-09-24 02:43:51,796 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=114200.0, ans=0.1 2026-09-24 02:43:57,260 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=114233.33333333333, ans=0.0 2026-09-24 02:43:59,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=114233.33333333333, ans=0.1 2026-09-24 02:44:00,127 INFO [train.py:1192] (1/2) Epoch 36, batch 750, loss[loss=0.2895, simple_loss=0.4054, pruned_loss=0.08681, over 24642.00 frames. ], tot_loss[loss=0.2795, simple_loss=0.3902, pruned_loss=0.08437, over 4725980.98 frames. ], batch size: 175, lr: 5.83e-03, grad_scale: 32.0 2026-09-24 02:44:03,997 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=114266.66666666667, ans=0.0 2026-09-24 02:44:06,917 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=114300.0, ans=0.125 2026-09-24 02:44:20,844 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=114366.66666666667, ans=0.5 2026-09-24 02:44:25,409 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=114400.0, ans=0.125 2026-09-24 02:44:26,931 INFO [train.py:1192] (1/2) Epoch 36, batch 800, loss[loss=0.237, simple_loss=0.3485, pruned_loss=0.06271, over 24547.00 frames. ], tot_loss[loss=0.2802, simple_loss=0.3907, pruned_loss=0.08489, over 4751924.52 frames. ], batch size: 137, lr: 5.83e-03, grad_scale: 32.0 2026-09-24 02:44:30,530 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.694e+02 3.229e+02 3.756e+02 4.190e+02 6.028e+02, threshold=7.512e+02, percent-clipped=0.0 2026-09-24 02:44:35,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=114466.66666666667, ans=0.1 2026-09-24 02:44:47,620 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:44:47,668 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=114533.33333333333, ans=0.125 2026-09-24 02:44:53,278 INFO [train.py:1192] (1/2) Epoch 36, batch 850, loss[loss=0.2884, simple_loss=0.408, pruned_loss=0.08439, over 24573.00 frames. ], tot_loss[loss=0.2797, simple_loss=0.3901, pruned_loss=0.08468, over 4770174.35 frames. ], batch size: 204, lr: 5.83e-03, grad_scale: 32.0 2026-09-24 02:44:55,733 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=114600.0, ans=0.125 2026-09-24 02:45:09,982 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.78 vs. limit=15.0 2026-09-24 02:45:15,160 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=114733.33333333333, ans=0.2 2026-09-24 02:45:18,030 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=114733.33333333333, ans=0.1 2026-09-24 02:45:19,370 INFO [train.py:1192] (1/2) Epoch 36, batch 900, loss[loss=0.2379, simple_loss=0.3528, pruned_loss=0.06157, over 24565.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3908, pruned_loss=0.08507, over 4781703.11 frames. ], batch size: 137, lr: 5.82e-03, grad_scale: 32.0 2026-09-24 02:45:22,088 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.15 vs. limit=22.5 2026-09-24 02:45:22,319 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.378e+02 3.202e+02 3.609e+02 4.015e+02 5.684e+02, threshold=7.217e+02, percent-clipped=0.0 2026-09-24 02:45:23,384 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=114766.66666666667, ans=0.125 2026-09-24 02:45:37,880 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.84 vs. limit=15.0 2026-09-24 02:45:41,449 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.57 vs. limit=10.0 2026-09-24 02:45:43,005 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=13.53 vs. limit=15.0 2026-09-24 02:45:44,362 INFO [train.py:1192] (1/2) Epoch 36, batch 950, loss[loss=0.3751, simple_loss=0.4334, pruned_loss=0.1584, over 11487.00 frames. ], tot_loss[loss=0.2816, simple_loss=0.3902, pruned_loss=0.08644, over 4709579.13 frames. ], batch size: 333, lr: 5.82e-03, grad_scale: 32.0 2026-09-24 02:45:54,757 INFO [train.py:1192] (1/2) Epoch 37, batch 0, loss[loss=0.2182, simple_loss=0.3386, pruned_loss=0.0489, over 24572.00 frames. ], tot_loss[loss=0.2182, simple_loss=0.3386, pruned_loss=0.0489, over 24572.00 frames. ], batch size: 137, lr: 5.74e-03, grad_scale: 32.0 2026-09-24 02:45:54,758 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 02:46:03,020 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2631, 3.9162, 3.7248, 3.2817], device='cuda:1') 2026-09-24 02:46:06,790 INFO [train.py:1224] (1/2) Epoch 37, validation: loss=0.1782, simple_loss=0.2965, pruned_loss=0.0299, over 2564189.00 frames. 2026-09-24 02:46:06,790 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 02:46:07,338 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=114960.0, ans=0.125 2026-09-24 02:46:07,787 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=114960.0, ans=0.1 2026-09-24 02:46:08,325 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.39 vs. limit=15.0 2026-09-24 02:46:08,672 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=114960.0, ans=0.025 2026-09-24 02:46:10,189 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=114960.0, ans=0.2 2026-09-24 02:46:15,890 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=114993.33333333333, ans=0.125 2026-09-24 02:46:15,913 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=114993.33333333333, ans=0.1 2026-09-24 02:46:25,146 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=115060.0, ans=0.2 2026-09-24 02:46:29,377 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=115093.33333333333, ans=0.07 2026-09-24 02:46:31,319 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.478e+02 3.261e+02 3.711e+02 4.242e+02 6.808e+02, threshold=7.422e+02, percent-clipped=0.0 2026-09-24 02:46:32,434 INFO [train.py:1192] (1/2) Epoch 37, batch 50, loss[loss=0.243, simple_loss=0.3487, pruned_loss=0.06869, over 24292.00 frames. ], tot_loss[loss=0.2843, simple_loss=0.3955, pruned_loss=0.08653, over 1082693.79 frames. ], batch size: 125, lr: 5.73e-03, grad_scale: 32.0 2026-09-24 02:46:35,536 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=115126.66666666667, ans=0.125 2026-09-24 02:46:44,381 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=115193.33333333333, ans=0.125 2026-09-24 02:46:58,407 INFO [train.py:1192] (1/2) Epoch 37, batch 100, loss[loss=0.2859, simple_loss=0.3915, pruned_loss=0.09021, over 24610.00 frames. ], tot_loss[loss=0.2854, simple_loss=0.3979, pruned_loss=0.08642, over 1915675.59 frames. ], batch size: 154, lr: 5.73e-03, grad_scale: 64.0 2026-09-24 02:46:58,522 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=115293.33333333333, ans=0.1 2026-09-24 02:47:19,945 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=115426.66666666667, ans=0.0 2026-09-24 02:47:20,384 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=115426.66666666667, ans=0.025 2026-09-24 02:47:23,774 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.684e+02 3.309e+02 3.683e+02 4.350e+02 6.360e+02, threshold=7.366e+02, percent-clipped=0.0 2026-09-24 02:47:24,713 INFO [train.py:1192] (1/2) Epoch 37, batch 150, loss[loss=0.2301, simple_loss=0.3359, pruned_loss=0.0622, over 24274.00 frames. ], tot_loss[loss=0.284, simple_loss=0.3952, pruned_loss=0.08636, over 2561208.03 frames. ], batch size: 125, lr: 5.72e-03, grad_scale: 64.0 2026-09-24 02:47:33,176 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=115493.33333333333, ans=0.125 2026-09-24 02:47:41,596 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=115560.0, ans=0.125 2026-09-24 02:47:45,317 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=115593.33333333333, ans=0.07 2026-09-24 02:47:49,289 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=115593.33333333333, ans=0.2 2026-09-24 02:47:50,713 INFO [train.py:1192] (1/2) Epoch 37, batch 200, loss[loss=0.2993, simple_loss=0.4209, pruned_loss=0.0889, over 24224.00 frames. ], tot_loss[loss=0.2815, simple_loss=0.393, pruned_loss=0.08505, over 3061626.22 frames. ], batch size: 257, lr: 5.72e-03, grad_scale: 32.0 2026-09-24 02:47:52,588 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=115626.66666666667, ans=0.0 2026-09-24 02:48:01,263 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=115693.33333333333, ans=0.125 2026-09-24 02:48:11,326 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=115760.0, ans=0.1 2026-09-24 02:48:12,537 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=115760.0, ans=0.125 2026-09-24 02:48:12,540 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=115760.0, ans=0.125 2026-09-24 02:48:16,229 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.586e+02 3.511e+02 4.062e+02 4.877e+02 7.585e+02, threshold=8.125e+02, percent-clipped=2.0 2026-09-24 02:48:16,746 INFO [train.py:1192] (1/2) Epoch 37, batch 250, loss[loss=0.3115, simple_loss=0.4271, pruned_loss=0.09793, over 24354.00 frames. ], tot_loss[loss=0.2813, simple_loss=0.3923, pruned_loss=0.08516, over 3444501.76 frames. ], batch size: 225, lr: 5.72e-03, grad_scale: 32.0 2026-09-24 02:48:21,122 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=16.63 vs. limit=22.5 2026-09-24 02:48:39,586 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=115926.66666666667, ans=0.125 2026-09-24 02:48:42,336 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=115960.0, ans=0.0 2026-09-24 02:48:42,694 INFO [train.py:1192] (1/2) Epoch 37, batch 300, loss[loss=0.2939, simple_loss=0.413, pruned_loss=0.08742, over 24537.00 frames. ], tot_loss[loss=0.2799, simple_loss=0.391, pruned_loss=0.0844, over 3758047.83 frames. ], batch size: 204, lr: 5.71e-03, grad_scale: 32.0 2026-09-24 02:48:49,672 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=115993.33333333333, ans=0.125 2026-09-24 02:48:58,317 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=116060.0, ans=0.125 2026-09-24 02:49:08,120 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.612e+02 3.098e+02 3.455e+02 3.844e+02 5.865e+02, threshold=6.911e+02, percent-clipped=0.0 2026-09-24 02:49:08,679 INFO [train.py:1192] (1/2) Epoch 37, batch 350, loss[loss=0.248, simple_loss=0.3562, pruned_loss=0.0699, over 24560.00 frames. ], tot_loss[loss=0.2804, simple_loss=0.3916, pruned_loss=0.08457, over 3998217.43 frames. ], batch size: 137, lr: 5.71e-03, grad_scale: 32.0 2026-09-24 02:49:13,732 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=116160.0, ans=0.125 2026-09-24 02:49:16,314 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.78 vs. limit=15.0 2026-09-24 02:49:17,617 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=116160.0, ans=0.0 2026-09-24 02:49:18,219 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=116160.0, ans=0.0 2026-09-24 02:49:22,176 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=116193.33333333333, ans=0.0 2026-09-24 02:49:29,156 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.88 vs. limit=6.0 2026-09-24 02:49:34,869 INFO [train.py:1192] (1/2) Epoch 37, batch 400, loss[loss=0.294, simple_loss=0.3961, pruned_loss=0.09596, over 24578.00 frames. ], tot_loss[loss=0.2802, simple_loss=0.391, pruned_loss=0.08468, over 4179503.26 frames. ], batch size: 170, lr: 5.70e-03, grad_scale: 32.0 2026-09-24 02:49:41,346 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=116326.66666666667, ans=0.2 2026-09-24 02:49:42,437 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=116326.66666666667, ans=0.125 2026-09-24 02:49:44,760 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=116326.66666666667, ans=0.0 2026-09-24 02:49:56,629 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=116426.66666666667, ans=0.0 2026-09-24 02:49:57,710 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=116426.66666666667, ans=0.125 2026-09-24 02:50:00,923 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.704e+02 3.336e+02 3.681e+02 4.206e+02 6.815e+02, threshold=7.363e+02, percent-clipped=0.0 2026-09-24 02:50:01,359 INFO [train.py:1192] (1/2) Epoch 37, batch 450, loss[loss=0.2859, simple_loss=0.399, pruned_loss=0.08643, over 24629.00 frames. ], tot_loss[loss=0.2811, simple_loss=0.3916, pruned_loss=0.08533, over 4319122.02 frames. ], batch size: 175, lr: 5.70e-03, grad_scale: 32.0 2026-09-24 02:50:02,600 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.36 vs. limit=15.0 2026-09-24 02:50:03,783 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.35 vs. limit=15.0 2026-09-24 02:50:08,065 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=116493.33333333333, ans=0.1 2026-09-24 02:50:15,551 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=116526.66666666667, ans=0.125 2026-09-24 02:50:16,583 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=116560.0, ans=0.2 2026-09-24 02:50:27,701 INFO [train.py:1192] (1/2) Epoch 37, batch 500, loss[loss=0.2986, simple_loss=0.415, pruned_loss=0.09104, over 24490.00 frames. ], tot_loss[loss=0.2807, simple_loss=0.3908, pruned_loss=0.08528, over 4437309.43 frames. ], batch size: 218, lr: 5.70e-03, grad_scale: 32.0 2026-09-24 02:50:32,456 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.08 vs. limit=15.0 2026-09-24 02:50:34,322 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=116660.0, ans=0.125 2026-09-24 02:50:36,805 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=116660.0, ans=0.125 2026-09-24 02:50:37,777 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=116693.33333333333, ans=0.125 2026-09-24 02:50:43,583 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=116726.66666666667, ans=0.0 2026-09-24 02:50:48,438 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=116760.0, ans=0.0 2026-09-24 02:50:50,364 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=116760.0, ans=0.1 2026-09-24 02:50:53,133 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=116760.0, ans=0.1 2026-09-24 02:50:53,468 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.663e+02 3.097e+02 3.515e+02 3.895e+02 6.020e+02, threshold=7.030e+02, percent-clipped=0.0 2026-09-24 02:50:53,929 INFO [train.py:1192] (1/2) Epoch 37, batch 550, loss[loss=0.3089, simple_loss=0.4258, pruned_loss=0.09595, over 24300.00 frames. ], tot_loss[loss=0.2805, simple_loss=0.3908, pruned_loss=0.08509, over 4522069.45 frames. ], batch size: 257, lr: 5.69e-03, grad_scale: 32.0 2026-09-24 02:51:01,398 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=116826.66666666667, ans=0.125 2026-09-24 02:51:20,203 INFO [train.py:1192] (1/2) Epoch 37, batch 600, loss[loss=0.287, simple_loss=0.4056, pruned_loss=0.08419, over 24358.00 frames. ], tot_loss[loss=0.2804, simple_loss=0.391, pruned_loss=0.08493, over 4588373.36 frames. ], batch size: 234, lr: 5.69e-03, grad_scale: 32.0 2026-09-24 02:51:33,703 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.73 vs. limit=15.0 2026-09-24 02:51:38,567 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=117060.0, ans=0.025 2026-09-24 02:51:45,009 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.43 vs. limit=10.0 2026-09-24 02:51:45,714 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.564e+02 3.262e+02 3.741e+02 4.492e+02 7.596e+02, threshold=7.483e+02, percent-clipped=1.0 2026-09-24 02:51:46,285 INFO [train.py:1192] (1/2) Epoch 37, batch 650, loss[loss=0.2925, simple_loss=0.3924, pruned_loss=0.0963, over 24594.00 frames. ], tot_loss[loss=0.28, simple_loss=0.3906, pruned_loss=0.08475, over 4653213.46 frames. ], batch size: 154, lr: 5.68e-03, grad_scale: 32.0 2026-09-24 02:51:55,232 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.67 vs. limit=22.5 2026-09-24 02:51:56,092 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=117193.33333333333, ans=0.125 2026-09-24 02:52:02,738 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=117226.66666666667, ans=0.125 2026-09-24 02:52:12,544 INFO [train.py:1192] (1/2) Epoch 37, batch 700, loss[loss=0.2888, simple_loss=0.3926, pruned_loss=0.09249, over 24552.00 frames. ], tot_loss[loss=0.281, simple_loss=0.3919, pruned_loss=0.08501, over 4688979.90 frames. ], batch size: 158, lr: 5.68e-03, grad_scale: 32.0 2026-09-24 02:52:19,214 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=117326.66666666667, ans=0.025 2026-09-24 02:52:21,520 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=117326.66666666667, ans=0.125 2026-09-24 02:52:28,567 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=117393.33333333333, ans=0.125 2026-09-24 02:52:36,192 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=117426.66666666667, ans=0.1 2026-09-24 02:52:37,365 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.586e+02 3.247e+02 3.506e+02 4.061e+02 6.119e+02, threshold=7.013e+02, percent-clipped=0.0 2026-09-24 02:52:37,869 INFO [train.py:1192] (1/2) Epoch 37, batch 750, loss[loss=0.2701, simple_loss=0.3911, pruned_loss=0.07454, over 24634.00 frames. ], tot_loss[loss=0.2787, simple_loss=0.3899, pruned_loss=0.08372, over 4725932.14 frames. ], batch size: 175, lr: 5.68e-03, grad_scale: 32.0 2026-09-24 02:52:43,764 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=117493.33333333333, ans=0.125 2026-09-24 02:52:46,186 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=117493.33333333333, ans=0.125 2026-09-24 02:52:50,980 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=117526.66666666667, ans=0.1 2026-09-24 02:53:03,672 INFO [train.py:1192] (1/2) Epoch 37, batch 800, loss[loss=0.2217, simple_loss=0.3362, pruned_loss=0.05361, over 24538.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3895, pruned_loss=0.08351, over 4751761.16 frames. ], batch size: 137, lr: 5.67e-03, grad_scale: 32.0 2026-09-24 02:53:06,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=117626.66666666667, ans=0.0 2026-09-24 02:53:10,613 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=117660.0, ans=0.04949747468305833 2026-09-24 02:53:13,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=117693.33333333333, ans=0.125 2026-09-24 02:53:16,315 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.56 vs. limit=15.0 2026-09-24 02:53:21,531 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:53:26,491 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=117760.0, ans=0.2 2026-09-24 02:53:29,074 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.564e+02 3.186e+02 3.597e+02 4.291e+02 6.249e+02, threshold=7.194e+02, percent-clipped=0.0 2026-09-24 02:53:29,505 INFO [train.py:1192] (1/2) Epoch 37, batch 850, loss[loss=0.2732, simple_loss=0.3953, pruned_loss=0.07553, over 24559.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3886, pruned_loss=0.083, over 4770183.27 frames. ], batch size: 204, lr: 5.67e-03, grad_scale: 32.0 2026-09-24 02:53:48,738 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 02:53:54,803 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=117960.0, ans=0.2 2026-09-24 02:53:55,223 INFO [train.py:1192] (1/2) Epoch 37, batch 900, loss[loss=0.2245, simple_loss=0.3401, pruned_loss=0.05446, over 24567.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3894, pruned_loss=0.08373, over 4780798.64 frames. ], batch size: 137, lr: 5.67e-03, grad_scale: 32.0 2026-09-24 02:54:01,034 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=117993.33333333333, ans=0.125 2026-09-24 02:54:02,541 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=117993.33333333333, ans=0.035 2026-09-24 02:54:09,426 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=118026.66666666667, ans=0.0 2026-09-24 02:54:16,671 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.87 vs. limit=15.0 2026-09-24 02:54:20,878 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.586e+02 3.397e+02 3.779e+02 4.305e+02 8.230e+02, threshold=7.558e+02, percent-clipped=2.0 2026-09-24 02:54:20,884 INFO [train.py:1192] (1/2) Epoch 37, batch 950, loss[loss=0.4018, simple_loss=0.4566, pruned_loss=0.1735, over 10908.00 frames. ], tot_loss[loss=0.2784, simple_loss=0.3878, pruned_loss=0.08449, over 4715807.81 frames. ], batch size: 333, lr: 5.66e-03, grad_scale: 16.0 2026-09-24 02:54:20,976 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=118126.66666666667, ans=0.1 2026-09-24 02:54:31,165 INFO [train.py:1192] (1/2) Epoch 38, batch 0, loss[loss=0.2318, simple_loss=0.3513, pruned_loss=0.05612, over 24551.00 frames. ], tot_loss[loss=0.2318, simple_loss=0.3513, pruned_loss=0.05612, over 24551.00 frames. ], batch size: 137, lr: 5.58e-03, grad_scale: 32.0 2026-09-24 02:54:31,168 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 02:54:42,984 INFO [train.py:1224] (1/2) Epoch 38, validation: loss=0.179, simple_loss=0.2974, pruned_loss=0.0303, over 2564189.00 frames. 2026-09-24 02:54:42,984 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 02:54:43,161 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.04 vs. limit=15.0 2026-09-24 02:54:48,691 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=118186.66666666667, ans=0.0 2026-09-24 02:54:52,792 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=118220.0, ans=0.125 2026-09-24 02:55:09,019 INFO [train.py:1192] (1/2) Epoch 38, batch 50, loss[loss=0.218, simple_loss=0.3198, pruned_loss=0.05813, over 24302.00 frames. ], tot_loss[loss=0.2849, simple_loss=0.3955, pruned_loss=0.08712, over 1082281.13 frames. ], batch size: 125, lr: 5.58e-03, grad_scale: 32.0 2026-09-24 02:55:16,318 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=118353.33333333333, ans=0.0 2026-09-24 02:55:24,384 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.66 vs. limit=15.0 2026-09-24 02:55:28,001 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=118420.0, ans=0.0 2026-09-24 02:55:30,410 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.646e+02 3.388e+02 3.790e+02 4.470e+02 6.487e+02, threshold=7.579e+02, percent-clipped=0.0 2026-09-24 02:55:33,593 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=118453.33333333333, ans=0.0 2026-09-24 02:55:34,354 INFO [train.py:1192] (1/2) Epoch 38, batch 100, loss[loss=0.252, simple_loss=0.3673, pruned_loss=0.06834, over 24604.00 frames. ], tot_loss[loss=0.2858, simple_loss=0.3979, pruned_loss=0.08683, over 1915544.41 frames. ], batch size: 154, lr: 5.58e-03, grad_scale: 32.0 2026-09-24 02:55:42,461 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=118520.0, ans=0.125 2026-09-24 02:55:45,005 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=118553.33333333333, ans=0.0 2026-09-24 02:55:47,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=118553.33333333333, ans=0.125 2026-09-24 02:56:00,440 INFO [train.py:1192] (1/2) Epoch 38, batch 150, loss[loss=0.2291, simple_loss=0.3352, pruned_loss=0.06148, over 24276.00 frames. ], tot_loss[loss=0.2806, simple_loss=0.3925, pruned_loss=0.0843, over 2560727.84 frames. ], batch size: 125, lr: 5.57e-03, grad_scale: 32.0 2026-09-24 02:56:07,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=118686.66666666667, ans=10.0 2026-09-24 02:56:21,948 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.483e+02 3.407e+02 3.926e+02 4.423e+02 6.071e+02, threshold=7.852e+02, percent-clipped=0.0 2026-09-24 02:56:23,439 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=118786.66666666667, ans=0.05 2026-09-24 02:56:25,873 INFO [train.py:1192] (1/2) Epoch 38, batch 200, loss[loss=0.3043, simple_loss=0.4288, pruned_loss=0.08986, over 24186.00 frames. ], tot_loss[loss=0.2777, simple_loss=0.3901, pruned_loss=0.0826, over 3060557.59 frames. ], batch size: 257, lr: 5.57e-03, grad_scale: 32.0 2026-09-24 02:56:31,756 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=118853.33333333333, ans=0.2 2026-09-24 02:56:32,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=118853.33333333333, ans=0.125 2026-09-24 02:56:48,257 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=118953.33333333333, ans=0.025 2026-09-24 02:56:49,138 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=118953.33333333333, ans=0.0 2026-09-24 02:56:51,599 INFO [train.py:1192] (1/2) Epoch 38, batch 250, loss[loss=0.2984, simple_loss=0.4204, pruned_loss=0.08819, over 24393.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3899, pruned_loss=0.08319, over 3443374.35 frames. ], batch size: 225, lr: 5.57e-03, grad_scale: 32.0 2026-09-24 02:57:11,683 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=119086.66666666667, ans=0.1 2026-09-24 02:57:13,772 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.465e+02 3.346e+02 3.899e+02 4.555e+02 7.441e+02, threshold=7.798e+02, percent-clipped=0.0 2026-09-24 02:57:14,731 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=119120.0, ans=0.0 2026-09-24 02:57:17,827 INFO [train.py:1192] (1/2) Epoch 38, batch 300, loss[loss=0.3137, simple_loss=0.4257, pruned_loss=0.1008, over 24552.00 frames. ], tot_loss[loss=0.2776, simple_loss=0.3893, pruned_loss=0.08289, over 3757311.28 frames. ], batch size: 204, lr: 5.56e-03, grad_scale: 32.0 2026-09-24 02:57:18,427 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=119153.33333333333, ans=0.0 2026-09-24 02:57:25,382 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.88 vs. limit=15.0 2026-09-24 02:57:40,785 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=119286.66666666667, ans=0.07 2026-09-24 02:57:43,983 INFO [train.py:1192] (1/2) Epoch 38, batch 350, loss[loss=0.2266, simple_loss=0.3356, pruned_loss=0.0588, over 24587.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.39, pruned_loss=0.08321, over 3997755.54 frames. ], batch size: 137, lr: 5.56e-03, grad_scale: 32.0 2026-09-24 02:57:45,131 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=119320.0, ans=0.125 2026-09-24 02:58:00,814 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=119420.0, ans=0.0 2026-09-24 02:58:03,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=119420.0, ans=0.125 2026-09-24 02:58:05,497 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.732e+02 3.234e+02 3.550e+02 4.047e+02 7.185e+02, threshold=7.100e+02, percent-clipped=0.0 2026-09-24 02:58:09,908 INFO [train.py:1192] (1/2) Epoch 38, batch 400, loss[loss=0.286, simple_loss=0.394, pruned_loss=0.08897, over 24587.00 frames. ], tot_loss[loss=0.2783, simple_loss=0.3898, pruned_loss=0.0834, over 4181957.47 frames. ], batch size: 170, lr: 5.55e-03, grad_scale: 32.0 2026-09-24 02:58:11,029 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=119486.66666666667, ans=0.2 2026-09-24 02:58:12,470 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=119486.66666666667, ans=0.125 2026-09-24 02:58:16,480 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=119520.0, ans=0.0 2026-09-24 02:58:18,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=119520.0, ans=0.0 2026-09-24 02:58:21,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=119553.33333333333, ans=0.0 2026-09-24 02:58:35,617 INFO [train.py:1192] (1/2) Epoch 38, batch 450, loss[loss=0.2826, simple_loss=0.397, pruned_loss=0.08411, over 24612.00 frames. ], tot_loss[loss=0.2779, simple_loss=0.3896, pruned_loss=0.08308, over 4318632.86 frames. ], batch size: 175, lr: 5.55e-03, grad_scale: 32.0 2026-09-24 02:58:45,157 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=119720.0, ans=0.125 2026-09-24 02:58:45,568 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=119720.0, ans=0.0 2026-09-24 02:58:56,192 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=119786.66666666667, ans=0.2 2026-09-24 02:58:57,084 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.476e+02 3.254e+02 3.716e+02 4.079e+02 6.258e+02, threshold=7.433e+02, percent-clipped=0.0 2026-09-24 02:59:01,215 INFO [train.py:1192] (1/2) Epoch 38, batch 500, loss[loss=0.3068, simple_loss=0.4202, pruned_loss=0.09668, over 24509.00 frames. ], tot_loss[loss=0.2772, simple_loss=0.3886, pruned_loss=0.0829, over 4437140.34 frames. ], batch size: 218, lr: 5.55e-03, grad_scale: 32.0 2026-09-24 02:59:06,967 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=119853.33333333333, ans=0.0 2026-09-24 02:59:09,359 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=119853.33333333333, ans=0.1 2026-09-24 02:59:09,371 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=119853.33333333333, ans=0.0 2026-09-24 02:59:15,102 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=119886.66666666667, ans=0.125 2026-09-24 02:59:19,469 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=119920.0, ans=0.1 2026-09-24 02:59:26,468 INFO [train.py:1192] (1/2) Epoch 38, batch 550, loss[loss=0.3399, simple_loss=0.4512, pruned_loss=0.1144, over 24283.00 frames. ], tot_loss[loss=0.2782, simple_loss=0.3894, pruned_loss=0.08351, over 4522230.68 frames. ], batch size: 257, lr: 5.54e-03, grad_scale: 32.0 2026-09-24 02:59:26,566 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=119986.66666666667, ans=0.125 2026-09-24 02:59:27,251 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.96 vs. limit=12.0 2026-09-24 02:59:27,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=119986.66666666667, ans=0.015 2026-09-24 02:59:30,226 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=119986.66666666667, ans=0.125 2026-09-24 02:59:38,956 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=120053.33333333333, ans=0.025 2026-09-24 02:59:42,174 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=120086.66666666667, ans=0.125 2026-09-24 02:59:46,684 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=120086.66666666667, ans=0.0 2026-09-24 02:59:48,621 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.442e+02 3.197e+02 3.513e+02 3.957e+02 5.244e+02, threshold=7.025e+02, percent-clipped=0.0 2026-09-24 02:59:52,895 INFO [train.py:1192] (1/2) Epoch 38, batch 600, loss[loss=0.3313, simple_loss=0.4411, pruned_loss=0.1108, over 24325.00 frames. ], tot_loss[loss=0.2779, simple_loss=0.3893, pruned_loss=0.08322, over 4589344.51 frames. ], batch size: 234, lr: 5.54e-03, grad_scale: 32.0 2026-09-24 02:59:53,500 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=120153.33333333333, ans=0.125 2026-09-24 02:59:59,742 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=120186.66666666667, ans=0.125 2026-09-24 03:00:05,520 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=120220.0, ans=0.125 2026-09-24 03:00:14,118 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=120286.66666666667, ans=0.2 2026-09-24 03:00:17,386 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=15.65 vs. limit=22.5 2026-09-24 03:00:18,492 INFO [train.py:1192] (1/2) Epoch 38, batch 650, loss[loss=0.2641, simple_loss=0.3807, pruned_loss=0.07373, over 24597.00 frames. ], tot_loss[loss=0.2768, simple_loss=0.3883, pruned_loss=0.08258, over 4654266.40 frames. ], batch size: 154, lr: 5.54e-03, grad_scale: 32.0 2026-09-24 03:00:21,443 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=6.23 vs. limit=15.0 2026-09-24 03:00:25,676 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=120353.33333333333, ans=0.1 2026-09-24 03:00:28,144 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=120353.33333333333, ans=0.2 2026-09-24 03:00:35,825 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.64 vs. limit=15.0 2026-09-24 03:00:36,093 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=120420.0, ans=0.5 2026-09-24 03:00:38,740 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=120420.0, ans=0.1 2026-09-24 03:00:40,702 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.639e+02 3.335e+02 3.771e+02 4.337e+02 7.675e+02, threshold=7.542e+02, percent-clipped=1.0 2026-09-24 03:00:44,700 INFO [train.py:1192] (1/2) Epoch 38, batch 700, loss[loss=0.2667, simple_loss=0.3765, pruned_loss=0.07846, over 24551.00 frames. ], tot_loss[loss=0.2778, simple_loss=0.3893, pruned_loss=0.08314, over 4689543.82 frames. ], batch size: 158, lr: 5.53e-03, grad_scale: 32.0 2026-09-24 03:01:06,148 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=120620.0, ans=0.2 2026-09-24 03:01:08,462 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=120620.0, ans=0.125 2026-09-24 03:01:10,248 INFO [train.py:1192] (1/2) Epoch 38, batch 750, loss[loss=0.2584, simple_loss=0.3795, pruned_loss=0.06867, over 24608.00 frames. ], tot_loss[loss=0.2777, simple_loss=0.389, pruned_loss=0.08326, over 4722656.84 frames. ], batch size: 175, lr: 5.53e-03, grad_scale: 32.0 2026-09-24 03:01:10,474 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=120653.33333333333, ans=10.0 2026-09-24 03:01:16,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=120686.66666666667, ans=0.125 2026-09-24 03:01:22,480 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=120720.0, ans=0.125 2026-09-24 03:01:32,091 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.547e+02 3.305e+02 3.709e+02 4.019e+02 5.996e+02, threshold=7.418e+02, percent-clipped=0.0 2026-09-24 03:01:33,536 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.61 vs. limit=12.0 2026-09-24 03:01:33,716 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.74 vs. limit=15.0 2026-09-24 03:01:36,221 INFO [train.py:1192] (1/2) Epoch 38, batch 800, loss[loss=0.2354, simple_loss=0.3478, pruned_loss=0.06151, over 24533.00 frames. ], tot_loss[loss=0.2778, simple_loss=0.3889, pruned_loss=0.08332, over 4750615.34 frames. ], batch size: 137, lr: 5.53e-03, grad_scale: 32.0 2026-09-24 03:01:37,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=120820.0, ans=0.1 2026-09-24 03:01:38,617 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.24 vs. limit=22.5 2026-09-24 03:01:44,229 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=120853.33333333333, ans=0.1 2026-09-24 03:01:55,605 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten.whitening_limit, batch_count=120920.0, ans=22.5 2026-09-24 03:01:57,996 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=120953.33333333333, ans=0.1 2026-09-24 03:02:02,174 INFO [train.py:1192] (1/2) Epoch 38, batch 850, loss[loss=0.3109, simple_loss=0.4217, pruned_loss=0.1, over 24565.00 frames. ], tot_loss[loss=0.2768, simple_loss=0.3881, pruned_loss=0.0827, over 4769714.24 frames. ], batch size: 204, lr: 5.52e-03, grad_scale: 32.0 2026-09-24 03:02:05,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=120986.66666666667, ans=0.0 2026-09-24 03:02:11,947 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=121020.0, ans=0.1 2026-09-24 03:02:16,275 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=121053.33333333333, ans=0.125 2026-09-24 03:02:19,411 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.43 vs. limit=10.0 2026-09-24 03:02:24,245 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.820e+02 3.178e+02 3.659e+02 4.130e+02 5.422e+02, threshold=7.319e+02, percent-clipped=0.0 2026-09-24 03:02:25,310 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=121120.0, ans=0.0 2026-09-24 03:02:28,152 INFO [train.py:1192] (1/2) Epoch 38, batch 900, loss[loss=0.2427, simple_loss=0.3566, pruned_loss=0.06443, over 24570.00 frames. ], tot_loss[loss=0.277, simple_loss=0.3884, pruned_loss=0.08275, over 4781139.80 frames. ], batch size: 137, lr: 5.52e-03, grad_scale: 32.0 2026-09-24 03:02:39,672 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=121220.0, ans=10.0 2026-09-24 03:02:42,545 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=121220.0, ans=0.125 2026-09-24 03:02:48,207 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=121286.66666666667, ans=0.2 2026-09-24 03:02:53,699 INFO [train.py:1192] (1/2) Epoch 38, batch 950, loss[loss=0.3848, simple_loss=0.4358, pruned_loss=0.167, over 10952.00 frames. ], tot_loss[loss=0.2778, simple_loss=0.3878, pruned_loss=0.08387, over 4715807.86 frames. ], batch size: 334, lr: 5.51e-03, grad_scale: 32.0 2026-09-24 03:03:03,988 INFO [train.py:1192] (1/2) Epoch 39, batch 0, loss[loss=0.2342, simple_loss=0.3473, pruned_loss=0.06057, over 24582.00 frames. ], tot_loss[loss=0.2342, simple_loss=0.3473, pruned_loss=0.06057, over 24582.00 frames. ], batch size: 137, lr: 5.44e-03, grad_scale: 32.0 2026-09-24 03:03:03,988 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 03:03:13,724 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6024, 2.7559, 3.1415, 2.7505, 2.5638, 3.0987, 1.8867, 2.5513], device='cuda:1') 2026-09-24 03:03:15,757 INFO [train.py:1224] (1/2) Epoch 39, validation: loss=0.177, simple_loss=0.2956, pruned_loss=0.02925, over 2564189.00 frames. 2026-09-24 03:03:15,757 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 03:03:16,824 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.86 vs. limit=15.0 2026-09-24 03:03:27,416 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.33 vs. limit=15.0 2026-09-24 03:03:33,307 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.743e+02 3.319e+02 3.713e+02 4.164e+02 7.088e+02, threshold=7.426e+02, percent-clipped=0.0 2026-09-24 03:03:38,644 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=121480.0, ans=0.2 2026-09-24 03:03:39,232 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.29 vs. limit=15.0 2026-09-24 03:03:40,996 INFO [train.py:1192] (1/2) Epoch 39, batch 50, loss[loss=0.2218, simple_loss=0.3285, pruned_loss=0.05753, over 24290.00 frames. ], tot_loss[loss=0.2819, simple_loss=0.3937, pruned_loss=0.08507, over 1082468.08 frames. ], batch size: 125, lr: 5.44e-03, grad_scale: 32.0 2026-09-24 03:03:46,281 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=121546.66666666667, ans=0.1 2026-09-24 03:03:47,658 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=121546.66666666667, ans=0.0 2026-09-24 03:04:07,128 INFO [train.py:1192] (1/2) Epoch 39, batch 100, loss[loss=0.2551, simple_loss=0.3638, pruned_loss=0.07319, over 24641.00 frames. ], tot_loss[loss=0.2839, simple_loss=0.3971, pruned_loss=0.08535, over 1915608.17 frames. ], batch size: 154, lr: 5.43e-03, grad_scale: 32.0 2026-09-24 03:04:11,037 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=121680.0, ans=0.0 2026-09-24 03:04:24,350 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.533e+02 3.287e+02 3.705e+02 4.110e+02 5.372e+02, threshold=7.409e+02, percent-clipped=0.0 2026-09-24 03:04:24,917 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=121780.0, ans=0.0 2026-09-24 03:04:26,436 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=121780.0, ans=0.125 2026-09-24 03:04:32,697 INFO [train.py:1192] (1/2) Epoch 39, batch 150, loss[loss=0.2195, simple_loss=0.3287, pruned_loss=0.05518, over 24238.00 frames. ], tot_loss[loss=0.279, simple_loss=0.3917, pruned_loss=0.08311, over 2560671.23 frames. ], batch size: 125, lr: 5.43e-03, grad_scale: 32.0 2026-09-24 03:04:34,192 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=121846.66666666667, ans=0.0 2026-09-24 03:04:37,396 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=121880.0, ans=0.125 2026-09-24 03:04:40,077 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.05 vs. limit=12.0 2026-09-24 03:04:42,877 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=121913.33333333333, ans=0.125 2026-09-24 03:04:46,166 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.40 vs. limit=12.0 2026-09-24 03:04:46,176 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.51 vs. limit=22.5 2026-09-24 03:04:50,377 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=121946.66666666667, ans=0.025 2026-09-24 03:04:55,947 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=121980.0, ans=0.125 2026-09-24 03:04:58,142 INFO [train.py:1192] (1/2) Epoch 39, batch 200, loss[loss=0.3189, simple_loss=0.4373, pruned_loss=0.1003, over 24222.00 frames. ], tot_loss[loss=0.2777, simple_loss=0.3902, pruned_loss=0.08256, over 3059175.10 frames. ], batch size: 257, lr: 5.43e-03, grad_scale: 32.0 2026-09-24 03:04:58,964 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.79 vs. limit=22.5 2026-09-24 03:05:15,481 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.822e+02 3.464e+02 3.893e+02 4.645e+02 6.817e+02, threshold=7.787e+02, percent-clipped=0.0 2026-09-24 03:05:21,293 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=122146.66666666667, ans=0.2 2026-09-24 03:05:24,147 INFO [train.py:1192] (1/2) Epoch 39, batch 250, loss[loss=0.3032, simple_loss=0.4239, pruned_loss=0.09129, over 24388.00 frames. ], tot_loss[loss=0.2774, simple_loss=0.3895, pruned_loss=0.08269, over 3442962.24 frames. ], batch size: 225, lr: 5.42e-03, grad_scale: 32.0 2026-09-24 03:05:49,186 INFO [train.py:1192] (1/2) Epoch 39, batch 300, loss[loss=0.2863, simple_loss=0.4096, pruned_loss=0.08143, over 24552.00 frames. ], tot_loss[loss=0.2764, simple_loss=0.3886, pruned_loss=0.08211, over 3756516.32 frames. ], batch size: 204, lr: 5.42e-03, grad_scale: 32.0 2026-09-24 03:05:54,927 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.81 vs. limit=15.0 2026-09-24 03:06:06,880 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.529e+02 3.291e+02 3.545e+02 3.950e+02 6.010e+02, threshold=7.091e+02, percent-clipped=0.0 2026-09-24 03:06:07,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=122446.66666666667, ans=0.125 2026-09-24 03:06:08,857 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=122446.66666666667, ans=0.1 2026-09-24 03:06:14,914 INFO [train.py:1192] (1/2) Epoch 39, batch 350, loss[loss=0.2295, simple_loss=0.3397, pruned_loss=0.05961, over 24575.00 frames. ], tot_loss[loss=0.2768, simple_loss=0.389, pruned_loss=0.08226, over 3997315.49 frames. ], batch size: 137, lr: 5.42e-03, grad_scale: 32.0 2026-09-24 03:06:18,849 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=122513.33333333333, ans=0.05 2026-09-24 03:06:24,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=122546.66666666667, ans=0.125 2026-09-24 03:06:40,564 INFO [train.py:1192] (1/2) Epoch 39, batch 400, loss[loss=0.3074, simple_loss=0.4147, pruned_loss=0.1, over 24565.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3879, pruned_loss=0.0817, over 4182179.22 frames. ], batch size: 170, lr: 5.41e-03, grad_scale: 32.0 2026-09-24 03:06:43,813 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=122680.0, ans=0.125 2026-09-24 03:06:45,837 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=122713.33333333333, ans=0.07 2026-09-24 03:06:50,437 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=122746.66666666667, ans=0.125 2026-09-24 03:06:55,722 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys.whitening_limit, batch_count=122780.0, ans=6.0 2026-09-24 03:06:57,820 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.331e+02 3.207e+02 3.695e+02 4.107e+02 6.392e+02, threshold=7.391e+02, percent-clipped=0.0 2026-09-24 03:06:58,369 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=122780.0, ans=0.125 2026-09-24 03:07:01,721 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=122813.33333333333, ans=0.125 2026-09-24 03:07:05,135 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=122813.33333333333, ans=0.125 2026-09-24 03:07:05,388 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.32 vs. limit=12.0 2026-09-24 03:07:06,106 INFO [train.py:1192] (1/2) Epoch 39, batch 450, loss[loss=0.2834, simple_loss=0.397, pruned_loss=0.08486, over 24643.00 frames. ], tot_loss[loss=0.2758, simple_loss=0.3881, pruned_loss=0.08176, over 4318856.37 frames. ], batch size: 175, lr: 5.41e-03, grad_scale: 32.0 2026-09-24 03:07:06,218 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=122846.66666666667, ans=0.125 2026-09-24 03:07:09,228 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=122846.66666666667, ans=0.2 2026-09-24 03:07:31,712 INFO [train.py:1192] (1/2) Epoch 39, batch 500, loss[loss=0.2776, simple_loss=0.4038, pruned_loss=0.07572, over 24527.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3869, pruned_loss=0.08148, over 4436818.87 frames. ], batch size: 218, lr: 5.41e-03, grad_scale: 32.0 2026-09-24 03:07:38,197 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=123046.66666666667, ans=0.0 2026-09-24 03:07:40,074 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=123046.66666666667, ans=10.0 2026-09-24 03:07:42,522 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.28 vs. limit=12.0 2026-09-24 03:07:49,208 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.615e+02 3.265e+02 3.601e+02 4.061e+02 6.133e+02, threshold=7.203e+02, percent-clipped=0.0 2026-09-24 03:07:50,266 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=123113.33333333333, ans=0.0 2026-09-24 03:07:53,155 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.23 vs. limit=15.0 2026-09-24 03:07:54,952 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=123146.66666666667, ans=0.125 2026-09-24 03:07:57,453 INFO [train.py:1192] (1/2) Epoch 39, batch 550, loss[loss=0.2895, simple_loss=0.4123, pruned_loss=0.08333, over 24272.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3876, pruned_loss=0.08183, over 4521904.21 frames. ], batch size: 257, lr: 5.40e-03, grad_scale: 32.0 2026-09-24 03:07:59,702 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=123180.0, ans=0.025 2026-09-24 03:08:05,827 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=2.623e-03 2026-09-24 03:08:21,453 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.77 vs. limit=15.0 2026-09-24 03:08:22,966 INFO [train.py:1192] (1/2) Epoch 39, batch 600, loss[loss=0.3198, simple_loss=0.4395, pruned_loss=0.1001, over 24446.00 frames. ], tot_loss[loss=0.276, simple_loss=0.388, pruned_loss=0.08206, over 4590183.38 frames. ], batch size: 235, lr: 5.40e-03, grad_scale: 32.0 2026-09-24 03:08:35,723 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=123413.33333333333, ans=0.1 2026-09-24 03:08:40,113 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.514e+02 3.211e+02 3.556e+02 3.999e+02 6.519e+02, threshold=7.112e+02, percent-clipped=0.0 2026-09-24 03:08:48,403 INFO [train.py:1192] (1/2) Epoch 39, batch 650, loss[loss=0.2572, simple_loss=0.366, pruned_loss=0.0742, over 24593.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3875, pruned_loss=0.08181, over 4654729.24 frames. ], batch size: 154, lr: 5.40e-03, grad_scale: 32.0 2026-09-24 03:08:48,971 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:08:56,093 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=123546.66666666667, ans=0.125 2026-09-24 03:09:13,271 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=123646.66666666667, ans=0.125 2026-09-24 03:09:14,146 INFO [train.py:1192] (1/2) Epoch 39, batch 700, loss[loss=0.279, simple_loss=0.386, pruned_loss=0.08598, over 24559.00 frames. ], tot_loss[loss=0.2764, simple_loss=0.3887, pruned_loss=0.082, over 4689568.78 frames. ], batch size: 158, lr: 5.39e-03, grad_scale: 32.0 2026-09-24 03:09:18,655 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.94 vs. limit=22.5 2026-09-24 03:09:18,879 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=123680.0, ans=0.1 2026-09-24 03:09:24,451 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=123746.66666666667, ans=0.0 2026-09-24 03:09:32,197 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.64 vs. limit=10.0 2026-09-24 03:09:32,402 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.634e+02 3.315e+02 3.820e+02 4.368e+02 6.372e+02, threshold=7.640e+02, percent-clipped=0.0 2026-09-24 03:09:36,023 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=123813.33333333333, ans=0.2 2026-09-24 03:09:40,290 INFO [train.py:1192] (1/2) Epoch 39, batch 750, loss[loss=0.2661, simple_loss=0.3863, pruned_loss=0.073, over 24609.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3882, pruned_loss=0.08204, over 4725474.46 frames. ], batch size: 175, lr: 5.39e-03, grad_scale: 32.0 2026-09-24 03:09:50,014 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.95 vs. limit=8.0 2026-09-24 03:09:56,827 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=123946.66666666667, ans=0.025 2026-09-24 03:09:57,823 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=123946.66666666667, ans=0.0 2026-09-24 03:10:03,446 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=123980.0, ans=0.1 2026-09-24 03:10:04,247 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.48 vs. limit=22.5 2026-09-24 03:10:05,325 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.95 vs. limit=15.0 2026-09-24 03:10:06,297 INFO [train.py:1192] (1/2) Epoch 39, batch 800, loss[loss=0.2448, simple_loss=0.3519, pruned_loss=0.06882, over 24540.00 frames. ], tot_loss[loss=0.2763, simple_loss=0.3882, pruned_loss=0.08223, over 4751146.04 frames. ], batch size: 137, lr: 5.39e-03, grad_scale: 32.0 2026-09-24 03:10:12,014 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=124046.66666666667, ans=0.1 2026-09-24 03:10:17,787 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=124080.0, ans=0.04949747468305833 2026-09-24 03:10:23,926 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten.whitening_limit, batch_count=124113.33333333333, ans=22.5 2026-09-24 03:10:24,160 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.717e+02 3.163e+02 3.551e+02 4.126e+02 5.357e+02, threshold=7.103e+02, percent-clipped=0.0 2026-09-24 03:10:31,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=124180.0, ans=0.0 2026-09-24 03:10:32,132 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.77 vs. limit=12.0 2026-09-24 03:10:32,370 INFO [train.py:1192] (1/2) Epoch 39, batch 850, loss[loss=0.3035, simple_loss=0.4203, pruned_loss=0.09332, over 24550.00 frames. ], tot_loss[loss=0.275, simple_loss=0.3869, pruned_loss=0.08157, over 4768987.56 frames. ], batch size: 204, lr: 5.38e-03, grad_scale: 32.0 2026-09-24 03:10:32,460 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=124180.0, ans=0.125 2026-09-24 03:10:33,568 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=124180.0, ans=0.2 2026-09-24 03:10:34,654 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.02 vs. limit=15.0 2026-09-24 03:10:49,819 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=124280.0, ans=0.125 2026-09-24 03:10:50,994 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=124280.0, ans=0.0 2026-09-24 03:10:58,698 INFO [train.py:1192] (1/2) Epoch 39, batch 900, loss[loss=0.2341, simple_loss=0.3485, pruned_loss=0.05982, over 24565.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3876, pruned_loss=0.08212, over 4779996.99 frames. ], batch size: 137, lr: 5.38e-03, grad_scale: 32.0 2026-09-24 03:11:05,326 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=124380.0, ans=0.125 2026-09-24 03:11:15,668 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.434e+02 3.204e+02 3.697e+02 4.181e+02 5.797e+02, threshold=7.393e+02, percent-clipped=0.0 2026-09-24 03:11:17,225 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=124446.66666666667, ans=0.0 2026-09-24 03:11:18,123 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=124480.0, ans=0.125 2026-09-24 03:11:24,009 INFO [train.py:1192] (1/2) Epoch 39, batch 950, loss[loss=0.377, simple_loss=0.4402, pruned_loss=0.1569, over 12320.00 frames. ], tot_loss[loss=0.2769, simple_loss=0.3871, pruned_loss=0.08338, over 4714604.87 frames. ], batch size: 333, lr: 5.37e-03, grad_scale: 32.0 2026-09-24 03:11:25,712 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=13.85 vs. limit=15.0 2026-09-24 03:11:35,755 INFO [train.py:1192] (1/2) Epoch 40, batch 0, loss[loss=0.2406, simple_loss=0.3471, pruned_loss=0.06703, over 24561.00 frames. ], tot_loss[loss=0.2406, simple_loss=0.3471, pruned_loss=0.06703, over 24561.00 frames. ], batch size: 137, lr: 5.31e-03, grad_scale: 32.0 2026-09-24 03:11:35,755 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 03:11:47,645 INFO [train.py:1224] (1/2) Epoch 40, validation: loss=0.176, simple_loss=0.2949, pruned_loss=0.0286, over 2564189.00 frames. 2026-09-24 03:11:47,645 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 03:11:53,741 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=124573.33333333333, ans=0.0 2026-09-24 03:11:57,804 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=124606.66666666667, ans=0.0 2026-09-24 03:11:59,495 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.73 vs. limit=15.0 2026-09-24 03:11:59,859 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=124606.66666666667, ans=0.125 2026-09-24 03:12:01,828 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=124606.66666666667, ans=0.125 2026-09-24 03:12:12,068 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=124673.33333333333, ans=0.1 2026-09-24 03:12:12,965 INFO [train.py:1192] (1/2) Epoch 40, batch 50, loss[loss=0.2098, simple_loss=0.3254, pruned_loss=0.04704, over 24255.00 frames. ], tot_loss[loss=0.2809, simple_loss=0.3932, pruned_loss=0.08429, over 1080286.28 frames. ], batch size: 125, lr: 5.30e-03, grad_scale: 32.0 2026-09-24 03:12:16,065 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=124706.66666666667, ans=0.0 2026-09-24 03:12:23,392 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=124773.33333333333, ans=0.025 2026-09-24 03:12:23,429 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=124773.33333333333, ans=0.0 2026-09-24 03:12:26,216 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.799e+02 3.387e+02 3.810e+02 4.289e+02 6.309e+02, threshold=7.620e+02, percent-clipped=0.0 2026-09-24 03:12:26,784 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=124773.33333333333, ans=0.0 2026-09-24 03:12:35,568 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.10 vs. limit=12.0 2026-09-24 03:12:38,644 INFO [train.py:1192] (1/2) Epoch 40, batch 100, loss[loss=0.2782, simple_loss=0.3874, pruned_loss=0.08447, over 24595.00 frames. ], tot_loss[loss=0.2847, simple_loss=0.3978, pruned_loss=0.08575, over 1915583.06 frames. ], batch size: 154, lr: 5.30e-03, grad_scale: 64.0 2026-09-24 03:12:38,730 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=124873.33333333333, ans=0.125 2026-09-24 03:12:41,682 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.76 vs. limit=22.5 2026-09-24 03:13:03,151 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=125006.66666666667, ans=0.0 2026-09-24 03:13:04,708 INFO [train.py:1192] (1/2) Epoch 40, batch 150, loss[loss=0.2147, simple_loss=0.3236, pruned_loss=0.05288, over 24237.00 frames. ], tot_loss[loss=0.28, simple_loss=0.3924, pruned_loss=0.08375, over 2561017.41 frames. ], batch size: 125, lr: 5.30e-03, grad_scale: 64.0 2026-09-24 03:13:12,527 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=14.52 vs. limit=22.5 2026-09-24 03:13:13,608 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=125073.33333333333, ans=0.1 2026-09-24 03:13:18,084 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.382e+02 3.300e+02 3.693e+02 4.054e+02 6.775e+02, threshold=7.385e+02, percent-clipped=0.0 2026-09-24 03:13:23,863 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.08 vs. limit=15.0 2026-09-24 03:13:29,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=125173.33333333333, ans=0.125 2026-09-24 03:13:30,588 INFO [train.py:1192] (1/2) Epoch 40, batch 200, loss[loss=0.2994, simple_loss=0.4285, pruned_loss=0.08512, over 24194.00 frames. ], tot_loss[loss=0.2769, simple_loss=0.3896, pruned_loss=0.08205, over 3059320.47 frames. ], batch size: 257, lr: 5.29e-03, grad_scale: 64.0 2026-09-24 03:13:32,006 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=125206.66666666667, ans=0.125 2026-09-24 03:13:32,007 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=125206.66666666667, ans=0.125 2026-09-24 03:13:34,311 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=125206.66666666667, ans=0.025 2026-09-24 03:13:34,585 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.30 vs. limit=15.0 2026-09-24 03:13:54,199 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=125340.0, ans=0.125 2026-09-24 03:13:54,206 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=125340.0, ans=0.125 2026-09-24 03:13:56,672 INFO [train.py:1192] (1/2) Epoch 40, batch 250, loss[loss=0.2993, simple_loss=0.4228, pruned_loss=0.08787, over 24356.00 frames. ], tot_loss[loss=0.2776, simple_loss=0.3897, pruned_loss=0.08279, over 3442635.44 frames. ], batch size: 225, lr: 5.29e-03, grad_scale: 64.0 2026-09-24 03:13:58,994 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.49 vs. limit=22.5 2026-09-24 03:14:10,309 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.416e+02 3.427e+02 3.987e+02 4.778e+02 6.902e+02, threshold=7.973e+02, percent-clipped=0.0 2026-09-24 03:14:18,512 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.99 vs. limit=15.0 2026-09-24 03:14:22,781 INFO [train.py:1192] (1/2) Epoch 40, batch 300, loss[loss=0.2972, simple_loss=0.4241, pruned_loss=0.0852, over 24522.00 frames. ], tot_loss[loss=0.2771, simple_loss=0.3891, pruned_loss=0.08251, over 3755647.78 frames. ], batch size: 204, lr: 5.28e-03, grad_scale: 64.0 2026-09-24 03:14:29,407 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=125573.33333333333, ans=0.0 2026-09-24 03:14:30,231 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=125573.33333333333, ans=0.125 2026-09-24 03:14:34,437 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=125606.66666666667, ans=0.125 2026-09-24 03:14:34,882 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=125606.66666666667, ans=0.125 2026-09-24 03:14:38,569 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:14:41,464 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.82 vs. limit=22.5 2026-09-24 03:14:42,248 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=125640.0, ans=0.5 2026-09-24 03:14:43,210 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=125673.33333333333, ans=0.0 2026-09-24 03:14:43,728 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=125673.33333333333, ans=0.0 2026-09-24 03:14:48,293 INFO [train.py:1192] (1/2) Epoch 40, batch 350, loss[loss=0.2393, simple_loss=0.347, pruned_loss=0.06579, over 24562.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3894, pruned_loss=0.08256, over 3996842.25 frames. ], batch size: 137, lr: 5.28e-03, grad_scale: 64.0 2026-09-24 03:14:54,259 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.28 vs. limit=15.0 2026-09-24 03:15:01,562 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.650e+02 3.199e+02 3.646e+02 4.083e+02 5.507e+02, threshold=7.292e+02, percent-clipped=0.0 2026-09-24 03:15:05,444 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=125806.66666666667, ans=0.125 2026-09-24 03:15:09,358 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=125840.0, ans=0.1 2026-09-24 03:15:13,997 INFO [train.py:1192] (1/2) Epoch 40, batch 400, loss[loss=0.2772, simple_loss=0.3828, pruned_loss=0.08581, over 24559.00 frames. ], tot_loss[loss=0.2763, simple_loss=0.3885, pruned_loss=0.08202, over 4181042.62 frames. ], batch size: 170, lr: 5.28e-03, grad_scale: 64.0 2026-09-24 03:15:15,456 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=125873.33333333333, ans=0.125 2026-09-24 03:15:26,512 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=125940.0, ans=0.0 2026-09-24 03:15:28,766 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=125940.0, ans=0.125 2026-09-24 03:15:39,668 INFO [train.py:1192] (1/2) Epoch 40, batch 450, loss[loss=0.2898, simple_loss=0.4014, pruned_loss=0.0891, over 24627.00 frames. ], tot_loss[loss=0.2771, simple_loss=0.3892, pruned_loss=0.08257, over 4321643.92 frames. ], batch size: 175, lr: 5.27e-03, grad_scale: 64.0 2026-09-24 03:15:53,262 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.616e+02 3.315e+02 3.764e+02 4.234e+02 7.619e+02, threshold=7.527e+02, percent-clipped=1.0 2026-09-24 03:15:56,468 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=126140.0, ans=0.125 2026-09-24 03:15:56,468 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=126140.0, ans=0.05 2026-09-24 03:16:02,168 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=126173.33333333333, ans=0.0 2026-09-24 03:16:02,405 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.08 vs. limit=22.5 2026-09-24 03:16:06,365 INFO [train.py:1192] (1/2) Epoch 40, batch 500, loss[loss=0.2949, simple_loss=0.4118, pruned_loss=0.08902, over 24521.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.3875, pruned_loss=0.0818, over 4439009.88 frames. ], batch size: 218, lr: 5.27e-03, grad_scale: 64.0 2026-09-24 03:16:10,438 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=126206.66666666667, ans=0.0 2026-09-24 03:16:24,184 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:16:30,830 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.29 vs. limit=6.0 2026-09-24 03:16:32,541 INFO [train.py:1192] (1/2) Epoch 40, batch 550, loss[loss=0.3127, simple_loss=0.4273, pruned_loss=0.0991, over 24301.00 frames. ], tot_loss[loss=0.2769, simple_loss=0.3886, pruned_loss=0.08255, over 4523502.63 frames. ], batch size: 257, lr: 5.27e-03, grad_scale: 64.0 2026-09-24 03:16:35,524 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=126373.33333333333, ans=0.125 2026-09-24 03:16:35,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=126373.33333333333, ans=0.125 2026-09-24 03:16:45,766 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.472e+02 3.255e+02 3.676e+02 4.301e+02 8.242e+02, threshold=7.351e+02, percent-clipped=1.0 2026-09-24 03:16:48,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=126473.33333333333, ans=0.125 2026-09-24 03:16:58,500 INFO [train.py:1192] (1/2) Epoch 40, batch 600, loss[loss=0.3068, simple_loss=0.4259, pruned_loss=0.09381, over 24341.00 frames. ], tot_loss[loss=0.2775, simple_loss=0.3891, pruned_loss=0.0829, over 4589718.81 frames. ], batch size: 234, lr: 5.26e-03, grad_scale: 64.0 2026-09-24 03:17:01,164 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=126540.0, ans=0.1 2026-09-24 03:17:04,507 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=126573.33333333333, ans=0.125 2026-09-24 03:17:05,372 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=126573.33333333333, ans=0.125 2026-09-24 03:17:13,031 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=126606.66666666667, ans=0.125 2026-09-24 03:17:13,468 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=126640.0, ans=0.125 2026-09-24 03:17:14,587 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=126640.0, ans=0.125 2026-09-24 03:17:19,202 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=126673.33333333333, ans=0.0 2026-09-24 03:17:20,359 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.46 vs. limit=15.0 2026-09-24 03:17:23,963 INFO [train.py:1192] (1/2) Epoch 40, batch 650, loss[loss=0.2704, simple_loss=0.3753, pruned_loss=0.08279, over 24562.00 frames. ], tot_loss[loss=0.276, simple_loss=0.3879, pruned_loss=0.08204, over 4654668.67 frames. ], batch size: 154, lr: 5.26e-03, grad_scale: 64.0 2026-09-24 03:17:26,886 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=126706.66666666667, ans=0.125 2026-09-24 03:17:28,261 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=126706.66666666667, ans=0.125 2026-09-24 03:17:29,205 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=126740.0, ans=0.0 2026-09-24 03:17:36,717 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=8.80 vs. limit=10.0 2026-09-24 03:17:37,286 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.662e+02 3.206e+02 3.623e+02 4.248e+02 7.318e+02, threshold=7.246e+02, percent-clipped=0.0 2026-09-24 03:17:37,390 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=126773.33333333333, ans=0.125 2026-09-24 03:17:38,342 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=126773.33333333333, ans=0.125 2026-09-24 03:17:45,257 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=126840.0, ans=0.125 2026-09-24 03:17:49,654 INFO [train.py:1192] (1/2) Epoch 40, batch 700, loss[loss=0.2804, simple_loss=0.3899, pruned_loss=0.08541, over 24555.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3885, pruned_loss=0.08192, over 4689756.13 frames. ], batch size: 158, lr: 5.26e-03, grad_scale: 64.0 2026-09-24 03:17:49,780 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=126873.33333333333, ans=0.125 2026-09-24 03:17:54,422 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.29 vs. limit=22.5 2026-09-24 03:18:15,795 INFO [train.py:1192] (1/2) Epoch 40, batch 750, loss[loss=0.2784, simple_loss=0.398, pruned_loss=0.07944, over 24627.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3872, pruned_loss=0.08145, over 4723364.78 frames. ], batch size: 175, lr: 5.25e-03, grad_scale: 64.0 2026-09-24 03:18:21,229 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=127073.33333333333, ans=0.0 2026-09-24 03:18:25,849 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=127073.33333333333, ans=0.125 2026-09-24 03:18:27,202 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=127106.66666666667, ans=0.0 2026-09-24 03:18:28,645 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten.whitening_limit, batch_count=127106.66666666667, ans=15.0 2026-09-24 03:18:29,577 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.517e+02 3.310e+02 3.828e+02 4.163e+02 6.054e+02, threshold=7.657e+02, percent-clipped=0.0 2026-09-24 03:18:34,069 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=127140.0, ans=0.125 2026-09-24 03:18:38,349 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=127173.33333333333, ans=0.0 2026-09-24 03:18:39,523 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.46 vs. limit=15.0 2026-09-24 03:18:41,726 INFO [train.py:1192] (1/2) Epoch 40, batch 800, loss[loss=0.2225, simple_loss=0.3346, pruned_loss=0.05522, over 24527.00 frames. ], tot_loss[loss=0.2752, simple_loss=0.3872, pruned_loss=0.0816, over 4749260.44 frames. ], batch size: 137, lr: 5.25e-03, grad_scale: 64.0 2026-09-24 03:18:42,355 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=127206.66666666667, ans=0.125 2026-09-24 03:18:49,132 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=127240.0, ans=0.1 2026-09-24 03:18:53,439 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=127273.33333333333, ans=0.0 2026-09-24 03:19:01,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=127340.0, ans=0.025 2026-09-24 03:19:02,822 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=127340.0, ans=0.1 2026-09-24 03:19:04,353 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=127340.0, ans=0.0 2026-09-24 03:19:05,576 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.00 vs. limit=22.5 2026-09-24 03:19:07,091 INFO [train.py:1192] (1/2) Epoch 40, batch 850, loss[loss=0.2998, simple_loss=0.4203, pruned_loss=0.08961, over 24546.00 frames. ], tot_loss[loss=0.2745, simple_loss=0.3869, pruned_loss=0.0811, over 4769952.54 frames. ], batch size: 204, lr: 5.25e-03, grad_scale: 64.0 2026-09-24 03:19:10,862 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=127373.33333333333, ans=0.125 2026-09-24 03:19:14,162 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.69 vs. limit=15.0 2026-09-24 03:19:20,519 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.749e+02 3.370e+02 3.782e+02 4.373e+02 5.808e+02, threshold=7.564e+02, percent-clipped=0.0 2026-09-24 03:19:23,659 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=17.06 vs. limit=15.0 2026-09-24 03:19:32,624 INFO [train.py:1192] (1/2) Epoch 40, batch 900, loss[loss=0.2413, simple_loss=0.3532, pruned_loss=0.06474, over 24563.00 frames. ], tot_loss[loss=0.2751, simple_loss=0.3873, pruned_loss=0.08148, over 4781189.38 frames. ], batch size: 137, lr: 5.24e-03, grad_scale: 64.0 2026-09-24 03:19:45,883 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.85 vs. limit=15.0 2026-09-24 03:19:49,329 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=127640.0, ans=0.125 2026-09-24 03:19:58,309 INFO [train.py:1192] (1/2) Epoch 40, batch 950, loss[loss=0.3853, simple_loss=0.4528, pruned_loss=0.1589, over 10734.00 frames. ], tot_loss[loss=0.2761, simple_loss=0.3867, pruned_loss=0.08271, over 4709472.41 frames. ], batch size: 334, lr: 5.24e-03, grad_scale: 32.0 2026-09-24 03:19:59,919 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.49 vs. limit=22.5 2026-09-24 03:20:09,889 INFO [train.py:1192] (1/2) Epoch 41, batch 0, loss[loss=0.2428, simple_loss=0.3561, pruned_loss=0.06479, over 24580.00 frames. ], tot_loss[loss=0.2428, simple_loss=0.3561, pruned_loss=0.06479, over 24580.00 frames. ], batch size: 137, lr: 5.18e-03, grad_scale: 32.0 2026-09-24 03:20:09,889 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 03:20:21,759 INFO [train.py:1224] (1/2) Epoch 41, validation: loss=0.1763, simple_loss=0.295, pruned_loss=0.02877, over 2564189.00 frames. 2026-09-24 03:20:21,760 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 03:20:21,853 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=127733.33333333333, ans=0.125 2026-09-24 03:20:29,001 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=127766.66666666667, ans=0.0 2026-09-24 03:20:29,498 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=127766.66666666667, ans=0.2 2026-09-24 03:20:31,164 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.577e+02 3.354e+02 3.712e+02 4.319e+02 6.875e+02, threshold=7.424e+02, percent-clipped=0.0 2026-09-24 03:20:35,550 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=127800.0, ans=0.125 2026-09-24 03:20:45,408 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=127866.66666666667, ans=0.1 2026-09-24 03:20:46,349 INFO [train.py:1192] (1/2) Epoch 41, batch 50, loss[loss=0.2207, simple_loss=0.3293, pruned_loss=0.05602, over 24301.00 frames. ], tot_loss[loss=0.2789, simple_loss=0.3908, pruned_loss=0.08346, over 1082284.20 frames. ], batch size: 125, lr: 5.17e-03, grad_scale: 32.0 2026-09-24 03:20:56,441 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.79 vs. limit=8.0 2026-09-24 03:21:04,501 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.34 vs. limit=12.0 2026-09-24 03:21:05,999 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.08 vs. limit=6.0 2026-09-24 03:21:07,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=128033.33333333333, ans=0.125 2026-09-24 03:21:11,880 INFO [train.py:1192] (1/2) Epoch 41, batch 100, loss[loss=0.2578, simple_loss=0.368, pruned_loss=0.07373, over 24553.00 frames. ], tot_loss[loss=0.2817, simple_loss=0.3951, pruned_loss=0.08419, over 1915173.37 frames. ], batch size: 154, lr: 5.17e-03, grad_scale: 32.0 2026-09-24 03:21:14,140 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.28 vs. limit=15.0 2026-09-24 03:21:18,031 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=128100.0, ans=0.0 2026-09-24 03:21:21,532 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.622e+02 3.322e+02 3.653e+02 4.087e+02 6.031e+02, threshold=7.305e+02, percent-clipped=0.0 2026-09-24 03:21:24,340 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.96 vs. limit=22.5 2026-09-24 03:21:36,894 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=128233.33333333333, ans=0.1 2026-09-24 03:21:37,300 INFO [train.py:1192] (1/2) Epoch 41, batch 150, loss[loss=0.2244, simple_loss=0.3325, pruned_loss=0.05813, over 24260.00 frames. ], tot_loss[loss=0.277, simple_loss=0.3899, pruned_loss=0.08199, over 2560178.90 frames. ], batch size: 125, lr: 5.17e-03, grad_scale: 32.0 2026-09-24 03:21:39,391 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.97 vs. limit=12.0 2026-09-24 03:21:42,173 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=128266.66666666667, ans=0.2 2026-09-24 03:21:59,342 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=128366.66666666667, ans=0.025 2026-09-24 03:22:03,508 INFO [train.py:1192] (1/2) Epoch 41, batch 200, loss[loss=0.294, simple_loss=0.4203, pruned_loss=0.08385, over 24202.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3879, pruned_loss=0.08079, over 3058799.25 frames. ], batch size: 257, lr: 5.16e-03, grad_scale: 32.0 2026-09-24 03:22:04,063 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=128400.0, ans=0.0 2026-09-24 03:22:08,980 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.13 vs. limit=6.0 2026-09-24 03:22:13,110 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.657e+02 3.570e+02 3.881e+02 4.626e+02 6.095e+02, threshold=7.762e+02, percent-clipped=0.0 2026-09-24 03:22:28,570 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=128533.33333333333, ans=0.1 2026-09-24 03:22:29,464 INFO [train.py:1192] (1/2) Epoch 41, batch 250, loss[loss=0.3039, simple_loss=0.4221, pruned_loss=0.09288, over 24367.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3876, pruned_loss=0.08113, over 3443099.46 frames. ], batch size: 225, lr: 5.16e-03, grad_scale: 32.0 2026-09-24 03:22:41,893 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.36 vs. limit=15.0 2026-09-24 03:22:46,193 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=128666.66666666667, ans=0.125 2026-09-24 03:22:46,830 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.84 vs. limit=22.5 2026-09-24 03:22:54,146 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=128700.0, ans=0.1 2026-09-24 03:22:55,051 INFO [train.py:1192] (1/2) Epoch 41, batch 300, loss[loss=0.2972, simple_loss=0.4166, pruned_loss=0.08896, over 24527.00 frames. ], tot_loss[loss=0.275, simple_loss=0.3874, pruned_loss=0.08128, over 3757356.81 frames. ], batch size: 204, lr: 5.16e-03, grad_scale: 32.0 2026-09-24 03:23:01,170 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.08 vs. limit=10.0 2026-09-24 03:23:01,547 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=128766.66666666667, ans=0.2 2026-09-24 03:23:03,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=128766.66666666667, ans=0.125 2026-09-24 03:23:05,200 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.682e+02 3.400e+02 3.675e+02 4.084e+02 5.232e+02, threshold=7.351e+02, percent-clipped=0.0 2026-09-24 03:23:14,701 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=10.38 vs. limit=15.0 2026-09-24 03:23:15,496 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:23:21,015 INFO [train.py:1192] (1/2) Epoch 41, batch 350, loss[loss=0.2187, simple_loss=0.3327, pruned_loss=0.05233, over 24601.00 frames. ], tot_loss[loss=0.2759, simple_loss=0.3881, pruned_loss=0.08187, over 3997782.51 frames. ], batch size: 137, lr: 5.15e-03, grad_scale: 32.0 2026-09-24 03:23:37,354 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.29 vs. limit=15.0 2026-09-24 03:23:46,516 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=129066.66666666667, ans=0.0 2026-09-24 03:23:46,955 INFO [train.py:1192] (1/2) Epoch 41, batch 400, loss[loss=0.2766, simple_loss=0.387, pruned_loss=0.08312, over 24559.00 frames. ], tot_loss[loss=0.2748, simple_loss=0.3872, pruned_loss=0.08125, over 4182663.94 frames. ], batch size: 170, lr: 5.15e-03, grad_scale: 32.0 2026-09-24 03:23:50,132 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=6.93 vs. limit=10.0 2026-09-24 03:23:56,978 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.582e+02 3.301e+02 3.573e+02 3.910e+02 6.479e+02, threshold=7.145e+02, percent-clipped=0.0 2026-09-24 03:23:57,590 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=129133.33333333333, ans=0.125 2026-09-24 03:23:59,138 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.81 vs. limit=22.5 2026-09-24 03:24:06,536 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=129166.66666666667, ans=0.125 2026-09-24 03:24:12,981 INFO [train.py:1192] (1/2) Epoch 41, batch 450, loss[loss=0.2669, simple_loss=0.3866, pruned_loss=0.07363, over 24638.00 frames. ], tot_loss[loss=0.2766, simple_loss=0.3883, pruned_loss=0.08248, over 4321777.72 frames. ], batch size: 175, lr: 5.15e-03, grad_scale: 32.0 2026-09-24 03:24:25,714 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=129300.0, ans=0.0 2026-09-24 03:24:29,178 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.47 vs. limit=10.0 2026-09-24 03:24:29,809 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.36 vs. limit=22.5 2026-09-24 03:24:34,870 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=129366.66666666667, ans=0.2 2026-09-24 03:24:35,874 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=129366.66666666667, ans=0.0 2026-09-24 03:24:37,006 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=129366.66666666667, ans=0.125 2026-09-24 03:24:39,063 INFO [train.py:1192] (1/2) Epoch 41, batch 500, loss[loss=0.2989, simple_loss=0.41, pruned_loss=0.09393, over 24514.00 frames. ], tot_loss[loss=0.2746, simple_loss=0.3865, pruned_loss=0.08135, over 4438229.80 frames. ], batch size: 218, lr: 5.14e-03, grad_scale: 32.0 2026-09-24 03:24:39,144 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=129400.0, ans=0.125 2026-09-24 03:24:42,437 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=129400.0, ans=0.2 2026-09-24 03:24:44,332 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=129433.33333333333, ans=0.1 2026-09-24 03:24:45,355 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=129433.33333333333, ans=0.0 2026-09-24 03:24:46,858 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=129433.33333333333, ans=0.09899494936611666 2026-09-24 03:24:48,631 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.368e+02 3.221e+02 3.628e+02 4.202e+02 5.845e+02, threshold=7.256e+02, percent-clipped=0.0 2026-09-24 03:25:03,227 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.45 vs. limit=15.0 2026-09-24 03:25:05,015 INFO [train.py:1192] (1/2) Epoch 41, batch 550, loss[loss=0.3413, simple_loss=0.4515, pruned_loss=0.1155, over 24262.00 frames. ], tot_loss[loss=0.2752, simple_loss=0.3873, pruned_loss=0.08158, over 4522668.36 frames. ], batch size: 257, lr: 5.14e-03, grad_scale: 32.0 2026-09-24 03:25:16,877 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=129633.33333333333, ans=0.025 2026-09-24 03:25:26,887 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=129700.0, ans=0.025 2026-09-24 03:25:30,675 INFO [train.py:1192] (1/2) Epoch 41, batch 600, loss[loss=0.329, simple_loss=0.4428, pruned_loss=0.1076, over 24331.00 frames. ], tot_loss[loss=0.2756, simple_loss=0.388, pruned_loss=0.08167, over 4589092.79 frames. ], batch size: 234, lr: 5.14e-03, grad_scale: 32.0 2026-09-24 03:25:31,208 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=129733.33333333333, ans=0.125 2026-09-24 03:25:33,521 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=129733.33333333333, ans=0.1 2026-09-24 03:25:34,077 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=129733.33333333333, ans=0.0 2026-09-24 03:25:39,861 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=129766.66666666667, ans=0.125 2026-09-24 03:25:40,286 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.644e+02 3.363e+02 3.684e+02 4.152e+02 7.573e+02, threshold=7.368e+02, percent-clipped=1.0 2026-09-24 03:25:50,313 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=129866.66666666667, ans=0.07 2026-09-24 03:25:55,733 INFO [train.py:1192] (1/2) Epoch 41, batch 650, loss[loss=0.2591, simple_loss=0.3668, pruned_loss=0.07573, over 24572.00 frames. ], tot_loss[loss=0.2736, simple_loss=0.3864, pruned_loss=0.08046, over 4653906.46 frames. ], batch size: 154, lr: 5.13e-03, grad_scale: 32.0 2026-09-24 03:25:59,226 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:26:01,276 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.11 vs. limit=15.0 2026-09-24 03:26:19,910 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=130033.33333333333, ans=0.0 2026-09-24 03:26:20,842 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=130033.33333333333, ans=0.2 2026-09-24 03:26:21,744 INFO [train.py:1192] (1/2) Epoch 41, batch 700, loss[loss=0.2685, simple_loss=0.3787, pruned_loss=0.07915, over 24560.00 frames. ], tot_loss[loss=0.275, simple_loss=0.3878, pruned_loss=0.08115, over 4688833.51 frames. ], batch size: 158, lr: 5.13e-03, grad_scale: 32.0 2026-09-24 03:26:23,633 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=130066.66666666667, ans=0.125 2026-09-24 03:26:31,725 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.677e+02 3.266e+02 3.752e+02 4.247e+02 5.971e+02, threshold=7.505e+02, percent-clipped=0.0 2026-09-24 03:26:33,620 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=130133.33333333333, ans=0.2 2026-09-24 03:26:36,696 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=130166.66666666667, ans=0.1 2026-09-24 03:26:37,711 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.02 vs. limit=6.0 2026-09-24 03:26:45,104 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.13 vs. limit=10.0 2026-09-24 03:26:46,819 INFO [train.py:1192] (1/2) Epoch 41, batch 750, loss[loss=0.2638, simple_loss=0.383, pruned_loss=0.07227, over 24648.00 frames. ], tot_loss[loss=0.2739, simple_loss=0.3865, pruned_loss=0.08069, over 4725863.65 frames. ], batch size: 175, lr: 5.13e-03, grad_scale: 32.0 2026-09-24 03:27:01,782 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=130333.33333333333, ans=0.125 2026-09-24 03:27:04,522 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.21 vs. limit=6.0 2026-09-24 03:27:06,502 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=130333.33333333333, ans=0.125 2026-09-24 03:27:12,708 INFO [train.py:1192] (1/2) Epoch 41, batch 800, loss[loss=0.2282, simple_loss=0.3413, pruned_loss=0.05752, over 24533.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3862, pruned_loss=0.08056, over 4752033.86 frames. ], batch size: 137, lr: 5.12e-03, grad_scale: 32.0 2026-09-24 03:27:22,723 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.416e+02 3.285e+02 3.718e+02 4.236e+02 6.146e+02, threshold=7.436e+02, percent-clipped=0.0 2026-09-24 03:27:27,486 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=130466.66666666667, ans=0.05 2026-09-24 03:27:29,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=130500.0, ans=0.0 2026-09-24 03:27:38,541 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=130566.66666666667, ans=0.0 2026-09-24 03:27:38,900 INFO [train.py:1192] (1/2) Epoch 41, batch 850, loss[loss=0.2975, simple_loss=0.4163, pruned_loss=0.0893, over 24545.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.3863, pruned_loss=0.08092, over 4771022.79 frames. ], batch size: 204, lr: 5.12e-03, grad_scale: 32.0 2026-09-24 03:27:46,577 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=130600.0, ans=0.125 2026-09-24 03:27:52,323 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:28:03,074 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.01 vs. limit=10.0 2026-09-24 03:28:03,610 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=130700.0, ans=0.0 2026-09-24 03:28:03,795 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.55 vs. limit=15.0 2026-09-24 03:28:05,006 INFO [train.py:1192] (1/2) Epoch 41, batch 900, loss[loss=0.2349, simple_loss=0.3497, pruned_loss=0.06004, over 24553.00 frames. ], tot_loss[loss=0.275, simple_loss=0.3871, pruned_loss=0.08142, over 4781813.24 frames. ], batch size: 137, lr: 5.12e-03, grad_scale: 32.0 2026-09-24 03:28:14,671 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.270e+02 3.061e+02 3.538e+02 4.190e+02 5.569e+02, threshold=7.076e+02, percent-clipped=0.0 2026-09-24 03:28:14,783 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=130800.0, ans=0.0 2026-09-24 03:28:17,062 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=130800.0, ans=0.0 2026-09-24 03:28:18,530 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=130800.0, ans=0.0 2026-09-24 03:28:20,472 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=130833.33333333333, ans=0.125 2026-09-24 03:28:21,432 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=130833.33333333333, ans=0.1 2026-09-24 03:28:26,432 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=130866.66666666667, ans=0.0 2026-09-24 03:28:30,318 INFO [train.py:1192] (1/2) Epoch 41, batch 950, loss[loss=0.3542, simple_loss=0.4158, pruned_loss=0.1463, over 11747.00 frames. ], tot_loss[loss=0.2749, simple_loss=0.3855, pruned_loss=0.08215, over 4717560.38 frames. ], batch size: 333, lr: 5.11e-03, grad_scale: 32.0 2026-09-24 03:28:30,854 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=130900.0, ans=15.0 2026-09-24 03:28:31,072 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=130900.0, ans=0.125 2026-09-24 03:28:41,255 INFO [train.py:1192] (1/2) Epoch 42, batch 0, loss[loss=0.232, simple_loss=0.3498, pruned_loss=0.05708, over 24589.00 frames. ], tot_loss[loss=0.232, simple_loss=0.3498, pruned_loss=0.05708, over 24589.00 frames. ], batch size: 137, lr: 5.05e-03, grad_scale: 32.0 2026-09-24 03:28:41,255 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 03:28:53,097 INFO [train.py:1224] (1/2) Epoch 42, validation: loss=0.1768, simple_loss=0.2955, pruned_loss=0.02906, over 2564189.00 frames. 2026-09-24 03:28:53,097 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 03:29:19,484 INFO [train.py:1192] (1/2) Epoch 42, batch 50, loss[loss=0.2499, simple_loss=0.3539, pruned_loss=0.07295, over 24282.00 frames. ], tot_loss[loss=0.2838, simple_loss=0.395, pruned_loss=0.08627, over 1081996.90 frames. ], batch size: 125, lr: 5.05e-03, grad_scale: 32.0 2026-09-24 03:29:23,024 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=131093.33333333334, ans=0.0 2026-09-24 03:29:24,802 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.770e+02 3.478e+02 3.914e+02 4.628e+02 6.581e+02, threshold=7.828e+02, percent-clipped=0.0 2026-09-24 03:29:31,231 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.02 vs. limit=15.0 2026-09-24 03:29:36,513 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=131193.33333333334, ans=0.1 2026-09-24 03:29:41,803 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=131226.66666666666, ans=0.125 2026-09-24 03:29:45,044 INFO [train.py:1192] (1/2) Epoch 42, batch 100, loss[loss=0.2739, simple_loss=0.3778, pruned_loss=0.08494, over 24620.00 frames. ], tot_loss[loss=0.2824, simple_loss=0.396, pruned_loss=0.08442, over 1916543.99 frames. ], batch size: 154, lr: 5.05e-03, grad_scale: 32.0 2026-09-24 03:29:58,199 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=131326.66666666666, ans=0.04949747468305833 2026-09-24 03:30:02,241 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=131360.0, ans=0.025 2026-09-24 03:30:10,621 INFO [train.py:1192] (1/2) Epoch 42, batch 150, loss[loss=0.2249, simple_loss=0.3252, pruned_loss=0.06229, over 24265.00 frames. ], tot_loss[loss=0.277, simple_loss=0.3898, pruned_loss=0.08206, over 2562198.48 frames. ], batch size: 125, lr: 5.04e-03, grad_scale: 32.0 2026-09-24 03:30:16,132 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.645e+02 3.318e+02 3.586e+02 4.260e+02 6.638e+02, threshold=7.171e+02, percent-clipped=0.0 2026-09-24 03:30:31,658 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=131560.0, ans=0.125 2026-09-24 03:30:36,151 INFO [train.py:1192] (1/2) Epoch 42, batch 200, loss[loss=0.3069, simple_loss=0.4253, pruned_loss=0.09421, over 24215.00 frames. ], tot_loss[loss=0.2726, simple_loss=0.3863, pruned_loss=0.07943, over 3062099.96 frames. ], batch size: 257, lr: 5.04e-03, grad_scale: 32.0 2026-09-24 03:30:43,645 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=131626.66666666666, ans=0.125 2026-09-24 03:30:44,530 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=131626.66666666666, ans=0.1 2026-09-24 03:30:50,866 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=131660.0, ans=0.0 2026-09-24 03:31:01,943 INFO [train.py:1192] (1/2) Epoch 42, batch 250, loss[loss=0.2961, simple_loss=0.4153, pruned_loss=0.08839, over 24376.00 frames. ], tot_loss[loss=0.2733, simple_loss=0.3865, pruned_loss=0.08005, over 3444261.58 frames. ], batch size: 225, lr: 5.04e-03, grad_scale: 32.0 2026-09-24 03:31:02,511 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.47 vs. limit=6.0 2026-09-24 03:31:07,513 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.589e+02 3.539e+02 4.041e+02 4.530e+02 7.446e+02, threshold=8.082e+02, percent-clipped=1.0 2026-09-24 03:31:11,791 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=131826.66666666666, ans=0.0 2026-09-24 03:31:20,899 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=131860.0, ans=0.0 2026-09-24 03:31:27,551 INFO [train.py:1192] (1/2) Epoch 42, batch 300, loss[loss=0.2791, simple_loss=0.4037, pruned_loss=0.07722, over 24556.00 frames. ], tot_loss[loss=0.2731, simple_loss=0.3862, pruned_loss=0.07994, over 3758065.26 frames. ], batch size: 204, lr: 5.03e-03, grad_scale: 32.0 2026-09-24 03:31:29,629 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=131926.66666666666, ans=0.125 2026-09-24 03:31:32,565 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=131960.0, ans=0.1 2026-09-24 03:31:36,179 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=131960.0, ans=0.125 2026-09-24 03:31:45,638 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=132026.66666666666, ans=0.2 2026-09-24 03:31:47,059 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=132026.66666666666, ans=0.0 2026-09-24 03:31:52,077 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=132060.0, ans=0.0 2026-09-24 03:31:53,357 INFO [train.py:1192] (1/2) Epoch 42, batch 350, loss[loss=0.2237, simple_loss=0.3355, pruned_loss=0.05596, over 24600.00 frames. ], tot_loss[loss=0.2735, simple_loss=0.3868, pruned_loss=0.08011, over 3998172.54 frames. ], batch size: 137, lr: 5.03e-03, grad_scale: 32.0 2026-09-24 03:31:58,552 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.397e+02 3.160e+02 3.538e+02 3.995e+02 6.562e+02, threshold=7.075e+02, percent-clipped=0.0 2026-09-24 03:32:01,165 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=14.13 vs. limit=22.5 2026-09-24 03:32:02,389 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=132126.66666666666, ans=0.125 2026-09-24 03:32:05,964 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=132160.0, ans=0.125 2026-09-24 03:32:19,030 INFO [train.py:1192] (1/2) Epoch 42, batch 400, loss[loss=0.2618, simple_loss=0.3766, pruned_loss=0.07351, over 24557.00 frames. ], tot_loss[loss=0.2727, simple_loss=0.3858, pruned_loss=0.07978, over 4182349.31 frames. ], batch size: 170, lr: 5.03e-03, grad_scale: 32.0 2026-09-24 03:32:20,699 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=132260.0, ans=0.2 2026-09-24 03:32:23,171 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=132260.0, ans=0.125 2026-09-24 03:32:29,051 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=132326.66666666666, ans=0.1 2026-09-24 03:32:32,698 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=132326.66666666666, ans=0.125 2026-09-24 03:32:41,839 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.16 vs. limit=15.0 2026-09-24 03:32:44,927 INFO [train.py:1192] (1/2) Epoch 42, batch 450, loss[loss=0.2775, simple_loss=0.3951, pruned_loss=0.07991, over 24605.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.387, pruned_loss=0.0806, over 4321586.13 frames. ], batch size: 175, lr: 5.02e-03, grad_scale: 32.0 2026-09-24 03:32:50,705 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.418e+02 3.287e+02 3.646e+02 4.125e+02 6.218e+02, threshold=7.292e+02, percent-clipped=0.0 2026-09-24 03:33:03,237 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=132526.66666666666, ans=0.125 2026-09-24 03:33:06,737 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.74 vs. limit=22.5 2026-09-24 03:33:09,469 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.24 vs. limit=15.0 2026-09-24 03:33:10,610 INFO [train.py:1192] (1/2) Epoch 42, batch 500, loss[loss=0.3045, simple_loss=0.4209, pruned_loss=0.09407, over 24499.00 frames. ], tot_loss[loss=0.2725, simple_loss=0.3852, pruned_loss=0.07985, over 4438994.28 frames. ], batch size: 218, lr: 5.02e-03, grad_scale: 32.0 2026-09-24 03:33:21,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=132660.0, ans=0.125 2026-09-24 03:33:23,085 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=132660.0, ans=0.1 2026-09-24 03:33:23,601 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=132660.0, ans=0.0 2026-09-24 03:33:32,796 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=132726.66666666666, ans=0.125 2026-09-24 03:33:36,588 INFO [train.py:1192] (1/2) Epoch 42, batch 550, loss[loss=0.2775, simple_loss=0.4075, pruned_loss=0.07369, over 24259.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3854, pruned_loss=0.07961, over 4523792.13 frames. ], batch size: 257, lr: 5.02e-03, grad_scale: 32.0 2026-09-24 03:33:38,094 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=132760.0, ans=0.0 2026-09-24 03:33:42,505 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.595e+02 3.112e+02 3.537e+02 3.986e+02 5.703e+02, threshold=7.073e+02, percent-clipped=0.0 2026-09-24 03:33:42,822 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.15 vs. limit=15.0 2026-09-24 03:33:43,104 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=132793.33333333334, ans=0.0 2026-09-24 03:33:44,527 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=132793.33333333334, ans=0.125 2026-09-24 03:34:02,757 INFO [train.py:1192] (1/2) Epoch 42, batch 600, loss[loss=0.29, simple_loss=0.4108, pruned_loss=0.08461, over 24337.00 frames. ], tot_loss[loss=0.2728, simple_loss=0.3859, pruned_loss=0.07979, over 4590325.23 frames. ], batch size: 234, lr: 5.02e-03, grad_scale: 32.0 2026-09-24 03:34:05,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=132926.66666666666, ans=0.125 2026-09-24 03:34:28,074 INFO [train.py:1192] (1/2) Epoch 42, batch 650, loss[loss=0.2786, simple_loss=0.383, pruned_loss=0.08715, over 24616.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3855, pruned_loss=0.07956, over 4654744.35 frames. ], batch size: 154, lr: 5.01e-03, grad_scale: 32.0 2026-09-24 03:34:33,630 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.432e+02 3.326e+02 3.659e+02 3.994e+02 6.136e+02, threshold=7.318e+02, percent-clipped=0.0 2026-09-24 03:34:40,982 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=133160.0, ans=0.2 2026-09-24 03:34:48,804 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.25 vs. limit=22.5 2026-09-24 03:34:53,865 INFO [train.py:1192] (1/2) Epoch 42, batch 700, loss[loss=0.2719, simple_loss=0.3843, pruned_loss=0.07969, over 24562.00 frames. ], tot_loss[loss=0.273, simple_loss=0.3864, pruned_loss=0.07974, over 4689500.04 frames. ], batch size: 158, lr: 5.01e-03, grad_scale: 32.0 2026-09-24 03:34:56,003 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=133260.0, ans=0.125 2026-09-24 03:35:02,512 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.57 vs. limit=12.0 2026-09-24 03:35:10,331 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.73 vs. limit=15.0 2026-09-24 03:35:14,222 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=133393.33333333334, ans=0.2 2026-09-24 03:35:19,963 INFO [train.py:1192] (1/2) Epoch 42, batch 750, loss[loss=0.2826, simple_loss=0.3992, pruned_loss=0.08296, over 24597.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3857, pruned_loss=0.07951, over 4726246.61 frames. ], batch size: 175, lr: 5.01e-03, grad_scale: 32.0 2026-09-24 03:35:25,862 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.650e+02 3.266e+02 3.751e+02 4.333e+02 6.246e+02, threshold=7.502e+02, percent-clipped=0.0 2026-09-24 03:35:31,556 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=133493.33333333334, ans=0.125 2026-09-24 03:35:33,002 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=133493.33333333334, ans=0.125 2026-09-24 03:35:35,544 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=133526.66666666666, ans=0.125 2026-09-24 03:35:36,515 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=133526.66666666666, ans=0.125 2026-09-24 03:35:45,904 INFO [train.py:1192] (1/2) Epoch 42, batch 800, loss[loss=0.2396, simple_loss=0.3493, pruned_loss=0.06492, over 24514.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3853, pruned_loss=0.07933, over 4753043.54 frames. ], batch size: 137, lr: 5.00e-03, grad_scale: 32.0 2026-09-24 03:35:47,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=133593.33333333334, ans=0.0 2026-09-24 03:35:49,062 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=133593.33333333334, ans=0.125 2026-09-24 03:35:51,576 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=133626.66666666666, ans=0.0 2026-09-24 03:36:11,451 INFO [train.py:1192] (1/2) Epoch 42, batch 850, loss[loss=0.2798, simple_loss=0.408, pruned_loss=0.07578, over 24550.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3846, pruned_loss=0.07892, over 4772310.37 frames. ], batch size: 204, lr: 5.00e-03, grad_scale: 32.0 2026-09-24 03:36:13,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=133760.0, ans=0.0 2026-09-24 03:36:17,257 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.579e+02 3.197e+02 3.653e+02 4.244e+02 6.667e+02, threshold=7.306e+02, percent-clipped=0.0 2026-09-24 03:36:29,013 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.69 vs. limit=22.5 2026-09-24 03:36:32,043 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=133893.33333333334, ans=0.125 2026-09-24 03:36:37,492 INFO [train.py:1192] (1/2) Epoch 42, batch 900, loss[loss=0.2289, simple_loss=0.3462, pruned_loss=0.05578, over 24574.00 frames. ], tot_loss[loss=0.2718, simple_loss=0.3852, pruned_loss=0.0792, over 4782981.23 frames. ], batch size: 137, lr: 5.00e-03, grad_scale: 32.0 2026-09-24 03:36:48,771 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=133993.33333333334, ans=0.1 2026-09-24 03:37:03,541 INFO [train.py:1192] (1/2) Epoch 42, batch 950, loss[loss=0.3423, simple_loss=0.4015, pruned_loss=0.1416, over 11345.00 frames. ], tot_loss[loss=0.2726, simple_loss=0.3843, pruned_loss=0.08046, over 4713367.79 frames. ], batch size: 333, lr: 4.99e-03, grad_scale: 32.0 2026-09-24 03:37:05,101 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=134093.33333333334, ans=0.125 2026-09-24 03:37:12,894 INFO [train.py:1192] (1/2) Epoch 43, batch 0, loss[loss=0.2185, simple_loss=0.3371, pruned_loss=0.04992, over 24545.00 frames. ], tot_loss[loss=0.2185, simple_loss=0.3371, pruned_loss=0.04992, over 24545.00 frames. ], batch size: 137, lr: 4.93e-03, grad_scale: 32.0 2026-09-24 03:37:12,894 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 03:37:17,673 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.8820, 2.5457, 2.2601, 1.9999], device='cuda:1') 2026-09-24 03:37:20,082 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6575, 2.2270, 2.5960, 1.5337], device='cuda:1') 2026-09-24 03:37:24,629 INFO [train.py:1224] (1/2) Epoch 43, validation: loss=0.1758, simple_loss=0.2945, pruned_loss=0.02853, over 2564189.00 frames. 2026-09-24 03:37:24,629 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 03:37:26,128 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.548e+02 3.430e+02 3.868e+02 4.424e+02 7.560e+02, threshold=7.736e+02, percent-clipped=1.0 2026-09-24 03:37:30,481 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=134153.33333333334, ans=0.125 2026-09-24 03:37:31,362 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=134153.33333333334, ans=0.5 2026-09-24 03:37:32,889 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=134153.33333333334, ans=0.0 2026-09-24 03:37:40,858 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=134220.0, ans=0.2 2026-09-24 03:37:42,276 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=134220.0, ans=0.125 2026-09-24 03:37:42,551 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=5.21 vs. limit=15.0 2026-09-24 03:37:43,201 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=134220.0, ans=0.125 2026-09-24 03:37:49,091 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=134253.33333333334, ans=0.125 2026-09-24 03:37:50,469 INFO [train.py:1192] (1/2) Epoch 43, batch 50, loss[loss=0.233, simple_loss=0.3385, pruned_loss=0.06377, over 24265.00 frames. ], tot_loss[loss=0.2787, simple_loss=0.3917, pruned_loss=0.08281, over 1080815.85 frames. ], batch size: 125, lr: 4.93e-03, grad_scale: 64.0 2026-09-24 03:37:59,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=134320.0, ans=0.0 2026-09-24 03:38:16,964 INFO [train.py:1192] (1/2) Epoch 43, batch 100, loss[loss=0.2623, simple_loss=0.373, pruned_loss=0.07578, over 24594.00 frames. ], tot_loss[loss=0.2811, simple_loss=0.3952, pruned_loss=0.0835, over 1914818.17 frames. ], batch size: 154, lr: 4.93e-03, grad_scale: 64.0 2026-09-24 03:38:18,350 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.349e+02 3.785e+02 4.307e+02 6.139e+02, threshold=7.569e+02, percent-clipped=0.0 2026-09-24 03:38:19,398 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=134453.33333333334, ans=0.1 2026-09-24 03:38:23,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=134486.66666666666, ans=0.125 2026-09-24 03:38:31,788 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=134553.33333333334, ans=0.0 2026-09-24 03:38:42,697 INFO [train.py:1192] (1/2) Epoch 43, batch 150, loss[loss=0.2428, simple_loss=0.3414, pruned_loss=0.07208, over 24276.00 frames. ], tot_loss[loss=0.2762, simple_loss=0.3896, pruned_loss=0.08141, over 2560023.43 frames. ], batch size: 125, lr: 4.93e-03, grad_scale: 64.0 2026-09-24 03:38:43,936 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=134620.0, ans=0.0 2026-09-24 03:38:50,572 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=134653.33333333334, ans=0.04949747468305833 2026-09-24 03:39:03,829 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=134753.33333333334, ans=0.2 2026-09-24 03:39:06,537 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=134753.33333333334, ans=0.025 2026-09-24 03:39:08,303 INFO [train.py:1192] (1/2) Epoch 43, batch 200, loss[loss=0.2886, simple_loss=0.4143, pruned_loss=0.08142, over 24197.00 frames. ], tot_loss[loss=0.273, simple_loss=0.3867, pruned_loss=0.07962, over 3058791.76 frames. ], batch size: 257, lr: 4.92e-03, grad_scale: 64.0 2026-09-24 03:39:09,612 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.539e+02 3.385e+02 3.870e+02 4.574e+02 6.427e+02, threshold=7.741e+02, percent-clipped=0.0 2026-09-24 03:39:23,933 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=134886.66666666666, ans=0.125 2026-09-24 03:39:31,176 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.73 vs. limit=15.0 2026-09-24 03:39:34,456 INFO [train.py:1192] (1/2) Epoch 43, batch 250, loss[loss=0.3246, simple_loss=0.4405, pruned_loss=0.1043, over 24369.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.3872, pruned_loss=0.08047, over 3442826.11 frames. ], batch size: 225, lr: 4.92e-03, grad_scale: 32.0 2026-09-24 03:39:36,555 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=134953.33333333334, ans=0.0 2026-09-24 03:39:41,851 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=134986.66666666666, ans=0.125 2026-09-24 03:39:42,910 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=134986.66666666666, ans=0.125 2026-09-24 03:39:44,545 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=135020.0, ans=0.125 2026-09-24 03:39:54,449 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=135053.33333333334, ans=0.025 2026-09-24 03:39:57,554 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=135086.66666666666, ans=0.2 2026-09-24 03:39:58,031 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=135086.66666666666, ans=0.125 2026-09-24 03:40:00,262 INFO [train.py:1192] (1/2) Epoch 43, batch 300, loss[loss=0.3247, simple_loss=0.4391, pruned_loss=0.1052, over 24554.00 frames. ], tot_loss[loss=0.2732, simple_loss=0.3864, pruned_loss=0.08005, over 3757091.97 frames. ], batch size: 204, lr: 4.92e-03, grad_scale: 32.0 2026-09-24 03:40:02,363 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.466e+02 3.257e+02 3.581e+02 4.184e+02 6.737e+02, threshold=7.162e+02, percent-clipped=0.0 2026-09-24 03:40:03,011 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=135120.0, ans=0.025 2026-09-24 03:40:06,626 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer_na.min_abs, batch_count=135153.33333333334, ans=0.02 2026-09-24 03:40:10,199 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.26 vs. limit=22.5 2026-09-24 03:40:11,014 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=135186.66666666666, ans=0.125 2026-09-24 03:40:11,466 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=135186.66666666666, ans=0.0 2026-09-24 03:40:15,625 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=135220.0, ans=0.2 2026-09-24 03:40:17,841 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=135220.0, ans=0.0 2026-09-24 03:40:20,145 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=135253.33333333334, ans=0.2 2026-09-24 03:40:21,162 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=135253.33333333334, ans=0.0 2026-09-24 03:40:25,733 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.41 vs. limit=22.5 2026-09-24 03:40:25,980 INFO [train.py:1192] (1/2) Epoch 43, batch 350, loss[loss=0.2289, simple_loss=0.3423, pruned_loss=0.05773, over 24562.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3868, pruned_loss=0.08026, over 3997642.30 frames. ], batch size: 137, lr: 4.91e-03, grad_scale: 32.0 2026-09-24 03:40:29,900 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=135286.66666666666, ans=0.125 2026-09-24 03:40:39,989 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=3.76 vs. limit=12.0 2026-09-24 03:40:45,776 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.55 vs. limit=22.5 2026-09-24 03:40:48,322 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.53 vs. limit=15.0 2026-09-24 03:40:51,785 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.32 vs. limit=22.5 2026-09-24 03:40:52,034 INFO [train.py:1192] (1/2) Epoch 43, batch 400, loss[loss=0.2615, simple_loss=0.3852, pruned_loss=0.06895, over 24559.00 frames. ], tot_loss[loss=0.2725, simple_loss=0.3858, pruned_loss=0.07962, over 4182758.56 frames. ], batch size: 170, lr: 4.91e-03, grad_scale: 32.0 2026-09-24 03:40:53,740 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.813e+02 3.350e+02 3.684e+02 4.267e+02 5.929e+02, threshold=7.368e+02, percent-clipped=0.0 2026-09-24 03:40:54,977 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=135453.33333333334, ans=0.0 2026-09-24 03:40:57,896 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=135486.66666666666, ans=0.125 2026-09-24 03:41:08,049 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=135553.33333333334, ans=0.125 2026-09-24 03:41:11,697 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.10 vs. limit=15.0 2026-09-24 03:41:16,345 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.15 vs. limit=10.0 2026-09-24 03:41:18,021 INFO [train.py:1192] (1/2) Epoch 43, batch 450, loss[loss=0.2681, simple_loss=0.3871, pruned_loss=0.07455, over 24646.00 frames. ], tot_loss[loss=0.2732, simple_loss=0.3864, pruned_loss=0.08004, over 4321615.93 frames. ], batch size: 175, lr: 4.91e-03, grad_scale: 32.0 2026-09-24 03:41:18,141 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=135620.0, ans=0.125 2026-09-24 03:41:23,710 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=135653.33333333334, ans=0.0 2026-09-24 03:41:29,356 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.08 vs. limit=12.0 2026-09-24 03:41:38,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=135753.33333333334, ans=0.0 2026-09-24 03:41:43,625 INFO [train.py:1192] (1/2) Epoch 43, batch 500, loss[loss=0.2849, simple_loss=0.412, pruned_loss=0.07885, over 24490.00 frames. ], tot_loss[loss=0.2717, simple_loss=0.3846, pruned_loss=0.07935, over 4439182.19 frames. ], batch size: 218, lr: 4.90e-03, grad_scale: 32.0 2026-09-24 03:41:44,276 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=135786.66666666666, ans=0.125 2026-09-24 03:41:45,632 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.508e+02 3.227e+02 3.679e+02 4.100e+02 5.797e+02, threshold=7.357e+02, percent-clipped=0.0 2026-09-24 03:41:49,255 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=135820.0, ans=0.125 2026-09-24 03:41:49,782 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=135820.0, ans=0.0 2026-09-24 03:42:09,392 INFO [train.py:1192] (1/2) Epoch 43, batch 550, loss[loss=0.3114, simple_loss=0.4296, pruned_loss=0.09656, over 24297.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3849, pruned_loss=0.07905, over 4523691.21 frames. ], batch size: 257, lr: 4.90e-03, grad_scale: 32.0 2026-09-24 03:42:11,653 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=135953.33333333334, ans=0.025 2026-09-24 03:42:11,762 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.17 vs. limit=15.0 2026-09-24 03:42:20,213 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=136020.0, ans=0.125 2026-09-24 03:42:28,396 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=136053.33333333334, ans=0.0 2026-09-24 03:42:32,584 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=136086.66666666666, ans=0.125 2026-09-24 03:42:34,375 INFO [train.py:1192] (1/2) Epoch 43, batch 600, loss[loss=0.279, simple_loss=0.4086, pruned_loss=0.07475, over 24325.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3856, pruned_loss=0.07918, over 4589938.92 frames. ], batch size: 234, lr: 4.90e-03, grad_scale: 32.0 2026-09-24 03:42:36,395 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.535e+02 3.174e+02 3.490e+02 3.947e+02 5.244e+02, threshold=6.979e+02, percent-clipped=0.0 2026-09-24 03:42:39,110 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=136153.33333333334, ans=0.125 2026-09-24 03:42:43,412 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.80 vs. limit=15.0 2026-09-24 03:42:44,227 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=136186.66666666666, ans=0.0 2026-09-24 03:42:46,367 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=136186.66666666666, ans=0.125 2026-09-24 03:42:59,734 INFO [train.py:1192] (1/2) Epoch 43, batch 650, loss[loss=0.29, simple_loss=0.3952, pruned_loss=0.09241, over 24598.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3843, pruned_loss=0.07827, over 4654714.81 frames. ], batch size: 154, lr: 4.90e-03, grad_scale: 32.0 2026-09-24 03:43:02,913 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=136286.66666666666, ans=0.125 2026-09-24 03:43:02,937 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=136286.66666666666, ans=0.0 2026-09-24 03:43:02,985 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.32 vs. limit=22.5 2026-09-24 03:43:04,710 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=136320.0, ans=0.2 2026-09-24 03:43:08,146 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=136320.0, ans=0.125 2026-09-24 03:43:18,412 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=136386.66666666666, ans=0.025 2026-09-24 03:43:20,989 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.14 vs. limit=15.0 2026-09-24 03:43:25,136 INFO [train.py:1192] (1/2) Epoch 43, batch 700, loss[loss=0.2751, simple_loss=0.3834, pruned_loss=0.08343, over 24558.00 frames. ], tot_loss[loss=0.2711, simple_loss=0.3851, pruned_loss=0.07855, over 4690161.77 frames. ], batch size: 158, lr: 4.89e-03, grad_scale: 32.0 2026-09-24 03:43:27,637 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.557e+02 3.247e+02 3.744e+02 4.318e+02 6.508e+02, threshold=7.488e+02, percent-clipped=0.0 2026-09-24 03:43:35,414 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.10 vs. limit=10.0 2026-09-24 03:43:45,925 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.12 vs. limit=15.0 2026-09-24 03:43:48,973 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=136586.66666666666, ans=0.2 2026-09-24 03:43:51,161 INFO [train.py:1192] (1/2) Epoch 43, batch 750, loss[loss=0.2728, simple_loss=0.3858, pruned_loss=0.07983, over 24633.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3844, pruned_loss=0.07874, over 4726416.98 frames. ], batch size: 175, lr: 4.89e-03, grad_scale: 32.0 2026-09-24 03:43:51,672 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:43:57,094 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.92 vs. limit=15.0 2026-09-24 03:44:09,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=136720.0, ans=0.125 2026-09-24 03:44:11,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=136753.33333333334, ans=0.0 2026-09-24 03:44:11,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=136753.33333333334, ans=0.2 2026-09-24 03:44:14,146 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.19 vs. limit=6.0 2026-09-24 03:44:17,093 INFO [train.py:1192] (1/2) Epoch 43, batch 800, loss[loss=0.2452, simple_loss=0.3535, pruned_loss=0.06849, over 24550.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3848, pruned_loss=0.07904, over 4752311.82 frames. ], batch size: 137, lr: 4.89e-03, grad_scale: 32.0 2026-09-24 03:44:17,668 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.95 vs. limit=15.0 2026-09-24 03:44:18,947 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.667e+02 3.416e+02 3.835e+02 4.566e+02 6.616e+02, threshold=7.670e+02, percent-clipped=0.0 2026-09-24 03:44:21,386 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=136820.0, ans=0.2 2026-09-24 03:44:22,271 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=136820.0, ans=0.125 2026-09-24 03:44:23,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=136820.0, ans=0.0 2026-09-24 03:44:23,811 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.95 vs. limit=10.0 2026-09-24 03:44:24,350 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.59 vs. limit=15.0 2026-09-24 03:44:42,274 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=136953.33333333334, ans=0.015 2026-09-24 03:44:42,631 INFO [train.py:1192] (1/2) Epoch 43, batch 850, loss[loss=0.2719, simple_loss=0.3976, pruned_loss=0.0731, over 24535.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3845, pruned_loss=0.07919, over 4771388.57 frames. ], batch size: 204, lr: 4.88e-03, grad_scale: 32.0 2026-09-24 03:44:48,724 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=136986.66666666666, ans=0.125 2026-09-24 03:44:50,092 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=136986.66666666666, ans=0.125 2026-09-24 03:44:51,431 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=136986.66666666666, ans=0.0 2026-09-24 03:45:01,977 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=137053.33333333334, ans=0.125 2026-09-24 03:45:08,783 INFO [train.py:1192] (1/2) Epoch 43, batch 900, loss[loss=0.2454, simple_loss=0.3563, pruned_loss=0.06721, over 24552.00 frames. ], tot_loss[loss=0.2726, simple_loss=0.3854, pruned_loss=0.07993, over 4782077.63 frames. ], batch size: 137, lr: 4.88e-03, grad_scale: 32.0 2026-09-24 03:45:10,011 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.49 vs. limit=10.0 2026-09-24 03:45:10,760 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.669e+02 3.203e+02 3.662e+02 4.207e+02 6.397e+02, threshold=7.324e+02, percent-clipped=0.0 2026-09-24 03:45:12,091 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.97 vs. limit=6.0 2026-09-24 03:45:12,566 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=137120.0, ans=0.0 2026-09-24 03:45:14,344 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.47 vs. limit=15.0 2026-09-24 03:45:14,868 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.08 vs. limit=15.0 2026-09-24 03:45:16,553 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.31 vs. limit=15.0 2026-09-24 03:45:23,775 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.28 vs. limit=22.5 2026-09-24 03:45:25,906 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=137220.0, ans=0.125 2026-09-24 03:45:28,447 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=137253.33333333334, ans=0.04949747468305833 2026-09-24 03:45:30,772 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=137253.33333333334, ans=0.125 2026-09-24 03:45:34,050 INFO [train.py:1192] (1/2) Epoch 43, batch 950, loss[loss=0.4179, simple_loss=0.4672, pruned_loss=0.1843, over 11611.00 frames. ], tot_loss[loss=0.2732, simple_loss=0.3844, pruned_loss=0.08104, over 4712322.52 frames. ], batch size: 334, lr: 4.88e-03, grad_scale: 32.0 2026-09-24 03:45:35,582 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=137286.66666666666, ans=0.125 2026-09-24 03:45:35,595 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=137286.66666666666, ans=0.125 2026-09-24 03:45:44,797 INFO [train.py:1192] (1/2) Epoch 44, batch 0, loss[loss=0.214, simple_loss=0.3355, pruned_loss=0.04621, over 24584.00 frames. ], tot_loss[loss=0.214, simple_loss=0.3355, pruned_loss=0.04621, over 24584.00 frames. ], batch size: 137, lr: 4.82e-03, grad_scale: 32.0 2026-09-24 03:45:44,797 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 03:45:56,518 INFO [train.py:1224] (1/2) Epoch 44, validation: loss=0.1759, simple_loss=0.2947, pruned_loss=0.02857, over 2564189.00 frames. 2026-09-24 03:45:56,518 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 03:46:00,443 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=137313.33333333334, ans=0.0 2026-09-24 03:46:03,296 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.16 vs. limit=22.5 2026-09-24 03:46:07,240 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=137380.0, ans=0.2 2026-09-24 03:46:14,899 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=137413.33333333334, ans=0.0 2026-09-24 03:46:19,386 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=137446.66666666666, ans=0.2 2026-09-24 03:46:20,388 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.625e+02 3.475e+02 3.849e+02 4.265e+02 7.177e+02, threshold=7.699e+02, percent-clipped=0.0 2026-09-24 03:46:22,255 INFO [train.py:1192] (1/2) Epoch 44, batch 50, loss[loss=0.2315, simple_loss=0.3392, pruned_loss=0.06194, over 24245.00 frames. ], tot_loss[loss=0.2797, simple_loss=0.3928, pruned_loss=0.08334, over 1082241.73 frames. ], batch size: 125, lr: 4.82e-03, grad_scale: 32.0 2026-09-24 03:46:31,144 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=137513.33333333334, ans=0.0 2026-09-24 03:46:32,091 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=137546.66666666666, ans=0.1 2026-09-24 03:46:33,955 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=137546.66666666666, ans=0.0 2026-09-24 03:46:34,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=137546.66666666666, ans=0.125 2026-09-24 03:46:37,655 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=137580.0, ans=0.0 2026-09-24 03:46:44,759 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=137613.33333333334, ans=0.125 2026-09-24 03:46:46,831 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=137613.33333333334, ans=0.125 2026-09-24 03:46:47,910 INFO [train.py:1192] (1/2) Epoch 44, batch 100, loss[loss=0.2592, simple_loss=0.3689, pruned_loss=0.07471, over 24595.00 frames. ], tot_loss[loss=0.2785, simple_loss=0.3937, pruned_loss=0.08161, over 1916454.59 frames. ], batch size: 154, lr: 4.82e-03, grad_scale: 32.0 2026-09-24 03:46:48,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=137646.66666666666, ans=0.125 2026-09-24 03:46:48,992 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=137646.66666666666, ans=0.125 2026-09-24 03:46:53,443 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=137680.0, ans=0.125 2026-09-24 03:46:56,318 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=137680.0, ans=0.1 2026-09-24 03:47:03,546 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=137746.66666666666, ans=0.1 2026-09-24 03:47:11,645 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.478e+02 3.527e+02 3.794e+02 4.347e+02 7.117e+02, threshold=7.588e+02, percent-clipped=0.0 2026-09-24 03:47:13,540 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=137813.33333333334, ans=0.125 2026-09-24 03:47:13,985 INFO [train.py:1192] (1/2) Epoch 44, batch 150, loss[loss=0.2368, simple_loss=0.3412, pruned_loss=0.06621, over 24245.00 frames. ], tot_loss[loss=0.2745, simple_loss=0.3885, pruned_loss=0.0802, over 2561366.43 frames. ], batch size: 125, lr: 4.81e-03, grad_scale: 32.0 2026-09-24 03:47:17,683 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.67 vs. limit=15.0 2026-09-24 03:47:24,274 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=137880.0, ans=0.1 2026-09-24 03:47:27,154 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=137880.0, ans=0.125 2026-09-24 03:47:28,542 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=137880.0, ans=0.0 2026-09-24 03:47:29,914 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=137913.33333333334, ans=0.025 2026-09-24 03:47:32,365 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=137913.33333333334, ans=0.035 2026-09-24 03:47:33,769 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=137946.66666666666, ans=0.125 2026-09-24 03:47:35,211 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=137946.66666666666, ans=0.035 2026-09-24 03:47:35,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=137946.66666666666, ans=0.1 2026-09-24 03:47:35,929 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.01 vs. limit=6.0 2026-09-24 03:47:38,082 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.27 vs. limit=6.0 2026-09-24 03:47:39,699 INFO [train.py:1192] (1/2) Epoch 44, batch 200, loss[loss=0.3263, simple_loss=0.445, pruned_loss=0.1038, over 24194.00 frames. ], tot_loss[loss=0.2724, simple_loss=0.3865, pruned_loss=0.07916, over 3059121.47 frames. ], batch size: 257, lr: 4.81e-03, grad_scale: 32.0 2026-09-24 03:47:39,973 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.76 vs. limit=15.0 2026-09-24 03:47:40,770 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=137980.0, ans=0.125 2026-09-24 03:47:51,655 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.84 vs. limit=10.0 2026-09-24 03:47:52,981 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=138046.66666666666, ans=0.0 2026-09-24 03:48:03,206 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.576e+02 3.480e+02 3.849e+02 4.558e+02 7.319e+02, threshold=7.698e+02, percent-clipped=0.0 2026-09-24 03:48:05,160 INFO [train.py:1192] (1/2) Epoch 44, batch 250, loss[loss=0.3077, simple_loss=0.4247, pruned_loss=0.0953, over 24370.00 frames. ], tot_loss[loss=0.2723, simple_loss=0.3861, pruned_loss=0.07925, over 3442844.42 frames. ], batch size: 225, lr: 4.81e-03, grad_scale: 32.0 2026-09-24 03:48:08,371 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.99 vs. limit=15.0 2026-09-24 03:48:22,089 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=138246.66666666666, ans=0.125 2026-09-24 03:48:27,560 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=138280.0, ans=0.0 2026-09-24 03:48:28,986 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.69 vs. limit=15.0 2026-09-24 03:48:30,553 INFO [train.py:1192] (1/2) Epoch 44, batch 300, loss[loss=0.2584, simple_loss=0.388, pruned_loss=0.06437, over 24558.00 frames. ], tot_loss[loss=0.2705, simple_loss=0.3846, pruned_loss=0.07821, over 3756275.79 frames. ], batch size: 204, lr: 4.80e-03, grad_scale: 32.0 2026-09-24 03:48:37,366 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.29 vs. limit=15.0 2026-09-24 03:48:37,649 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=138346.66666666666, ans=0.125 2026-09-24 03:48:41,462 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=138380.0, ans=0.125 2026-09-24 03:48:42,874 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=138380.0, ans=0.1 2026-09-24 03:48:42,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=138380.0, ans=0.125 2026-09-24 03:48:48,370 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=4.04 vs. limit=5.0 2026-09-24 03:48:53,674 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.116e+02 3.558e+02 3.992e+02 7.971e+02, threshold=7.116e+02, percent-clipped=1.0 2026-09-24 03:48:55,528 INFO [train.py:1192] (1/2) Epoch 44, batch 350, loss[loss=0.2154, simple_loss=0.3347, pruned_loss=0.04808, over 24583.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3851, pruned_loss=0.07849, over 3996866.94 frames. ], batch size: 137, lr: 4.80e-03, grad_scale: 32.0 2026-09-24 03:48:58,815 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=138480.0, ans=0.125 2026-09-24 03:49:00,749 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=10.64 vs. limit=15.0 2026-09-24 03:49:07,699 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=138546.66666666666, ans=0.025 2026-09-24 03:49:20,845 INFO [train.py:1192] (1/2) Epoch 44, batch 400, loss[loss=0.2813, simple_loss=0.3936, pruned_loss=0.08444, over 24559.00 frames. ], tot_loss[loss=0.2703, simple_loss=0.3842, pruned_loss=0.07821, over 4182685.42 frames. ], batch size: 170, lr: 4.80e-03, grad_scale: 32.0 2026-09-24 03:49:26,192 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.24 vs. limit=10.0 2026-09-24 03:49:26,386 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=7.63 vs. limit=10.0 2026-09-24 03:49:43,227 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=138780.0, ans=0.1 2026-09-24 03:49:44,346 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.485e+02 3.290e+02 3.759e+02 4.377e+02 6.505e+02, threshold=7.518e+02, percent-clipped=0.0 2026-09-24 03:49:46,474 INFO [train.py:1192] (1/2) Epoch 44, batch 450, loss[loss=0.2727, simple_loss=0.3931, pruned_loss=0.07611, over 24639.00 frames. ], tot_loss[loss=0.271, simple_loss=0.3847, pruned_loss=0.07864, over 4321104.99 frames. ], batch size: 175, lr: 4.80e-03, grad_scale: 32.0 2026-09-24 03:49:46,725 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.57 vs. limit=12.0 2026-09-24 03:49:51,529 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=138846.66666666666, ans=0.0 2026-09-24 03:50:00,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=138880.0, ans=0.2 2026-09-24 03:50:02,186 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=138913.33333333334, ans=0.125 2026-09-24 03:50:11,672 INFO [train.py:1192] (1/2) Epoch 44, batch 500, loss[loss=0.2612, simple_loss=0.3903, pruned_loss=0.06601, over 24507.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3837, pruned_loss=0.07836, over 4438876.37 frames. ], batch size: 218, lr: 4.79e-03, grad_scale: 32.0 2026-09-24 03:50:22,490 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=139046.66666666666, ans=0.125 2026-09-24 03:50:23,684 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=139046.66666666666, ans=0.0 2026-09-24 03:50:35,231 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.334e+02 3.160e+02 3.480e+02 3.855e+02 6.130e+02, threshold=6.959e+02, percent-clipped=0.0 2026-09-24 03:50:35,332 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=139113.33333333334, ans=0.125 2026-09-24 03:50:37,416 INFO [train.py:1192] (1/2) Epoch 44, batch 550, loss[loss=0.2931, simple_loss=0.4102, pruned_loss=0.08802, over 24315.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.3845, pruned_loss=0.07914, over 4523444.95 frames. ], batch size: 257, lr: 4.79e-03, grad_scale: 32.0 2026-09-24 03:50:47,304 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=139213.33333333334, ans=0.125 2026-09-24 03:50:55,895 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.23 vs. limit=15.0 2026-09-24 03:51:03,146 INFO [train.py:1192] (1/2) Epoch 44, batch 600, loss[loss=0.2974, simple_loss=0.4134, pruned_loss=0.09074, over 24329.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3852, pruned_loss=0.07936, over 4590002.73 frames. ], batch size: 234, lr: 4.79e-03, grad_scale: 32.0 2026-09-24 03:51:15,448 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=139380.0, ans=0.0 2026-09-24 03:51:26,653 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.702e+02 3.328e+02 3.826e+02 4.343e+02 7.892e+02, threshold=7.652e+02, percent-clipped=1.0 2026-09-24 03:51:28,608 INFO [train.py:1192] (1/2) Epoch 44, batch 650, loss[loss=0.2848, simple_loss=0.3847, pruned_loss=0.09248, over 24607.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3837, pruned_loss=0.07831, over 4654733.36 frames. ], batch size: 154, lr: 4.79e-03, grad_scale: 32.0 2026-09-24 03:51:38,957 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=139546.66666666666, ans=0.0 2026-09-24 03:51:39,480 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=139546.66666666666, ans=0.1 2026-09-24 03:51:51,657 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.29 vs. limit=15.0 2026-09-24 03:51:54,229 INFO [train.py:1192] (1/2) Epoch 44, batch 700, loss[loss=0.2617, simple_loss=0.376, pruned_loss=0.07373, over 24546.00 frames. ], tot_loss[loss=0.2714, simple_loss=0.3853, pruned_loss=0.07871, over 4690336.81 frames. ], batch size: 158, lr: 4.78e-03, grad_scale: 32.0 2026-09-24 03:51:57,749 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=139646.66666666666, ans=0.0 2026-09-24 03:51:58,781 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=139646.66666666666, ans=0.0 2026-09-24 03:52:00,219 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=139680.0, ans=0.0 2026-09-24 03:52:10,820 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=139746.66666666666, ans=0.0 2026-09-24 03:52:17,790 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.781e+02 3.272e+02 3.702e+02 4.229e+02 6.362e+02, threshold=7.404e+02, percent-clipped=0.0 2026-09-24 03:52:19,837 INFO [train.py:1192] (1/2) Epoch 44, batch 750, loss[loss=0.2757, simple_loss=0.3879, pruned_loss=0.08176, over 24639.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3844, pruned_loss=0.07858, over 4726557.88 frames. ], batch size: 175, lr: 4.78e-03, grad_scale: 32.0 2026-09-24 03:52:39,488 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.73 vs. limit=15.0 2026-09-24 03:52:44,614 INFO [train.py:1192] (1/2) Epoch 44, batch 800, loss[loss=0.2377, simple_loss=0.3454, pruned_loss=0.06502, over 24555.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3843, pruned_loss=0.07857, over 4751870.70 frames. ], batch size: 137, lr: 4.78e-03, grad_scale: 32.0 2026-09-24 03:53:02,353 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=140080.0, ans=0.0 2026-09-24 03:53:08,110 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.700e+02 3.287e+02 3.727e+02 4.320e+02 6.374e+02, threshold=7.454e+02, percent-clipped=0.0 2026-09-24 03:53:10,435 INFO [train.py:1192] (1/2) Epoch 44, batch 850, loss[loss=0.31, simple_loss=0.4226, pruned_loss=0.0987, over 24551.00 frames. ], tot_loss[loss=0.2705, simple_loss=0.3839, pruned_loss=0.07855, over 4770349.80 frames. ], batch size: 204, lr: 4.77e-03, grad_scale: 32.0 2026-09-24 03:53:25,754 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=140246.66666666666, ans=0.0 2026-09-24 03:53:26,276 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=140246.66666666666, ans=0.0 2026-09-24 03:53:26,305 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=140246.66666666666, ans=0.125 2026-09-24 03:53:36,114 INFO [train.py:1192] (1/2) Epoch 44, batch 900, loss[loss=0.2165, simple_loss=0.336, pruned_loss=0.0485, over 24577.00 frames. ], tot_loss[loss=0.2709, simple_loss=0.3842, pruned_loss=0.07878, over 4780683.70 frames. ], batch size: 137, lr: 4.77e-03, grad_scale: 32.0 2026-09-24 03:53:37,683 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=140313.33333333334, ans=0.125 2026-09-24 03:53:50,276 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=140380.0, ans=0.2 2026-09-24 03:53:59,151 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.855e+02 3.359e+02 3.856e+02 4.342e+02 7.075e+02, threshold=7.713e+02, percent-clipped=0.0 2026-09-24 03:54:01,406 INFO [train.py:1192] (1/2) Epoch 44, batch 950, loss[loss=0.4004, simple_loss=0.4449, pruned_loss=0.1779, over 10933.00 frames. ], tot_loss[loss=0.2718, simple_loss=0.3834, pruned_loss=0.08014, over 4716706.62 frames. ], batch size: 334, lr: 4.77e-03, grad_scale: 32.0 2026-09-24 03:54:11,293 INFO [train.py:1192] (1/2) Epoch 45, batch 0, loss[loss=0.2105, simple_loss=0.3335, pruned_loss=0.04378, over 24570.00 frames. ], tot_loss[loss=0.2105, simple_loss=0.3335, pruned_loss=0.04378, over 24570.00 frames. ], batch size: 137, lr: 4.71e-03, grad_scale: 32.0 2026-09-24 03:54:11,293 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 03:54:22,191 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.0213, 2.2398, 3.3522, 1.3412], device='cuda:1') 2026-09-24 03:54:23,061 INFO [train.py:1224] (1/2) Epoch 45, validation: loss=0.175, simple_loss=0.2937, pruned_loss=0.02817, over 2564189.00 frames. 2026-09-24 03:54:23,062 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 03:54:23,623 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=140506.66666666666, ans=0.0 2026-09-24 03:54:30,353 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer_ff2.min_abs, batch_count=140540.0, ans=0.1 2026-09-24 03:54:42,194 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=18.33 vs. limit=22.5 2026-09-24 03:54:42,829 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=140640.0, ans=0.125 2026-09-24 03:54:44,532 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=140640.0, ans=0.0 2026-09-24 03:54:45,088 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=140640.0, ans=0.125 2026-09-24 03:54:46,670 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=140640.0, ans=0.0 2026-09-24 03:54:48,564 INFO [train.py:1192] (1/2) Epoch 45, batch 50, loss[loss=0.2211, simple_loss=0.3317, pruned_loss=0.05524, over 24236.00 frames. ], tot_loss[loss=0.2767, simple_loss=0.389, pruned_loss=0.08226, over 1082260.14 frames. ], batch size: 125, lr: 4.71e-03, grad_scale: 32.0 2026-09-24 03:54:48,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=140673.33333333334, ans=0.0 2026-09-24 03:54:49,179 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=140673.33333333334, ans=0.125 2026-09-24 03:55:04,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=140773.33333333334, ans=0.1 2026-09-24 03:55:06,915 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=140773.33333333334, ans=0.125 2026-09-24 03:55:08,145 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.716e+02 3.456e+02 3.884e+02 4.368e+02 7.152e+02, threshold=7.768e+02, percent-clipped=0.0 2026-09-24 03:55:08,859 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.22 vs. limit=6.0 2026-09-24 03:55:13,709 INFO [train.py:1192] (1/2) Epoch 45, batch 100, loss[loss=0.2807, simple_loss=0.3904, pruned_loss=0.0855, over 24606.00 frames. ], tot_loss[loss=0.2774, simple_loss=0.3921, pruned_loss=0.08136, over 1915746.41 frames. ], batch size: 154, lr: 4.71e-03, grad_scale: 32.0 2026-09-24 03:55:14,359 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=140840.0, ans=0.125 2026-09-24 03:55:20,942 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=140873.33333333334, ans=0.125 2026-09-24 03:55:26,403 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=140906.66666666666, ans=0.2 2026-09-24 03:55:27,300 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=140906.66666666666, ans=0.125 2026-09-24 03:55:28,692 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.whiten.whitening_limit, batch_count=140906.66666666666, ans=12.0 2026-09-24 03:55:38,940 INFO [train.py:1192] (1/2) Epoch 45, batch 150, loss[loss=0.2103, simple_loss=0.3217, pruned_loss=0.04949, over 24288.00 frames. ], tot_loss[loss=0.2725, simple_loss=0.3869, pruned_loss=0.0791, over 2560607.72 frames. ], batch size: 125, lr: 4.71e-03, grad_scale: 32.0 2026-09-24 03:55:43,363 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=141040.0, ans=0.125 2026-09-24 03:55:49,293 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=141073.33333333334, ans=0.125 2026-09-24 03:55:54,753 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=141106.66666666666, ans=0.0 2026-09-24 03:55:57,760 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=141106.66666666666, ans=0.0 2026-09-24 03:55:58,176 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.550e+02 3.376e+02 3.915e+02 4.425e+02 7.469e+02, threshold=7.829e+02, percent-clipped=0.0 2026-09-24 03:56:04,582 INFO [train.py:1192] (1/2) Epoch 45, batch 200, loss[loss=0.3154, simple_loss=0.4329, pruned_loss=0.09891, over 24234.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3858, pruned_loss=0.07844, over 3059089.67 frames. ], batch size: 257, lr: 4.70e-03, grad_scale: 32.0 2026-09-24 03:56:04,666 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=141173.33333333334, ans=0.125 2026-09-24 03:56:28,449 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=141306.66666666666, ans=0.0 2026-09-24 03:56:30,216 INFO [train.py:1192] (1/2) Epoch 45, batch 250, loss[loss=0.2964, simple_loss=0.4185, pruned_loss=0.08713, over 24391.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3852, pruned_loss=0.07861, over 3444428.80 frames. ], batch size: 225, lr: 4.70e-03, grad_scale: 32.0 2026-09-24 03:56:37,702 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=141373.33333333334, ans=0.125 2026-09-24 03:56:49,001 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.566e+02 3.289e+02 3.843e+02 4.322e+02 7.099e+02, threshold=7.687e+02, percent-clipped=0.0 2026-09-24 03:56:50,608 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=141473.33333333334, ans=0.125 2026-09-24 03:56:51,793 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=141473.33333333334, ans=0.1 2026-09-24 03:56:52,326 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.94 vs. limit=22.5 2026-09-24 03:56:55,383 INFO [train.py:1192] (1/2) Epoch 45, batch 300, loss[loss=0.2622, simple_loss=0.3885, pruned_loss=0.06793, over 24533.00 frames. ], tot_loss[loss=0.2698, simple_loss=0.384, pruned_loss=0.07785, over 3757136.04 frames. ], batch size: 204, lr: 4.70e-03, grad_scale: 32.0 2026-09-24 03:56:58,034 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.10 vs. limit=10.0 2026-09-24 03:56:59,551 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.95 vs. limit=10.0 2026-09-24 03:57:01,068 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.07 vs. limit=15.0 2026-09-24 03:57:12,952 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=141606.66666666666, ans=0.125 2026-09-24 03:57:14,771 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=141606.66666666666, ans=0.125 2026-09-24 03:57:17,471 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=141640.0, ans=0.025 2026-09-24 03:57:20,658 INFO [train.py:1192] (1/2) Epoch 45, batch 350, loss[loss=0.2179, simple_loss=0.3274, pruned_loss=0.05415, over 24580.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3847, pruned_loss=0.07809, over 3998990.73 frames. ], batch size: 137, lr: 4.70e-03, grad_scale: 64.0 2026-09-24 03:57:22,798 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=141673.33333333334, ans=0.1 2026-09-24 03:57:24,729 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=141673.33333333334, ans=0.0 2026-09-24 03:57:40,562 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.533e+02 3.226e+02 3.567e+02 4.126e+02 5.906e+02, threshold=7.134e+02, percent-clipped=0.0 2026-09-24 03:57:46,216 INFO [train.py:1192] (1/2) Epoch 45, batch 400, loss[loss=0.2802, simple_loss=0.3908, pruned_loss=0.08481, over 24555.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.3837, pruned_loss=0.07774, over 4184157.29 frames. ], batch size: 170, lr: 4.69e-03, grad_scale: 32.0 2026-09-24 03:57:46,829 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=141840.0, ans=0.1 2026-09-24 03:57:47,063 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.90 vs. limit=15.0 2026-09-24 03:57:59,759 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=141906.66666666666, ans=0.1 2026-09-24 03:58:11,586 INFO [train.py:1192] (1/2) Epoch 45, batch 450, loss[loss=0.3108, simple_loss=0.4159, pruned_loss=0.1029, over 24624.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3842, pruned_loss=0.07823, over 4323445.76 frames. ], batch size: 175, lr: 4.69e-03, grad_scale: 32.0 2026-09-24 03:58:16,521 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.16 vs. limit=6.0 2026-09-24 03:58:17,343 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:58:17,350 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=142040.0, ans=0.0 2026-09-24 03:58:31,733 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.402e+02 3.304e+02 3.778e+02 4.404e+02 6.390e+02, threshold=7.556e+02, percent-clipped=0.0 2026-09-24 03:58:33,673 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=142140.0, ans=0.125 2026-09-24 03:58:37,062 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=142173.33333333334, ans=0.125 2026-09-24 03:58:37,488 INFO [train.py:1192] (1/2) Epoch 45, batch 500, loss[loss=0.2742, simple_loss=0.4021, pruned_loss=0.07312, over 24531.00 frames. ], tot_loss[loss=0.2689, simple_loss=0.3828, pruned_loss=0.0775, over 4440366.99 frames. ], batch size: 218, lr: 4.69e-03, grad_scale: 32.0 2026-09-24 03:58:39,049 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.77 vs. limit=15.0 2026-09-24 03:58:40,817 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=142173.33333333334, ans=0.125 2026-09-24 03:58:54,040 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:58:57,367 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=142306.66666666666, ans=0.025 2026-09-24 03:59:03,181 INFO [train.py:1192] (1/2) Epoch 45, batch 550, loss[loss=0.28, simple_loss=0.4107, pruned_loss=0.07462, over 24271.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3837, pruned_loss=0.07805, over 4524545.62 frames. ], batch size: 257, lr: 4.68e-03, grad_scale: 32.0 2026-09-24 03:59:03,790 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=142340.0, ans=0.125 2026-09-24 03:59:10,970 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=142373.33333333334, ans=0.1 2026-09-24 03:59:22,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=142440.0, ans=0.1 2026-09-24 03:59:23,080 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.702e+02 3.255e+02 3.555e+02 3.956e+02 6.275e+02, threshold=7.109e+02, percent-clipped=0.0 2026-09-24 03:59:29,342 INFO [train.py:1192] (1/2) Epoch 45, batch 600, loss[loss=0.2825, simple_loss=0.4127, pruned_loss=0.07611, over 24280.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3849, pruned_loss=0.0787, over 4590778.81 frames. ], batch size: 234, lr: 4.68e-03, grad_scale: 32.0 2026-09-24 03:59:42,617 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 03:59:43,152 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=142573.33333333334, ans=15.0 2026-09-24 03:59:49,992 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=142640.0, ans=0.125 2026-09-24 03:59:54,107 INFO [train.py:1192] (1/2) Epoch 45, batch 650, loss[loss=0.2441, simple_loss=0.3633, pruned_loss=0.06248, over 24582.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3838, pruned_loss=0.07784, over 4655322.33 frames. ], batch size: 154, lr: 4.68e-03, grad_scale: 32.0 2026-09-24 04:00:04,874 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=9.15 vs. limit=15.0 2026-09-24 04:00:06,755 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=142740.0, ans=0.1 2026-09-24 04:00:06,870 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.28 vs. limit=22.5 2026-09-24 04:00:08,899 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.37 vs. limit=10.0 2026-09-24 04:00:09,482 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=142773.33333333334, ans=0.125 2026-09-24 04:00:13,790 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.583e+02 3.274e+02 3.633e+02 4.554e+02 7.149e+02, threshold=7.266e+02, percent-clipped=1.0 2026-09-24 04:00:19,430 INFO [train.py:1192] (1/2) Epoch 45, batch 700, loss[loss=0.2827, simple_loss=0.3878, pruned_loss=0.08883, over 24556.00 frames. ], tot_loss[loss=0.2705, simple_loss=0.3847, pruned_loss=0.07814, over 4691682.31 frames. ], batch size: 158, lr: 4.68e-03, grad_scale: 16.0 2026-09-24 04:00:24,678 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=142873.33333333334, ans=0.125 2026-09-24 04:00:28,931 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=142873.33333333334, ans=0.0 2026-09-24 04:00:35,973 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=142940.0, ans=0.0 2026-09-24 04:00:37,000 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=142940.0, ans=0.2 2026-09-24 04:00:44,292 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=142973.33333333334, ans=0.0 2026-09-24 04:00:44,309 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=142973.33333333334, ans=0.125 2026-09-24 04:00:45,206 INFO [train.py:1192] (1/2) Epoch 45, batch 750, loss[loss=0.2847, simple_loss=0.4011, pruned_loss=0.08417, over 24627.00 frames. ], tot_loss[loss=0.2706, simple_loss=0.3844, pruned_loss=0.07842, over 4727736.27 frames. ], batch size: 175, lr: 4.67e-03, grad_scale: 16.0 2026-09-24 04:00:53,072 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=143040.0, ans=0.0 2026-09-24 04:01:05,937 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.482e+02 3.353e+02 3.726e+02 4.298e+02 5.582e+02, threshold=7.452e+02, percent-clipped=0.0 2026-09-24 04:01:10,795 INFO [train.py:1192] (1/2) Epoch 45, batch 800, loss[loss=0.2428, simple_loss=0.3524, pruned_loss=0.06663, over 24559.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3841, pruned_loss=0.07815, over 4753294.18 frames. ], batch size: 137, lr: 4.67e-03, grad_scale: 32.0 2026-09-24 04:01:14,135 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=143173.33333333334, ans=0.04949747468305833 2026-09-24 04:01:16,081 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=143206.66666666666, ans=0.2 2026-09-24 04:01:17,429 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=143206.66666666666, ans=0.125 2026-09-24 04:01:23,355 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=143240.0, ans=0.1 2026-09-24 04:01:35,906 INFO [train.py:1192] (1/2) Epoch 45, batch 850, loss[loss=0.2953, simple_loss=0.4135, pruned_loss=0.08853, over 24540.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3836, pruned_loss=0.07793, over 4772716.46 frames. ], batch size: 204, lr: 4.67e-03, grad_scale: 32.0 2026-09-24 04:01:50,716 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=143440.0, ans=0.2 2026-09-24 04:01:54,976 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=143440.0, ans=0.1 2026-09-24 04:01:56,442 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.722e+02 3.280e+02 3.837e+02 4.261e+02 5.745e+02, threshold=7.675e+02, percent-clipped=0.0 2026-09-24 04:02:01,673 INFO [train.py:1192] (1/2) Epoch 45, batch 900, loss[loss=0.2444, simple_loss=0.3544, pruned_loss=0.06721, over 24543.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.3839, pruned_loss=0.07817, over 4783285.26 frames. ], batch size: 137, lr: 4.67e-03, grad_scale: 32.0 2026-09-24 04:02:15,168 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=143573.33333333334, ans=0.125 2026-09-24 04:02:20,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=143606.66666666666, ans=0.125 2026-09-24 04:02:25,186 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=143640.0, ans=0.125 2026-09-24 04:02:26,410 INFO [train.py:1192] (1/2) Epoch 45, batch 950, loss[loss=0.3371, simple_loss=0.418, pruned_loss=0.1281, over 11219.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3825, pruned_loss=0.07894, over 4715753.74 frames. ], batch size: 333, lr: 4.66e-03, grad_scale: 32.0 2026-09-24 04:02:37,994 INFO [train.py:1192] (1/2) Epoch 46, batch 0, loss[loss=0.2131, simple_loss=0.3343, pruned_loss=0.04593, over 24582.00 frames. ], tot_loss[loss=0.2131, simple_loss=0.3343, pruned_loss=0.04593, over 24582.00 frames. ], batch size: 137, lr: 4.61e-03, grad_scale: 32.0 2026-09-24 04:02:37,994 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 04:02:47,223 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.1192, 2.5596, 2.8372, 2.1573, 2.9907, 2.6292, 2.8979, 2.0947], device='cuda:1') 2026-09-24 04:02:49,713 INFO [train.py:1224] (1/2) Epoch 46, validation: loss=0.1736, simple_loss=0.2924, pruned_loss=0.02741, over 2564189.00 frames. 2026-09-24 04:02:49,713 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 04:02:49,981 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.59 vs. limit=15.0 2026-09-24 04:02:56,304 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.whiten.whitening_limit, batch_count=143733.33333333334, ans=12.0 2026-09-24 04:02:58,902 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=143733.33333333334, ans=0.2 2026-09-24 04:03:05,940 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.612e+02 3.424e+02 3.959e+02 4.444e+02 9.934e+02, threshold=7.917e+02, percent-clipped=1.0 2026-09-24 04:03:13,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=143833.33333333334, ans=0.1 2026-09-24 04:03:14,024 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=143833.33333333334, ans=0.125 2026-09-24 04:03:15,316 INFO [train.py:1192] (1/2) Epoch 46, batch 50, loss[loss=0.2168, simple_loss=0.3294, pruned_loss=0.05213, over 24261.00 frames. ], tot_loss[loss=0.279, simple_loss=0.3916, pruned_loss=0.08318, over 1081171.17 frames. ], batch size: 125, lr: 4.61e-03, grad_scale: 32.0 2026-09-24 04:03:15,389 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=143866.66666666666, ans=0.125 2026-09-24 04:03:21,637 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=143900.0, ans=0.0 2026-09-24 04:03:31,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=143966.66666666666, ans=0.125 2026-09-24 04:03:40,814 INFO [train.py:1192] (1/2) Epoch 46, batch 100, loss[loss=0.2574, simple_loss=0.3725, pruned_loss=0.07113, over 24600.00 frames. ], tot_loss[loss=0.2804, simple_loss=0.3945, pruned_loss=0.08312, over 1914496.44 frames. ], batch size: 154, lr: 4.61e-03, grad_scale: 32.0 2026-09-24 04:03:41,387 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=144033.33333333334, ans=0.125 2026-09-24 04:03:52,975 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=144100.0, ans=0.0 2026-09-24 04:03:53,933 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=144100.0, ans=0.125 2026-09-24 04:03:57,493 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.558e+02 3.314e+02 3.738e+02 4.223e+02 7.597e+02, threshold=7.476e+02, percent-clipped=0.0 2026-09-24 04:03:58,035 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=144133.33333333334, ans=0.0 2026-09-24 04:03:59,304 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.15 vs. limit=6.0 2026-09-24 04:04:06,755 INFO [train.py:1192] (1/2) Epoch 46, batch 150, loss[loss=0.2216, simple_loss=0.3292, pruned_loss=0.05703, over 24290.00 frames. ], tot_loss[loss=0.274, simple_loss=0.388, pruned_loss=0.07999, over 2559976.46 frames. ], batch size: 125, lr: 4.60e-03, grad_scale: 32.0 2026-09-24 04:04:20,314 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=12.81 vs. limit=15.0 2026-09-24 04:04:31,991 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=144366.66666666666, ans=0.125 2026-09-24 04:04:32,396 INFO [train.py:1192] (1/2) Epoch 46, batch 200, loss[loss=0.3005, simple_loss=0.4278, pruned_loss=0.08661, over 24237.00 frames. ], tot_loss[loss=0.2712, simple_loss=0.3856, pruned_loss=0.07843, over 3059429.03 frames. ], batch size: 257, lr: 4.60e-03, grad_scale: 32.0 2026-09-24 04:04:33,552 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=144366.66666666666, ans=0.0 2026-09-24 04:04:39,368 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=144400.0, ans=0.125 2026-09-24 04:04:39,389 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=144400.0, ans=0.125 2026-09-24 04:04:48,269 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=144466.66666666666, ans=0.1 2026-09-24 04:04:49,082 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.682e+02 3.354e+02 3.903e+02 4.472e+02 8.657e+02, threshold=7.805e+02, percent-clipped=1.0 2026-09-24 04:04:50,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=144466.66666666666, ans=0.015 2026-09-24 04:04:58,389 INFO [train.py:1192] (1/2) Epoch 46, batch 250, loss[loss=0.2939, simple_loss=0.4173, pruned_loss=0.08523, over 24385.00 frames. ], tot_loss[loss=0.2715, simple_loss=0.3854, pruned_loss=0.07882, over 3442505.73 frames. ], batch size: 225, lr: 4.60e-03, grad_scale: 32.0 2026-09-24 04:05:08,790 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:05:24,421 INFO [train.py:1192] (1/2) Epoch 46, batch 300, loss[loss=0.3218, simple_loss=0.4381, pruned_loss=0.1028, over 24543.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3846, pruned_loss=0.07844, over 3756357.67 frames. ], batch size: 204, lr: 4.60e-03, grad_scale: 32.0 2026-09-24 04:05:29,779 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=144733.33333333334, ans=0.2 2026-09-24 04:05:33,072 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=144733.33333333334, ans=0.0 2026-09-24 04:05:35,351 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=144766.66666666666, ans=0.125 2026-09-24 04:05:37,222 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=144766.66666666666, ans=0.1 2026-09-24 04:05:40,629 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.777e+02 3.310e+02 3.640e+02 4.094e+02 5.743e+02, threshold=7.281e+02, percent-clipped=0.0 2026-09-24 04:05:41,280 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=144800.0, ans=0.1 2026-09-24 04:05:44,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=144833.33333333334, ans=0.5 2026-09-24 04:05:49,883 INFO [train.py:1192] (1/2) Epoch 46, batch 350, loss[loss=0.2479, simple_loss=0.3537, pruned_loss=0.07105, over 24576.00 frames. ], tot_loss[loss=0.2716, simple_loss=0.3856, pruned_loss=0.07876, over 3996989.43 frames. ], batch size: 137, lr: 4.59e-03, grad_scale: 32.0 2026-09-24 04:05:54,861 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:05:57,581 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=144900.0, ans=0.125 2026-09-24 04:05:58,606 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=144900.0, ans=0.1 2026-09-24 04:06:00,299 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.29 vs. limit=12.0 2026-09-24 04:06:04,943 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=144966.66666666666, ans=0.025 2026-09-24 04:06:12,012 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=145000.0, ans=0.0 2026-09-24 04:06:15,263 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=145033.33333333334, ans=0.0 2026-09-24 04:06:15,695 INFO [train.py:1192] (1/2) Epoch 46, batch 400, loss[loss=0.2522, simple_loss=0.3691, pruned_loss=0.06771, over 24570.00 frames. ], tot_loss[loss=0.2697, simple_loss=0.3839, pruned_loss=0.07779, over 4179263.09 frames. ], batch size: 170, lr: 4.59e-03, grad_scale: 32.0 2026-09-24 04:06:17,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=145033.33333333334, ans=0.0 2026-09-24 04:06:18,281 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=145033.33333333334, ans=0.0 2026-09-24 04:06:20,764 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=145066.66666666666, ans=0.1 2026-09-24 04:06:31,863 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.676e+02 3.294e+02 3.676e+02 4.081e+02 6.453e+02, threshold=7.352e+02, percent-clipped=0.0 2026-09-24 04:06:34,748 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:06:41,589 INFO [train.py:1192] (1/2) Epoch 46, batch 450, loss[loss=0.2782, simple_loss=0.394, pruned_loss=0.08118, over 24627.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3841, pruned_loss=0.07791, over 4319764.76 frames. ], batch size: 175, lr: 4.59e-03, grad_scale: 32.0 2026-09-24 04:06:50,107 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=145233.33333333334, ans=0.125 2026-09-24 04:06:53,788 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=145266.66666666666, ans=0.09899494936611666 2026-09-24 04:06:55,769 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=145300.0, ans=0.125 2026-09-24 04:07:06,922 INFO [train.py:1192] (1/2) Epoch 46, batch 500, loss[loss=0.3167, simple_loss=0.4302, pruned_loss=0.1016, over 24492.00 frames. ], tot_loss[loss=0.2692, simple_loss=0.3833, pruned_loss=0.07761, over 4437492.62 frames. ], batch size: 218, lr: 4.59e-03, grad_scale: 32.0 2026-09-24 04:07:10,857 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.01 vs. limit=6.0 2026-09-24 04:07:17,059 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=145433.33333333334, ans=0.0 2026-09-24 04:07:17,962 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=145433.33333333334, ans=0.125 2026-09-24 04:07:18,491 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=145433.33333333334, ans=0.2 2026-09-24 04:07:23,927 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.597e+02 3.252e+02 3.691e+02 4.584e+02 6.780e+02, threshold=7.382e+02, percent-clipped=0.0 2026-09-24 04:07:25,065 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:07:26,909 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=145466.66666666666, ans=0.0 2026-09-24 04:07:32,260 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=145500.0, ans=0.2 2026-09-24 04:07:33,063 INFO [train.py:1192] (1/2) Epoch 46, batch 550, loss[loss=0.2929, simple_loss=0.4169, pruned_loss=0.08445, over 24267.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3842, pruned_loss=0.07808, over 4522617.50 frames. ], batch size: 257, lr: 4.58e-03, grad_scale: 32.0 2026-09-24 04:07:40,960 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.14 vs. limit=15.0 2026-09-24 04:07:44,956 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=145600.0, ans=0.125 2026-09-24 04:07:49,784 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=145633.33333333334, ans=0.0 2026-09-24 04:07:58,898 INFO [train.py:1192] (1/2) Epoch 46, batch 600, loss[loss=0.2976, simple_loss=0.4173, pruned_loss=0.08894, over 24308.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3842, pruned_loss=0.0778, over 4590962.69 frames. ], batch size: 234, lr: 4.58e-03, grad_scale: 32.0 2026-09-24 04:07:59,169 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.93 vs. limit=15.0 2026-09-24 04:08:04,786 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.46 vs. limit=15.0 2026-09-24 04:08:05,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=145733.33333333334, ans=0.2 2026-09-24 04:08:08,644 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.21 vs. limit=15.0 2026-09-24 04:08:15,073 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.576e+02 3.178e+02 3.584e+02 4.061e+02 7.593e+02, threshold=7.168e+02, percent-clipped=1.0 2026-09-24 04:08:15,190 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=145800.0, ans=0.125 2026-09-24 04:08:24,004 INFO [train.py:1192] (1/2) Epoch 46, batch 650, loss[loss=0.286, simple_loss=0.3905, pruned_loss=0.09079, over 24610.00 frames. ], tot_loss[loss=0.2687, simple_loss=0.3832, pruned_loss=0.07713, over 4655565.96 frames. ], batch size: 154, lr: 4.58e-03, grad_scale: 32.0 2026-09-24 04:08:26,539 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:08:31,347 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=145900.0, ans=0.2 2026-09-24 04:08:50,129 INFO [train.py:1192] (1/2) Epoch 46, batch 700, loss[loss=0.2545, simple_loss=0.3708, pruned_loss=0.06908, over 24551.00 frames. ], tot_loss[loss=0.2695, simple_loss=0.3842, pruned_loss=0.07742, over 4690635.15 frames. ], batch size: 158, lr: 4.58e-03, grad_scale: 32.0 2026-09-24 04:08:50,951 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=146033.33333333334, ans=10.0 2026-09-24 04:08:56,550 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.15 vs. limit=22.5 2026-09-24 04:09:06,411 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.700e+02 3.386e+02 3.749e+02 4.522e+02 6.408e+02, threshold=7.499e+02, percent-clipped=0.0 2026-09-24 04:09:11,170 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=146166.66666666666, ans=0.1 2026-09-24 04:09:15,855 INFO [train.py:1192] (1/2) Epoch 46, batch 750, loss[loss=0.2729, simple_loss=0.3928, pruned_loss=0.07651, over 24634.00 frames. ], tot_loss[loss=0.2689, simple_loss=0.3835, pruned_loss=0.0771, over 4727003.30 frames. ], batch size: 175, lr: 4.57e-03, grad_scale: 32.0 2026-09-24 04:09:21,456 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.74 vs. limit=15.0 2026-09-24 04:09:29,063 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=146266.66666666666, ans=0.1 2026-09-24 04:09:39,414 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=146333.33333333334, ans=0.0 2026-09-24 04:09:41,938 INFO [train.py:1192] (1/2) Epoch 46, batch 800, loss[loss=0.2257, simple_loss=0.3412, pruned_loss=0.05513, over 24557.00 frames. ], tot_loss[loss=0.2691, simple_loss=0.3836, pruned_loss=0.07732, over 4753925.42 frames. ], batch size: 137, lr: 4.57e-03, grad_scale: 32.0 2026-09-24 04:09:50,234 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=146400.0, ans=0.05 2026-09-24 04:09:52,883 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.28 vs. limit=6.0 2026-09-24 04:09:53,187 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=146433.33333333334, ans=0.125 2026-09-24 04:09:55,524 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=146433.33333333334, ans=0.025 2026-09-24 04:09:56,544 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=146466.66666666666, ans=0.1 2026-09-24 04:09:58,205 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.506e+02 3.341e+02 3.701e+02 4.189e+02 6.774e+02, threshold=7.403e+02, percent-clipped=0.0 2026-09-24 04:10:07,166 INFO [train.py:1192] (1/2) Epoch 46, batch 850, loss[loss=0.2797, simple_loss=0.4002, pruned_loss=0.07956, over 24572.00 frames. ], tot_loss[loss=0.2681, simple_loss=0.3825, pruned_loss=0.07683, over 4772807.12 frames. ], batch size: 204, lr: 4.57e-03, grad_scale: 32.0 2026-09-24 04:10:08,724 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=146533.33333333334, ans=0.1 2026-09-24 04:10:09,195 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=146533.33333333334, ans=0.125 2026-09-24 04:10:09,748 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=146533.33333333334, ans=0.1 2026-09-24 04:10:12,985 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.59 vs. limit=22.5 2026-09-24 04:10:32,767 INFO [train.py:1192] (1/2) Epoch 46, batch 900, loss[loss=0.212, simple_loss=0.326, pruned_loss=0.04899, over 24566.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.3828, pruned_loss=0.07691, over 4782874.50 frames. ], batch size: 137, lr: 4.57e-03, grad_scale: 32.0 2026-09-24 04:10:42,072 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.60 vs. limit=6.0 2026-09-24 04:10:43,087 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.68 vs. limit=15.0 2026-09-24 04:10:48,460 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=146800.0, ans=0.0 2026-09-24 04:10:49,531 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.635e+02 3.272e+02 3.689e+02 4.102e+02 7.758e+02, threshold=7.378e+02, percent-clipped=1.0 2026-09-24 04:10:54,472 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=146833.33333333334, ans=0.125 2026-09-24 04:10:58,350 INFO [train.py:1192] (1/2) Epoch 46, batch 950, loss[loss=0.3784, simple_loss=0.4376, pruned_loss=0.1596, over 11062.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3826, pruned_loss=0.07861, over 4705282.47 frames. ], batch size: 333, lr: 4.56e-03, grad_scale: 32.0 2026-09-24 04:11:09,912 INFO [train.py:1192] (1/2) Epoch 47, batch 0, loss[loss=0.2172, simple_loss=0.3382, pruned_loss=0.04816, over 24562.00 frames. ], tot_loss[loss=0.2172, simple_loss=0.3382, pruned_loss=0.04816, over 24562.00 frames. ], batch size: 137, lr: 4.51e-03, grad_scale: 32.0 2026-09-24 04:11:09,913 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 04:11:12,457 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9399, 1.5594, 2.7655, 1.6420], device='cuda:1') 2026-09-24 04:11:18,567 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.0577, 2.4038, 2.7781, 2.0764, 2.9480, 2.5351, 2.7799, 2.1308], device='cuda:1') 2026-09-24 04:11:21,603 INFO [train.py:1224] (1/2) Epoch 47, validation: loss=0.1733, simple_loss=0.2922, pruned_loss=0.02718, over 2564189.00 frames. 2026-09-24 04:11:21,604 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 04:11:33,551 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=146960.0, ans=0.1 2026-09-24 04:11:43,220 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=18.63 vs. limit=22.5 2026-09-24 04:11:43,534 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=147026.66666666666, ans=0.125 2026-09-24 04:11:44,022 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=147026.66666666666, ans=0.125 2026-09-24 04:11:47,142 INFO [train.py:1192] (1/2) Epoch 47, batch 50, loss[loss=0.2053, simple_loss=0.3179, pruned_loss=0.04638, over 24261.00 frames. ], tot_loss[loss=0.2774, simple_loss=0.3912, pruned_loss=0.08176, over 1080921.26 frames. ], batch size: 125, lr: 4.51e-03, grad_scale: 32.0 2026-09-24 04:11:48,167 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=147060.0, ans=0.1 2026-09-24 04:11:54,938 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=147093.33333333334, ans=0.0 2026-09-24 04:11:55,460 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=147093.33333333334, ans=0.125 2026-09-24 04:11:59,574 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.653e+02 3.466e+02 3.856e+02 4.341e+02 7.987e+02, threshold=7.712e+02, percent-clipped=1.0 2026-09-24 04:12:12,481 INFO [train.py:1192] (1/2) Epoch 47, batch 100, loss[loss=0.2697, simple_loss=0.3797, pruned_loss=0.07985, over 24619.00 frames. ], tot_loss[loss=0.2775, simple_loss=0.393, pruned_loss=0.08101, over 1914196.03 frames. ], batch size: 154, lr: 4.51e-03, grad_scale: 32.0 2026-09-24 04:12:15,279 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=147226.66666666666, ans=0.025 2026-09-24 04:12:22,699 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=147293.33333333334, ans=0.0 2026-09-24 04:12:31,784 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=147326.66666666666, ans=0.125 2026-09-24 04:12:37,840 INFO [train.py:1192] (1/2) Epoch 47, batch 150, loss[loss=0.2103, simple_loss=0.3236, pruned_loss=0.04848, over 24243.00 frames. ], tot_loss[loss=0.2731, simple_loss=0.3874, pruned_loss=0.07936, over 2560191.18 frames. ], batch size: 125, lr: 4.51e-03, grad_scale: 32.0 2026-09-24 04:12:46,348 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=147426.66666666666, ans=0.0 2026-09-24 04:12:49,690 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=147460.0, ans=0.0 2026-09-24 04:12:50,036 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.765e+02 3.439e+02 3.895e+02 4.306e+02 6.653e+02, threshold=7.789e+02, percent-clipped=0.0 2026-09-24 04:12:50,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=147460.0, ans=0.125 2026-09-24 04:12:53,007 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=147493.33333333334, ans=0.2 2026-09-24 04:12:54,040 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=147493.33333333334, ans=0.0 2026-09-24 04:13:03,211 INFO [train.py:1192] (1/2) Epoch 47, batch 200, loss[loss=0.2769, simple_loss=0.4032, pruned_loss=0.0753, over 24222.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3847, pruned_loss=0.07763, over 3059113.47 frames. ], batch size: 257, lr: 4.50e-03, grad_scale: 32.0 2026-09-24 04:13:04,832 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.60 vs. limit=12.0 2026-09-24 04:13:07,801 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:13:13,933 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=147626.66666666666, ans=0.025 2026-09-24 04:13:20,293 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=147660.0, ans=0.125 2026-09-24 04:13:21,897 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=147660.0, ans=0.125 2026-09-24 04:13:28,518 INFO [train.py:1192] (1/2) Epoch 47, batch 250, loss[loss=0.2816, simple_loss=0.407, pruned_loss=0.07805, over 24368.00 frames. ], tot_loss[loss=0.2692, simple_loss=0.3838, pruned_loss=0.07726, over 3444198.42 frames. ], batch size: 225, lr: 4.50e-03, grad_scale: 32.0 2026-09-24 04:13:36,238 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=147760.0, ans=0.025 2026-09-24 04:13:40,674 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.617e+02 3.262e+02 3.790e+02 4.435e+02 6.838e+02, threshold=7.580e+02, percent-clipped=0.0 2026-09-24 04:13:49,638 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=5.07 vs. limit=6.0 2026-09-24 04:13:53,711 INFO [train.py:1192] (1/2) Epoch 47, batch 300, loss[loss=0.2949, simple_loss=0.417, pruned_loss=0.08634, over 24556.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.383, pruned_loss=0.07685, over 3757543.43 frames. ], batch size: 204, lr: 4.50e-03, grad_scale: 32.0 2026-09-24 04:13:55,492 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=147893.33333333334, ans=0.0 2026-09-24 04:13:56,871 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=147893.33333333334, ans=0.1 2026-09-24 04:13:57,817 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.82 vs. limit=15.0 2026-09-24 04:14:11,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=147993.33333333334, ans=0.125 2026-09-24 04:14:11,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=147993.33333333334, ans=0.125 2026-09-24 04:14:18,901 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.27 vs. limit=15.0 2026-09-24 04:14:19,257 INFO [train.py:1192] (1/2) Epoch 47, batch 350, loss[loss=0.2487, simple_loss=0.3506, pruned_loss=0.07343, over 24554.00 frames. ], tot_loss[loss=0.2692, simple_loss=0.3838, pruned_loss=0.07733, over 3999501.21 frames. ], batch size: 137, lr: 4.50e-03, grad_scale: 32.0 2026-09-24 04:14:20,385 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=148060.0, ans=0.2 2026-09-24 04:14:31,313 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.466e+02 3.317e+02 3.744e+02 4.508e+02 7.087e+02, threshold=7.487e+02, percent-clipped=0.0 2026-09-24 04:14:31,418 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=148126.66666666666, ans=0.2 2026-09-24 04:14:42,669 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=148193.33333333334, ans=0.125 2026-09-24 04:14:44,388 INFO [train.py:1192] (1/2) Epoch 47, batch 400, loss[loss=0.2948, simple_loss=0.4099, pruned_loss=0.08987, over 24585.00 frames. ], tot_loss[loss=0.2687, simple_loss=0.3833, pruned_loss=0.07701, over 4184685.54 frames. ], batch size: 170, lr: 4.49e-03, grad_scale: 32.0 2026-09-24 04:15:05,653 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=148360.0, ans=0.1 2026-09-24 04:15:09,636 INFO [train.py:1192] (1/2) Epoch 47, batch 450, loss[loss=0.2722, simple_loss=0.3942, pruned_loss=0.07507, over 24613.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3842, pruned_loss=0.07776, over 4321591.74 frames. ], batch size: 175, lr: 4.49e-03, grad_scale: 32.0 2026-09-24 04:15:09,743 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=148393.33333333334, ans=0.125 2026-09-24 04:15:13,806 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.29 vs. limit=6.0 2026-09-24 04:15:21,773 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.549e+02 3.364e+02 3.852e+02 4.437e+02 6.702e+02, threshold=7.704e+02, percent-clipped=0.0 2026-09-24 04:15:32,010 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=148526.66666666666, ans=0.0 2026-09-24 04:15:33,200 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=148526.66666666666, ans=0.0 2026-09-24 04:15:35,027 INFO [train.py:1192] (1/2) Epoch 47, batch 500, loss[loss=0.3206, simple_loss=0.428, pruned_loss=0.1066, over 24485.00 frames. ], tot_loss[loss=0.2687, simple_loss=0.3827, pruned_loss=0.07731, over 4438996.14 frames. ], batch size: 218, lr: 4.49e-03, grad_scale: 32.0 2026-09-24 04:15:40,925 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=148593.33333333334, ans=0.0 2026-09-24 04:15:45,563 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=148626.66666666666, ans=0.2 2026-09-24 04:15:49,850 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=148626.66666666666, ans=0.125 2026-09-24 04:15:51,303 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=148660.0, ans=0.0 2026-09-24 04:15:51,308 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=148660.0, ans=0.0 2026-09-24 04:15:53,505 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=148660.0, ans=0.04949747468305833 2026-09-24 04:16:00,748 INFO [train.py:1192] (1/2) Epoch 47, batch 550, loss[loss=0.2784, simple_loss=0.4032, pruned_loss=0.07682, over 24264.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3838, pruned_loss=0.07796, over 4523953.96 frames. ], batch size: 257, lr: 4.49e-03, grad_scale: 32.0 2026-09-24 04:16:04,180 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.63 vs. limit=15.0 2026-09-24 04:16:10,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=148793.33333333334, ans=0.125 2026-09-24 04:16:10,727 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=148793.33333333334, ans=0.025 2026-09-24 04:16:12,774 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.497e+02 3.250e+02 3.585e+02 3.951e+02 5.774e+02, threshold=7.171e+02, percent-clipped=0.0 2026-09-24 04:16:13,832 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=148793.33333333334, ans=0.0 2026-09-24 04:16:22,599 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.39 vs. limit=10.0 2026-09-24 04:16:25,891 INFO [train.py:1192] (1/2) Epoch 47, batch 600, loss[loss=0.2733, simple_loss=0.4053, pruned_loss=0.07064, over 24325.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3843, pruned_loss=0.07806, over 4590418.20 frames. ], batch size: 234, lr: 4.48e-03, grad_scale: 32.0 2026-09-24 04:16:29,191 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=148893.33333333334, ans=0.025 2026-09-24 04:16:34,349 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=148926.66666666666, ans=0.1 2026-09-24 04:16:36,458 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=148960.0, ans=0.125 2026-09-24 04:16:43,174 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=148993.33333333334, ans=0.125 2026-09-24 04:16:45,249 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=148993.33333333334, ans=0.0 2026-09-24 04:16:47,812 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=149026.66666666666, ans=0.2 2026-09-24 04:16:51,792 INFO [train.py:1192] (1/2) Epoch 47, batch 650, loss[loss=0.2583, simple_loss=0.3688, pruned_loss=0.0739, over 24592.00 frames. ], tot_loss[loss=0.2692, simple_loss=0.3834, pruned_loss=0.07749, over 4654992.61 frames. ], batch size: 154, lr: 4.48e-03, grad_scale: 32.0 2026-09-24 04:16:56,661 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=149093.33333333334, ans=0.125 2026-09-24 04:17:02,020 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=149126.66666666666, ans=0.125 2026-09-24 04:17:03,723 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.89 vs. limit=15.0 2026-09-24 04:17:03,998 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.790e+02 3.432e+02 3.795e+02 4.283e+02 7.994e+02, threshold=7.590e+02, percent-clipped=1.0 2026-09-24 04:17:07,800 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:17:18,159 INFO [train.py:1192] (1/2) Epoch 47, batch 700, loss[loss=0.2567, simple_loss=0.3734, pruned_loss=0.07001, over 24552.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3845, pruned_loss=0.07769, over 4691882.53 frames. ], batch size: 158, lr: 4.48e-03, grad_scale: 32.0 2026-09-24 04:17:24,694 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=149260.0, ans=0.0 2026-09-24 04:17:31,601 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=149293.33333333334, ans=0.0 2026-09-24 04:17:38,372 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.91 vs. limit=6.0 2026-09-24 04:17:40,586 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=149360.0, ans=0.125 2026-09-24 04:17:43,632 INFO [train.py:1192] (1/2) Epoch 47, batch 750, loss[loss=0.279, simple_loss=0.3925, pruned_loss=0.08277, over 24614.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.3837, pruned_loss=0.07771, over 4727685.54 frames. ], batch size: 175, lr: 4.48e-03, grad_scale: 32.0 2026-09-24 04:17:55,785 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=149460.0, ans=0.025 2026-09-24 04:17:56,196 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.544e+02 3.424e+02 3.890e+02 4.427e+02 7.116e+02, threshold=7.781e+02, percent-clipped=0.0 2026-09-24 04:17:56,297 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=149460.0, ans=0.07 2026-09-24 04:17:59,188 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=149493.33333333334, ans=0.125 2026-09-24 04:18:03,012 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=149493.33333333334, ans=0.0 2026-09-24 04:18:09,201 INFO [train.py:1192] (1/2) Epoch 47, batch 800, loss[loss=0.2141, simple_loss=0.3349, pruned_loss=0.0466, over 24570.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3831, pruned_loss=0.07741, over 4753989.30 frames. ], batch size: 137, lr: 4.47e-03, grad_scale: 64.0 2026-09-24 04:18:11,724 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=149560.0, ans=0.0 2026-09-24 04:18:23,965 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=149660.0, ans=0.125 2026-09-24 04:18:35,065 INFO [train.py:1192] (1/2) Epoch 47, batch 850, loss[loss=0.294, simple_loss=0.4117, pruned_loss=0.08818, over 24565.00 frames. ], tot_loss[loss=0.2687, simple_loss=0.3828, pruned_loss=0.07729, over 4772505.59 frames. ], batch size: 204, lr: 4.47e-03, grad_scale: 64.0 2026-09-24 04:18:35,260 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.35 vs. limit=22.5 2026-09-24 04:18:36,303 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.23 vs. limit=10.0 2026-09-24 04:18:45,923 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=149793.33333333334, ans=10.0 2026-09-24 04:18:47,893 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.652e+02 3.362e+02 3.668e+02 4.158e+02 7.639e+02, threshold=7.335e+02, percent-clipped=0.0 2026-09-24 04:18:48,459 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=149793.33333333334, ans=0.1 2026-09-24 04:18:55,516 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=149826.66666666666, ans=0.2 2026-09-24 04:19:01,471 INFO [train.py:1192] (1/2) Epoch 47, batch 900, loss[loss=0.2318, simple_loss=0.3474, pruned_loss=0.05813, over 24557.00 frames. ], tot_loss[loss=0.269, simple_loss=0.383, pruned_loss=0.07751, over 4782545.27 frames. ], batch size: 137, lr: 4.47e-03, grad_scale: 64.0 2026-09-24 04:19:06,054 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=149893.33333333334, ans=0.125 2026-09-24 04:19:08,028 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:19:10,475 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=149926.66666666666, ans=0.07 2026-09-24 04:19:11,122 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.64 vs. limit=15.0 2026-09-24 04:19:15,928 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=149960.0, ans=0.125 2026-09-24 04:19:18,238 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.99 vs. limit=6.0 2026-09-24 04:19:18,668 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=149993.33333333334, ans=0.2 2026-09-24 04:19:26,901 INFO [train.py:1192] (1/2) Epoch 47, batch 950, loss[loss=0.3516, simple_loss=0.4284, pruned_loss=0.1374, over 11276.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3826, pruned_loss=0.07907, over 4711028.39 frames. ], batch size: 333, lr: 4.47e-03, grad_scale: 32.0 2026-09-24 04:19:28,090 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=150060.0, ans=0.125 2026-09-24 04:19:28,550 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=150060.0, ans=0.0 2026-09-24 04:19:35,823 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.89 vs. limit=15.0 2026-09-24 04:19:38,545 INFO [train.py:1192] (1/2) Epoch 48, batch 0, loss[loss=0.2267, simple_loss=0.3399, pruned_loss=0.05672, over 24527.00 frames. ], tot_loss[loss=0.2267, simple_loss=0.3399, pruned_loss=0.05672, over 24527.00 frames. ], batch size: 137, lr: 4.42e-03, grad_scale: 32.0 2026-09-24 04:19:38,546 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 04:19:50,024 INFO [train.py:1224] (1/2) Epoch 48, validation: loss=0.1731, simple_loss=0.2919, pruned_loss=0.02714, over 2564189.00 frames. 2026-09-24 04:19:50,024 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 04:19:56,631 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=150120.0, ans=0.1 2026-09-24 04:19:58,772 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.755e+02 3.407e+02 3.828e+02 4.280e+02 6.595e+02, threshold=7.656e+02, percent-clipped=0.0 2026-09-24 04:20:15,532 INFO [train.py:1192] (1/2) Epoch 48, batch 50, loss[loss=0.2214, simple_loss=0.3303, pruned_loss=0.05622, over 24175.00 frames. ], tot_loss[loss=0.2741, simple_loss=0.3881, pruned_loss=0.08005, over 1082834.82 frames. ], batch size: 125, lr: 4.42e-03, grad_scale: 32.0 2026-09-24 04:20:17,088 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=150253.33333333334, ans=0.05 2026-09-24 04:20:24,962 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=150286.66666666666, ans=0.125 2026-09-24 04:20:29,268 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.19 vs. limit=15.0 2026-09-24 04:20:36,803 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=150386.66666666666, ans=0.1 2026-09-24 04:20:37,283 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=150386.66666666666, ans=0.125 2026-09-24 04:20:38,411 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.89 vs. limit=15.0 2026-09-24 04:20:41,304 INFO [train.py:1192] (1/2) Epoch 48, batch 100, loss[loss=0.2737, simple_loss=0.3834, pruned_loss=0.08203, over 24604.00 frames. ], tot_loss[loss=0.2763, simple_loss=0.3918, pruned_loss=0.08041, over 1916715.80 frames. ], batch size: 154, lr: 4.41e-03, grad_scale: 32.0 2026-09-24 04:20:43,749 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.25 vs. limit=12.0 2026-09-24 04:20:47,234 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=150453.33333333334, ans=0.125 2026-09-24 04:20:49,515 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.639e+02 3.321e+02 3.619e+02 4.205e+02 5.847e+02, threshold=7.239e+02, percent-clipped=0.0 2026-09-24 04:20:57,038 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.55 vs. limit=15.0 2026-09-24 04:21:03,088 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=150553.33333333334, ans=0.0 2026-09-24 04:21:04,696 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.min_abs, batch_count=150553.33333333334, ans=0.5 2026-09-24 04:21:06,297 INFO [train.py:1192] (1/2) Epoch 48, batch 150, loss[loss=0.2476, simple_loss=0.3475, pruned_loss=0.07387, over 24252.00 frames. ], tot_loss[loss=0.2707, simple_loss=0.3857, pruned_loss=0.07786, over 2562138.76 frames. ], batch size: 125, lr: 4.41e-03, grad_scale: 32.0 2026-09-24 04:21:12,995 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=150620.0, ans=0.1 2026-09-24 04:21:15,507 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.72 vs. limit=15.0 2026-09-24 04:21:19,371 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=150653.33333333334, ans=0.125 2026-09-24 04:21:25,221 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.76 vs. limit=6.0 2026-09-24 04:21:25,940 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=150686.66666666666, ans=0.2 2026-09-24 04:21:27,244 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=150720.0, ans=0.025 2026-09-24 04:21:31,956 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=150753.33333333334, ans=0.2 2026-09-24 04:21:32,297 INFO [train.py:1192] (1/2) Epoch 48, batch 200, loss[loss=0.2846, simple_loss=0.413, pruned_loss=0.07811, over 24229.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3839, pruned_loss=0.07708, over 3060243.89 frames. ], batch size: 257, lr: 4.41e-03, grad_scale: 32.0 2026-09-24 04:21:40,828 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.610e+02 3.325e+02 3.930e+02 4.573e+02 7.027e+02, threshold=7.861e+02, percent-clipped=0.0 2026-09-24 04:21:41,866 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=150820.0, ans=0.05 2026-09-24 04:21:57,389 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.33 vs. limit=6.0 2026-09-24 04:21:57,570 INFO [train.py:1192] (1/2) Epoch 48, batch 250, loss[loss=0.284, simple_loss=0.411, pruned_loss=0.07856, over 24401.00 frames. ], tot_loss[loss=0.2695, simple_loss=0.3839, pruned_loss=0.07754, over 3444614.24 frames. ], batch size: 225, lr: 4.41e-03, grad_scale: 32.0 2026-09-24 04:21:58,653 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=150920.0, ans=0.025 2026-09-24 04:22:13,219 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.22 vs. limit=15.0 2026-09-24 04:22:20,915 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=151053.33333333334, ans=0.125 2026-09-24 04:22:20,980 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=151053.33333333334, ans=0.125 2026-09-24 04:22:22,758 INFO [train.py:1192] (1/2) Epoch 48, batch 300, loss[loss=0.2616, simple_loss=0.3858, pruned_loss=0.06866, over 24524.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3827, pruned_loss=0.07667, over 3758508.30 frames. ], batch size: 204, lr: 4.40e-03, grad_scale: 32.0 2026-09-24 04:22:31,874 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.428e+02 3.339e+02 3.787e+02 4.147e+02 5.561e+02, threshold=7.574e+02, percent-clipped=0.0 2026-09-24 04:22:36,414 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=151153.33333333334, ans=0.025 2026-09-24 04:22:38,700 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=151186.66666666666, ans=0.0 2026-09-24 04:22:46,972 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=151220.0, ans=0.125 2026-09-24 04:22:48,688 INFO [train.py:1192] (1/2) Epoch 48, batch 350, loss[loss=0.2183, simple_loss=0.3308, pruned_loss=0.05297, over 24581.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3831, pruned_loss=0.07663, over 3999921.01 frames. ], batch size: 137, lr: 4.40e-03, grad_scale: 32.0 2026-09-24 04:22:50,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=151253.33333333334, ans=0.125 2026-09-24 04:23:06,179 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=151353.33333333334, ans=0.035 2026-09-24 04:23:13,160 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=151420.0, ans=0.2 2026-09-24 04:23:13,577 INFO [train.py:1192] (1/2) Epoch 48, batch 400, loss[loss=0.2877, simple_loss=0.4011, pruned_loss=0.08718, over 24560.00 frames. ], tot_loss[loss=0.2677, simple_loss=0.3826, pruned_loss=0.07638, over 4181457.95 frames. ], batch size: 170, lr: 4.40e-03, grad_scale: 32.0 2026-09-24 04:23:15,425 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=151420.0, ans=0.125 2026-09-24 04:23:16,172 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.31 vs. limit=22.5 2026-09-24 04:23:17,008 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=151420.0, ans=0.1 2026-09-24 04:23:20,481 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.min_positive, batch_count=151453.33333333334, ans=0.05 2026-09-24 04:23:22,736 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.837e+02 3.335e+02 3.676e+02 4.305e+02 6.049e+02, threshold=7.352e+02, percent-clipped=0.0 2026-09-24 04:23:28,396 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=151486.66666666666, ans=0.125 2026-09-24 04:23:29,534 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.37 vs. limit=22.5 2026-09-24 04:23:31,880 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=151520.0, ans=0.0 2026-09-24 04:23:39,845 INFO [train.py:1192] (1/2) Epoch 48, batch 450, loss[loss=0.285, simple_loss=0.4031, pruned_loss=0.08342, over 24631.00 frames. ], tot_loss[loss=0.2695, simple_loss=0.384, pruned_loss=0.07753, over 4320598.26 frames. ], batch size: 175, lr: 4.40e-03, grad_scale: 32.0 2026-09-24 04:23:45,794 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.56 vs. limit=22.5 2026-09-24 04:23:56,824 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=151686.66666666666, ans=0.0 2026-09-24 04:24:04,826 INFO [train.py:1192] (1/2) Epoch 48, batch 500, loss[loss=0.2824, simple_loss=0.4059, pruned_loss=0.07942, over 24508.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.382, pruned_loss=0.07632, over 4438073.71 frames. ], batch size: 218, lr: 4.39e-03, grad_scale: 32.0 2026-09-24 04:24:07,329 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=151753.33333333334, ans=0.07 2026-09-24 04:24:13,349 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.540e+02 3.286e+02 3.673e+02 4.043e+02 7.538e+02, threshold=7.347e+02, percent-clipped=1.0 2026-09-24 04:24:19,885 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=151853.33333333334, ans=0.0 2026-09-24 04:24:20,060 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.37 vs. limit=15.0 2026-09-24 04:24:22,533 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=151853.33333333334, ans=0.125 2026-09-24 04:24:29,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=151886.66666666666, ans=0.0 2026-09-24 04:24:30,563 INFO [train.py:1192] (1/2) Epoch 48, batch 550, loss[loss=0.3068, simple_loss=0.4302, pruned_loss=0.09165, over 24282.00 frames. ], tot_loss[loss=0.269, simple_loss=0.3834, pruned_loss=0.07732, over 4523067.36 frames. ], batch size: 257, lr: 4.39e-03, grad_scale: 32.0 2026-09-24 04:24:33,014 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=151920.0, ans=0.125 2026-09-24 04:24:34,045 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=151920.0, ans=0.125 2026-09-24 04:24:39,325 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=151953.33333333334, ans=0.125 2026-09-24 04:24:55,518 INFO [train.py:1192] (1/2) Epoch 48, batch 600, loss[loss=0.2685, simple_loss=0.3969, pruned_loss=0.07002, over 24320.00 frames. ], tot_loss[loss=0.2688, simple_loss=0.3836, pruned_loss=0.077, over 4588658.03 frames. ], batch size: 234, lr: 4.39e-03, grad_scale: 32.0 2026-09-24 04:24:55,611 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=152086.66666666666, ans=0.0 2026-09-24 04:24:55,835 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.79 vs. limit=15.0 2026-09-24 04:24:59,908 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.91 vs. limit=22.5 2026-09-24 04:25:02,140 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=152120.0, ans=0.125 2026-09-24 04:25:03,863 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.649e+02 3.267e+02 3.673e+02 4.105e+02 6.858e+02, threshold=7.345e+02, percent-clipped=0.0 2026-09-24 04:25:15,392 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=152220.0, ans=0.2 2026-09-24 04:25:20,876 INFO [train.py:1192] (1/2) Epoch 48, batch 650, loss[loss=0.2532, simple_loss=0.3687, pruned_loss=0.06883, over 24636.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3826, pruned_loss=0.07632, over 4653720.47 frames. ], batch size: 154, lr: 4.39e-03, grad_scale: 32.0 2026-09-24 04:25:35,396 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=152320.0, ans=0.0 2026-09-24 04:25:35,896 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=152353.33333333334, ans=0.1 2026-09-24 04:25:40,008 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=152353.33333333334, ans=0.125 2026-09-24 04:25:46,278 INFO [train.py:1192] (1/2) Epoch 48, batch 700, loss[loss=0.2458, simple_loss=0.3622, pruned_loss=0.06467, over 24555.00 frames. ], tot_loss[loss=0.2685, simple_loss=0.3836, pruned_loss=0.07667, over 4688433.91 frames. ], batch size: 158, lr: 4.39e-03, grad_scale: 32.0 2026-09-24 04:25:50,946 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=152453.33333333334, ans=0.1 2026-09-24 04:25:54,378 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=152453.33333333334, ans=0.0 2026-09-24 04:25:54,862 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.695e+02 3.461e+02 3.802e+02 4.262e+02 6.575e+02, threshold=7.604e+02, percent-clipped=0.0 2026-09-24 04:26:07,894 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.04 vs. limit=12.0 2026-09-24 04:26:11,637 INFO [train.py:1192] (1/2) Epoch 48, batch 750, loss[loss=0.2806, simple_loss=0.3928, pruned_loss=0.0842, over 24628.00 frames. ], tot_loss[loss=0.2675, simple_loss=0.3826, pruned_loss=0.07618, over 4721620.74 frames. ], batch size: 175, lr: 4.38e-03, grad_scale: 32.0 2026-09-24 04:26:32,438 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=152720.0, ans=0.1 2026-09-24 04:26:36,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=152753.33333333334, ans=0.0 2026-09-24 04:26:37,351 INFO [train.py:1192] (1/2) Epoch 48, batch 800, loss[loss=0.2163, simple_loss=0.3335, pruned_loss=0.04958, over 24595.00 frames. ], tot_loss[loss=0.267, simple_loss=0.3822, pruned_loss=0.07594, over 4747697.10 frames. ], batch size: 137, lr: 4.38e-03, grad_scale: 32.0 2026-09-24 04:26:40,683 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=152753.33333333334, ans=0.125 2026-09-24 04:26:45,995 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.614e+02 3.351e+02 3.680e+02 4.144e+02 5.789e+02, threshold=7.360e+02, percent-clipped=0.0 2026-09-24 04:26:50,343 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:26:53,025 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=152853.33333333334, ans=0.125 2026-09-24 04:27:03,093 INFO [train.py:1192] (1/2) Epoch 48, batch 850, loss[loss=0.2911, simple_loss=0.4179, pruned_loss=0.08211, over 24529.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3814, pruned_loss=0.07542, over 4767636.03 frames. ], batch size: 204, lr: 4.38e-03, grad_scale: 32.0 2026-09-24 04:27:03,658 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=152920.0, ans=0.0 2026-09-24 04:27:28,540 INFO [train.py:1192] (1/2) Epoch 48, batch 900, loss[loss=0.2471, simple_loss=0.3588, pruned_loss=0.06771, over 24571.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3817, pruned_loss=0.07576, over 4778658.08 frames. ], batch size: 137, lr: 4.38e-03, grad_scale: 32.0 2026-09-24 04:27:37,269 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.574e+02 3.308e+02 3.681e+02 3.992e+02 6.338e+02, threshold=7.363e+02, percent-clipped=0.0 2026-09-24 04:27:37,872 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=153120.0, ans=0.125 2026-09-24 04:27:40,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=153153.33333333334, ans=0.125 2026-09-24 04:27:48,171 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=153220.0, ans=0.125 2026-09-24 04:27:48,299 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.01 vs. limit=10.0 2026-09-24 04:27:53,768 INFO [train.py:1192] (1/2) Epoch 48, batch 950, loss[loss=0.3586, simple_loss=0.4331, pruned_loss=0.142, over 10603.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3805, pruned_loss=0.077, over 4708367.38 frames. ], batch size: 334, lr: 4.37e-03, grad_scale: 32.0 2026-09-24 04:27:54,807 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=153253.33333333334, ans=0.125 2026-09-24 04:28:05,084 INFO [train.py:1192] (1/2) Epoch 49, batch 0, loss[loss=0.2411, simple_loss=0.3539, pruned_loss=0.06411, over 24568.00 frames. ], tot_loss[loss=0.2411, simple_loss=0.3539, pruned_loss=0.06411, over 24568.00 frames. ], batch size: 137, lr: 4.33e-03, grad_scale: 32.0 2026-09-24 04:28:05,084 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 04:28:16,654 INFO [train.py:1224] (1/2) Epoch 49, validation: loss=0.1742, simple_loss=0.2928, pruned_loss=0.02777, over 2564189.00 frames. 2026-09-24 04:28:16,654 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 04:28:20,791 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=153313.33333333334, ans=0.07 2026-09-24 04:28:21,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=153313.33333333334, ans=0.1 2026-09-24 04:28:24,496 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=153313.33333333334, ans=0.125 2026-09-24 04:28:31,463 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=153380.0, ans=0.2 2026-09-24 04:28:32,912 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=153380.0, ans=0.125 2026-09-24 04:28:41,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=153446.66666666666, ans=0.125 2026-09-24 04:28:41,629 INFO [train.py:1192] (1/2) Epoch 49, batch 50, loss[loss=0.2062, simple_loss=0.321, pruned_loss=0.04567, over 24302.00 frames. ], tot_loss[loss=0.276, simple_loss=0.3898, pruned_loss=0.08115, over 1082301.96 frames. ], batch size: 125, lr: 4.33e-03, grad_scale: 32.0 2026-09-24 04:28:45,222 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.98 vs. limit=15.0 2026-09-24 04:28:45,909 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.772e+02 3.411e+02 3.790e+02 4.372e+02 7.575e+02, threshold=7.579e+02, percent-clipped=1.0 2026-09-24 04:28:57,386 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.67 vs. limit=15.0 2026-09-24 04:29:06,955 INFO [train.py:1192] (1/2) Epoch 49, batch 100, loss[loss=0.2557, simple_loss=0.3721, pruned_loss=0.0696, over 24592.00 frames. ], tot_loss[loss=0.2764, simple_loss=0.3921, pruned_loss=0.0804, over 1916196.07 frames. ], batch size: 154, lr: 4.32e-03, grad_scale: 32.0 2026-09-24 04:29:13,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=153646.66666666666, ans=0.1 2026-09-24 04:29:32,904 INFO [train.py:1192] (1/2) Epoch 49, batch 150, loss[loss=0.2366, simple_loss=0.3377, pruned_loss=0.06774, over 24251.00 frames. ], tot_loss[loss=0.272, simple_loss=0.3868, pruned_loss=0.07863, over 2561188.83 frames. ], batch size: 125, lr: 4.32e-03, grad_scale: 32.0 2026-09-24 04:29:37,156 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.713e+02 3.405e+02 3.569e+02 4.203e+02 6.730e+02, threshold=7.137e+02, percent-clipped=0.0 2026-09-24 04:29:51,595 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.93 vs. limit=10.0 2026-09-24 04:29:51,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=153880.0, ans=0.125 2026-09-24 04:29:54,107 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=153913.33333333334, ans=0.125 2026-09-24 04:29:58,648 INFO [train.py:1192] (1/2) Epoch 49, batch 200, loss[loss=0.3009, simple_loss=0.4228, pruned_loss=0.08953, over 24216.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3834, pruned_loss=0.07671, over 3060444.70 frames. ], batch size: 257, lr: 4.32e-03, grad_scale: 32.0 2026-09-24 04:30:02,189 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:30:24,608 INFO [train.py:1192] (1/2) Epoch 49, batch 250, loss[loss=0.2793, simple_loss=0.4089, pruned_loss=0.07486, over 24379.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.384, pruned_loss=0.07761, over 3444430.07 frames. ], batch size: 225, lr: 4.32e-03, grad_scale: 32.0 2026-09-24 04:30:26,562 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=154113.33333333334, ans=0.125 2026-09-24 04:30:27,539 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:30:28,990 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.668e+02 3.483e+02 3.921e+02 4.551e+02 6.304e+02, threshold=7.842e+02, percent-clipped=0.0 2026-09-24 04:30:36,408 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=154180.0, ans=0.07 2026-09-24 04:30:40,191 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:30:41,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=154213.33333333334, ans=0.125 2026-09-24 04:30:50,198 INFO [train.py:1192] (1/2) Epoch 49, batch 300, loss[loss=0.2819, simple_loss=0.4104, pruned_loss=0.07674, over 24579.00 frames. ], tot_loss[loss=0.2677, simple_loss=0.3825, pruned_loss=0.07647, over 3757361.92 frames. ], batch size: 204, lr: 4.31e-03, grad_scale: 32.0 2026-09-24 04:30:55,027 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=154313.33333333334, ans=0.2 2026-09-24 04:30:55,980 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=154313.33333333334, ans=0.125 2026-09-24 04:30:56,144 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.45 vs. limit=10.0 2026-09-24 04:30:56,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=154313.33333333334, ans=0.1 2026-09-24 04:31:04,814 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=154380.0, ans=0.025 2026-09-24 04:31:10,864 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=154413.33333333334, ans=0.0 2026-09-24 04:31:12,772 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.82 vs. limit=22.5 2026-09-24 04:31:15,198 INFO [train.py:1192] (1/2) Epoch 49, batch 350, loss[loss=0.2237, simple_loss=0.3391, pruned_loss=0.05419, over 24570.00 frames. ], tot_loss[loss=0.2691, simple_loss=0.3838, pruned_loss=0.07721, over 3998604.11 frames. ], batch size: 137, lr: 4.31e-03, grad_scale: 32.0 2026-09-24 04:31:15,767 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=154446.66666666666, ans=0.025 2026-09-24 04:31:19,892 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.470e+02 3.382e+02 3.658e+02 4.103e+02 5.631e+02, threshold=7.315e+02, percent-clipped=0.0 2026-09-24 04:31:20,010 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=154480.0, ans=0.125 2026-09-24 04:31:20,988 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=154480.0, ans=0.125 2026-09-24 04:31:26,813 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=154513.33333333334, ans=0.1 2026-09-24 04:31:40,921 INFO [train.py:1192] (1/2) Epoch 49, batch 400, loss[loss=0.2584, simple_loss=0.3793, pruned_loss=0.06872, over 24549.00 frames. ], tot_loss[loss=0.2675, simple_loss=0.3825, pruned_loss=0.07631, over 4183518.39 frames. ], batch size: 170, lr: 4.31e-03, grad_scale: 32.0 2026-09-24 04:31:46,835 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=154646.66666666666, ans=0.125 2026-09-24 04:31:54,670 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=7.85 vs. limit=15.0 2026-09-24 04:31:59,354 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.78 vs. limit=15.0 2026-09-24 04:32:00,101 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=154713.33333333334, ans=0.0 2026-09-24 04:32:04,771 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:32:06,541 INFO [train.py:1192] (1/2) Epoch 49, batch 450, loss[loss=0.2876, simple_loss=0.4048, pruned_loss=0.08518, over 24634.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3827, pruned_loss=0.07666, over 4322011.60 frames. ], batch size: 175, lr: 4.31e-03, grad_scale: 32.0 2026-09-24 04:32:07,147 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=154780.0, ans=0.0 2026-09-24 04:32:09,586 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=154780.0, ans=0.025 2026-09-24 04:32:11,433 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.823e+02 3.406e+02 3.821e+02 4.417e+02 7.384e+02, threshold=7.641e+02, percent-clipped=1.0 2026-09-24 04:32:12,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=154813.33333333334, ans=0.125 2026-09-24 04:32:17,018 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.19 vs. limit=15.0 2026-09-24 04:32:24,091 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=154880.0, ans=0.125 2026-09-24 04:32:31,780 INFO [train.py:1192] (1/2) Epoch 49, batch 500, loss[loss=0.2954, simple_loss=0.4154, pruned_loss=0.08765, over 24542.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3812, pruned_loss=0.07613, over 4439990.39 frames. ], batch size: 218, lr: 4.31e-03, grad_scale: 32.0 2026-09-24 04:32:42,899 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=155013.33333333334, ans=0.0 2026-09-24 04:32:47,278 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=155046.66666666666, ans=0.125 2026-09-24 04:32:47,293 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=155046.66666666666, ans=0.0 2026-09-24 04:32:48,195 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=155046.66666666666, ans=0.125 2026-09-24 04:32:49,121 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=155046.66666666666, ans=0.125 2026-09-24 04:32:49,156 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=155046.66666666666, ans=0.0 2026-09-24 04:32:49,644 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=155046.66666666666, ans=0.125 2026-09-24 04:32:50,038 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=155046.66666666666, ans=0.125 2026-09-24 04:32:52,411 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=155080.0, ans=0.0 2026-09-24 04:32:56,337 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=155080.0, ans=0.0 2026-09-24 04:32:57,669 INFO [train.py:1192] (1/2) Epoch 49, batch 550, loss[loss=0.2888, simple_loss=0.4131, pruned_loss=0.08222, over 24266.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3824, pruned_loss=0.07684, over 4524785.69 frames. ], batch size: 257, lr: 4.30e-03, grad_scale: 32.0 2026-09-24 04:33:01,151 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=155113.33333333334, ans=0.125 2026-09-24 04:33:01,922 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.674e+02 3.320e+02 3.622e+02 4.066e+02 5.897e+02, threshold=7.244e+02, percent-clipped=0.0 2026-09-24 04:33:04,426 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=155146.66666666666, ans=0.015 2026-09-24 04:33:05,992 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=155146.66666666666, ans=0.1 2026-09-24 04:33:07,330 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=155180.0, ans=0.125 2026-09-24 04:33:16,668 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=155213.33333333334, ans=0.07 2026-09-24 04:33:20,909 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.44 vs. limit=15.0 2026-09-24 04:33:23,652 INFO [train.py:1192] (1/2) Epoch 49, batch 600, loss[loss=0.3074, simple_loss=0.4272, pruned_loss=0.09376, over 24319.00 frames. ], tot_loss[loss=0.2691, simple_loss=0.3834, pruned_loss=0.0774, over 4591082.72 frames. ], batch size: 234, lr: 4.30e-03, grad_scale: 32.0 2026-09-24 04:33:30,343 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=155313.33333333334, ans=0.07 2026-09-24 04:33:37,265 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=155346.66666666666, ans=0.1 2026-09-24 04:33:40,110 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=155380.0, ans=0.07 2026-09-24 04:33:47,929 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=155413.33333333334, ans=0.125 2026-09-24 04:33:48,806 INFO [train.py:1192] (1/2) Epoch 49, batch 650, loss[loss=0.2585, simple_loss=0.3734, pruned_loss=0.07176, over 24615.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3822, pruned_loss=0.07649, over 4655755.82 frames. ], batch size: 154, lr: 4.30e-03, grad_scale: 32.0 2026-09-24 04:33:52,645 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=155446.66666666666, ans=0.0 2026-09-24 04:33:52,681 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=155446.66666666666, ans=0.125 2026-09-24 04:33:53,474 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.674e+02 3.285e+02 3.739e+02 4.267e+02 7.018e+02, threshold=7.478e+02, percent-clipped=0.0 2026-09-24 04:33:54,920 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=155480.0, ans=0.1 2026-09-24 04:34:00,738 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.12 vs. limit=15.0 2026-09-24 04:34:02,051 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=155513.33333333334, ans=0.2 2026-09-24 04:34:04,212 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=155546.66666666666, ans=0.125 2026-09-24 04:34:09,851 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=155580.0, ans=0.125 2026-09-24 04:34:12,764 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=155580.0, ans=0.95 2026-09-24 04:34:12,771 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=155580.0, ans=0.1 2026-09-24 04:34:12,950 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.48 vs. limit=15.0 2026-09-24 04:34:14,580 INFO [train.py:1192] (1/2) Epoch 49, batch 700, loss[loss=0.2547, simple_loss=0.3687, pruned_loss=0.07034, over 24551.00 frames. ], tot_loss[loss=0.2675, simple_loss=0.3827, pruned_loss=0.07617, over 4690794.50 frames. ], batch size: 158, lr: 4.30e-03, grad_scale: 32.0 2026-09-24 04:34:15,205 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten.whitening_limit, batch_count=155613.33333333334, ans=15.0 2026-09-24 04:34:25,399 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=155680.0, ans=0.125 2026-09-24 04:34:27,112 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.54 vs. limit=22.5 2026-09-24 04:34:32,755 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.71 vs. limit=15.0 2026-09-24 04:34:39,548 INFO [train.py:1192] (1/2) Epoch 49, batch 750, loss[loss=0.2719, simple_loss=0.3879, pruned_loss=0.07795, over 24619.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3817, pruned_loss=0.07577, over 4726933.14 frames. ], batch size: 175, lr: 4.29e-03, grad_scale: 32.0 2026-09-24 04:34:44,235 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.581e+02 3.365e+02 3.787e+02 4.317e+02 7.388e+02, threshold=7.575e+02, percent-clipped=0.0 2026-09-24 04:34:48,732 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=155813.33333333334, ans=0.125 2026-09-24 04:34:50,164 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=155846.66666666666, ans=0.0 2026-09-24 04:34:53,888 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=155846.66666666666, ans=0.125 2026-09-24 04:34:57,749 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=155880.0, ans=0.0 2026-09-24 04:35:03,997 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=155913.33333333334, ans=0.0 2026-09-24 04:35:04,012 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=155913.33333333334, ans=0.125 2026-09-24 04:35:04,932 INFO [train.py:1192] (1/2) Epoch 49, batch 800, loss[loss=0.2216, simple_loss=0.3352, pruned_loss=0.05397, over 24573.00 frames. ], tot_loss[loss=0.2664, simple_loss=0.3815, pruned_loss=0.07564, over 4753857.68 frames. ], batch size: 137, lr: 4.29e-03, grad_scale: 32.0 2026-09-24 04:35:11,586 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=155980.0, ans=0.035 2026-09-24 04:35:13,562 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.73 vs. limit=15.0 2026-09-24 04:35:13,940 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=155980.0, ans=0.025 2026-09-24 04:35:22,734 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.71 vs. limit=6.0 2026-09-24 04:35:29,440 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=156080.0, ans=0.0 2026-09-24 04:35:30,728 INFO [train.py:1192] (1/2) Epoch 49, batch 850, loss[loss=0.3101, simple_loss=0.4274, pruned_loss=0.0964, over 24569.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3812, pruned_loss=0.07552, over 4773502.30 frames. ], batch size: 204, lr: 4.29e-03, grad_scale: 32.0 2026-09-24 04:35:32,767 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.86 vs. limit=6.0 2026-09-24 04:35:35,323 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=156113.33333333334, ans=0.0 2026-09-24 04:35:35,610 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.754e+02 3.290e+02 3.634e+02 4.203e+02 5.240e+02, threshold=7.268e+02, percent-clipped=0.0 2026-09-24 04:35:41,790 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=156180.0, ans=0.125 2026-09-24 04:35:43,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=156180.0, ans=0.0 2026-09-24 04:35:56,331 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=156280.0, ans=0.125 2026-09-24 04:35:56,851 INFO [train.py:1192] (1/2) Epoch 49, batch 900, loss[loss=0.2206, simple_loss=0.3363, pruned_loss=0.05239, over 24583.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3815, pruned_loss=0.07572, over 4783531.30 frames. ], batch size: 137, lr: 4.29e-03, grad_scale: 32.0 2026-09-24 04:35:57,835 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=156280.0, ans=0.125 2026-09-24 04:36:00,811 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=156280.0, ans=0.125 2026-09-24 04:36:03,614 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.prob, batch_count=156313.33333333334, ans=0.125 2026-09-24 04:36:05,594 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=156313.33333333334, ans=0.025 2026-09-24 04:36:12,819 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=156380.0, ans=0.0 2026-09-24 04:36:21,642 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=156446.66666666666, ans=0.1 2026-09-24 04:36:22,053 INFO [train.py:1192] (1/2) Epoch 49, batch 950, loss[loss=0.3559, simple_loss=0.4125, pruned_loss=0.1497, over 11882.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3806, pruned_loss=0.07679, over 4716757.62 frames. ], batch size: 333, lr: 4.28e-03, grad_scale: 32.0 2026-09-24 04:36:31,064 INFO [train.py:1192] (1/2) Epoch 50, batch 0, loss[loss=0.2279, simple_loss=0.3428, pruned_loss=0.05644, over 24577.00 frames. ], tot_loss[loss=0.2279, simple_loss=0.3428, pruned_loss=0.05644, over 24577.00 frames. ], batch size: 137, lr: 4.24e-03, grad_scale: 32.0 2026-09-24 04:36:31,064 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 04:36:42,794 INFO [train.py:1224] (1/2) Epoch 50, validation: loss=0.1711, simple_loss=0.2901, pruned_loss=0.02606, over 2564189.00 frames. 2026-09-24 04:36:42,795 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 04:36:43,678 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.433e+02 3.385e+02 3.879e+02 4.350e+02 6.538e+02, threshold=7.759e+02, percent-clipped=0.0 2026-09-24 04:36:53,940 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=156540.0, ans=0.0 2026-09-24 04:37:01,289 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=156573.33333333334, ans=0.125 2026-09-24 04:37:03,918 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=156606.66666666666, ans=0.125 2026-09-24 04:37:08,434 INFO [train.py:1192] (1/2) Epoch 50, batch 50, loss[loss=0.2218, simple_loss=0.3284, pruned_loss=0.05758, over 24274.00 frames. ], tot_loss[loss=0.2747, simple_loss=0.3884, pruned_loss=0.0805, over 1081219.56 frames. ], batch size: 125, lr: 4.24e-03, grad_scale: 32.0 2026-09-24 04:37:10,402 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=156640.0, ans=0.0 2026-09-24 04:37:15,292 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.37 vs. limit=22.5 2026-09-24 04:37:29,487 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.73 vs. limit=15.0 2026-09-24 04:37:29,733 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=156773.33333333334, ans=0.0 2026-09-24 04:37:31,638 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=156773.33333333334, ans=0.025 2026-09-24 04:37:34,293 INFO [train.py:1192] (1/2) Epoch 50, batch 100, loss[loss=0.2844, simple_loss=0.3822, pruned_loss=0.09331, over 24578.00 frames. ], tot_loss[loss=0.2773, simple_loss=0.3926, pruned_loss=0.08102, over 1916210.79 frames. ], batch size: 154, lr: 4.24e-03, grad_scale: 32.0 2026-09-24 04:37:35,234 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.636e+02 3.396e+02 3.722e+02 4.209e+02 6.213e+02, threshold=7.444e+02, percent-clipped=0.0 2026-09-24 04:37:45,877 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.09 vs. limit=15.0 2026-09-24 04:37:56,518 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=156940.0, ans=0.05 2026-09-24 04:37:59,478 INFO [train.py:1192] (1/2) Epoch 50, batch 150, loss[loss=0.2387, simple_loss=0.3412, pruned_loss=0.06809, over 24259.00 frames. ], tot_loss[loss=0.2713, simple_loss=0.3864, pruned_loss=0.07811, over 2561473.16 frames. ], batch size: 125, lr: 4.23e-03, grad_scale: 32.0 2026-09-24 04:38:17,476 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=157073.33333333334, ans=0.125 2026-09-24 04:38:19,262 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=157106.66666666666, ans=0.0 2026-09-24 04:38:19,984 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=157106.66666666666, ans=0.1 2026-09-24 04:38:24,990 INFO [train.py:1192] (1/2) Epoch 50, batch 200, loss[loss=0.2834, simple_loss=0.4091, pruned_loss=0.07885, over 24238.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3831, pruned_loss=0.07606, over 3061179.26 frames. ], batch size: 257, lr: 4.23e-03, grad_scale: 32.0 2026-09-24 04:38:26,000 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.389e+02 3.343e+02 3.775e+02 4.499e+02 6.276e+02, threshold=7.550e+02, percent-clipped=0.0 2026-09-24 04:38:39,150 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=11.24 vs. limit=15.0 2026-09-24 04:38:44,121 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=157240.0, ans=0.125 2026-09-24 04:38:51,434 INFO [train.py:1192] (1/2) Epoch 50, batch 250, loss[loss=0.3004, simple_loss=0.4254, pruned_loss=0.08773, over 24365.00 frames. ], tot_loss[loss=0.2693, simple_loss=0.384, pruned_loss=0.07726, over 3443963.59 frames. ], batch size: 225, lr: 4.23e-03, grad_scale: 32.0 2026-09-24 04:39:00,439 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=157340.0, ans=0.1 2026-09-24 04:39:02,260 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=157373.33333333334, ans=0.125 2026-09-24 04:39:09,120 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=157406.66666666666, ans=0.025 2026-09-24 04:39:11,003 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=157440.0, ans=0.0 2026-09-24 04:39:16,948 INFO [train.py:1192] (1/2) Epoch 50, batch 300, loss[loss=0.2869, simple_loss=0.4063, pruned_loss=0.08374, over 24558.00 frames. ], tot_loss[loss=0.2684, simple_loss=0.3832, pruned_loss=0.07687, over 3758036.10 frames. ], batch size: 204, lr: 4.23e-03, grad_scale: 32.0 2026-09-24 04:39:18,000 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.829e+02 3.385e+02 3.880e+02 4.348e+02 7.252e+02, threshold=7.759e+02, percent-clipped=0.0 2026-09-24 04:39:24,212 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.79 vs. limit=15.0 2026-09-24 04:39:25,925 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=157506.66666666666, ans=0.0 2026-09-24 04:39:26,867 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:39:35,407 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=157573.33333333334, ans=0.0 2026-09-24 04:39:36,992 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=157606.66666666666, ans=0.125 2026-09-24 04:39:42,199 INFO [train.py:1192] (1/2) Epoch 50, batch 350, loss[loss=0.2252, simple_loss=0.3389, pruned_loss=0.05571, over 24577.00 frames. ], tot_loss[loss=0.2678, simple_loss=0.383, pruned_loss=0.07629, over 3998434.22 frames. ], batch size: 137, lr: 4.23e-03, grad_scale: 32.0 2026-09-24 04:39:45,720 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=157640.0, ans=0.125 2026-09-24 04:39:47,855 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=157673.33333333334, ans=0.125 2026-09-24 04:39:53,338 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=157706.66666666666, ans=0.1 2026-09-24 04:39:55,274 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=157706.66666666666, ans=0.1 2026-09-24 04:39:57,564 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.34 vs. limit=15.0 2026-09-24 04:39:59,376 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.40 vs. limit=15.0 2026-09-24 04:40:00,687 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=157740.0, ans=0.125 2026-09-24 04:40:06,340 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=157773.33333333334, ans=0.1 2026-09-24 04:40:08,043 INFO [train.py:1192] (1/2) Epoch 50, batch 400, loss[loss=0.264, simple_loss=0.3824, pruned_loss=0.07281, over 24561.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3824, pruned_loss=0.07606, over 4183533.23 frames. ], batch size: 170, lr: 4.22e-03, grad_scale: 32.0 2026-09-24 04:40:09,092 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.477e+02 3.343e+02 3.833e+02 4.625e+02 8.353e+02, threshold=7.667e+02, percent-clipped=1.0 2026-09-24 04:40:15,255 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=157840.0, ans=0.125 2026-09-24 04:40:32,622 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.43 vs. limit=15.0 2026-09-24 04:40:33,871 INFO [train.py:1192] (1/2) Epoch 50, batch 450, loss[loss=0.2532, simple_loss=0.3758, pruned_loss=0.06527, over 24638.00 frames. ], tot_loss[loss=0.2674, simple_loss=0.3824, pruned_loss=0.07617, over 4319989.64 frames. ], batch size: 175, lr: 4.22e-03, grad_scale: 32.0 2026-09-24 04:40:35,469 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=157973.33333333334, ans=0.0 2026-09-24 04:40:39,299 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=158006.66666666666, ans=0.125 2026-09-24 04:40:40,249 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=158006.66666666666, ans=0.1 2026-09-24 04:40:40,263 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=158006.66666666666, ans=0.1 2026-09-24 04:40:49,214 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=158073.33333333334, ans=0.125 2026-09-24 04:40:52,518 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.47 vs. limit=6.0 2026-09-24 04:40:55,139 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=4.86 vs. limit=15.0 2026-09-24 04:40:56,793 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=158106.66666666666, ans=0.0 2026-09-24 04:40:58,422 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=158106.66666666666, ans=0.1 2026-09-24 04:40:58,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=158106.66666666666, ans=0.0 2026-09-24 04:40:59,742 INFO [train.py:1192] (1/2) Epoch 50, batch 500, loss[loss=0.3017, simple_loss=0.4188, pruned_loss=0.09228, over 24515.00 frames. ], tot_loss[loss=0.266, simple_loss=0.3809, pruned_loss=0.07551, over 4437912.47 frames. ], batch size: 218, lr: 4.22e-03, grad_scale: 32.0 2026-09-24 04:41:00,681 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.605e+02 3.299e+02 3.819e+02 4.283e+02 6.545e+02, threshold=7.637e+02, percent-clipped=0.0 2026-09-24 04:41:01,844 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=158140.0, ans=0.125 2026-09-24 04:41:02,701 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=158140.0, ans=0.0 2026-09-24 04:41:23,772 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:41:23,773 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=158273.33333333334, ans=0.1 2026-09-24 04:41:24,702 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=158273.33333333334, ans=0.125 2026-09-24 04:41:25,803 INFO [train.py:1192] (1/2) Epoch 50, batch 550, loss[loss=0.2821, simple_loss=0.4061, pruned_loss=0.07912, over 24287.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3817, pruned_loss=0.07589, over 4522762.15 frames. ], batch size: 257, lr: 4.22e-03, grad_scale: 32.0 2026-09-24 04:41:38,610 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=158373.33333333334, ans=0.0 2026-09-24 04:41:51,765 INFO [train.py:1192] (1/2) Epoch 50, batch 600, loss[loss=0.2574, simple_loss=0.3806, pruned_loss=0.06706, over 24323.00 frames. ], tot_loss[loss=0.267, simple_loss=0.3822, pruned_loss=0.07587, over 4589021.36 frames. ], batch size: 234, lr: 4.21e-03, grad_scale: 32.0 2026-09-24 04:41:52,797 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.780e+02 3.282e+02 3.695e+02 4.091e+02 6.346e+02, threshold=7.389e+02, percent-clipped=0.0 2026-09-24 04:42:05,649 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=158540.0, ans=0.0 2026-09-24 04:42:16,987 INFO [train.py:1192] (1/2) Epoch 50, batch 650, loss[loss=0.2556, simple_loss=0.3674, pruned_loss=0.07189, over 24611.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3812, pruned_loss=0.07546, over 4653733.32 frames. ], batch size: 154, lr: 4.21e-03, grad_scale: 32.0 2026-09-24 04:42:19,991 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.63 vs. limit=15.0 2026-09-24 04:42:25,486 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.12 vs. limit=22.5 2026-09-24 04:42:39,934 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.17 vs. limit=6.0 2026-09-24 04:42:42,470 INFO [train.py:1192] (1/2) Epoch 50, batch 700, loss[loss=0.2505, simple_loss=0.3655, pruned_loss=0.06776, over 24561.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3825, pruned_loss=0.07597, over 4689950.25 frames. ], batch size: 158, lr: 4.21e-03, grad_scale: 32.0 2026-09-24 04:42:43,768 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.653e+02 3.394e+02 3.848e+02 4.389e+02 7.114e+02, threshold=7.696e+02, percent-clipped=0.0 2026-09-24 04:42:45,240 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=158806.66666666666, ans=0.1 2026-09-24 04:42:50,594 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.82 vs. limit=15.0 2026-09-24 04:42:51,648 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=12.81 vs. limit=15.0 2026-09-24 04:42:54,700 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.11 vs. limit=22.5 2026-09-24 04:43:08,562 INFO [train.py:1192] (1/2) Epoch 50, batch 750, loss[loss=0.2755, simple_loss=0.3933, pruned_loss=0.07879, over 24618.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3822, pruned_loss=0.07616, over 4726320.18 frames. ], batch size: 175, lr: 4.21e-03, grad_scale: 32.0 2026-09-24 04:43:19,047 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=159040.0, ans=0.0 2026-09-24 04:43:21,463 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=159040.0, ans=0.125 2026-09-24 04:43:33,941 INFO [train.py:1192] (1/2) Epoch 50, batch 800, loss[loss=0.2184, simple_loss=0.3374, pruned_loss=0.04973, over 24545.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3816, pruned_loss=0.07592, over 4751656.94 frames. ], batch size: 137, lr: 4.21e-03, grad_scale: 32.0 2026-09-24 04:43:34,776 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.450e+02 3.348e+02 3.704e+02 4.242e+02 6.225e+02, threshold=7.409e+02, percent-clipped=0.0 2026-09-24 04:43:34,893 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=159140.0, ans=0.125 2026-09-24 04:43:58,854 INFO [train.py:1192] (1/2) Epoch 50, batch 850, loss[loss=0.2772, simple_loss=0.3997, pruned_loss=0.07733, over 24566.00 frames. ], tot_loss[loss=0.2657, simple_loss=0.3809, pruned_loss=0.07528, over 4770263.91 frames. ], batch size: 204, lr: 4.20e-03, grad_scale: 32.0 2026-09-24 04:44:02,872 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=159306.66666666666, ans=0.125 2026-09-24 04:44:17,618 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.94 vs. limit=15.0 2026-09-24 04:44:21,980 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=159440.0, ans=0.125 2026-09-24 04:44:23,820 INFO [train.py:1192] (1/2) Epoch 50, batch 900, loss[loss=0.221, simple_loss=0.3391, pruned_loss=0.05146, over 24558.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3812, pruned_loss=0.07551, over 4780731.18 frames. ], batch size: 137, lr: 4.20e-03, grad_scale: 32.0 2026-09-24 04:44:25,032 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.433e+02 3.308e+02 3.810e+02 4.336e+02 6.433e+02, threshold=7.620e+02, percent-clipped=0.0 2026-09-24 04:44:25,749 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.76 vs. limit=22.5 2026-09-24 04:44:29,309 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=159506.66666666666, ans=0.025 2026-09-24 04:44:33,829 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.34 vs. limit=15.0 2026-09-24 04:44:34,769 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=159540.0, ans=0.2 2026-09-24 04:44:35,764 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=13.71 vs. limit=22.5 2026-09-24 04:44:36,487 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=159540.0, ans=0.125 2026-09-24 04:44:46,000 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.50 vs. limit=22.5 2026-09-24 04:44:48,738 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=159640.0, ans=0.025 2026-09-24 04:44:49,058 INFO [train.py:1192] (1/2) Epoch 50, batch 950, loss[loss=0.3648, simple_loss=0.4295, pruned_loss=0.15, over 11307.00 frames. ], tot_loss[loss=0.2658, simple_loss=0.3795, pruned_loss=0.07604, over 4711417.60 frames. ], batch size: 333, lr: 4.20e-03, grad_scale: 32.0 2026-09-24 04:45:01,168 INFO [train.py:1192] (1/2) Epoch 51, batch 0, loss[loss=0.2087, simple_loss=0.3315, pruned_loss=0.043, over 24583.00 frames. ], tot_loss[loss=0.2087, simple_loss=0.3315, pruned_loss=0.043, over 24583.00 frames. ], batch size: 137, lr: 4.16e-03, grad_scale: 32.0 2026-09-24 04:45:01,168 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 04:45:11,765 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3832, 4.6789, 4.3419, 4.2288], device='cuda:1') 2026-09-24 04:45:12,820 INFO [train.py:1224] (1/2) Epoch 51, validation: loss=0.1742, simple_loss=0.2926, pruned_loss=0.02794, over 2564189.00 frames. 2026-09-24 04:45:12,821 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 04:45:13,926 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.70 vs. limit=15.0 2026-09-24 04:45:19,486 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.87 vs. limit=15.0 2026-09-24 04:45:22,301 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=21.16 vs. limit=22.5 2026-09-24 04:45:23,720 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.44 vs. limit=15.0 2026-09-24 04:45:32,670 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=159800.0, ans=0.0 2026-09-24 04:45:34,748 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=159800.0, ans=0.125 2026-09-24 04:45:35,099 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.751e+02 3.480e+02 3.802e+02 4.431e+02 6.259e+02, threshold=7.604e+02, percent-clipped=0.0 2026-09-24 04:45:35,685 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=159800.0, ans=0.1 2026-09-24 04:45:35,756 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=159800.0, ans=0.125 2026-09-24 04:45:38,323 INFO [train.py:1192] (1/2) Epoch 51, batch 50, loss[loss=0.2155, simple_loss=0.3281, pruned_loss=0.05143, over 24243.00 frames. ], tot_loss[loss=0.2696, simple_loss=0.3851, pruned_loss=0.0771, over 1081252.72 frames. ], batch size: 125, lr: 4.16e-03, grad_scale: 32.0 2026-09-24 04:45:43,436 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.63 vs. limit=15.0 2026-09-24 04:45:56,192 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=159933.33333333334, ans=0.2 2026-09-24 04:46:01,683 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=159966.66666666666, ans=0.125 2026-09-24 04:46:03,741 INFO [train.py:1192] (1/2) Epoch 51, batch 100, loss[loss=0.2588, simple_loss=0.3736, pruned_loss=0.07201, over 24602.00 frames. ], tot_loss[loss=0.2727, simple_loss=0.3897, pruned_loss=0.07785, over 1914170.87 frames. ], batch size: 154, lr: 4.15e-03, grad_scale: 32.0 2026-09-24 04:46:08,320 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=160033.33333333334, ans=0.125 2026-09-24 04:46:11,930 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=160033.33333333334, ans=0.1 2026-09-24 04:46:15,748 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.93 vs. limit=6.0 2026-09-24 04:46:22,418 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=160100.0, ans=0.2 2026-09-24 04:46:26,900 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.485e+02 3.390e+02 3.893e+02 4.368e+02 6.652e+02, threshold=7.786e+02, percent-clipped=0.0 2026-09-24 04:46:27,026 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=160133.33333333334, ans=0.0 2026-09-24 04:46:30,050 INFO [train.py:1192] (1/2) Epoch 51, batch 150, loss[loss=0.2367, simple_loss=0.3387, pruned_loss=0.06736, over 24288.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3857, pruned_loss=0.07713, over 2559757.81 frames. ], batch size: 125, lr: 4.15e-03, grad_scale: 32.0 2026-09-24 04:46:32,901 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=160166.66666666666, ans=0.0 2026-09-24 04:46:35,546 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:46:42,206 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=160233.33333333334, ans=0.125 2026-09-24 04:46:47,436 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=160266.66666666666, ans=0.125 2026-09-24 04:46:55,492 INFO [train.py:1192] (1/2) Epoch 51, batch 200, loss[loss=0.3039, simple_loss=0.423, pruned_loss=0.09237, over 24205.00 frames. ], tot_loss[loss=0.2679, simple_loss=0.3835, pruned_loss=0.07615, over 3059223.72 frames. ], batch size: 257, lr: 4.15e-03, grad_scale: 32.0 2026-09-24 04:46:56,067 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=160333.33333333334, ans=0.1 2026-09-24 04:46:56,537 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=160333.33333333334, ans=0.2 2026-09-24 04:47:02,749 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=160366.66666666666, ans=0.0 2026-09-24 04:47:04,374 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=160366.66666666666, ans=0.125 2026-09-24 04:47:08,708 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=160400.0, ans=0.04949747468305833 2026-09-24 04:47:17,477 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.506e+02 3.435e+02 3.856e+02 4.490e+02 6.082e+02, threshold=7.712e+02, percent-clipped=0.0 2026-09-24 04:47:20,317 INFO [train.py:1192] (1/2) Epoch 51, batch 250, loss[loss=0.2942, simple_loss=0.4149, pruned_loss=0.08679, over 24369.00 frames. ], tot_loss[loss=0.268, simple_loss=0.3833, pruned_loss=0.07634, over 3444228.47 frames. ], batch size: 225, lr: 4.15e-03, grad_scale: 32.0 2026-09-24 04:47:28,910 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=160533.33333333334, ans=0.2 2026-09-24 04:47:45,607 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=160633.33333333334, ans=0.125 2026-09-24 04:47:46,481 INFO [train.py:1192] (1/2) Epoch 51, batch 300, loss[loss=0.26, simple_loss=0.3889, pruned_loss=0.06558, over 24560.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3825, pruned_loss=0.07597, over 3757379.98 frames. ], batch size: 204, lr: 4.14e-03, grad_scale: 32.0 2026-09-24 04:47:53,100 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=160700.0, ans=0.0 2026-09-24 04:47:56,944 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=160733.33333333334, ans=0.125 2026-09-24 04:48:01,567 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.75 vs. limit=15.0 2026-09-24 04:48:08,887 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.699e+02 3.332e+02 3.581e+02 4.177e+02 6.227e+02, threshold=7.162e+02, percent-clipped=0.0 2026-09-24 04:48:09,119 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.66 vs. limit=15.0 2026-09-24 04:48:12,028 INFO [train.py:1192] (1/2) Epoch 51, batch 350, loss[loss=0.2355, simple_loss=0.3442, pruned_loss=0.06344, over 24549.00 frames. ], tot_loss[loss=0.2673, simple_loss=0.3829, pruned_loss=0.07586, over 3998042.87 frames. ], batch size: 137, lr: 4.14e-03, grad_scale: 32.0 2026-09-24 04:48:14,811 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=160833.33333333334, ans=0.125 2026-09-24 04:48:17,842 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:48:19,276 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=160866.66666666666, ans=0.1 2026-09-24 04:48:19,807 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=160866.66666666666, ans=0.025 2026-09-24 04:48:37,663 INFO [train.py:1192] (1/2) Epoch 51, batch 400, loss[loss=0.2777, simple_loss=0.3877, pruned_loss=0.0839, over 24555.00 frames. ], tot_loss[loss=0.2667, simple_loss=0.3821, pruned_loss=0.07564, over 4182982.73 frames. ], batch size: 170, lr: 4.14e-03, grad_scale: 32.0 2026-09-24 04:48:42,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=161033.33333333334, ans=0.2 2026-09-24 04:48:54,787 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:48:59,333 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=161133.33333333334, ans=0.125 2026-09-24 04:48:59,370 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=161133.33333333334, ans=0.125 2026-09-24 04:48:59,948 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.681e+02 3.395e+02 3.852e+02 4.592e+02 6.632e+02, threshold=7.704e+02, percent-clipped=0.0 2026-09-24 04:49:00,060 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=161133.33333333334, ans=0.0 2026-09-24 04:49:02,798 INFO [train.py:1192] (1/2) Epoch 51, batch 450, loss[loss=0.2642, simple_loss=0.3869, pruned_loss=0.07079, over 24634.00 frames. ], tot_loss[loss=0.2663, simple_loss=0.3815, pruned_loss=0.07549, over 4323059.79 frames. ], batch size: 175, lr: 4.14e-03, grad_scale: 32.0 2026-09-24 04:49:04,350 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=161166.66666666666, ans=0.0 2026-09-24 04:49:10,886 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=161200.0, ans=0.2 2026-09-24 04:49:15,747 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=161233.33333333334, ans=0.025 2026-09-24 04:49:25,845 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=161300.0, ans=0.125 2026-09-24 04:49:28,285 INFO [train.py:1192] (1/2) Epoch 51, batch 500, loss[loss=0.2705, simple_loss=0.3992, pruned_loss=0.07092, over 24510.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.38, pruned_loss=0.07475, over 4439964.42 frames. ], batch size: 218, lr: 4.14e-03, grad_scale: 32.0 2026-09-24 04:49:31,593 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=161333.33333333334, ans=0.125 2026-09-24 04:49:32,118 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=161333.33333333334, ans=0.0 2026-09-24 04:49:36,272 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=161366.66666666666, ans=0.0 2026-09-24 04:49:50,390 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.493e+02 3.341e+02 3.648e+02 4.164e+02 5.336e+02, threshold=7.296e+02, percent-clipped=0.0 2026-09-24 04:49:53,806 INFO [train.py:1192] (1/2) Epoch 51, batch 550, loss[loss=0.2747, simple_loss=0.4022, pruned_loss=0.07358, over 24269.00 frames. ], tot_loss[loss=0.2658, simple_loss=0.3812, pruned_loss=0.07524, over 4524268.49 frames. ], batch size: 257, lr: 4.13e-03, grad_scale: 32.0 2026-09-24 04:50:17,268 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=161633.33333333334, ans=10.0 2026-09-24 04:50:18,579 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=161633.33333333334, ans=0.0 2026-09-24 04:50:19,491 INFO [train.py:1192] (1/2) Epoch 51, batch 600, loss[loss=0.2763, simple_loss=0.4068, pruned_loss=0.07288, over 24311.00 frames. ], tot_loss[loss=0.2665, simple_loss=0.3818, pruned_loss=0.07562, over 4589933.99 frames. ], batch size: 234, lr: 4.13e-03, grad_scale: 32.0 2026-09-24 04:50:22,914 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 04:50:24,741 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=161700.0, ans=0.125 2026-09-24 04:50:27,745 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=161700.0, ans=0.5 2026-09-24 04:50:38,462 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=161766.66666666666, ans=0.035 2026-09-24 04:50:41,557 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.597e+02 3.326e+02 3.679e+02 4.212e+02 6.429e+02, threshold=7.358e+02, percent-clipped=0.0 2026-09-24 04:50:43,125 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=161800.0, ans=0.125 2026-09-24 04:50:44,524 INFO [train.py:1192] (1/2) Epoch 51, batch 650, loss[loss=0.2571, simple_loss=0.3724, pruned_loss=0.07085, over 24591.00 frames. ], tot_loss[loss=0.2653, simple_loss=0.3808, pruned_loss=0.07491, over 4654429.53 frames. ], batch size: 154, lr: 4.13e-03, grad_scale: 32.0 2026-09-24 04:50:46,932 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=161833.33333333334, ans=0.125 2026-09-24 04:50:49,426 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=161866.66666666666, ans=0.0 2026-09-24 04:50:51,784 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=161866.66666666666, ans=10.0 2026-09-24 04:50:57,224 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=161900.0, ans=0.2 2026-09-24 04:51:03,298 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=161933.33333333334, ans=0.2 2026-09-24 04:51:09,866 INFO [train.py:1192] (1/2) Epoch 51, batch 700, loss[loss=0.2436, simple_loss=0.3628, pruned_loss=0.06216, over 24558.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.3812, pruned_loss=0.07463, over 4690641.26 frames. ], batch size: 158, lr: 4.13e-03, grad_scale: 32.0 2026-09-24 04:51:12,985 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=162000.0, ans=0.125 2026-09-24 04:51:27,276 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=162100.0, ans=0.0 2026-09-24 04:51:30,953 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=162133.33333333334, ans=0.125 2026-09-24 04:51:30,957 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=162133.33333333334, ans=0.0 2026-09-24 04:51:32,180 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.650e+02 3.501e+02 3.853e+02 4.434e+02 7.375e+02, threshold=7.706e+02, percent-clipped=1.0 2026-09-24 04:51:35,132 INFO [train.py:1192] (1/2) Epoch 51, batch 750, loss[loss=0.288, simple_loss=0.4039, pruned_loss=0.08603, over 24627.00 frames. ], tot_loss[loss=0.2649, simple_loss=0.3806, pruned_loss=0.07456, over 4726875.14 frames. ], batch size: 175, lr: 4.13e-03, grad_scale: 32.0 2026-09-24 04:51:42,548 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.66 vs. limit=8.0 2026-09-24 04:51:43,645 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=162200.0, ans=0.125 2026-09-24 04:52:00,970 INFO [train.py:1192] (1/2) Epoch 51, batch 800, loss[loss=0.2258, simple_loss=0.343, pruned_loss=0.05427, over 24546.00 frames. ], tot_loss[loss=0.2643, simple_loss=0.3802, pruned_loss=0.07422, over 4752582.24 frames. ], batch size: 137, lr: 4.12e-03, grad_scale: 32.0 2026-09-24 04:52:09,589 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=162366.66666666666, ans=0.125 2026-09-24 04:52:10,140 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=162366.66666666666, ans=0.125 2026-09-24 04:52:13,894 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=162400.0, ans=0.0 2026-09-24 04:52:15,002 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=162400.0, ans=0.125 2026-09-24 04:52:17,756 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=162433.33333333334, ans=0.1 2026-09-24 04:52:23,769 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.585e+02 3.311e+02 3.770e+02 4.405e+02 6.210e+02, threshold=7.541e+02, percent-clipped=0.0 2026-09-24 04:52:23,880 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=162466.66666666666, ans=0.125 2026-09-24 04:52:26,427 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.convnext.out_whiten, num_groups=1, num_channels=128, metric=3.76 vs. limit=5.0 2026-09-24 04:52:26,556 INFO [train.py:1192] (1/2) Epoch 51, batch 850, loss[loss=0.2799, simple_loss=0.4044, pruned_loss=0.07766, over 24524.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.38, pruned_loss=0.07444, over 4772369.71 frames. ], batch size: 204, lr: 4.12e-03, grad_scale: 32.0 2026-09-24 04:52:27,691 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=162500.0, ans=0.125 2026-09-24 04:52:33,291 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=162533.33333333334, ans=0.125 2026-09-24 04:52:33,725 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=162533.33333333334, ans=0.07 2026-09-24 04:52:41,352 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=162566.66666666666, ans=0.125 2026-09-24 04:52:52,884 INFO [train.py:1192] (1/2) Epoch 51, batch 900, loss[loss=0.2193, simple_loss=0.3373, pruned_loss=0.0506, over 24541.00 frames. ], tot_loss[loss=0.2654, simple_loss=0.3806, pruned_loss=0.07509, over 4783150.78 frames. ], batch size: 137, lr: 4.12e-03, grad_scale: 32.0 2026-09-24 04:53:05,906 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=162733.33333333334, ans=0.125 2026-09-24 04:53:15,092 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.707e+02 3.515e+02 3.906e+02 4.366e+02 6.602e+02, threshold=7.811e+02, percent-clipped=0.0 2026-09-24 04:53:16,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=162800.0, ans=0.125 2026-09-24 04:53:17,471 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=162833.33333333334, ans=0.125 2026-09-24 04:53:17,799 INFO [train.py:1192] (1/2) Epoch 51, batch 950, loss[loss=0.3611, simple_loss=0.423, pruned_loss=0.1496, over 12001.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3799, pruned_loss=0.07623, over 4712442.90 frames. ], batch size: 333, lr: 4.12e-03, grad_scale: 32.0 2026-09-24 04:53:18,013 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=162833.33333333334, ans=22.5 2026-09-24 04:53:43,616 INFO [train.py:1192] (1/2) Epoch 52, batch 0, loss[loss=0.2127, simple_loss=0.332, pruned_loss=0.04667, over 24585.00 frames. ], tot_loss[loss=0.2127, simple_loss=0.332, pruned_loss=0.04667, over 24585.00 frames. ], batch size: 137, lr: 4.08e-03, grad_scale: 32.0 2026-09-24 04:53:43,617 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 04:53:51,457 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0434, 2.0540, 2.4161, 2.1122, 1.7831, 2.3297, 1.2764, 1.9655], device='cuda:1') 2026-09-24 04:53:52,781 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6819, 2.2581, 3.8294, 2.3281], device='cuda:1') 2026-09-24 04:53:54,259 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.1869, 2.8522, 2.8602, 1.9812], device='cuda:1') 2026-09-24 04:53:55,114 INFO [train.py:1224] (1/2) Epoch 52, validation: loss=0.1726, simple_loss=0.2914, pruned_loss=0.02685, over 2564189.00 frames. 2026-09-24 04:53:55,114 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 04:54:00,699 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=162893.33333333334, ans=0.125 2026-09-24 04:54:04,738 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=162926.66666666666, ans=0.0 2026-09-24 04:54:10,553 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.35 vs. limit=10.0 2026-09-24 04:54:12,392 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=162960.0, ans=0.09899494936611666 2026-09-24 04:54:14,779 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=162993.33333333334, ans=0.0 2026-09-24 04:54:20,331 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.71 vs. limit=6.0 2026-09-24 04:54:20,528 INFO [train.py:1192] (1/2) Epoch 52, batch 50, loss[loss=0.228, simple_loss=0.3306, pruned_loss=0.06268, over 24256.00 frames. ], tot_loss[loss=0.27, simple_loss=0.3849, pruned_loss=0.07757, over 1082217.32 frames. ], batch size: 125, lr: 4.07e-03, grad_scale: 32.0 2026-09-24 04:54:26,270 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=9.05 vs. limit=10.0 2026-09-24 04:54:38,535 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.799e+02 3.476e+02 3.824e+02 4.146e+02 5.729e+02, threshold=7.649e+02, percent-clipped=0.0 2026-09-24 04:54:45,738 INFO [train.py:1192] (1/2) Epoch 52, batch 100, loss[loss=0.2807, simple_loss=0.3847, pruned_loss=0.08833, over 24583.00 frames. ], tot_loss[loss=0.273, simple_loss=0.3895, pruned_loss=0.07825, over 1915668.74 frames. ], batch size: 154, lr: 4.07e-03, grad_scale: 64.0 2026-09-24 04:54:58,630 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=163260.0, ans=0.125 2026-09-24 04:55:11,079 INFO [train.py:1192] (1/2) Epoch 52, batch 150, loss[loss=0.2175, simple_loss=0.3275, pruned_loss=0.05372, over 24258.00 frames. ], tot_loss[loss=0.2686, simple_loss=0.3847, pruned_loss=0.07626, over 2561391.01 frames. ], batch size: 125, lr: 4.07e-03, grad_scale: 64.0 2026-09-24 04:55:22,373 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.12 vs. limit=15.0 2026-09-24 04:55:22,728 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=163426.66666666666, ans=0.125 2026-09-24 04:55:25,740 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=163460.0, ans=0.0 2026-09-24 04:55:28,705 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.605e+02 3.284e+02 3.723e+02 4.398e+02 6.163e+02, threshold=7.447e+02, percent-clipped=0.0 2026-09-24 04:55:33,854 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=163493.33333333334, ans=0.125 2026-09-24 04:55:36,225 INFO [train.py:1192] (1/2) Epoch 52, batch 200, loss[loss=0.3209, simple_loss=0.4419, pruned_loss=0.09992, over 24208.00 frames. ], tot_loss[loss=0.2663, simple_loss=0.3828, pruned_loss=0.07493, over 3060111.13 frames. ], batch size: 257, lr: 4.07e-03, grad_scale: 64.0 2026-09-24 04:55:36,386 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.53 vs. limit=15.0 2026-09-24 04:55:45,696 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=163560.0, ans=0.125 2026-09-24 04:55:47,851 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=163593.33333333334, ans=0.1 2026-09-24 04:55:54,477 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=163626.66666666666, ans=0.125 2026-09-24 04:55:55,907 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=163626.66666666666, ans=0.0 2026-09-24 04:56:01,902 INFO [train.py:1192] (1/2) Epoch 52, batch 250, loss[loss=0.3077, simple_loss=0.4308, pruned_loss=0.09233, over 24366.00 frames. ], tot_loss[loss=0.267, simple_loss=0.3828, pruned_loss=0.07558, over 3443780.67 frames. ], batch size: 225, lr: 4.07e-03, grad_scale: 64.0 2026-09-24 04:56:05,580 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=163693.33333333334, ans=0.2 2026-09-24 04:56:05,732 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=163693.33333333334, ans=10.0 2026-09-24 04:56:15,568 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=163760.0, ans=0.125 2026-09-24 04:56:20,171 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.910e+02 3.359e+02 3.886e+02 4.376e+02 6.164e+02, threshold=7.772e+02, percent-clipped=0.0 2026-09-24 04:56:23,316 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=163826.66666666666, ans=0.025 2026-09-24 04:56:24,278 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.24 vs. limit=15.0 2026-09-24 04:56:27,292 INFO [train.py:1192] (1/2) Epoch 52, batch 300, loss[loss=0.2856, simple_loss=0.4083, pruned_loss=0.08146, over 24561.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3817, pruned_loss=0.07473, over 3757085.81 frames. ], batch size: 204, lr: 4.06e-03, grad_scale: 64.0 2026-09-24 04:56:32,198 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=163893.33333333334, ans=0.025 2026-09-24 04:56:35,614 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=163893.33333333334, ans=0.125 2026-09-24 04:56:37,304 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=163926.66666666666, ans=0.125 2026-09-24 04:56:39,797 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=163926.66666666666, ans=0.0 2026-09-24 04:56:45,564 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=163960.0, ans=0.04949747468305833 2026-09-24 04:56:47,501 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=163993.33333333334, ans=0.125 2026-09-24 04:56:50,282 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.17 vs. limit=6.0 2026-09-24 04:56:53,604 INFO [train.py:1192] (1/2) Epoch 52, batch 350, loss[loss=0.2073, simple_loss=0.3258, pruned_loss=0.04442, over 24563.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3827, pruned_loss=0.0753, over 3998461.08 frames. ], batch size: 137, lr: 4.06e-03, grad_scale: 64.0 2026-09-24 04:56:55,290 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=164026.66666666666, ans=0.125 2026-09-24 04:56:57,696 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.33 vs. limit=15.0 2026-09-24 04:57:01,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=164060.0, ans=0.125 2026-09-24 04:57:04,632 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=8.48 vs. limit=15.0 2026-09-24 04:57:06,952 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.24 vs. limit=12.0 2026-09-24 04:57:09,948 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=164126.66666666666, ans=0.0 2026-09-24 04:57:11,792 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.781e+02 3.431e+02 3.812e+02 4.631e+02 6.402e+02, threshold=7.623e+02, percent-clipped=0.0 2026-09-24 04:57:14,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=164160.0, ans=0.125 2026-09-24 04:57:19,352 INFO [train.py:1192] (1/2) Epoch 52, batch 400, loss[loss=0.2486, simple_loss=0.3676, pruned_loss=0.06476, over 24564.00 frames. ], tot_loss[loss=0.2663, simple_loss=0.3818, pruned_loss=0.07535, over 4184398.49 frames. ], batch size: 170, lr: 4.06e-03, grad_scale: 64.0 2026-09-24 04:57:37,479 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=8.77 vs. limit=15.0 2026-09-24 04:57:37,511 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=17.98 vs. limit=22.5 2026-09-24 04:57:44,883 INFO [train.py:1192] (1/2) Epoch 52, batch 450, loss[loss=0.283, simple_loss=0.3994, pruned_loss=0.08326, over 24630.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3822, pruned_loss=0.07552, over 4323075.12 frames. ], batch size: 175, lr: 4.06e-03, grad_scale: 64.0 2026-09-24 04:57:59,072 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.whiten.whitening_limit, batch_count=164426.66666666666, ans=15.0 2026-09-24 04:57:59,435 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.83 vs. limit=15.0 2026-09-24 04:58:02,627 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.537e+02 3.288e+02 3.722e+02 4.227e+02 6.045e+02, threshold=7.443e+02, percent-clipped=0.0 2026-09-24 04:58:06,472 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.87 vs. limit=22.5 2026-09-24 04:58:09,871 INFO [train.py:1192] (1/2) Epoch 52, batch 500, loss[loss=0.2892, simple_loss=0.4135, pruned_loss=0.08244, over 24480.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.3806, pruned_loss=0.07489, over 4440135.83 frames. ], batch size: 218, lr: 4.06e-03, grad_scale: 64.0 2026-09-24 04:58:10,420 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=164526.66666666666, ans=0.0 2026-09-24 04:58:17,115 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.85 vs. limit=10.0 2026-09-24 04:58:35,343 INFO [train.py:1192] (1/2) Epoch 52, batch 550, loss[loss=0.2818, simple_loss=0.4069, pruned_loss=0.07833, over 24282.00 frames. ], tot_loss[loss=0.2651, simple_loss=0.3806, pruned_loss=0.07474, over 4526944.87 frames. ], batch size: 257, lr: 4.05e-03, grad_scale: 64.0 2026-09-24 04:58:38,877 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=164693.33333333334, ans=0.125 2026-09-24 04:58:40,172 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=164726.66666666666, ans=0.125 2026-09-24 04:58:41,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=164726.66666666666, ans=0.2 2026-09-24 04:58:42,545 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=164726.66666666666, ans=0.125 2026-09-24 04:58:44,039 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_na.min_abs, batch_count=164726.66666666666, ans=0.02 2026-09-24 04:58:53,757 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.511e+02 3.288e+02 3.632e+02 3.999e+02 5.951e+02, threshold=7.265e+02, percent-clipped=0.0 2026-09-24 04:59:00,947 INFO [train.py:1192] (1/2) Epoch 52, batch 600, loss[loss=0.2842, simple_loss=0.4175, pruned_loss=0.0755, over 24408.00 frames. ], tot_loss[loss=0.2654, simple_loss=0.3812, pruned_loss=0.07481, over 4592219.88 frames. ], batch size: 235, lr: 4.05e-03, grad_scale: 64.0 2026-09-24 04:59:13,780 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=164926.66666666666, ans=0.0 2026-09-24 04:59:17,530 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=164960.0, ans=0.1 2026-09-24 04:59:19,361 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=164960.0, ans=0.5 2026-09-24 04:59:26,565 INFO [train.py:1192] (1/2) Epoch 52, batch 650, loss[loss=0.2485, simple_loss=0.3688, pruned_loss=0.06409, over 24589.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.3802, pruned_loss=0.07437, over 4656176.22 frames. ], batch size: 154, lr: 4.05e-03, grad_scale: 64.0 2026-09-24 04:59:26,652 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=165026.66666666666, ans=0.125 2026-09-24 04:59:34,832 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=165060.0, ans=0.125 2026-09-24 04:59:35,357 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=165060.0, ans=0.0 2026-09-24 04:59:45,092 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.682e+02 3.336e+02 3.710e+02 4.177e+02 6.779e+02, threshold=7.421e+02, percent-clipped=0.0 2026-09-24 04:59:47,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=165160.0, ans=0.125 2026-09-24 04:59:51,005 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=165160.0, ans=0.0 2026-09-24 04:59:51,910 INFO [train.py:1192] (1/2) Epoch 52, batch 700, loss[loss=0.264, simple_loss=0.3768, pruned_loss=0.07556, over 24551.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.3814, pruned_loss=0.07457, over 4690988.99 frames. ], batch size: 158, lr: 4.05e-03, grad_scale: 64.0 2026-09-24 04:59:58,605 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.63 vs. limit=22.5 2026-09-24 05:00:06,257 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=165260.0, ans=0.035 2026-09-24 05:00:06,295 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=165260.0, ans=0.2 2026-09-24 05:00:10,495 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=165293.33333333334, ans=0.0 2026-09-24 05:00:10,946 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=165293.33333333334, ans=0.1 2026-09-24 05:00:17,521 INFO [train.py:1192] (1/2) Epoch 52, batch 750, loss[loss=0.256, simple_loss=0.3789, pruned_loss=0.06657, over 24609.00 frames. ], tot_loss[loss=0.264, simple_loss=0.38, pruned_loss=0.074, over 4727345.82 frames. ], batch size: 175, lr: 4.05e-03, grad_scale: 64.0 2026-09-24 05:00:28,485 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.77 vs. limit=15.0 2026-09-24 05:00:31,195 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=165426.66666666666, ans=0.125 2026-09-24 05:00:34,794 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=165460.0, ans=0.0 2026-09-24 05:00:36,100 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.774e+02 3.253e+02 3.701e+02 4.355e+02 6.161e+02, threshold=7.402e+02, percent-clipped=0.0 2026-09-24 05:00:36,228 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=165460.0, ans=0.125 2026-09-24 05:00:43,550 INFO [train.py:1192] (1/2) Epoch 52, batch 800, loss[loss=0.2262, simple_loss=0.3404, pruned_loss=0.05599, over 24529.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3797, pruned_loss=0.07382, over 4751775.32 frames. ], batch size: 137, lr: 4.04e-03, grad_scale: 64.0 2026-09-24 05:01:00,448 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.34 vs. limit=15.0 2026-09-24 05:01:02,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=165626.66666666666, ans=0.125 2026-09-24 05:01:03,552 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=165660.0, ans=0.125 2026-09-24 05:01:03,576 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=165660.0, ans=0.2 2026-09-24 05:01:08,876 INFO [train.py:1192] (1/2) Epoch 52, batch 850, loss[loss=0.2846, simple_loss=0.4084, pruned_loss=0.08045, over 24552.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.379, pruned_loss=0.07338, over 4770649.55 frames. ], batch size: 204, lr: 4.04e-03, grad_scale: 64.0 2026-09-24 05:01:16,745 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.18 vs. limit=10.0 2026-09-24 05:01:19,882 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:01:21,447 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=165760.0, ans=0.0 2026-09-24 05:01:27,954 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.431e+02 3.277e+02 3.764e+02 4.341e+02 6.225e+02, threshold=7.527e+02, percent-clipped=0.0 2026-09-24 05:01:30,925 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=165826.66666666666, ans=0.0 2026-09-24 05:01:34,859 INFO [train.py:1192] (1/2) Epoch 52, batch 900, loss[loss=0.2296, simple_loss=0.3441, pruned_loss=0.05755, over 24542.00 frames. ], tot_loss[loss=0.2644, simple_loss=0.3802, pruned_loss=0.07426, over 4781686.52 frames. ], batch size: 137, lr: 4.04e-03, grad_scale: 32.0 2026-09-24 05:01:37,346 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=165860.0, ans=0.125 2026-09-24 05:01:44,677 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=165926.66666666666, ans=0.1 2026-09-24 05:01:50,536 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=165960.0, ans=0.0 2026-09-24 05:01:53,815 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.55 vs. limit=15.0 2026-09-24 05:02:00,372 INFO [train.py:1192] (1/2) Epoch 52, batch 950, loss[loss=0.3671, simple_loss=0.4318, pruned_loss=0.1513, over 11869.00 frames. ], tot_loss[loss=0.2656, simple_loss=0.3796, pruned_loss=0.07583, over 4701063.43 frames. ], batch size: 334, lr: 4.04e-03, grad_scale: 16.0 2026-09-24 05:02:10,250 INFO [train.py:1192] (1/2) Epoch 53, batch 0, loss[loss=0.2251, simple_loss=0.3401, pruned_loss=0.05503, over 24565.00 frames. ], tot_loss[loss=0.2251, simple_loss=0.3401, pruned_loss=0.05503, over 24565.00 frames. ], batch size: 137, lr: 4.00e-03, grad_scale: 32.0 2026-09-24 05:02:10,251 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 05:02:15,846 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2948, 2.1435, 1.8019, 2.5194], device='cuda:1') 2026-09-24 05:02:21,902 INFO [train.py:1224] (1/2) Epoch 53, validation: loss=0.171, simple_loss=0.2896, pruned_loss=0.0262, over 2564189.00 frames. 2026-09-24 05:02:21,902 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 05:02:35,313 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=166120.0, ans=0.125 2026-09-24 05:02:37,248 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.655e+02 3.515e+02 4.091e+02 4.690e+02 6.599e+02, threshold=8.183e+02, percent-clipped=0.0 2026-09-24 05:02:37,455 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten.whitening_limit, batch_count=166153.33333333334, ans=22.5 2026-09-24 05:02:37,816 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=166153.33333333334, ans=0.2 2026-09-24 05:02:39,212 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=166153.33333333334, ans=0.125 2026-09-24 05:02:47,175 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=166220.0, ans=0.125 2026-09-24 05:02:47,651 INFO [train.py:1192] (1/2) Epoch 53, batch 50, loss[loss=0.215, simple_loss=0.327, pruned_loss=0.05149, over 24319.00 frames. ], tot_loss[loss=0.2704, simple_loss=0.3849, pruned_loss=0.07789, over 1082612.69 frames. ], batch size: 125, lr: 4.00e-03, grad_scale: 32.0 2026-09-24 05:03:10,281 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=166353.33333333334, ans=0.0 2026-09-24 05:03:13,183 INFO [train.py:1192] (1/2) Epoch 53, batch 100, loss[loss=0.2596, simple_loss=0.3715, pruned_loss=0.07382, over 24592.00 frames. ], tot_loss[loss=0.2737, simple_loss=0.3896, pruned_loss=0.07885, over 1915150.77 frames. ], batch size: 154, lr: 4.00e-03, grad_scale: 32.0 2026-09-24 05:03:20,393 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=166420.0, ans=0.0 2026-09-24 05:03:28,099 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.355e+02 3.302e+02 3.763e+02 4.279e+02 8.783e+02, threshold=7.526e+02, percent-clipped=1.0 2026-09-24 05:03:33,122 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=166520.0, ans=0.2 2026-09-24 05:03:33,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=166520.0, ans=0.95 2026-09-24 05:03:36,997 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=166520.0, ans=0.0 2026-09-24 05:03:37,420 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:03:38,262 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:03:38,532 INFO [train.py:1192] (1/2) Epoch 53, batch 150, loss[loss=0.2154, simple_loss=0.3235, pruned_loss=0.05368, over 24274.00 frames. ], tot_loss[loss=0.2671, simple_loss=0.3834, pruned_loss=0.07542, over 2561544.24 frames. ], batch size: 125, lr: 3.99e-03, grad_scale: 32.0 2026-09-24 05:03:48,526 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=166620.0, ans=0.125 2026-09-24 05:03:49,207 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.68 vs. limit=22.5 2026-09-24 05:03:54,236 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=166653.33333333334, ans=0.1 2026-09-24 05:04:01,605 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=166686.66666666666, ans=0.125 2026-09-24 05:04:02,588 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=166686.66666666666, ans=0.125 2026-09-24 05:04:03,896 INFO [train.py:1192] (1/2) Epoch 53, batch 200, loss[loss=0.3063, simple_loss=0.4333, pruned_loss=0.08968, over 24233.00 frames. ], tot_loss[loss=0.2643, simple_loss=0.3807, pruned_loss=0.07396, over 3061009.37 frames. ], batch size: 257, lr: 3.99e-03, grad_scale: 32.0 2026-09-24 05:04:11,286 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.03 vs. limit=15.0 2026-09-24 05:04:16,066 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=166786.66666666666, ans=0.0 2026-09-24 05:04:18,896 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.437e+02 3.418e+02 3.818e+02 4.297e+02 6.430e+02, threshold=7.637e+02, percent-clipped=0.0 2026-09-24 05:04:20,061 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:04:29,152 INFO [train.py:1192] (1/2) Epoch 53, batch 250, loss[loss=0.2861, simple_loss=0.4086, pruned_loss=0.08186, over 24391.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.3809, pruned_loss=0.07477, over 3446119.42 frames. ], batch size: 225, lr: 3.99e-03, grad_scale: 32.0 2026-09-24 05:04:35,074 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=166920.0, ans=0.125 2026-09-24 05:04:42,435 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=166953.33333333334, ans=0.035 2026-09-24 05:04:43,373 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=166953.33333333334, ans=0.025 2026-09-24 05:04:54,228 INFO [train.py:1192] (1/2) Epoch 53, batch 300, loss[loss=0.2896, simple_loss=0.4048, pruned_loss=0.08717, over 24532.00 frames. ], tot_loss[loss=0.2642, simple_loss=0.38, pruned_loss=0.07421, over 3759662.27 frames. ], batch size: 204, lr: 3.99e-03, grad_scale: 32.0 2026-09-24 05:04:55,272 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=167053.33333333334, ans=0.025 2026-09-24 05:05:02,852 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=167086.66666666666, ans=0.125 2026-09-24 05:05:09,122 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.681e+02 3.368e+02 3.702e+02 4.143e+02 5.571e+02, threshold=7.403e+02, percent-clipped=0.0 2026-09-24 05:05:19,228 INFO [train.py:1192] (1/2) Epoch 53, batch 350, loss[loss=0.2026, simple_loss=0.3248, pruned_loss=0.04022, over 24565.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3808, pruned_loss=0.07457, over 4000140.65 frames. ], batch size: 137, lr: 3.99e-03, grad_scale: 32.0 2026-09-24 05:05:21,388 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=167220.0, ans=0.1 2026-09-24 05:05:37,934 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=167320.0, ans=0.125 2026-09-24 05:05:45,351 INFO [train.py:1192] (1/2) Epoch 53, batch 400, loss[loss=0.2715, simple_loss=0.3914, pruned_loss=0.07581, over 24541.00 frames. ], tot_loss[loss=0.2646, simple_loss=0.3804, pruned_loss=0.07442, over 4183289.66 frames. ], batch size: 170, lr: 3.98e-03, grad_scale: 32.0 2026-09-24 05:05:46,656 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=9.14 vs. limit=15.0 2026-09-24 05:06:00,880 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.584e+02 3.416e+02 3.853e+02 4.638e+02 5.855e+02, threshold=7.707e+02, percent-clipped=0.0 2026-09-24 05:06:11,079 INFO [train.py:1192] (1/2) Epoch 53, batch 450, loss[loss=0.2541, simple_loss=0.3837, pruned_loss=0.06229, over 24601.00 frames. ], tot_loss[loss=0.2645, simple_loss=0.3804, pruned_loss=0.07431, over 4322910.08 frames. ], batch size: 175, lr: 3.98e-03, grad_scale: 32.0 2026-09-24 05:06:15,836 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=167586.66666666666, ans=0.125 2026-09-24 05:06:19,622 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=167586.66666666666, ans=0.0 2026-09-24 05:06:27,572 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.39 vs. limit=22.5 2026-09-24 05:06:31,195 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.86 vs. limit=15.0 2026-09-24 05:06:32,951 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=167686.66666666666, ans=0.125 2026-09-24 05:06:34,474 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=167686.66666666666, ans=0.125 2026-09-24 05:06:36,748 INFO [train.py:1192] (1/2) Epoch 53, batch 500, loss[loss=0.282, simple_loss=0.4002, pruned_loss=0.08186, over 24523.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3788, pruned_loss=0.07374, over 4439585.02 frames. ], batch size: 218, lr: 3.98e-03, grad_scale: 32.0 2026-09-24 05:06:52,084 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.590e+02 3.274e+02 3.651e+02 4.065e+02 5.653e+02, threshold=7.301e+02, percent-clipped=0.0 2026-09-24 05:06:52,738 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=167820.0, ans=0.125 2026-09-24 05:07:00,216 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.03 vs. limit=15.0 2026-09-24 05:07:02,155 INFO [train.py:1192] (1/2) Epoch 53, batch 550, loss[loss=0.2805, simple_loss=0.4044, pruned_loss=0.07827, over 24257.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3796, pruned_loss=0.07401, over 4524131.29 frames. ], batch size: 257, lr: 3.98e-03, grad_scale: 32.0 2026-09-24 05:07:23,733 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=168020.0, ans=0.125 2026-09-24 05:07:24,693 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=168020.0, ans=0.125 2026-09-24 05:07:27,640 INFO [train.py:1192] (1/2) Epoch 53, batch 600, loss[loss=0.3226, simple_loss=0.4373, pruned_loss=0.104, over 24317.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3808, pruned_loss=0.07464, over 4590717.85 frames. ], batch size: 234, lr: 3.98e-03, grad_scale: 32.0 2026-09-24 05:07:28,248 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=168053.33333333334, ans=0.125 2026-09-24 05:07:36,848 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:07:42,731 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.643e+02 3.309e+02 3.591e+02 3.976e+02 6.557e+02, threshold=7.181e+02, percent-clipped=0.0 2026-09-24 05:07:46,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=168153.33333333334, ans=0.1 2026-09-24 05:07:47,384 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=168186.66666666666, ans=0.05 2026-09-24 05:07:48,732 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=168186.66666666666, ans=0.0 2026-09-24 05:07:51,830 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=168186.66666666666, ans=0.1 2026-09-24 05:07:52,652 INFO [train.py:1192] (1/2) Epoch 53, batch 650, loss[loss=0.2615, simple_loss=0.375, pruned_loss=0.07399, over 24576.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3797, pruned_loss=0.07401, over 4655038.13 frames. ], batch size: 154, lr: 3.97e-03, grad_scale: 32.0 2026-09-24 05:07:58,033 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=168253.33333333334, ans=0.2 2026-09-24 05:07:59,334 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.15 vs. limit=15.0 2026-09-24 05:08:08,582 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.37 vs. limit=22.5 2026-09-24 05:08:18,293 INFO [train.py:1192] (1/2) Epoch 53, batch 700, loss[loss=0.2526, simple_loss=0.3679, pruned_loss=0.06859, over 24551.00 frames. ], tot_loss[loss=0.264, simple_loss=0.3804, pruned_loss=0.07383, over 4691340.01 frames. ], batch size: 158, lr: 3.97e-03, grad_scale: 32.0 2026-09-24 05:08:24,424 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=168420.0, ans=0.125 2026-09-24 05:08:27,088 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=168420.0, ans=0.2 2026-09-24 05:08:31,947 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=168453.33333333334, ans=10.0 2026-09-24 05:08:33,536 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.683e+02 3.418e+02 3.806e+02 4.319e+02 6.633e+02, threshold=7.612e+02, percent-clipped=0.0 2026-09-24 05:08:36,004 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:08:43,138 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.69 vs. limit=15.0 2026-09-24 05:08:43,353 INFO [train.py:1192] (1/2) Epoch 53, batch 750, loss[loss=0.2652, simple_loss=0.3848, pruned_loss=0.07281, over 24628.00 frames. ], tot_loss[loss=0.2631, simple_loss=0.3795, pruned_loss=0.07337, over 4727402.01 frames. ], batch size: 175, lr: 3.97e-03, grad_scale: 32.0 2026-09-24 05:08:49,016 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=168586.66666666666, ans=0.2 2026-09-24 05:08:49,027 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=168586.66666666666, ans=0.125 2026-09-24 05:09:08,889 INFO [train.py:1192] (1/2) Epoch 53, batch 800, loss[loss=0.2278, simple_loss=0.3414, pruned_loss=0.0571, over 24515.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.3792, pruned_loss=0.07316, over 4753251.15 frames. ], batch size: 137, lr: 3.97e-03, grad_scale: 32.0 2026-09-24 05:09:24,341 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.559e+02 3.349e+02 3.707e+02 4.167e+02 6.733e+02, threshold=7.413e+02, percent-clipped=0.0 2026-09-24 05:09:34,556 INFO [train.py:1192] (1/2) Epoch 53, batch 850, loss[loss=0.3117, simple_loss=0.4259, pruned_loss=0.09878, over 24578.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3785, pruned_loss=0.07295, over 4772445.74 frames. ], batch size: 204, lr: 3.97e-03, grad_scale: 32.0 2026-09-24 05:10:00,563 INFO [train.py:1192] (1/2) Epoch 53, batch 900, loss[loss=0.2328, simple_loss=0.3475, pruned_loss=0.05906, over 24570.00 frames. ], tot_loss[loss=0.2618, simple_loss=0.3782, pruned_loss=0.07268, over 4783132.08 frames. ], batch size: 137, lr: 3.96e-03, grad_scale: 32.0 2026-09-24 05:10:01,677 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=169053.33333333334, ans=0.0 2026-09-24 05:10:15,702 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.552e+02 3.417e+02 3.958e+02 4.535e+02 7.546e+02, threshold=7.916e+02, percent-clipped=1.0 2026-09-24 05:10:17,945 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=169153.33333333334, ans=0.1 2026-09-24 05:10:25,369 INFO [train.py:1192] (1/2) Epoch 53, batch 950, loss[loss=0.3481, simple_loss=0.4188, pruned_loss=0.1386, over 11383.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3773, pruned_loss=0.07375, over 4712884.74 frames. ], batch size: 334, lr: 3.96e-03, grad_scale: 32.0 2026-09-24 05:10:37,219 INFO [train.py:1192] (1/2) Epoch 54, batch 0, loss[loss=0.2293, simple_loss=0.3431, pruned_loss=0.05773, over 24550.00 frames. ], tot_loss[loss=0.2293, simple_loss=0.3431, pruned_loss=0.05773, over 24550.00 frames. ], batch size: 137, lr: 3.93e-03, grad_scale: 32.0 2026-09-24 05:10:37,220 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 05:10:48,902 INFO [train.py:1224] (1/2) Epoch 54, validation: loss=0.1725, simple_loss=0.2912, pruned_loss=0.02692, over 2564189.00 frames. 2026-09-24 05:10:48,902 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 05:11:08,573 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=169380.0, ans=0.0 2026-09-24 05:11:13,890 INFO [train.py:1192] (1/2) Epoch 54, batch 50, loss[loss=0.2082, simple_loss=0.3207, pruned_loss=0.04788, over 24285.00 frames. ], tot_loss[loss=0.2675, simple_loss=0.383, pruned_loss=0.076, over 1081110.05 frames. ], batch size: 125, lr: 3.92e-03, grad_scale: 32.0 2026-09-24 05:11:15,748 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.94 vs. limit=12.0 2026-09-24 05:11:21,705 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=169446.66666666666, ans=0.125 2026-09-24 05:11:25,120 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.865e+02 3.454e+02 4.001e+02 4.532e+02 7.399e+02, threshold=8.002e+02, percent-clipped=0.0 2026-09-24 05:11:25,752 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=169480.0, ans=0.0 2026-09-24 05:11:36,107 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=169546.66666666666, ans=0.125 2026-09-24 05:11:39,437 INFO [train.py:1192] (1/2) Epoch 54, batch 100, loss[loss=0.2628, simple_loss=0.3732, pruned_loss=0.07619, over 24579.00 frames. ], tot_loss[loss=0.2702, simple_loss=0.3871, pruned_loss=0.0766, over 1915401.44 frames. ], batch size: 154, lr: 3.92e-03, grad_scale: 32.0 2026-09-24 05:11:44,493 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=169613.33333333334, ans=0.0 2026-09-24 05:11:46,077 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.08 vs. limit=15.0 2026-09-24 05:11:46,837 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=169613.33333333334, ans=0.125 2026-09-24 05:12:05,050 INFO [train.py:1192] (1/2) Epoch 54, batch 150, loss[loss=0.1982, simple_loss=0.313, pruned_loss=0.04163, over 24294.00 frames. ], tot_loss[loss=0.2675, simple_loss=0.3836, pruned_loss=0.07568, over 2560944.99 frames. ], batch size: 125, lr: 3.92e-03, grad_scale: 32.0 2026-09-24 05:12:16,182 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.710e+02 3.321e+02 3.761e+02 4.264e+02 6.680e+02, threshold=7.522e+02, percent-clipped=0.0 2026-09-24 05:12:30,902 INFO [train.py:1192] (1/2) Epoch 54, batch 200, loss[loss=0.2865, simple_loss=0.4121, pruned_loss=0.08043, over 24202.00 frames. ], tot_loss[loss=0.2642, simple_loss=0.3807, pruned_loss=0.07386, over 3060192.87 frames. ], batch size: 257, lr: 3.92e-03, grad_scale: 32.0 2026-09-24 05:12:35,975 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=169946.66666666666, ans=0.1 2026-09-24 05:12:38,046 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=169946.66666666666, ans=0.125 2026-09-24 05:12:38,577 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.72 vs. limit=22.5 2026-09-24 05:12:42,961 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=169980.0, ans=0.125 2026-09-24 05:12:50,355 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=170013.33333333334, ans=0.0 2026-09-24 05:12:56,743 INFO [train.py:1192] (1/2) Epoch 54, batch 250, loss[loss=0.2772, simple_loss=0.404, pruned_loss=0.07526, over 24389.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3806, pruned_loss=0.07446, over 3444042.43 frames. ], batch size: 225, lr: 3.92e-03, grad_scale: 32.0 2026-09-24 05:12:57,849 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=170080.0, ans=0.125 2026-09-24 05:13:01,358 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=170113.33333333334, ans=0.125 2026-09-24 05:13:02,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=170113.33333333334, ans=0.1 2026-09-24 05:13:08,134 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.630e+02 3.409e+02 3.936e+02 4.619e+02 6.254e+02, threshold=7.872e+02, percent-clipped=0.0 2026-09-24 05:13:09,441 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.28 vs. limit=12.0 2026-09-24 05:13:09,692 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=170146.66666666666, ans=0.125 2026-09-24 05:13:10,238 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=170146.66666666666, ans=0.125 2026-09-24 05:13:11,177 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=170146.66666666666, ans=0.2 2026-09-24 05:13:11,186 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=170146.66666666666, ans=0.0 2026-09-24 05:13:16,159 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=170180.0, ans=0.1 2026-09-24 05:13:17,565 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.22 vs. limit=15.0 2026-09-24 05:13:22,279 INFO [train.py:1192] (1/2) Epoch 54, batch 300, loss[loss=0.2939, simple_loss=0.4214, pruned_loss=0.08322, over 24557.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3798, pruned_loss=0.07405, over 3757001.93 frames. ], batch size: 204, lr: 3.91e-03, grad_scale: 32.0 2026-09-24 05:13:26,858 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=170280.0, ans=0.125 2026-09-24 05:13:41,715 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=170346.66666666666, ans=0.1 2026-09-24 05:13:41,844 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.81 vs. limit=15.0 2026-09-24 05:13:46,026 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=170380.0, ans=0.1 2026-09-24 05:13:47,882 INFO [train.py:1192] (1/2) Epoch 54, batch 350, loss[loss=0.2155, simple_loss=0.3345, pruned_loss=0.04829, over 24566.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.381, pruned_loss=0.07467, over 3997165.20 frames. ], batch size: 137, lr: 3.91e-03, grad_scale: 32.0 2026-09-24 05:13:59,591 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.512e+02 3.353e+02 3.675e+02 4.247e+02 6.490e+02, threshold=7.351e+02, percent-clipped=0.0 2026-09-24 05:14:09,839 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=170546.66666666666, ans=0.0 2026-09-24 05:14:13,364 INFO [train.py:1192] (1/2) Epoch 54, batch 400, loss[loss=0.2788, simple_loss=0.3935, pruned_loss=0.08209, over 24565.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3804, pruned_loss=0.07449, over 4182070.55 frames. ], batch size: 170, lr: 3.91e-03, grad_scale: 32.0 2026-09-24 05:14:13,894 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=170580.0, ans=0.125 2026-09-24 05:14:33,810 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=12.31 vs. limit=22.5 2026-09-24 05:14:36,491 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.72 vs. limit=6.0 2026-09-24 05:14:37,298 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=170713.33333333334, ans=0.125 2026-09-24 05:14:37,691 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=170713.33333333334, ans=0.0 2026-09-24 05:14:38,545 INFO [train.py:1192] (1/2) Epoch 54, batch 450, loss[loss=0.2773, simple_loss=0.3925, pruned_loss=0.08108, over 24622.00 frames. ], tot_loss[loss=0.2646, simple_loss=0.3805, pruned_loss=0.07435, over 4317753.67 frames. ], batch size: 175, lr: 3.91e-03, grad_scale: 32.0 2026-09-24 05:14:50,272 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.675e+02 3.328e+02 3.837e+02 4.286e+02 5.852e+02, threshold=7.675e+02, percent-clipped=0.0 2026-09-24 05:14:52,301 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=170813.33333333334, ans=0.125 2026-09-24 05:14:53,711 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=170846.66666666666, ans=0.125 2026-09-24 05:15:00,868 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=170880.0, ans=0.125 2026-09-24 05:15:04,410 INFO [train.py:1192] (1/2) Epoch 54, batch 500, loss[loss=0.2736, simple_loss=0.3986, pruned_loss=0.07433, over 24519.00 frames. ], tot_loss[loss=0.264, simple_loss=0.3797, pruned_loss=0.07416, over 4436372.58 frames. ], batch size: 218, lr: 3.91e-03, grad_scale: 32.0 2026-09-24 05:15:08,297 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=170913.33333333334, ans=0.04949747468305833 2026-09-24 05:15:15,092 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=170980.0, ans=0.025 2026-09-24 05:15:22,513 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=171013.33333333334, ans=0.0 2026-09-24 05:15:23,474 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=171013.33333333334, ans=0.04949747468305833 2026-09-24 05:15:30,148 INFO [train.py:1192] (1/2) Epoch 54, batch 550, loss[loss=0.2946, simple_loss=0.4173, pruned_loss=0.08593, over 24288.00 frames. ], tot_loss[loss=0.2641, simple_loss=0.3798, pruned_loss=0.07422, over 4522002.73 frames. ], batch size: 257, lr: 3.90e-03, grad_scale: 32.0 2026-09-24 05:15:30,272 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=171080.0, ans=0.04949747468305833 2026-09-24 05:15:33,827 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=171080.0, ans=0.125 2026-09-24 05:15:41,308 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.598e+02 3.316e+02 3.672e+02 4.016e+02 8.596e+02, threshold=7.345e+02, percent-clipped=1.0 2026-09-24 05:15:46,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=171180.0, ans=0.125 2026-09-24 05:15:56,053 INFO [train.py:1192] (1/2) Epoch 54, batch 600, loss[loss=0.2871, simple_loss=0.4104, pruned_loss=0.0819, over 24329.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3805, pruned_loss=0.07445, over 4589976.53 frames. ], batch size: 234, lr: 3.90e-03, grad_scale: 32.0 2026-09-24 05:15:59,925 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=171246.66666666666, ans=0.0 2026-09-24 05:16:21,027 INFO [train.py:1192] (1/2) Epoch 54, batch 650, loss[loss=0.247, simple_loss=0.3567, pruned_loss=0.06861, over 24596.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3793, pruned_loss=0.07358, over 4654728.43 frames. ], batch size: 154, lr: 3.90e-03, grad_scale: 32.0 2026-09-24 05:16:25,497 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=171413.33333333334, ans=0.125 2026-09-24 05:16:31,275 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=171480.0, ans=0.125 2026-09-24 05:16:32,625 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.684e+02 3.310e+02 3.698e+02 4.173e+02 6.242e+02, threshold=7.396e+02, percent-clipped=0.0 2026-09-24 05:16:39,145 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=171513.33333333334, ans=0.125 2026-09-24 05:16:42,056 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=7.42 vs. limit=15.0 2026-09-24 05:16:47,141 INFO [train.py:1192] (1/2) Epoch 54, batch 700, loss[loss=0.2563, simple_loss=0.3709, pruned_loss=0.07081, over 24552.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3803, pruned_loss=0.07375, over 4689610.49 frames. ], batch size: 158, lr: 3.90e-03, grad_scale: 32.0 2026-09-24 05:16:51,748 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=171613.33333333334, ans=0.1 2026-09-24 05:16:52,590 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=171613.33333333334, ans=0.125 2026-09-24 05:16:54,941 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=9.20 vs. limit=12.0 2026-09-24 05:16:56,534 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=171613.33333333334, ans=0.125 2026-09-24 05:16:58,175 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.66 vs. limit=15.0 2026-09-24 05:16:58,617 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.31 vs. limit=15.0 2026-09-24 05:17:02,710 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=171680.0, ans=0.2 2026-09-24 05:17:09,967 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.78 vs. limit=15.0 2026-09-24 05:17:11,805 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=171713.33333333334, ans=0.0 2026-09-24 05:17:12,647 INFO [train.py:1192] (1/2) Epoch 54, batch 750, loss[loss=0.2529, simple_loss=0.3762, pruned_loss=0.06478, over 24642.00 frames. ], tot_loss[loss=0.2635, simple_loss=0.3795, pruned_loss=0.07373, over 4726003.73 frames. ], batch size: 175, lr: 3.90e-03, grad_scale: 32.0 2026-09-24 05:17:16,603 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=171746.66666666666, ans=0.0 2026-09-24 05:17:23,171 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=5.82 vs. limit=15.0 2026-09-24 05:17:23,790 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.549e+02 3.380e+02 3.817e+02 4.305e+02 6.996e+02, threshold=7.634e+02, percent-clipped=0.0 2026-09-24 05:17:24,985 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=171813.33333333334, ans=0.1 2026-09-24 05:17:27,468 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=171846.66666666666, ans=0.0 2026-09-24 05:17:38,400 INFO [train.py:1192] (1/2) Epoch 54, batch 800, loss[loss=0.2179, simple_loss=0.3316, pruned_loss=0.05214, over 24549.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3793, pruned_loss=0.07361, over 4752046.48 frames. ], batch size: 137, lr: 3.90e-03, grad_scale: 32.0 2026-09-24 05:18:04,224 INFO [train.py:1192] (1/2) Epoch 54, batch 850, loss[loss=0.283, simple_loss=0.4017, pruned_loss=0.08221, over 24557.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3785, pruned_loss=0.07322, over 4769864.24 frames. ], batch size: 204, lr: 3.89e-03, grad_scale: 32.0 2026-09-24 05:18:04,318 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=172080.0, ans=0.0 2026-09-24 05:18:07,587 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=172080.0, ans=0.125 2026-09-24 05:18:15,652 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.670e+02 3.373e+02 3.656e+02 4.269e+02 5.742e+02, threshold=7.313e+02, percent-clipped=0.0 2026-09-24 05:18:25,379 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=172213.33333333334, ans=0.0 2026-09-24 05:18:29,109 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=172213.33333333334, ans=0.0 2026-09-24 05:18:30,019 INFO [train.py:1192] (1/2) Epoch 54, batch 900, loss[loss=0.223, simple_loss=0.3369, pruned_loss=0.05451, over 24565.00 frames. ], tot_loss[loss=0.263, simple_loss=0.3789, pruned_loss=0.07357, over 4781110.95 frames. ], batch size: 137, lr: 3.89e-03, grad_scale: 32.0 2026-09-24 05:18:33,282 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=172246.66666666666, ans=0.2 2026-09-24 05:18:45,395 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=172346.66666666666, ans=0.04949747468305833 2026-09-24 05:18:55,144 INFO [train.py:1192] (1/2) Epoch 54, batch 950, loss[loss=0.3813, simple_loss=0.4397, pruned_loss=0.1615, over 11123.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3775, pruned_loss=0.07415, over 4716243.21 frames. ], batch size: 333, lr: 3.89e-03, grad_scale: 32.0 2026-09-24 05:19:05,387 INFO [train.py:1192] (1/2) Epoch 55, batch 0, loss[loss=0.2388, simple_loss=0.3554, pruned_loss=0.06107, over 24590.00 frames. ], tot_loss[loss=0.2388, simple_loss=0.3554, pruned_loss=0.06107, over 24590.00 frames. ], batch size: 137, lr: 3.85e-03, grad_scale: 32.0 2026-09-24 05:19:05,387 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 05:19:12,077 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.7013, 3.2823, 3.6465, 3.2799], device='cuda:1') 2026-09-24 05:19:17,034 INFO [train.py:1224] (1/2) Epoch 55, validation: loss=0.1719, simple_loss=0.2905, pruned_loss=0.02667, over 2564189.00 frames. 2026-09-24 05:19:17,034 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13020MB 2026-09-24 05:19:24,274 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 3.542e+02 4.212e+02 4.755e+02 6.619e+02, threshold=8.424e+02, percent-clipped=0.0 2026-09-24 05:19:28,387 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=172506.66666666666, ans=0.04949747468305833 2026-09-24 05:19:32,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=172540.0, ans=0.0 2026-09-24 05:19:33,709 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten.whitening_limit, batch_count=172540.0, ans=15.0 2026-09-24 05:19:39,104 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.22 vs. limit=6.0 2026-09-24 05:19:42,850 INFO [train.py:1192] (1/2) Epoch 55, batch 50, loss[loss=0.2239, simple_loss=0.3343, pruned_loss=0.05674, over 24274.00 frames. ], tot_loss[loss=0.2694, simple_loss=0.3849, pruned_loss=0.07695, over 1081562.03 frames. ], batch size: 125, lr: 3.85e-03, grad_scale: 32.0 2026-09-24 05:19:50,292 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:19:59,717 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=172706.66666666666, ans=0.125 2026-09-24 05:20:08,618 INFO [train.py:1192] (1/2) Epoch 55, batch 100, loss[loss=0.2578, simple_loss=0.3703, pruned_loss=0.07271, over 24621.00 frames. ], tot_loss[loss=0.2711, simple_loss=0.3883, pruned_loss=0.07693, over 1915222.06 frames. ], batch size: 154, lr: 3.85e-03, grad_scale: 64.0 2026-09-24 05:20:13,088 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=172806.66666666666, ans=0.125 2026-09-24 05:20:15,586 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=172806.66666666666, ans=0.2 2026-09-24 05:20:15,920 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.943e+02 3.406e+02 3.885e+02 4.242e+02 5.650e+02, threshold=7.770e+02, percent-clipped=0.0 2026-09-24 05:20:20,296 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=172840.0, ans=0.0 2026-09-24 05:20:25,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=172873.33333333334, ans=0.125 2026-09-24 05:20:28,831 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.00 vs. limit=22.5 2026-09-24 05:20:32,815 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=172906.66666666666, ans=0.0 2026-09-24 05:20:34,109 INFO [train.py:1192] (1/2) Epoch 55, batch 150, loss[loss=0.2276, simple_loss=0.3359, pruned_loss=0.05963, over 24283.00 frames. ], tot_loss[loss=0.2682, simple_loss=0.3844, pruned_loss=0.07603, over 2560326.58 frames. ], batch size: 125, lr: 3.85e-03, grad_scale: 64.0 2026-09-24 05:20:37,037 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=172940.0, ans=0.09899494936611666 2026-09-24 05:20:49,382 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.56 vs. limit=15.0 2026-09-24 05:20:52,576 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.59 vs. limit=12.0 2026-09-24 05:20:54,599 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=173073.33333333334, ans=0.025 2026-09-24 05:20:59,261 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=173106.66666666666, ans=0.125 2026-09-24 05:20:59,612 INFO [train.py:1192] (1/2) Epoch 55, batch 200, loss[loss=0.2859, simple_loss=0.411, pruned_loss=0.08043, over 24233.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3813, pruned_loss=0.07411, over 3058665.59 frames. ], batch size: 257, lr: 3.85e-03, grad_scale: 64.0 2026-09-24 05:21:03,426 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=173106.66666666666, ans=0.1 2026-09-24 05:21:07,127 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.318e+02 3.421e+02 3.801e+02 4.436e+02 7.383e+02, threshold=7.602e+02, percent-clipped=0.0 2026-09-24 05:21:19,044 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff3_skip_rate, batch_count=173206.66666666666, ans=0.0 2026-09-24 05:21:24,965 INFO [train.py:1192] (1/2) Epoch 55, batch 250, loss[loss=0.307, simple_loss=0.4257, pruned_loss=0.09415, over 24383.00 frames. ], tot_loss[loss=0.2647, simple_loss=0.3808, pruned_loss=0.07431, over 3441795.56 frames. ], batch size: 225, lr: 3.84e-03, grad_scale: 64.0 2026-09-24 05:21:26,815 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=173273.33333333334, ans=0.2 2026-09-24 05:21:28,217 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.09 vs. limit=15.0 2026-09-24 05:21:34,845 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=173306.66666666666, ans=0.125 2026-09-24 05:21:36,325 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=173340.0, ans=0.125 2026-09-24 05:21:49,640 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=173406.66666666666, ans=0.0 2026-09-24 05:21:51,004 INFO [train.py:1192] (1/2) Epoch 55, batch 300, loss[loss=0.2923, simple_loss=0.4124, pruned_loss=0.08608, over 24555.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3809, pruned_loss=0.07457, over 3756488.86 frames. ], batch size: 204, lr: 3.84e-03, grad_scale: 64.0 2026-09-24 05:21:54,160 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=173440.0, ans=0.0 2026-09-24 05:21:55,034 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=173440.0, ans=0.125 2026-09-24 05:21:58,132 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.734e+02 3.486e+02 3.863e+02 4.417e+02 5.821e+02, threshold=7.726e+02, percent-clipped=0.0 2026-09-24 05:22:01,122 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=173506.66666666666, ans=0.125 2026-09-24 05:22:09,360 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=173540.0, ans=0.0 2026-09-24 05:22:16,130 INFO [train.py:1192] (1/2) Epoch 55, batch 350, loss[loss=0.2122, simple_loss=0.3288, pruned_loss=0.04773, over 24607.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3812, pruned_loss=0.07439, over 3997159.29 frames. ], batch size: 137, lr: 3.84e-03, grad_scale: 64.0 2026-09-24 05:22:17,244 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=173606.66666666666, ans=0.2 2026-09-24 05:22:18,685 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.45 vs. limit=15.0 2026-09-24 05:22:23,495 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=173640.0, ans=0.125 2026-09-24 05:22:25,988 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.96 vs. limit=15.0 2026-09-24 05:22:30,726 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=173673.33333333334, ans=0.1 2026-09-24 05:22:31,577 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=173706.66666666666, ans=0.125 2026-09-24 05:22:34,983 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=173706.66666666666, ans=0.125 2026-09-24 05:22:35,425 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=173706.66666666666, ans=0.125 2026-09-24 05:22:41,390 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=173773.33333333334, ans=0.125 2026-09-24 05:22:41,735 INFO [train.py:1192] (1/2) Epoch 55, batch 400, loss[loss=0.2469, simple_loss=0.3743, pruned_loss=0.05976, over 24567.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.3799, pruned_loss=0.07342, over 4181506.65 frames. ], batch size: 170, lr: 3.84e-03, grad_scale: 64.0 2026-09-24 05:22:43,277 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=173773.33333333334, ans=0.0 2026-09-24 05:22:43,783 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=173773.33333333334, ans=0.125 2026-09-24 05:22:44,850 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=173773.33333333334, ans=0.0 2026-09-24 05:22:48,943 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.557e+02 3.486e+02 3.848e+02 4.662e+02 6.190e+02, threshold=7.697e+02, percent-clipped=0.0 2026-09-24 05:22:50,395 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=173806.66666666666, ans=0.2 2026-09-24 05:22:54,060 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=173840.0, ans=0.2 2026-09-24 05:22:57,101 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=173873.33333333334, ans=0.2 2026-09-24 05:23:07,716 INFO [train.py:1192] (1/2) Epoch 55, batch 450, loss[loss=0.2747, simple_loss=0.3963, pruned_loss=0.07658, over 24634.00 frames. ], tot_loss[loss=0.2645, simple_loss=0.3806, pruned_loss=0.07414, over 4320793.72 frames. ], batch size: 175, lr: 3.84e-03, grad_scale: 64.0 2026-09-24 05:23:16,916 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=173973.33333333334, ans=0.025 2026-09-24 05:23:18,195 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=174006.66666666666, ans=0.125 2026-09-24 05:23:32,077 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=174106.66666666666, ans=0.125 2026-09-24 05:23:32,423 INFO [train.py:1192] (1/2) Epoch 55, batch 500, loss[loss=0.2838, simple_loss=0.4094, pruned_loss=0.07904, over 24504.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.3793, pruned_loss=0.0737, over 4439183.74 frames. ], batch size: 218, lr: 3.84e-03, grad_scale: 64.0 2026-09-24 05:23:33,524 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=174106.66666666666, ans=0.125 2026-09-24 05:23:36,593 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=174106.66666666666, ans=0.0 2026-09-24 05:23:39,755 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.453e+02 3.364e+02 3.704e+02 4.452e+02 5.986e+02, threshold=7.409e+02, percent-clipped=0.0 2026-09-24 05:23:41,723 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.10 vs. limit=15.0 2026-09-24 05:23:51,056 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=174206.66666666666, ans=0.0 2026-09-24 05:23:53,455 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=174240.0, ans=0.09899494936611666 2026-09-24 05:23:56,221 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=174240.0, ans=0.125 2026-09-24 05:23:56,616 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=174240.0, ans=0.125 2026-09-24 05:23:58,230 INFO [train.py:1192] (1/2) Epoch 55, batch 550, loss[loss=0.2943, simple_loss=0.4174, pruned_loss=0.08561, over 24283.00 frames. ], tot_loss[loss=0.265, simple_loss=0.3805, pruned_loss=0.07476, over 4524526.82 frames. ], batch size: 257, lr: 3.83e-03, grad_scale: 64.0 2026-09-24 05:24:05,681 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=174306.66666666666, ans=0.0 2026-09-24 05:24:24,184 INFO [train.py:1192] (1/2) Epoch 55, batch 600, loss[loss=0.3035, simple_loss=0.4308, pruned_loss=0.08805, over 24318.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3811, pruned_loss=0.07497, over 4589832.01 frames. ], batch size: 234, lr: 3.83e-03, grad_scale: 32.0 2026-09-24 05:24:25,560 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=14.19 vs. limit=15.0 2026-09-24 05:24:27,027 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=174440.0, ans=0.0 2026-09-24 05:24:28,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=174440.0, ans=0.0 2026-09-24 05:24:28,399 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=174440.0, ans=0.0 2026-09-24 05:24:31,742 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.760e+02 3.340e+02 3.684e+02 4.234e+02 5.839e+02, threshold=7.369e+02, percent-clipped=0.0 2026-09-24 05:24:35,315 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.09 vs. limit=10.0 2026-09-24 05:24:39,050 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=174506.66666666666, ans=0.1 2026-09-24 05:24:40,994 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=174540.0, ans=0.125 2026-09-24 05:24:44,872 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=174573.33333333334, ans=0.0 2026-09-24 05:24:50,236 INFO [train.py:1192] (1/2) Epoch 55, batch 650, loss[loss=0.2499, simple_loss=0.3647, pruned_loss=0.06756, over 24606.00 frames. ], tot_loss[loss=0.2637, simple_loss=0.3795, pruned_loss=0.07396, over 4654676.26 frames. ], batch size: 154, lr: 3.83e-03, grad_scale: 32.0 2026-09-24 05:24:58,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=174640.0, ans=0.125 2026-09-24 05:25:10,534 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=174740.0, ans=0.0 2026-09-24 05:25:11,511 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=174740.0, ans=0.2 2026-09-24 05:25:14,879 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=174773.33333333334, ans=0.125 2026-09-24 05:25:15,225 INFO [train.py:1192] (1/2) Epoch 55, batch 700, loss[loss=0.2572, simple_loss=0.3719, pruned_loss=0.07122, over 24551.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3798, pruned_loss=0.07393, over 4690124.03 frames. ], batch size: 158, lr: 3.83e-03, grad_scale: 32.0 2026-09-24 05:25:15,632 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.35 vs. limit=6.0 2026-09-24 05:25:22,403 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.835e+02 3.386e+02 3.666e+02 3.991e+02 5.815e+02, threshold=7.332e+02, percent-clipped=0.0 2026-09-24 05:25:26,440 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=174840.0, ans=0.125 2026-09-24 05:25:29,816 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=174873.33333333334, ans=0.2 2026-09-24 05:25:40,795 INFO [train.py:1192] (1/2) Epoch 55, batch 750, loss[loss=0.268, simple_loss=0.3929, pruned_loss=0.07154, over 24620.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.379, pruned_loss=0.07376, over 4726358.53 frames. ], batch size: 175, lr: 3.83e-03, grad_scale: 32.0 2026-09-24 05:25:43,840 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=174940.0, ans=0.1 2026-09-24 05:26:05,631 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=175073.33333333334, ans=0.125 2026-09-24 05:26:06,048 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=175106.66666666666, ans=0.125 2026-09-24 05:26:06,493 INFO [train.py:1192] (1/2) Epoch 55, batch 800, loss[loss=0.2061, simple_loss=0.3241, pruned_loss=0.04405, over 24546.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3789, pruned_loss=0.0737, over 4751945.65 frames. ], batch size: 137, lr: 3.82e-03, grad_scale: 32.0 2026-09-24 05:26:11,968 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=175140.0, ans=0.125 2026-09-24 05:26:14,557 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.433e+02 3.429e+02 3.889e+02 4.225e+02 6.137e+02, threshold=7.777e+02, percent-clipped=0.0 2026-09-24 05:26:17,071 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=175173.33333333334, ans=0.0 2026-09-24 05:26:19,789 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=175173.33333333334, ans=0.025 2026-09-24 05:26:32,233 INFO [train.py:1192] (1/2) Epoch 55, batch 850, loss[loss=0.2733, simple_loss=0.398, pruned_loss=0.07428, over 24552.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3779, pruned_loss=0.07281, over 4770450.67 frames. ], batch size: 204, lr: 3.82e-03, grad_scale: 32.0 2026-09-24 05:26:40,700 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=175306.66666666666, ans=0.0 2026-09-24 05:26:41,212 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=175306.66666666666, ans=0.125 2026-09-24 05:26:45,961 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=3.32 vs. limit=12.0 2026-09-24 05:26:54,337 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=175406.66666666666, ans=0.125 2026-09-24 05:26:57,632 INFO [train.py:1192] (1/2) Epoch 55, batch 900, loss[loss=0.2168, simple_loss=0.3316, pruned_loss=0.05098, over 24574.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3783, pruned_loss=0.07305, over 4780972.99 frames. ], batch size: 137, lr: 3.82e-03, grad_scale: 32.0 2026-09-24 05:27:03,712 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=175473.33333333334, ans=0.125 2026-09-24 05:27:04,083 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=175473.33333333334, ans=0.0 2026-09-24 05:27:05,096 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=175473.33333333334, ans=0.125 2026-09-24 05:27:05,927 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.592e+02 3.331e+02 3.802e+02 4.333e+02 6.243e+02, threshold=7.604e+02, percent-clipped=0.0 2026-09-24 05:27:08,925 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=175506.66666666666, ans=0.2 2026-09-24 05:27:20,255 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=175573.33333333334, ans=0.025 2026-09-24 05:27:23,441 INFO [train.py:1192] (1/2) Epoch 55, batch 950, loss[loss=0.3244, simple_loss=0.4023, pruned_loss=0.1232, over 11619.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3771, pruned_loss=0.07388, over 4714968.00 frames. ], batch size: 334, lr: 3.82e-03, grad_scale: 32.0 2026-09-24 05:27:25,595 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=175606.66666666666, ans=0.1 2026-09-24 05:27:33,773 INFO [train.py:1192] (1/2) Epoch 56, batch 0, loss[loss=0.2421, simple_loss=0.3622, pruned_loss=0.06098, over 24602.00 frames. ], tot_loss[loss=0.2421, simple_loss=0.3622, pruned_loss=0.06098, over 24602.00 frames. ], batch size: 137, lr: 3.78e-03, grad_scale: 32.0 2026-09-24 05:27:33,773 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 05:27:45,129 INFO [train.py:1224] (1/2) Epoch 56, validation: loss=0.1714, simple_loss=0.2901, pruned_loss=0.02629, over 2564189.00 frames. 2026-09-24 05:27:45,129 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 05:27:59,606 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.44 vs. limit=15.0 2026-09-24 05:28:07,627 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.max_positive, batch_count=175766.66666666666, ans=0.95 2026-09-24 05:28:10,396 INFO [train.py:1192] (1/2) Epoch 56, batch 50, loss[loss=0.2174, simple_loss=0.3289, pruned_loss=0.05295, over 24283.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3836, pruned_loss=0.0754, over 1082538.06 frames. ], batch size: 125, lr: 3.78e-03, grad_scale: 32.0 2026-09-24 05:28:13,967 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.642e+02 3.572e+02 4.045e+02 4.731e+02 9.270e+02, threshold=8.089e+02, percent-clipped=2.0 2026-09-24 05:28:16,358 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=175833.33333333334, ans=0.09899494936611666 2026-09-24 05:28:23,315 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=175866.66666666666, ans=0.125 2026-09-24 05:28:27,344 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=175900.0, ans=0.125 2026-09-24 05:28:36,933 INFO [train.py:1192] (1/2) Epoch 56, batch 100, loss[loss=0.2644, simple_loss=0.3706, pruned_loss=0.07907, over 24612.00 frames. ], tot_loss[loss=0.2721, simple_loss=0.3893, pruned_loss=0.07747, over 1915131.43 frames. ], batch size: 154, lr: 3.78e-03, grad_scale: 32.0 2026-09-24 05:28:37,045 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=175966.66666666666, ans=0.0 2026-09-24 05:28:51,814 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=176066.66666666666, ans=0.1 2026-09-24 05:28:55,226 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=176066.66666666666, ans=0.025 2026-09-24 05:29:02,513 INFO [train.py:1192] (1/2) Epoch 56, batch 150, loss[loss=0.2081, simple_loss=0.3199, pruned_loss=0.04812, over 24268.00 frames. ], tot_loss[loss=0.2659, simple_loss=0.3828, pruned_loss=0.07452, over 2560036.95 frames. ], batch size: 125, lr: 3.78e-03, grad_scale: 32.0 2026-09-24 05:29:05,893 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.706e+02 3.375e+02 3.744e+02 4.234e+02 6.625e+02, threshold=7.487e+02, percent-clipped=0.0 2026-09-24 05:29:12,011 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=176200.0, ans=0.125 2026-09-24 05:29:23,900 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=176266.66666666666, ans=0.025 2026-09-24 05:29:25,896 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=176266.66666666666, ans=0.125 2026-09-24 05:29:27,742 INFO [train.py:1192] (1/2) Epoch 56, batch 200, loss[loss=0.3013, simple_loss=0.4219, pruned_loss=0.09032, over 24245.00 frames. ], tot_loss[loss=0.2625, simple_loss=0.3795, pruned_loss=0.07272, over 3059116.64 frames. ], batch size: 257, lr: 3.78e-03, grad_scale: 32.0 2026-09-24 05:29:30,773 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=176300.0, ans=0.125 2026-09-24 05:29:36,463 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:29:43,070 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=3.88 vs. limit=12.0 2026-09-24 05:29:47,090 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=176400.0, ans=0.0 2026-09-24 05:29:53,237 INFO [train.py:1192] (1/2) Epoch 56, batch 250, loss[loss=0.2752, simple_loss=0.4032, pruned_loss=0.07359, over 24358.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3792, pruned_loss=0.07269, over 3442341.54 frames. ], batch size: 225, lr: 3.78e-03, grad_scale: 32.0 2026-09-24 05:29:57,044 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.699e+02 3.326e+02 3.793e+02 4.270e+02 6.257e+02, threshold=7.587e+02, percent-clipped=0.0 2026-09-24 05:29:58,481 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=176500.0, ans=0.05 2026-09-24 05:30:13,797 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.54 vs. limit=22.5 2026-09-24 05:30:18,516 INFO [train.py:1192] (1/2) Epoch 56, batch 300, loss[loss=0.2948, simple_loss=0.4139, pruned_loss=0.08785, over 24547.00 frames. ], tot_loss[loss=0.2616, simple_loss=0.3783, pruned_loss=0.07243, over 3756098.65 frames. ], batch size: 204, lr: 3.77e-03, grad_scale: 32.0 2026-09-24 05:30:25,828 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=176666.66666666666, ans=0.025 2026-09-24 05:30:26,955 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=176666.66666666666, ans=0.0 2026-09-24 05:30:28,488 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=176700.0, ans=0.0 2026-09-24 05:30:37,334 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.99 vs. limit=15.0 2026-09-24 05:30:38,621 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=176766.66666666666, ans=0.125 2026-09-24 05:30:40,202 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.22 vs. limit=22.5 2026-09-24 05:30:41,086 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.28 vs. limit=15.0 2026-09-24 05:30:43,697 INFO [train.py:1192] (1/2) Epoch 56, batch 350, loss[loss=0.2176, simple_loss=0.3347, pruned_loss=0.05028, over 24581.00 frames. ], tot_loss[loss=0.2616, simple_loss=0.3788, pruned_loss=0.07224, over 3996905.36 frames. ], batch size: 137, lr: 3.77e-03, grad_scale: 32.0 2026-09-24 05:30:44,262 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=176800.0, ans=0.125 2026-09-24 05:30:47,106 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.508e+02 3.265e+02 3.602e+02 4.252e+02 5.165e+02, threshold=7.204e+02, percent-clipped=0.0 2026-09-24 05:31:09,370 INFO [train.py:1192] (1/2) Epoch 56, batch 400, loss[loss=0.2533, simple_loss=0.3735, pruned_loss=0.06655, over 24570.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3786, pruned_loss=0.07222, over 4181945.64 frames. ], batch size: 170, lr: 3.77e-03, grad_scale: 32.0 2026-09-24 05:31:10,970 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=176966.66666666666, ans=0.125 2026-09-24 05:31:16,152 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=177000.0, ans=0.1 2026-09-24 05:31:22,405 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=10.20 vs. limit=15.0 2026-09-24 05:31:27,202 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.35 vs. limit=15.0 2026-09-24 05:31:31,086 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.36 vs. limit=15.0 2026-09-24 05:31:32,798 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=177100.0, ans=0.025 2026-09-24 05:31:34,646 INFO [train.py:1192] (1/2) Epoch 56, batch 450, loss[loss=0.2765, simple_loss=0.395, pruned_loss=0.07896, over 24620.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3787, pruned_loss=0.0726, over 4322062.36 frames. ], batch size: 175, lr: 3.77e-03, grad_scale: 32.0 2026-09-24 05:31:34,760 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=177133.33333333334, ans=0.2 2026-09-24 05:31:37,866 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.807e+02 3.424e+02 3.853e+02 4.456e+02 6.389e+02, threshold=7.705e+02, percent-clipped=0.0 2026-09-24 05:31:38,420 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=177133.33333333334, ans=0.125 2026-09-24 05:31:45,295 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=177200.0, ans=0.1 2026-09-24 05:31:58,946 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=177300.0, ans=0.0 2026-09-24 05:31:59,294 INFO [train.py:1192] (1/2) Epoch 56, batch 500, loss[loss=0.2759, simple_loss=0.4016, pruned_loss=0.07512, over 24525.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3775, pruned_loss=0.07202, over 4439341.30 frames. ], batch size: 218, lr: 3.77e-03, grad_scale: 32.0 2026-09-24 05:32:02,489 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=177300.0, ans=0.125 2026-09-24 05:32:03,575 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=177300.0, ans=0.0 2026-09-24 05:32:09,340 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=177366.66666666666, ans=0.125 2026-09-24 05:32:16,493 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=177400.0, ans=0.125 2026-09-24 05:32:19,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=177433.33333333334, ans=0.0 2026-09-24 05:32:22,015 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=177433.33333333334, ans=0.1 2026-09-24 05:32:24,903 INFO [train.py:1192] (1/2) Epoch 56, batch 550, loss[loss=0.3, simple_loss=0.4178, pruned_loss=0.09112, over 24261.00 frames. ], tot_loss[loss=0.2614, simple_loss=0.378, pruned_loss=0.07239, over 4523566.18 frames. ], batch size: 257, lr: 3.76e-03, grad_scale: 32.0 2026-09-24 05:32:28,151 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.644e+02 3.338e+02 3.718e+02 4.019e+02 5.234e+02, threshold=7.435e+02, percent-clipped=0.0 2026-09-24 05:32:29,533 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=177500.0, ans=0.2 2026-09-24 05:32:35,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=177533.33333333334, ans=0.0 2026-09-24 05:32:40,612 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=177566.66666666666, ans=0.0 2026-09-24 05:32:41,614 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=177566.66666666666, ans=0.125 2026-09-24 05:32:45,015 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=177600.0, ans=0.1 2026-09-24 05:32:50,685 INFO [train.py:1192] (1/2) Epoch 56, batch 600, loss[loss=0.2562, simple_loss=0.3867, pruned_loss=0.06289, over 24406.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3787, pruned_loss=0.07273, over 4592337.69 frames. ], batch size: 235, lr: 3.76e-03, grad_scale: 32.0 2026-09-24 05:32:59,494 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=177666.66666666666, ans=0.025 2026-09-24 05:33:08,889 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=177733.33333333334, ans=0.0 2026-09-24 05:33:09,974 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.90 vs. limit=22.5 2026-09-24 05:33:16,211 INFO [train.py:1192] (1/2) Epoch 56, batch 650, loss[loss=0.2554, simple_loss=0.3697, pruned_loss=0.07051, over 24577.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3775, pruned_loss=0.072, over 4656311.14 frames. ], batch size: 154, lr: 3.76e-03, grad_scale: 32.0 2026-09-24 05:33:18,584 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=177800.0, ans=0.125 2026-09-24 05:33:19,819 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.562e+02 3.403e+02 3.720e+02 4.184e+02 6.335e+02, threshold=7.440e+02, percent-clipped=0.0 2026-09-24 05:33:27,852 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=177866.66666666666, ans=0.125 2026-09-24 05:33:35,373 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=177900.0, ans=0.0 2026-09-24 05:33:42,130 INFO [train.py:1192] (1/2) Epoch 56, batch 700, loss[loss=0.2597, simple_loss=0.3729, pruned_loss=0.07326, over 24544.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3784, pruned_loss=0.07213, over 4690806.82 frames. ], batch size: 158, lr: 3.76e-03, grad_scale: 32.0 2026-09-24 05:33:42,227 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=177966.66666666666, ans=0.2 2026-09-24 05:33:46,413 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=178000.0, ans=0.0 2026-09-24 05:34:07,517 INFO [train.py:1192] (1/2) Epoch 56, batch 750, loss[loss=0.2947, simple_loss=0.4058, pruned_loss=0.09179, over 24641.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3773, pruned_loss=0.07195, over 4723619.70 frames. ], batch size: 175, lr: 3.76e-03, grad_scale: 32.0 2026-09-24 05:34:11,078 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.813e+02 3.441e+02 3.869e+02 4.291e+02 6.518e+02, threshold=7.738e+02, percent-clipped=0.0 2026-09-24 05:34:16,043 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=178166.66666666666, ans=0.0 2026-09-24 05:34:26,610 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=178233.33333333334, ans=0.125 2026-09-24 05:34:27,000 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=178266.66666666666, ans=0.125 2026-09-24 05:34:28,324 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.min_positive, batch_count=178266.66666666666, ans=0.025 2026-09-24 05:34:32,697 INFO [train.py:1192] (1/2) Epoch 56, batch 800, loss[loss=0.2045, simple_loss=0.3218, pruned_loss=0.04357, over 24568.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3777, pruned_loss=0.0723, over 4750330.20 frames. ], batch size: 137, lr: 3.76e-03, grad_scale: 32.0 2026-09-24 05:34:37,042 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=178333.33333333334, ans=0.125 2026-09-24 05:34:39,444 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=178333.33333333334, ans=0.1 2026-09-24 05:34:51,811 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=178400.0, ans=0.1 2026-09-24 05:34:58,005 INFO [train.py:1192] (1/2) Epoch 56, batch 850, loss[loss=0.32, simple_loss=0.4303, pruned_loss=0.1049, over 24539.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3774, pruned_loss=0.07235, over 4769488.27 frames. ], batch size: 204, lr: 3.75e-03, grad_scale: 32.0 2026-09-24 05:35:01,692 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.451e+02 3.249e+02 3.700e+02 4.107e+02 5.695e+02, threshold=7.400e+02, percent-clipped=0.0 2026-09-24 05:35:04,007 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=178500.0, ans=0.1 2026-09-24 05:35:08,120 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:35:17,168 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=178566.66666666666, ans=0.2 2026-09-24 05:35:20,440 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=178600.0, ans=0.1 2026-09-24 05:35:23,536 INFO [train.py:1192] (1/2) Epoch 56, batch 900, loss[loss=0.247, simple_loss=0.3571, pruned_loss=0.06846, over 24514.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3786, pruned_loss=0.07306, over 4779593.10 frames. ], batch size: 137, lr: 3.75e-03, grad_scale: 32.0 2026-09-24 05:35:36,704 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=178700.0, ans=0.125 2026-09-24 05:35:41,841 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=178733.33333333334, ans=0.125 2026-09-24 05:35:44,151 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=6.01 vs. limit=8.0 2026-09-24 05:35:47,911 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=17.81 vs. limit=15.0 2026-09-24 05:35:49,015 INFO [train.py:1192] (1/2) Epoch 56, batch 950, loss[loss=0.3525, simple_loss=0.4269, pruned_loss=0.139, over 10978.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3769, pruned_loss=0.07347, over 4710055.16 frames. ], batch size: 333, lr: 3.75e-03, grad_scale: 32.0 2026-09-24 05:35:51,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=178800.0, ans=0.05 2026-09-24 05:35:52,352 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.987e+02 3.455e+02 4.002e+02 4.893e+02 8.245e+02, threshold=8.003e+02, percent-clipped=1.0 2026-09-24 05:35:59,334 INFO [train.py:1192] (1/2) Epoch 57, batch 0, loss[loss=0.2241, simple_loss=0.3373, pruned_loss=0.05546, over 24594.00 frames. ], tot_loss[loss=0.2241, simple_loss=0.3373, pruned_loss=0.05546, over 24594.00 frames. ], batch size: 137, lr: 3.72e-03, grad_scale: 32.0 2026-09-24 05:35:59,334 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 05:36:10,802 INFO [train.py:1224] (1/2) Epoch 57, validation: loss=0.1707, simple_loss=0.2894, pruned_loss=0.02596, over 2564189.00 frames. 2026-09-24 05:36:10,803 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 05:36:14,065 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=178826.66666666666, ans=0.125 2026-09-24 05:36:21,861 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.min_abs, batch_count=178893.33333333334, ans=0.5 2026-09-24 05:36:24,875 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=178893.33333333334, ans=0.0 2026-09-24 05:36:36,822 INFO [train.py:1192] (1/2) Epoch 57, batch 50, loss[loss=0.2012, simple_loss=0.3137, pruned_loss=0.04433, over 24278.00 frames. ], tot_loss[loss=0.2661, simple_loss=0.3824, pruned_loss=0.07491, over 1082452.05 frames. ], batch size: 125, lr: 3.72e-03, grad_scale: 32.0 2026-09-24 05:36:37,767 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=178993.33333333334, ans=0.125 2026-09-24 05:36:53,759 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=179093.33333333334, ans=0.0 2026-09-24 05:36:56,287 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.26 vs. limit=6.0 2026-09-24 05:37:01,832 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.678e+02 3.460e+02 3.786e+02 4.432e+02 6.420e+02, threshold=7.573e+02, percent-clipped=0.0 2026-09-24 05:37:02,588 INFO [train.py:1192] (1/2) Epoch 57, batch 100, loss[loss=0.2733, simple_loss=0.3806, pruned_loss=0.08303, over 24593.00 frames. ], tot_loss[loss=0.2689, simple_loss=0.3869, pruned_loss=0.07541, over 1916362.87 frames. ], batch size: 154, lr: 3.71e-03, grad_scale: 32.0 2026-09-24 05:37:05,235 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.75 vs. limit=12.0 2026-09-24 05:37:14,390 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=179226.66666666666, ans=0.125 2026-09-24 05:37:22,504 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=179293.33333333334, ans=0.125 2026-09-24 05:37:27,900 INFO [train.py:1192] (1/2) Epoch 57, batch 150, loss[loss=0.2035, simple_loss=0.3143, pruned_loss=0.04631, over 24245.00 frames. ], tot_loss[loss=0.2645, simple_loss=0.3815, pruned_loss=0.07376, over 2561440.87 frames. ], batch size: 125, lr: 3.71e-03, grad_scale: 32.0 2026-09-24 05:37:27,980 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=179326.66666666666, ans=0.025 2026-09-24 05:37:30,530 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=179326.66666666666, ans=0.0 2026-09-24 05:37:35,128 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=179360.0, ans=0.0 2026-09-24 05:37:35,131 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=179360.0, ans=0.1 2026-09-24 05:37:36,449 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=179360.0, ans=0.0 2026-09-24 05:37:38,071 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.23 vs. limit=15.0 2026-09-24 05:37:40,145 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=179393.33333333334, ans=0.125 2026-09-24 05:37:45,674 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=179426.66666666666, ans=0.2 2026-09-24 05:37:48,065 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=179460.0, ans=0.2 2026-09-24 05:37:53,207 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.516e+02 3.407e+02 3.900e+02 4.333e+02 6.747e+02, threshold=7.800e+02, percent-clipped=0.0 2026-09-24 05:37:53,699 INFO [train.py:1192] (1/2) Epoch 57, batch 200, loss[loss=0.2766, simple_loss=0.4038, pruned_loss=0.07475, over 24206.00 frames. ], tot_loss[loss=0.2626, simple_loss=0.3796, pruned_loss=0.07279, over 3060730.63 frames. ], batch size: 257, lr: 3.71e-03, grad_scale: 32.0 2026-09-24 05:37:53,810 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=179493.33333333334, ans=0.125 2026-09-24 05:37:54,339 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=179493.33333333334, ans=0.125 2026-09-24 05:37:56,757 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=179493.33333333334, ans=0.025 2026-09-24 05:37:58,163 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=179526.66666666666, ans=0.125 2026-09-24 05:38:08,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=179593.33333333334, ans=0.2 2026-09-24 05:38:11,226 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=179593.33333333334, ans=0.1 2026-09-24 05:38:16,828 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.36 vs. limit=15.0 2026-09-24 05:38:18,997 INFO [train.py:1192] (1/2) Epoch 57, batch 250, loss[loss=0.3039, simple_loss=0.4235, pruned_loss=0.09216, over 24367.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.3793, pruned_loss=0.07312, over 3444261.11 frames. ], batch size: 225, lr: 3.71e-03, grad_scale: 32.0 2026-09-24 05:38:23,238 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=4.69 vs. limit=15.0 2026-09-24 05:38:29,291 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:38:34,736 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.44 vs. limit=15.0 2026-09-24 05:38:43,576 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.470e+02 3.240e+02 3.731e+02 4.326e+02 5.664e+02, threshold=7.462e+02, percent-clipped=0.0 2026-09-24 05:38:44,049 INFO [train.py:1192] (1/2) Epoch 57, batch 300, loss[loss=0.2745, simple_loss=0.3941, pruned_loss=0.07747, over 24534.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3789, pruned_loss=0.07269, over 3758472.58 frames. ], batch size: 204, lr: 3.71e-03, grad_scale: 32.0 2026-09-24 05:38:49,481 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.35 vs. limit=10.0 2026-09-24 05:39:04,596 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=179960.0, ans=10.0 2026-09-24 05:39:10,028 INFO [train.py:1192] (1/2) Epoch 57, batch 350, loss[loss=0.2143, simple_loss=0.3267, pruned_loss=0.05097, over 24562.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.38, pruned_loss=0.07339, over 3998681.75 frames. ], batch size: 137, lr: 3.71e-03, grad_scale: 32.0 2026-09-24 05:39:20,506 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.80 vs. limit=15.0 2026-09-24 05:39:27,548 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=180093.33333333334, ans=0.2 2026-09-24 05:39:32,809 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.98 vs. limit=15.0 2026-09-24 05:39:35,104 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.864e+02 3.559e+02 4.088e+02 4.633e+02 6.508e+02, threshold=8.176e+02, percent-clipped=0.0 2026-09-24 05:39:35,581 INFO [train.py:1192] (1/2) Epoch 57, batch 400, loss[loss=0.2596, simple_loss=0.3801, pruned_loss=0.06952, over 24574.00 frames. ], tot_loss[loss=0.2632, simple_loss=0.3798, pruned_loss=0.07332, over 4181370.07 frames. ], batch size: 170, lr: 3.70e-03, grad_scale: 32.0 2026-09-24 05:39:35,695 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:39:39,307 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=180160.0, ans=0.125 2026-09-24 05:39:53,998 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:39:57,134 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=8.86 vs. limit=22.5 2026-09-24 05:39:58,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=180293.33333333334, ans=0.1 2026-09-24 05:40:00,533 INFO [train.py:1192] (1/2) Epoch 57, batch 450, loss[loss=0.2583, simple_loss=0.3837, pruned_loss=0.06643, over 24625.00 frames. ], tot_loss[loss=0.2629, simple_loss=0.3795, pruned_loss=0.07316, over 4321508.29 frames. ], batch size: 175, lr: 3.70e-03, grad_scale: 32.0 2026-09-24 05:40:05,423 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.32 vs. limit=12.0 2026-09-24 05:40:14,272 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.06 vs. limit=15.0 2026-09-24 05:40:16,575 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=180426.66666666666, ans=0.1 2026-09-24 05:40:25,548 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.658e+02 3.346e+02 3.702e+02 4.138e+02 6.456e+02, threshold=7.404e+02, percent-clipped=0.0 2026-09-24 05:40:26,097 INFO [train.py:1192] (1/2) Epoch 57, batch 500, loss[loss=0.2655, simple_loss=0.4006, pruned_loss=0.06519, over 24506.00 frames. ], tot_loss[loss=0.261, simple_loss=0.3776, pruned_loss=0.07221, over 4438931.30 frames. ], batch size: 218, lr: 3.70e-03, grad_scale: 32.0 2026-09-24 05:40:28,187 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=180493.33333333334, ans=0.05 2026-09-24 05:40:38,488 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=180560.0, ans=0.07 2026-09-24 05:40:42,754 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.21 vs. limit=15.0 2026-09-24 05:40:44,904 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=180593.33333333334, ans=0.0 2026-09-24 05:40:50,856 INFO [train.py:1192] (1/2) Epoch 57, batch 550, loss[loss=0.2899, simple_loss=0.4085, pruned_loss=0.08568, over 24303.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3782, pruned_loss=0.0726, over 4524129.18 frames. ], batch size: 257, lr: 3.70e-03, grad_scale: 32.0 2026-09-24 05:40:50,984 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:40:54,761 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=180660.0, ans=0.125 2026-09-24 05:40:55,360 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=180693.33333333334, ans=0.125 2026-09-24 05:40:57,389 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=180693.33333333334, ans=0.025 2026-09-24 05:41:05,096 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=180726.66666666666, ans=0.1 2026-09-24 05:41:16,221 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.732e+02 3.299e+02 3.672e+02 4.156e+02 5.943e+02, threshold=7.343e+02, percent-clipped=0.0 2026-09-24 05:41:16,799 INFO [train.py:1192] (1/2) Epoch 57, batch 600, loss[loss=0.2866, simple_loss=0.4129, pruned_loss=0.08017, over 24309.00 frames. ], tot_loss[loss=0.263, simple_loss=0.3795, pruned_loss=0.07321, over 4589894.07 frames. ], batch size: 234, lr: 3.70e-03, grad_scale: 32.0 2026-09-24 05:41:42,014 INFO [train.py:1192] (1/2) Epoch 57, batch 650, loss[loss=0.2645, simple_loss=0.3734, pruned_loss=0.07785, over 24592.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3777, pruned_loss=0.07194, over 4654710.93 frames. ], batch size: 154, lr: 3.70e-03, grad_scale: 32.0 2026-09-24 05:41:55,525 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.61 vs. limit=15.0 2026-09-24 05:42:00,704 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.79 vs. limit=22.5 2026-09-24 05:42:02,108 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=181126.66666666666, ans=0.1 2026-09-24 05:42:07,960 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.698e+02 3.523e+02 3.824e+02 4.374e+02 7.238e+02, threshold=7.648e+02, percent-clipped=0.0 2026-09-24 05:42:07,966 INFO [train.py:1192] (1/2) Epoch 57, batch 700, loss[loss=0.2772, simple_loss=0.3863, pruned_loss=0.08404, over 24552.00 frames. ], tot_loss[loss=0.2616, simple_loss=0.3787, pruned_loss=0.07227, over 4691118.00 frames. ], batch size: 158, lr: 3.69e-03, grad_scale: 32.0 2026-09-24 05:42:08,092 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=181160.0, ans=0.125 2026-09-24 05:42:13,615 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.24 vs. limit=10.0 2026-09-24 05:42:14,520 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.41 vs. limit=15.0 2026-09-24 05:42:18,214 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=181226.66666666666, ans=0.0 2026-09-24 05:42:21,005 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=181226.66666666666, ans=0.0 2026-09-24 05:42:27,478 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=181260.0, ans=0.125 2026-09-24 05:42:31,354 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=181293.33333333334, ans=0.0 2026-09-24 05:42:33,038 INFO [train.py:1192] (1/2) Epoch 57, batch 750, loss[loss=0.2585, simple_loss=0.3806, pruned_loss=0.06822, over 24620.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3777, pruned_loss=0.07192, over 4727331.99 frames. ], batch size: 175, lr: 3.69e-03, grad_scale: 32.0 2026-09-24 05:42:36,032 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=181326.66666666666, ans=0.125 2026-09-24 05:42:37,846 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=181360.0, ans=10.0 2026-09-24 05:42:41,012 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.56 vs. limit=15.0 2026-09-24 05:42:41,376 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=181360.0, ans=0.125 2026-09-24 05:42:45,295 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=181393.33333333334, ans=0.025 2026-09-24 05:42:46,840 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=181393.33333333334, ans=0.125 2026-09-24 05:42:58,300 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.714e+02 3.444e+02 3.874e+02 4.390e+02 6.289e+02, threshold=7.748e+02, percent-clipped=0.0 2026-09-24 05:42:58,306 INFO [train.py:1192] (1/2) Epoch 57, batch 800, loss[loss=0.2307, simple_loss=0.3467, pruned_loss=0.05736, over 24515.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3777, pruned_loss=0.07197, over 4752847.44 frames. ], batch size: 137, lr: 3.69e-03, grad_scale: 32.0 2026-09-24 05:43:00,418 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=181493.33333333334, ans=0.125 2026-09-24 05:43:03,188 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=181526.66666666666, ans=0.125 2026-09-24 05:43:11,816 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=181560.0, ans=0.125 2026-09-24 05:43:18,782 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=181626.66666666666, ans=0.07 2026-09-24 05:43:20,212 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=181626.66666666666, ans=0.1 2026-09-24 05:43:23,938 INFO [train.py:1192] (1/2) Epoch 57, batch 850, loss[loss=0.2712, simple_loss=0.3932, pruned_loss=0.07458, over 24547.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.3771, pruned_loss=0.07183, over 4772295.04 frames. ], batch size: 204, lr: 3.69e-03, grad_scale: 32.0 2026-09-24 05:43:24,196 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.01 vs. limit=10.0 2026-09-24 05:43:26,331 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=181660.0, ans=0.125 2026-09-24 05:43:49,093 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=181826.66666666666, ans=0.0 2026-09-24 05:43:49,505 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.660e+02 3.443e+02 3.791e+02 4.284e+02 6.953e+02, threshold=7.582e+02, percent-clipped=0.0 2026-09-24 05:43:49,511 INFO [train.py:1192] (1/2) Epoch 57, batch 900, loss[loss=0.2335, simple_loss=0.3504, pruned_loss=0.05836, over 24567.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3775, pruned_loss=0.07199, over 4783067.76 frames. ], batch size: 137, lr: 3.69e-03, grad_scale: 32.0 2026-09-24 05:43:59,630 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=181893.33333333334, ans=0.1 2026-09-24 05:44:01,105 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.55 vs. limit=6.0 2026-09-24 05:44:07,890 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=181926.66666666666, ans=0.035 2026-09-24 05:44:14,494 INFO [train.py:1192] (1/2) Epoch 57, batch 950, loss[loss=0.349, simple_loss=0.4256, pruned_loss=0.1362, over 10920.00 frames. ], tot_loss[loss=0.2616, simple_loss=0.3766, pruned_loss=0.07331, over 4715559.06 frames. ], batch size: 334, lr: 3.69e-03, grad_scale: 32.0 2026-09-24 05:44:25,449 INFO [train.py:1192] (1/2) Epoch 58, batch 0, loss[loss=0.2175, simple_loss=0.3385, pruned_loss=0.04829, over 24566.00 frames. ], tot_loss[loss=0.2175, simple_loss=0.3385, pruned_loss=0.04829, over 24566.00 frames. ], batch size: 137, lr: 3.65e-03, grad_scale: 32.0 2026-09-24 05:44:25,449 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 05:44:27,690 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.2536, 3.2906, 3.1298, 2.2442], device='cuda:1') 2026-09-24 05:44:33,316 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0850, 2.9429, 2.5429, 2.2350], device='cuda:1') 2026-09-24 05:44:36,101 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.3910, 4.5096, 4.2234, 4.0596], device='cuda:1') 2026-09-24 05:44:36,239 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.0499, 3.7631, 3.7197, 3.9798], device='cuda:1') 2026-09-24 05:44:37,022 INFO [train.py:1224] (1/2) Epoch 58, validation: loss=0.1721, simple_loss=0.2908, pruned_loss=0.02669, over 2564189.00 frames. 2026-09-24 05:44:37,022 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 05:44:38,319 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.57 vs. limit=15.0 2026-09-24 05:44:45,836 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.99 vs. limit=10.0 2026-09-24 05:44:51,449 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=182086.66666666666, ans=0.2 2026-09-24 05:44:57,509 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=182153.33333333334, ans=0.09899494936611666 2026-09-24 05:44:59,008 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.814e+02 3.593e+02 4.032e+02 4.676e+02 7.320e+02, threshold=8.064e+02, percent-clipped=0.0 2026-09-24 05:45:02,914 INFO [train.py:1192] (1/2) Epoch 58, batch 50, loss[loss=0.2055, simple_loss=0.3175, pruned_loss=0.04669, over 24333.00 frames. ], tot_loss[loss=0.2699, simple_loss=0.3852, pruned_loss=0.07735, over 1081941.15 frames. ], batch size: 125, lr: 3.65e-03, grad_scale: 32.0 2026-09-24 05:45:06,736 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=182186.66666666666, ans=15.0 2026-09-24 05:45:13,388 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=182253.33333333334, ans=0.125 2026-09-24 05:45:16,049 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=182253.33333333334, ans=0.125 2026-09-24 05:45:17,595 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:45:21,325 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=182286.66666666666, ans=0.0 2026-09-24 05:45:24,841 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=182320.0, ans=0.025 2026-09-24 05:45:28,136 INFO [train.py:1192] (1/2) Epoch 58, batch 100, loss[loss=0.2447, simple_loss=0.3568, pruned_loss=0.06634, over 24622.00 frames. ], tot_loss[loss=0.2701, simple_loss=0.3874, pruned_loss=0.07644, over 1915279.52 frames. ], batch size: 154, lr: 3.65e-03, grad_scale: 32.0 2026-09-24 05:45:33,199 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=182386.66666666666, ans=0.2 2026-09-24 05:45:37,391 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=182386.66666666666, ans=0.125 2026-09-24 05:45:45,572 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:45:47,422 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=182453.33333333334, ans=0.125 2026-09-24 05:45:49,736 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.618e+02 3.344e+02 3.795e+02 4.291e+02 6.356e+02, threshold=7.589e+02, percent-clipped=0.0 2026-09-24 05:45:53,690 INFO [train.py:1192] (1/2) Epoch 58, batch 150, loss[loss=0.2203, simple_loss=0.3315, pruned_loss=0.05458, over 24258.00 frames. ], tot_loss[loss=0.2655, simple_loss=0.3824, pruned_loss=0.07424, over 2560856.41 frames. ], batch size: 125, lr: 3.65e-03, grad_scale: 32.0 2026-09-24 05:45:55,222 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=182520.0, ans=0.025 2026-09-24 05:45:55,582 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=182520.0, ans=0.0 2026-09-24 05:46:01,401 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=182553.33333333334, ans=0.0 2026-09-24 05:46:02,746 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=182553.33333333334, ans=0.1 2026-09-24 05:46:02,866 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.01 vs. limit=15.0 2026-09-24 05:46:06,779 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:46:11,036 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=182620.0, ans=0.0 2026-09-24 05:46:12,799 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=182620.0, ans=0.125 2026-09-24 05:46:19,926 INFO [train.py:1192] (1/2) Epoch 58, batch 200, loss[loss=0.2872, simple_loss=0.4148, pruned_loss=0.07984, over 24219.00 frames. ], tot_loss[loss=0.2633, simple_loss=0.3804, pruned_loss=0.07313, over 3059263.46 frames. ], batch size: 257, lr: 3.65e-03, grad_scale: 32.0 2026-09-24 05:46:41,401 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.696e+02 3.457e+02 3.741e+02 4.445e+02 6.120e+02, threshold=7.483e+02, percent-clipped=0.0 2026-09-24 05:46:43,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=182820.0, ans=0.125 2026-09-24 05:46:45,727 INFO [train.py:1192] (1/2) Epoch 58, batch 250, loss[loss=0.305, simple_loss=0.4233, pruned_loss=0.09332, over 24383.00 frames. ], tot_loss[loss=0.2631, simple_loss=0.3797, pruned_loss=0.07323, over 3442553.31 frames. ], batch size: 225, lr: 3.64e-03, grad_scale: 32.0 2026-09-24 05:46:45,809 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=182853.33333333334, ans=0.125 2026-09-24 05:46:50,638 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:46:52,951 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=182886.66666666666, ans=0.5 2026-09-24 05:47:00,787 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=182953.33333333334, ans=0.125 2026-09-24 05:47:07,682 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=182986.66666666666, ans=0.125 2026-09-24 05:47:11,806 INFO [train.py:1192] (1/2) Epoch 58, batch 300, loss[loss=0.2999, simple_loss=0.4114, pruned_loss=0.09425, over 24523.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.3787, pruned_loss=0.07255, over 3755939.28 frames. ], batch size: 204, lr: 3.64e-03, grad_scale: 32.0 2026-09-24 05:47:19,044 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.50 vs. limit=15.0 2026-09-24 05:47:21,988 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=183086.66666666666, ans=0.0 2026-09-24 05:47:23,943 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=183086.66666666666, ans=0.125 2026-09-24 05:47:33,099 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.515e+02 3.314e+02 3.805e+02 4.215e+02 6.606e+02, threshold=7.609e+02, percent-clipped=0.0 2026-09-24 05:47:37,117 INFO [train.py:1192] (1/2) Epoch 58, batch 350, loss[loss=0.2111, simple_loss=0.3262, pruned_loss=0.04802, over 24577.00 frames. ], tot_loss[loss=0.263, simple_loss=0.3798, pruned_loss=0.07308, over 3993536.77 frames. ], batch size: 137, lr: 3.64e-03, grad_scale: 32.0 2026-09-24 05:47:38,184 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=183186.66666666666, ans=0.125 2026-09-24 05:47:38,679 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=183186.66666666666, ans=0.2 2026-09-24 05:47:46,867 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=183253.33333333334, ans=0.04949747468305833 2026-09-24 05:47:52,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=183286.66666666666, ans=0.125 2026-09-24 05:48:02,034 INFO [train.py:1192] (1/2) Epoch 58, batch 400, loss[loss=0.2401, simple_loss=0.3608, pruned_loss=0.05971, over 24540.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3781, pruned_loss=0.07224, over 4179410.24 frames. ], batch size: 170, lr: 3.64e-03, grad_scale: 32.0 2026-09-24 05:48:03,726 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.70 vs. limit=15.0 2026-09-24 05:48:10,636 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.36 vs. limit=10.0 2026-09-24 05:48:21,468 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.40 vs. limit=6.0 2026-09-24 05:48:23,412 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.871e+02 3.378e+02 3.805e+02 4.366e+02 6.990e+02, threshold=7.609e+02, percent-clipped=0.0 2026-09-24 05:48:27,156 INFO [train.py:1192] (1/2) Epoch 58, batch 450, loss[loss=0.2874, simple_loss=0.4049, pruned_loss=0.08493, over 24638.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3781, pruned_loss=0.07202, over 4317614.46 frames. ], batch size: 175, lr: 3.64e-03, grad_scale: 32.0 2026-09-24 05:48:33,628 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.43 vs. limit=12.0 2026-09-24 05:48:41,631 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=183586.66666666666, ans=0.125 2026-09-24 05:48:52,868 INFO [train.py:1192] (1/2) Epoch 58, batch 500, loss[loss=0.2925, simple_loss=0.4181, pruned_loss=0.08348, over 24525.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3766, pruned_loss=0.07155, over 4436498.33 frames. ], batch size: 218, lr: 3.64e-03, grad_scale: 32.0 2026-09-24 05:48:53,425 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=183686.66666666666, ans=0.0 2026-09-24 05:48:59,449 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=183720.0, ans=0.025 2026-09-24 05:49:00,093 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.99 vs. limit=6.0 2026-09-24 05:49:08,783 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=183786.66666666666, ans=0.125 2026-09-24 05:49:12,873 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer2.prob, batch_count=183820.0, ans=0.125 2026-09-24 05:49:14,455 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.501e+02 3.193e+02 3.750e+02 4.124e+02 5.753e+02, threshold=7.500e+02, percent-clipped=0.0 2026-09-24 05:49:17,148 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=183820.0, ans=0.125 2026-09-24 05:49:18,789 INFO [train.py:1192] (1/2) Epoch 58, batch 550, loss[loss=0.2682, simple_loss=0.3969, pruned_loss=0.0697, over 24206.00 frames. ], tot_loss[loss=0.2616, simple_loss=0.3781, pruned_loss=0.07252, over 4521342.95 frames. ], batch size: 257, lr: 3.63e-03, grad_scale: 32.0 2026-09-24 05:49:34,118 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=183953.33333333334, ans=0.125 2026-09-24 05:49:44,582 INFO [train.py:1192] (1/2) Epoch 58, batch 600, loss[loss=0.2931, simple_loss=0.4197, pruned_loss=0.08322, over 24348.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3787, pruned_loss=0.07276, over 4588240.38 frames. ], batch size: 234, lr: 3.63e-03, grad_scale: 32.0 2026-09-24 05:49:58,822 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.36 vs. limit=15.0 2026-09-24 05:50:03,924 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.78 vs. limit=15.0 2026-09-24 05:50:05,768 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.673e+02 3.291e+02 3.709e+02 4.176e+02 6.941e+02, threshold=7.419e+02, percent-clipped=0.0 2026-09-24 05:50:08,218 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=184153.33333333334, ans=0.125 2026-09-24 05:50:10,282 INFO [train.py:1192] (1/2) Epoch 58, batch 650, loss[loss=0.2649, simple_loss=0.3733, pruned_loss=0.07825, over 24585.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.378, pruned_loss=0.07224, over 4653397.87 frames. ], batch size: 154, lr: 3.63e-03, grad_scale: 32.0 2026-09-24 05:50:12,075 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=184186.66666666666, ans=0.0 2026-09-24 05:50:22,209 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=184253.33333333334, ans=0.125 2026-09-24 05:50:36,104 INFO [train.py:1192] (1/2) Epoch 58, batch 700, loss[loss=0.2638, simple_loss=0.3737, pruned_loss=0.07694, over 24551.00 frames. ], tot_loss[loss=0.262, simple_loss=0.379, pruned_loss=0.07249, over 4689825.77 frames. ], batch size: 158, lr: 3.63e-03, grad_scale: 32.0 2026-09-24 05:50:39,687 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:50:49,045 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=184420.0, ans=0.07 2026-09-24 05:50:57,734 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.707e+02 3.433e+02 3.912e+02 4.428e+02 7.211e+02, threshold=7.824e+02, percent-clipped=0.0 2026-09-24 05:51:01,757 INFO [train.py:1192] (1/2) Epoch 58, batch 750, loss[loss=0.2535, simple_loss=0.3755, pruned_loss=0.06571, over 24629.00 frames. ], tot_loss[loss=0.2624, simple_loss=0.3788, pruned_loss=0.07297, over 4726515.51 frames. ], batch size: 175, lr: 3.63e-03, grad_scale: 32.0 2026-09-24 05:51:05,888 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=184520.0, ans=0.0 2026-09-24 05:51:08,965 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=184553.33333333334, ans=0.125 2026-09-24 05:51:18,816 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=184620.0, ans=0.125 2026-09-24 05:51:27,058 INFO [train.py:1192] (1/2) Epoch 58, batch 800, loss[loss=0.2397, simple_loss=0.3433, pruned_loss=0.0681, over 24565.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3783, pruned_loss=0.07258, over 4752240.71 frames. ], batch size: 137, lr: 3.63e-03, grad_scale: 32.0 2026-09-24 05:51:27,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=184686.66666666666, ans=0.025 2026-09-24 05:51:41,906 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=184786.66666666666, ans=0.025 2026-09-24 05:51:47,386 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=184820.0, ans=0.0 2026-09-24 05:51:48,213 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.583e+02 3.476e+02 3.817e+02 4.450e+02 5.594e+02, threshold=7.634e+02, percent-clipped=0.0 2026-09-24 05:51:51,247 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=184820.0, ans=0.125 2026-09-24 05:51:51,984 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.35 vs. limit=10.0 2026-09-24 05:51:52,209 INFO [train.py:1192] (1/2) Epoch 58, batch 850, loss[loss=0.2976, simple_loss=0.4165, pruned_loss=0.08932, over 24548.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3777, pruned_loss=0.07225, over 4770597.62 frames. ], batch size: 204, lr: 3.63e-03, grad_scale: 32.0 2026-09-24 05:52:04,931 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=184920.0, ans=0.0 2026-09-24 05:52:11,309 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=184953.33333333334, ans=0.2 2026-09-24 05:52:17,220 INFO [train.py:1192] (1/2) Epoch 58, batch 900, loss[loss=0.2243, simple_loss=0.3414, pruned_loss=0.05355, over 24558.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3777, pruned_loss=0.072, over 4781117.41 frames. ], batch size: 137, lr: 3.62e-03, grad_scale: 32.0 2026-09-24 05:52:21,353 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=185020.0, ans=0.125 2026-09-24 05:52:22,128 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.67 vs. limit=15.0 2026-09-24 05:52:38,729 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.652e+02 3.344e+02 3.958e+02 4.577e+02 6.010e+02, threshold=7.916e+02, percent-clipped=0.0 2026-09-24 05:52:39,234 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=185153.33333333334, ans=0.0 2026-09-24 05:52:39,717 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=185153.33333333334, ans=0.0 2026-09-24 05:52:42,580 INFO [train.py:1192] (1/2) Epoch 58, batch 950, loss[loss=0.3553, simple_loss=0.4209, pruned_loss=0.1449, over 10942.00 frames. ], tot_loss[loss=0.2615, simple_loss=0.3768, pruned_loss=0.07315, over 4717212.13 frames. ], batch size: 334, lr: 3.62e-03, grad_scale: 32.0 2026-09-24 05:52:43,960 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=11.95 vs. limit=12.0 2026-09-24 05:52:45,122 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=185186.66666666666, ans=0.2 2026-09-24 05:52:52,719 INFO [train.py:1192] (1/2) Epoch 59, batch 0, loss[loss=0.2156, simple_loss=0.3371, pruned_loss=0.04705, over 24565.00 frames. ], tot_loss[loss=0.2156, simple_loss=0.3371, pruned_loss=0.04705, over 24565.00 frames. ], batch size: 137, lr: 3.59e-03, grad_scale: 32.0 2026-09-24 05:52:52,720 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 05:52:54,308 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.0.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.4369, 4.5456, 4.2333, 4.0544], device='cuda:1') 2026-09-24 05:53:04,541 INFO [train.py:1224] (1/2) Epoch 59, validation: loss=0.171, simple_loss=0.2895, pruned_loss=0.02627, over 2564189.00 frames. 2026-09-24 05:53:04,541 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 05:53:05,887 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=185213.33333333334, ans=0.2 2026-09-24 05:53:06,846 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=185213.33333333334, ans=0.0 2026-09-24 05:53:12,442 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=185246.66666666666, ans=0.025 2026-09-24 05:53:30,308 INFO [train.py:1192] (1/2) Epoch 59, batch 50, loss[loss=0.2132, simple_loss=0.3247, pruned_loss=0.05084, over 24286.00 frames. ], tot_loss[loss=0.2642, simple_loss=0.3816, pruned_loss=0.0734, over 1082644.59 frames. ], batch size: 125, lr: 3.59e-03, grad_scale: 32.0 2026-09-24 05:53:33,722 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=185380.0, ans=0.1 2026-09-24 05:53:47,501 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.691e+02 3.534e+02 3.979e+02 4.502e+02 6.245e+02, threshold=7.957e+02, percent-clipped=0.0 2026-09-24 05:53:48,570 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:53:52,111 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=185513.33333333334, ans=0.125 2026-09-24 05:53:53,510 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=185513.33333333334, ans=0.0 2026-09-24 05:53:55,373 INFO [train.py:1192] (1/2) Epoch 59, batch 100, loss[loss=0.2505, simple_loss=0.3658, pruned_loss=0.06762, over 24638.00 frames. ], tot_loss[loss=0.2683, simple_loss=0.3863, pruned_loss=0.07512, over 1914743.51 frames. ], batch size: 154, lr: 3.59e-03, grad_scale: 32.0 2026-09-24 05:54:05,217 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.64 vs. limit=8.0 2026-09-24 05:54:07,765 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=185613.33333333334, ans=0.0 2026-09-24 05:54:21,465 INFO [train.py:1192] (1/2) Epoch 59, batch 150, loss[loss=0.2199, simple_loss=0.3303, pruned_loss=0.05477, over 24312.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3818, pruned_loss=0.07391, over 2560207.42 frames. ], batch size: 125, lr: 3.59e-03, grad_scale: 32.0 2026-09-24 05:54:22,108 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=185713.33333333334, ans=0.125 2026-09-24 05:54:32,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=185780.0, ans=0.1 2026-09-24 05:54:36,666 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=185813.33333333334, ans=0.05 2026-09-24 05:54:36,803 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.84 vs. limit=15.0 2026-09-24 05:54:38,851 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.662e+02 3.284e+02 3.697e+02 4.244e+02 6.453e+02, threshold=7.394e+02, percent-clipped=0.0 2026-09-24 05:54:44,369 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=185846.66666666666, ans=0.125 2026-09-24 05:54:45,418 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=185846.66666666666, ans=0.0 2026-09-24 05:54:45,422 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=185846.66666666666, ans=0.125 2026-09-24 05:54:47,286 INFO [train.py:1192] (1/2) Epoch 59, batch 200, loss[loss=0.2896, simple_loss=0.4122, pruned_loss=0.08353, over 24199.00 frames. ], tot_loss[loss=0.263, simple_loss=0.38, pruned_loss=0.07305, over 3059561.05 frames. ], batch size: 257, lr: 3.58e-03, grad_scale: 32.0 2026-09-24 05:54:50,714 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.14 vs. limit=10.0 2026-09-24 05:54:52,216 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=15.04 vs. limit=15.0 2026-09-24 05:54:54,446 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=185913.33333333334, ans=0.0 2026-09-24 05:54:59,656 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=185946.66666666666, ans=0.1 2026-09-24 05:55:07,302 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=186013.33333333334, ans=0.0 2026-09-24 05:55:12,730 INFO [train.py:1192] (1/2) Epoch 59, batch 250, loss[loss=0.2835, simple_loss=0.4051, pruned_loss=0.08093, over 24396.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3788, pruned_loss=0.0727, over 3443868.26 frames. ], batch size: 225, lr: 3.58e-03, grad_scale: 32.0 2026-09-24 05:55:21,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=186080.0, ans=0.025 2026-09-24 05:55:30,137 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.774e+02 3.471e+02 3.847e+02 4.480e+02 8.835e+02, threshold=7.694e+02, percent-clipped=1.0 2026-09-24 05:55:33,756 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 05:55:35,030 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=186180.0, ans=0.125 2026-09-24 05:55:38,443 INFO [train.py:1192] (1/2) Epoch 59, batch 300, loss[loss=0.255, simple_loss=0.39, pruned_loss=0.06003, over 24540.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3783, pruned_loss=0.07219, over 3754662.19 frames. ], batch size: 204, lr: 3.58e-03, grad_scale: 32.0 2026-09-24 05:55:44,864 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=186246.66666666666, ans=0.1 2026-09-24 05:56:03,678 INFO [train.py:1192] (1/2) Epoch 59, batch 350, loss[loss=0.2011, simple_loss=0.3193, pruned_loss=0.04145, over 24580.00 frames. ], tot_loss[loss=0.2622, simple_loss=0.3793, pruned_loss=0.07256, over 3995432.49 frames. ], batch size: 137, lr: 3.58e-03, grad_scale: 32.0 2026-09-24 05:56:10,395 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=186413.33333333334, ans=0.125 2026-09-24 05:56:20,967 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.677e+02 3.293e+02 3.742e+02 4.328e+02 7.534e+02, threshold=7.484e+02, percent-clipped=0.0 2026-09-24 05:56:24,056 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.05 vs. limit=15.0 2026-09-24 05:56:29,151 INFO [train.py:1192] (1/2) Epoch 59, batch 400, loss[loss=0.2812, simple_loss=0.3916, pruned_loss=0.08535, over 24569.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3779, pruned_loss=0.07172, over 4182578.58 frames. ], batch size: 170, lr: 3.58e-03, grad_scale: 32.0 2026-09-24 05:56:39,015 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.21 vs. limit=15.0 2026-09-24 05:56:42,262 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=186613.33333333334, ans=0.125 2026-09-24 05:56:54,934 INFO [train.py:1192] (1/2) Epoch 59, batch 450, loss[loss=0.2793, simple_loss=0.3938, pruned_loss=0.08239, over 24622.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.3781, pruned_loss=0.0719, over 4321300.28 frames. ], batch size: 175, lr: 3.58e-03, grad_scale: 32.0 2026-09-24 05:56:56,887 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=186713.33333333334, ans=0.95 2026-09-24 05:56:59,282 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=186746.66666666666, ans=0.0 2026-09-24 05:57:08,273 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.04 vs. limit=22.5 2026-09-24 05:57:11,989 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.672e+02 3.301e+02 3.744e+02 4.193e+02 6.017e+02, threshold=7.488e+02, percent-clipped=0.0 2026-09-24 05:57:18,672 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=186846.66666666666, ans=0.035 2026-09-24 05:57:20,273 INFO [train.py:1192] (1/2) Epoch 59, batch 500, loss[loss=0.2618, simple_loss=0.3848, pruned_loss=0.06945, over 24499.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3768, pruned_loss=0.07156, over 4438451.96 frames. ], batch size: 218, lr: 3.57e-03, grad_scale: 32.0 2026-09-24 05:57:33,147 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=186946.66666666666, ans=0.125 2026-09-24 05:57:41,905 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=187013.33333333334, ans=0.125 2026-09-24 05:57:45,403 INFO [train.py:1192] (1/2) Epoch 59, batch 550, loss[loss=0.27, simple_loss=0.4024, pruned_loss=0.06884, over 24241.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3769, pruned_loss=0.0713, over 4523655.04 frames. ], batch size: 257, lr: 3.57e-03, grad_scale: 32.0 2026-09-24 05:57:45,478 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=187046.66666666666, ans=0.125 2026-09-24 05:57:50,286 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=187080.0, ans=0.0 2026-09-24 05:57:52,596 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=187080.0, ans=0.125 2026-09-24 05:58:02,550 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.630e+02 3.301e+02 3.638e+02 4.319e+02 6.236e+02, threshold=7.276e+02, percent-clipped=0.0 2026-09-24 05:58:05,047 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.99 vs. limit=15.0 2026-09-24 05:58:05,764 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=187180.0, ans=0.0 2026-09-24 05:58:10,786 INFO [train.py:1192] (1/2) Epoch 59, batch 600, loss[loss=0.2846, simple_loss=0.4159, pruned_loss=0.07667, over 24456.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.3775, pruned_loss=0.07153, over 4590425.74 frames. ], batch size: 235, lr: 3.57e-03, grad_scale: 32.0 2026-09-24 05:58:23,582 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=187280.0, ans=0.025 2026-09-24 05:58:32,721 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=187346.66666666666, ans=0.1 2026-09-24 05:58:36,333 INFO [train.py:1192] (1/2) Epoch 59, batch 650, loss[loss=0.2392, simple_loss=0.3581, pruned_loss=0.06018, over 24615.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3771, pruned_loss=0.07161, over 4655253.10 frames. ], batch size: 154, lr: 3.57e-03, grad_scale: 32.0 2026-09-24 05:58:36,867 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=187380.0, ans=0.125 2026-09-24 05:58:39,505 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=187380.0, ans=0.125 2026-09-24 05:58:53,470 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.591e+02 3.411e+02 3.788e+02 4.386e+02 6.509e+02, threshold=7.576e+02, percent-clipped=0.0 2026-09-24 05:59:01,734 INFO [train.py:1192] (1/2) Epoch 59, batch 700, loss[loss=0.2363, simple_loss=0.3581, pruned_loss=0.05726, over 24562.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3781, pruned_loss=0.07159, over 4690051.69 frames. ], batch size: 158, lr: 3.57e-03, grad_scale: 32.0 2026-09-24 05:59:10,742 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=187580.0, ans=0.125 2026-09-24 05:59:21,625 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=187646.66666666666, ans=0.1 2026-09-24 05:59:25,012 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.82 vs. limit=15.0 2026-09-24 05:59:27,692 INFO [train.py:1192] (1/2) Epoch 59, batch 750, loss[loss=0.2804, simple_loss=0.397, pruned_loss=0.08191, over 24643.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.3774, pruned_loss=0.07157, over 4726280.84 frames. ], batch size: 175, lr: 3.57e-03, grad_scale: 32.0 2026-09-24 05:59:31,146 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=187713.33333333334, ans=0.0 2026-09-24 05:59:38,617 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=187780.0, ans=0.05 2026-09-24 05:59:42,607 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=187813.33333333334, ans=0.1 2026-09-24 05:59:43,129 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=187813.33333333334, ans=0.125 2026-09-24 05:59:45,330 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.834e+02 3.595e+02 4.005e+02 4.409e+02 6.130e+02, threshold=8.011e+02, percent-clipped=0.0 2026-09-24 05:59:48,991 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=187846.66666666666, ans=0.1 2026-09-24 05:59:53,264 INFO [train.py:1192] (1/2) Epoch 59, batch 800, loss[loss=0.2173, simple_loss=0.3362, pruned_loss=0.04921, over 24553.00 frames. ], tot_loss[loss=0.2602, simple_loss=0.3773, pruned_loss=0.07155, over 4751992.26 frames. ], batch size: 137, lr: 3.57e-03, grad_scale: 64.0 2026-09-24 05:59:53,863 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=187880.0, ans=0.0 2026-09-24 06:00:03,051 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.63 vs. limit=15.0 2026-09-24 06:00:12,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=187980.0, ans=0.0 2026-09-24 06:00:18,896 INFO [train.py:1192] (1/2) Epoch 59, batch 850, loss[loss=0.2984, simple_loss=0.4164, pruned_loss=0.09027, over 24552.00 frames. ], tot_loss[loss=0.2596, simple_loss=0.3767, pruned_loss=0.07129, over 4771597.14 frames. ], batch size: 204, lr: 3.56e-03, grad_scale: 64.0 2026-09-24 06:00:21,256 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=188046.66666666666, ans=0.125 2026-09-24 06:00:27,588 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=188080.0, ans=0.125 2026-09-24 06:00:32,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=188113.33333333334, ans=0.1 2026-09-24 06:00:36,162 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.731e+02 3.395e+02 3.830e+02 4.384e+02 6.066e+02, threshold=7.661e+02, percent-clipped=0.0 2026-09-24 06:00:40,839 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=12.61 vs. limit=22.5 2026-09-24 06:00:41,850 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.53 vs. limit=15.0 2026-09-24 06:00:43,800 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.95 vs. limit=10.0 2026-09-24 06:00:44,043 INFO [train.py:1192] (1/2) Epoch 59, batch 900, loss[loss=0.2226, simple_loss=0.339, pruned_loss=0.05317, over 24565.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3765, pruned_loss=0.07102, over 4781917.14 frames. ], batch size: 137, lr: 3.56e-03, grad_scale: 64.0 2026-09-24 06:00:54,171 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=188280.0, ans=0.125 2026-09-24 06:01:06,796 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=188346.66666666666, ans=0.125 2026-09-24 06:01:08,622 INFO [train.py:1192] (1/2) Epoch 59, batch 950, loss[loss=0.3612, simple_loss=0.4246, pruned_loss=0.1489, over 11390.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3753, pruned_loss=0.07178, over 4712643.68 frames. ], batch size: 334, lr: 3.56e-03, grad_scale: 32.0 2026-09-24 06:01:17,701 INFO [train.py:1192] (1/2) Epoch 60, batch 0, loss[loss=0.211, simple_loss=0.3344, pruned_loss=0.04378, over 24570.00 frames. ], tot_loss[loss=0.211, simple_loss=0.3344, pruned_loss=0.04378, over 24570.00 frames. ], batch size: 137, lr: 3.53e-03, grad_scale: 32.0 2026-09-24 06:01:17,701 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 06:01:28,329 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.6664, 4.2661, 4.4316, 4.2762], device='cuda:1') 2026-09-24 06:01:29,341 INFO [train.py:1224] (1/2) Epoch 60, validation: loss=0.1697, simple_loss=0.2883, pruned_loss=0.0255, over 2564189.00 frames. 2026-09-24 06:01:29,341 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 06:01:37,311 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=188440.0, ans=0.125 2026-09-24 06:01:42,911 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.788e+02 3.464e+02 3.986e+02 4.513e+02 6.611e+02, threshold=7.972e+02, percent-clipped=0.0 2026-09-24 06:01:45,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=188506.66666666666, ans=0.025 2026-09-24 06:01:45,681 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=188506.66666666666, ans=0.125 2026-09-24 06:01:51,020 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=188540.0, ans=0.125 2026-09-24 06:01:52,194 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=188540.0, ans=0.1 2026-09-24 06:01:54,595 INFO [train.py:1192] (1/2) Epoch 60, batch 50, loss[loss=0.2139, simple_loss=0.3302, pruned_loss=0.04878, over 24257.00 frames. ], tot_loss[loss=0.2652, simple_loss=0.3825, pruned_loss=0.07399, over 1082400.98 frames. ], batch size: 125, lr: 3.53e-03, grad_scale: 32.0 2026-09-24 06:01:58,132 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer1.prob, batch_count=188573.33333333334, ans=0.125 2026-09-24 06:02:11,230 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:02:15,001 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=188706.66666666666, ans=0.0 2026-09-24 06:02:19,925 INFO [train.py:1192] (1/2) Epoch 60, batch 100, loss[loss=0.2623, simple_loss=0.3727, pruned_loss=0.07599, over 24602.00 frames. ], tot_loss[loss=0.2672, simple_loss=0.3855, pruned_loss=0.07441, over 1915915.11 frames. ], batch size: 154, lr: 3.53e-03, grad_scale: 32.0 2026-09-24 06:02:21,510 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=188740.0, ans=0.2 2026-09-24 06:02:22,073 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.scale_min, batch_count=188740.0, ans=0.2 2026-09-24 06:02:29,777 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=188806.66666666666, ans=0.1 2026-09-24 06:02:33,607 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.858e+02 3.594e+02 3.944e+02 4.374e+02 5.505e+02, threshold=7.888e+02, percent-clipped=0.0 2026-09-24 06:02:33,729 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=188806.66666666666, ans=0.5 2026-09-24 06:02:35,107 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=188840.0, ans=0.0 2026-09-24 06:02:37,500 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=188840.0, ans=0.2 2026-09-24 06:02:44,870 INFO [train.py:1192] (1/2) Epoch 60, batch 150, loss[loss=0.1983, simple_loss=0.3136, pruned_loss=0.04152, over 24250.00 frames. ], tot_loss[loss=0.2621, simple_loss=0.3798, pruned_loss=0.07219, over 2561476.91 frames. ], batch size: 125, lr: 3.53e-03, grad_scale: 32.0 2026-09-24 06:03:05,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=189040.0, ans=0.025 2026-09-24 06:03:10,719 INFO [train.py:1192] (1/2) Epoch 60, batch 200, loss[loss=0.3008, simple_loss=0.4252, pruned_loss=0.08814, over 24222.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3786, pruned_loss=0.07177, over 3059962.81 frames. ], batch size: 257, lr: 3.52e-03, grad_scale: 32.0 2026-09-24 06:03:15,874 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=189106.66666666666, ans=0.125 2026-09-24 06:03:16,399 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=189106.66666666666, ans=0.0 2026-09-24 06:03:24,458 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.616e+02 3.276e+02 3.806e+02 4.339e+02 6.683e+02, threshold=7.612e+02, percent-clipped=0.0 2026-09-24 06:03:25,537 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=189173.33333333334, ans=0.125 2026-09-24 06:03:34,034 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=189206.66666666666, ans=0.0 2026-09-24 06:03:36,158 INFO [train.py:1192] (1/2) Epoch 60, batch 250, loss[loss=0.293, simple_loss=0.4167, pruned_loss=0.08469, over 24385.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3789, pruned_loss=0.07225, over 3443593.10 frames. ], batch size: 225, lr: 3.52e-03, grad_scale: 32.0 2026-09-24 06:03:46,650 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.13 vs. limit=6.0 2026-09-24 06:03:52,525 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=10.19 vs. limit=15.0 2026-09-24 06:03:55,009 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=8.99 vs. limit=12.0 2026-09-24 06:03:57,392 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=189373.33333333334, ans=0.125 2026-09-24 06:04:01,877 INFO [train.py:1192] (1/2) Epoch 60, batch 300, loss[loss=0.259, simple_loss=0.3892, pruned_loss=0.06439, over 24555.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3779, pruned_loss=0.07157, over 3756693.58 frames. ], batch size: 204, lr: 3.52e-03, grad_scale: 32.0 2026-09-24 06:04:15,631 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.845e+02 3.363e+02 3.702e+02 4.210e+02 5.489e+02, threshold=7.404e+02, percent-clipped=0.0 2026-09-24 06:04:27,313 INFO [train.py:1192] (1/2) Epoch 60, batch 350, loss[loss=0.2349, simple_loss=0.3474, pruned_loss=0.06119, over 24558.00 frames. ], tot_loss[loss=0.261, simple_loss=0.3783, pruned_loss=0.07179, over 3998089.36 frames. ], batch size: 137, lr: 3.52e-03, grad_scale: 32.0 2026-09-24 06:04:42,401 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=189673.33333333334, ans=0.0 2026-09-24 06:04:45,787 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=189673.33333333334, ans=0.125 2026-09-24 06:04:49,121 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=189706.66666666666, ans=0.125 2026-09-24 06:04:52,395 INFO [train.py:1192] (1/2) Epoch 60, batch 400, loss[loss=0.2728, simple_loss=0.388, pruned_loss=0.0788, over 24564.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3775, pruned_loss=0.07126, over 4181468.00 frames. ], batch size: 170, lr: 3.52e-03, grad_scale: 32.0 2026-09-24 06:04:53,482 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=189740.0, ans=0.95 2026-09-24 06:04:56,884 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:05:06,102 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.726e+02 3.383e+02 3.799e+02 4.527e+02 6.363e+02, threshold=7.598e+02, percent-clipped=0.0 2026-09-24 06:05:11,297 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=189840.0, ans=0.035 2026-09-24 06:05:13,307 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=189873.33333333334, ans=0.0 2026-09-24 06:05:14,572 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=189873.33333333334, ans=0.125 2026-09-24 06:05:18,082 INFO [train.py:1192] (1/2) Epoch 60, batch 450, loss[loss=0.2576, simple_loss=0.3869, pruned_loss=0.06417, over 24625.00 frames. ], tot_loss[loss=0.2609, simple_loss=0.378, pruned_loss=0.07186, over 4319466.60 frames. ], batch size: 175, lr: 3.52e-03, grad_scale: 32.0 2026-09-24 06:05:23,642 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=189940.0, ans=0.125 2026-09-24 06:05:26,439 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.11 vs. limit=15.0 2026-09-24 06:05:35,797 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=190006.66666666666, ans=0.125 2026-09-24 06:05:38,662 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=190040.0, ans=0.025 2026-09-24 06:05:39,532 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=190040.0, ans=0.125 2026-09-24 06:05:39,758 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.14 vs. limit=10.0 2026-09-24 06:05:42,720 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=190040.0, ans=0.1 2026-09-24 06:05:43,568 INFO [train.py:1192] (1/2) Epoch 60, batch 500, loss[loss=0.2544, simple_loss=0.3869, pruned_loss=0.06094, over 24509.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3768, pruned_loss=0.07154, over 4436939.36 frames. ], batch size: 218, lr: 3.52e-03, grad_scale: 32.0 2026-09-24 06:05:50,349 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=190106.66666666666, ans=0.0 2026-09-24 06:05:51,788 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=190106.66666666666, ans=0.09899494936611666 2026-09-24 06:05:57,319 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.393e+02 3.216e+02 3.484e+02 4.055e+02 5.792e+02, threshold=6.968e+02, percent-clipped=0.0 2026-09-24 06:06:00,861 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=190173.33333333334, ans=0.125 2026-09-24 06:06:02,566 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=190173.33333333334, ans=0.05 2026-09-24 06:06:09,332 INFO [train.py:1192] (1/2) Epoch 60, batch 550, loss[loss=0.3157, simple_loss=0.4371, pruned_loss=0.0972, over 24295.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3771, pruned_loss=0.07158, over 4521937.58 frames. ], batch size: 257, lr: 3.51e-03, grad_scale: 32.0 2026-09-24 06:06:14,703 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=190273.33333333334, ans=0.0 2026-09-24 06:06:15,579 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=190273.33333333334, ans=0.1 2026-09-24 06:06:20,277 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=190306.66666666666, ans=0.125 2026-09-24 06:06:21,188 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=190306.66666666666, ans=0.025 2026-09-24 06:06:26,353 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.00 vs. limit=12.0 2026-09-24 06:06:33,288 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=190373.33333333334, ans=10.0 2026-09-24 06:06:34,617 INFO [train.py:1192] (1/2) Epoch 60, batch 600, loss[loss=0.298, simple_loss=0.4245, pruned_loss=0.08574, over 24434.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.3776, pruned_loss=0.07169, over 4588928.66 frames. ], batch size: 235, lr: 3.51e-03, grad_scale: 32.0 2026-09-24 06:06:38,036 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=190406.66666666666, ans=0.0 2026-09-24 06:06:41,915 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=190440.0, ans=0.09899494936611666 2026-09-24 06:06:44,923 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=190473.33333333334, ans=0.0 2026-09-24 06:06:46,316 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=190473.33333333334, ans=0.04949747468305833 2026-09-24 06:06:48,511 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.621e+02 3.279e+02 3.656e+02 4.277e+02 6.065e+02, threshold=7.312e+02, percent-clipped=0.0 2026-09-24 06:07:00,284 INFO [train.py:1192] (1/2) Epoch 60, batch 650, loss[loss=0.2665, simple_loss=0.3702, pruned_loss=0.08143, over 24613.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.3759, pruned_loss=0.07077, over 4653848.00 frames. ], batch size: 154, lr: 3.51e-03, grad_scale: 32.0 2026-09-24 06:07:04,749 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=190573.33333333334, ans=0.125 2026-09-24 06:07:13,330 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=190640.0, ans=0.025 2026-09-24 06:07:19,265 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.79 vs. limit=15.0 2026-09-24 06:07:25,600 INFO [train.py:1192] (1/2) Epoch 60, batch 700, loss[loss=0.2503, simple_loss=0.3658, pruned_loss=0.06743, over 24549.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3768, pruned_loss=0.07094, over 4689688.66 frames. ], batch size: 158, lr: 3.51e-03, grad_scale: 32.0 2026-09-24 06:07:30,060 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=190740.0, ans=0.2 2026-09-24 06:07:30,066 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=190740.0, ans=0.125 2026-09-24 06:07:39,150 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=190806.66666666666, ans=0.0 2026-09-24 06:07:39,500 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.842e+02 3.414e+02 3.718e+02 4.175e+02 5.828e+02, threshold=7.437e+02, percent-clipped=0.0 2026-09-24 06:07:50,270 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=190906.66666666666, ans=0.125 2026-09-24 06:07:50,671 INFO [train.py:1192] (1/2) Epoch 60, batch 750, loss[loss=0.2545, simple_loss=0.3792, pruned_loss=0.06494, over 24625.00 frames. ], tot_loss[loss=0.2597, simple_loss=0.3767, pruned_loss=0.07129, over 4725793.25 frames. ], batch size: 175, lr: 3.51e-03, grad_scale: 32.0 2026-09-24 06:07:56,899 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=190940.0, ans=0.025 2026-09-24 06:08:07,876 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer_ff3.min_abs, batch_count=191006.66666666666, ans=0.2 2026-09-24 06:08:12,333 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=191040.0, ans=0.025 2026-09-24 06:08:15,835 INFO [train.py:1192] (1/2) Epoch 60, batch 800, loss[loss=0.1962, simple_loss=0.3172, pruned_loss=0.0376, over 24519.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.3761, pruned_loss=0.07076, over 4751785.30 frames. ], batch size: 137, lr: 3.51e-03, grad_scale: 32.0 2026-09-24 06:08:23,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=191106.66666666666, ans=0.0 2026-09-24 06:08:27,724 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=191140.0, ans=0.125 2026-09-24 06:08:29,585 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.454e+02 3.407e+02 3.822e+02 4.313e+02 6.694e+02, threshold=7.644e+02, percent-clipped=0.0 2026-09-24 06:08:41,209 INFO [train.py:1192] (1/2) Epoch 60, batch 850, loss[loss=0.2927, simple_loss=0.414, pruned_loss=0.08572, over 24541.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3758, pruned_loss=0.07073, over 4770668.55 frames. ], batch size: 204, lr: 3.50e-03, grad_scale: 32.0 2026-09-24 06:08:43,802 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=191240.0, ans=0.025 2026-09-24 06:09:01,057 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=191340.0, ans=0.0 2026-09-24 06:09:01,527 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=191373.33333333334, ans=0.125 2026-09-24 06:09:01,884 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=191373.33333333334, ans=0.125 2026-09-24 06:09:06,696 INFO [train.py:1192] (1/2) Epoch 60, batch 900, loss[loss=0.2122, simple_loss=0.331, pruned_loss=0.0467, over 24552.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3762, pruned_loss=0.07081, over 4781681.69 frames. ], batch size: 137, lr: 3.50e-03, grad_scale: 32.0 2026-09-24 06:09:19,813 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.616e+02 3.471e+02 3.853e+02 4.480e+02 6.109e+02, threshold=7.707e+02, percent-clipped=0.0 2026-09-24 06:09:26,318 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=191540.0, ans=0.1 2026-09-24 06:09:31,555 INFO [train.py:1192] (1/2) Epoch 60, batch 950, loss[loss=0.3487, simple_loss=0.4201, pruned_loss=0.1387, over 11706.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3755, pruned_loss=0.07224, over 4717156.59 frames. ], batch size: 333, lr: 3.50e-03, grad_scale: 32.0 2026-09-24 06:09:41,445 INFO [train.py:1192] (1/2) Epoch 61, batch 0, loss[loss=0.2184, simple_loss=0.3381, pruned_loss=0.04937, over 24570.00 frames. ], tot_loss[loss=0.2184, simple_loss=0.3381, pruned_loss=0.04937, over 24570.00 frames. ], batch size: 137, lr: 3.47e-03, grad_scale: 32.0 2026-09-24 06:09:41,446 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 06:09:51,750 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([3.3274, 2.4252, 3.8247, 1.5378], device='cuda:1') 2026-09-24 06:09:53,105 INFO [train.py:1224] (1/2) Epoch 61, validation: loss=0.17, simple_loss=0.2887, pruned_loss=0.02562, over 2564189.00 frames. 2026-09-24 06:09:53,105 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 06:10:04,360 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=191666.66666666666, ans=0.125 2026-09-24 06:10:07,002 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.51 vs. limit=10.0 2026-09-24 06:10:18,025 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=191766.66666666666, ans=0.1 2026-09-24 06:10:18,462 INFO [train.py:1192] (1/2) Epoch 61, batch 50, loss[loss=0.2036, simple_loss=0.3165, pruned_loss=0.04533, over 24292.00 frames. ], tot_loss[loss=0.2617, simple_loss=0.3791, pruned_loss=0.07213, over 1080467.66 frames. ], batch size: 125, lr: 3.47e-03, grad_scale: 32.0 2026-09-24 06:10:21,191 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=13.96 vs. limit=22.5 2026-09-24 06:10:28,374 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.687e+02 3.624e+02 4.017e+02 4.823e+02 1.136e+03, threshold=8.035e+02, percent-clipped=1.0 2026-09-24 06:10:37,362 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=191866.66666666666, ans=0.2 2026-09-24 06:10:37,806 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=191866.66666666666, ans=0.125 2026-09-24 06:10:38,514 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=191900.0, ans=0.0 2026-09-24 06:10:44,281 INFO [train.py:1192] (1/2) Epoch 61, batch 100, loss[loss=0.2453, simple_loss=0.3629, pruned_loss=0.06383, over 24601.00 frames. ], tot_loss[loss=0.2668, simple_loss=0.3853, pruned_loss=0.07412, over 1915106.11 frames. ], batch size: 154, lr: 3.47e-03, grad_scale: 32.0 2026-09-24 06:10:57,524 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.29 vs. limit=15.0 2026-09-24 06:11:10,555 INFO [train.py:1192] (1/2) Epoch 61, batch 150, loss[loss=0.2092, simple_loss=0.3213, pruned_loss=0.04861, over 24258.00 frames. ], tot_loss[loss=0.2639, simple_loss=0.3814, pruned_loss=0.07316, over 2560804.22 frames. ], batch size: 125, lr: 3.47e-03, grad_scale: 32.0 2026-09-24 06:11:13,223 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=192100.0, ans=0.1 2026-09-24 06:11:13,621 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=192100.0, ans=0.1 2026-09-24 06:11:19,765 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.514e+02 3.411e+02 3.849e+02 4.309e+02 6.280e+02, threshold=7.698e+02, percent-clipped=0.0 2026-09-24 06:11:23,544 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.95 vs. limit=15.0 2026-09-24 06:11:35,626 INFO [train.py:1192] (1/2) Epoch 61, batch 200, loss[loss=0.2987, simple_loss=0.425, pruned_loss=0.08621, over 24199.00 frames. ], tot_loss[loss=0.2607, simple_loss=0.3784, pruned_loss=0.07145, over 3058512.30 frames. ], batch size: 257, lr: 3.47e-03, grad_scale: 32.0 2026-09-24 06:11:39,578 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:11:52,598 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=192366.66666666666, ans=0.125 2026-09-24 06:11:59,182 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=192400.0, ans=0.0 2026-09-24 06:12:01,365 INFO [train.py:1192] (1/2) Epoch 61, batch 250, loss[loss=0.2928, simple_loss=0.4163, pruned_loss=0.08462, over 24380.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3771, pruned_loss=0.07062, over 3443163.95 frames. ], batch size: 225, lr: 3.46e-03, grad_scale: 32.0 2026-09-24 06:12:10,444 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=192466.66666666666, ans=0.125 2026-09-24 06:12:11,212 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.616e+02 3.434e+02 3.850e+02 4.566e+02 6.399e+02, threshold=7.699e+02, percent-clipped=0.0 2026-09-24 06:12:16,448 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=192533.33333333334, ans=0.1 2026-09-24 06:12:16,598 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.61 vs. limit=15.0 2026-09-24 06:12:26,500 INFO [train.py:1192] (1/2) Epoch 61, batch 300, loss[loss=0.2892, simple_loss=0.414, pruned_loss=0.08225, over 24553.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.377, pruned_loss=0.07086, over 3756008.83 frames. ], batch size: 204, lr: 3.46e-03, grad_scale: 32.0 2026-09-24 06:12:27,716 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=192600.0, ans=0.1 2026-09-24 06:12:50,353 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=192733.33333333334, ans=0.125 2026-09-24 06:12:52,166 INFO [train.py:1192] (1/2) Epoch 61, batch 350, loss[loss=0.2263, simple_loss=0.337, pruned_loss=0.05777, over 24576.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.3777, pruned_loss=0.07149, over 3994488.95 frames. ], batch size: 137, lr: 3.46e-03, grad_scale: 32.0 2026-09-24 06:12:59,186 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.dropout.p, batch_count=192800.0, ans=0.1 2026-09-24 06:12:59,431 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=17.83 vs. limit=22.5 2026-09-24 06:13:01,848 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.749e+02 3.413e+02 3.727e+02 4.235e+02 7.272e+02, threshold=7.455e+02, percent-clipped=0.0 2026-09-24 06:13:06,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=192866.66666666666, ans=0.1 2026-09-24 06:13:08,983 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.min_positive, batch_count=192866.66666666666, ans=0.025 2026-09-24 06:13:17,078 INFO [train.py:1192] (1/2) Epoch 61, batch 400, loss[loss=0.2598, simple_loss=0.3749, pruned_loss=0.07231, over 24551.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3773, pruned_loss=0.07113, over 4177990.76 frames. ], batch size: 170, lr: 3.46e-03, grad_scale: 32.0 2026-09-24 06:13:36,897 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=193033.33333333334, ans=0.0 2026-09-24 06:13:43,515 INFO [train.py:1192] (1/2) Epoch 61, batch 450, loss[loss=0.2894, simple_loss=0.4034, pruned_loss=0.08773, over 24623.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.3776, pruned_loss=0.07157, over 4319487.17 frames. ], batch size: 175, lr: 3.46e-03, grad_scale: 32.0 2026-09-24 06:13:46,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=193100.0, ans=0.05 2026-09-24 06:13:47,221 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=4.024e-02 2026-09-24 06:13:48,539 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=193133.33333333334, ans=0.2 2026-09-24 06:13:49,101 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=193133.33333333334, ans=0.2 2026-09-24 06:13:52,171 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=193133.33333333334, ans=0.2 2026-09-24 06:13:52,184 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_na.min_abs, batch_count=193133.33333333334, ans=0.02 2026-09-24 06:13:53,054 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.637e+02 3.431e+02 3.745e+02 4.317e+02 6.858e+02, threshold=7.490e+02, percent-clipped=0.0 2026-09-24 06:13:56,863 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=193166.66666666666, ans=0.0 2026-09-24 06:13:59,284 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=193200.0, ans=0.125 2026-09-24 06:14:07,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=193233.33333333334, ans=0.0 2026-09-24 06:14:09,330 INFO [train.py:1192] (1/2) Epoch 61, batch 500, loss[loss=0.3197, simple_loss=0.4348, pruned_loss=0.1023, over 24503.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3767, pruned_loss=0.07154, over 4437107.65 frames. ], batch size: 218, lr: 3.46e-03, grad_scale: 32.0 2026-09-24 06:14:12,509 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.23 vs. limit=10.0 2026-09-24 06:14:26,344 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=193366.66666666666, ans=0.025 2026-09-24 06:14:34,750 INFO [train.py:1192] (1/2) Epoch 61, batch 550, loss[loss=0.2668, simple_loss=0.394, pruned_loss=0.0698, over 24257.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.377, pruned_loss=0.07139, over 4521962.43 frames. ], batch size: 257, lr: 3.46e-03, grad_scale: 32.0 2026-09-24 06:14:35,335 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=193433.33333333334, ans=0.125 2026-09-24 06:14:35,852 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=193433.33333333334, ans=0.1 2026-09-24 06:14:44,570 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.586e+02 3.162e+02 3.590e+02 3.957e+02 6.076e+02, threshold=7.179e+02, percent-clipped=0.0 2026-09-24 06:15:00,352 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=193600.0, ans=0.125 2026-09-24 06:15:00,776 INFO [train.py:1192] (1/2) Epoch 61, batch 600, loss[loss=0.2919, simple_loss=0.4088, pruned_loss=0.08746, over 24328.00 frames. ], tot_loss[loss=0.2608, simple_loss=0.3778, pruned_loss=0.07192, over 4589188.88 frames. ], batch size: 234, lr: 3.45e-03, grad_scale: 32.0 2026-09-24 06:15:14,306 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer2.prob, batch_count=193666.66666666666, ans=0.125 2026-09-24 06:15:15,851 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=193700.0, ans=0.125 2026-09-24 06:15:18,722 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=7.91 vs. limit=15.0 2026-09-24 06:15:26,176 INFO [train.py:1192] (1/2) Epoch 61, batch 650, loss[loss=0.2894, simple_loss=0.3902, pruned_loss=0.09427, over 24637.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3769, pruned_loss=0.07133, over 4654168.46 frames. ], batch size: 154, lr: 3.45e-03, grad_scale: 16.0 2026-09-24 06:15:30,027 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=193766.66666666666, ans=0.125 2026-09-24 06:15:35,190 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=193800.0, ans=0.2 2026-09-24 06:15:36,529 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.685e+02 3.370e+02 3.618e+02 4.011e+02 6.223e+02, threshold=7.236e+02, percent-clipped=0.0 2026-09-24 06:15:45,388 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.77 vs. limit=15.0 2026-09-24 06:15:52,128 INFO [train.py:1192] (1/2) Epoch 61, batch 700, loss[loss=0.2593, simple_loss=0.3755, pruned_loss=0.07159, over 24556.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3775, pruned_loss=0.07122, over 4688814.95 frames. ], batch size: 158, lr: 3.45e-03, grad_scale: 16.0 2026-09-24 06:15:56,691 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=193933.33333333334, ans=0.125 2026-09-24 06:16:01,119 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.92 vs. limit=10.0 2026-09-24 06:16:02,600 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=194000.0, ans=0.125 2026-09-24 06:16:06,819 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=194000.0, ans=0.0 2026-09-24 06:16:07,377 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=194033.33333333334, ans=0.0 2026-09-24 06:16:10,865 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=9.75 vs. limit=15.0 2026-09-24 06:16:17,965 INFO [train.py:1192] (1/2) Epoch 61, batch 750, loss[loss=0.2634, simple_loss=0.3844, pruned_loss=0.07114, over 24624.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3765, pruned_loss=0.07111, over 4725499.31 frames. ], batch size: 175, lr: 3.45e-03, grad_scale: 16.0 2026-09-24 06:16:28,057 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.911e+02 3.503e+02 4.008e+02 4.518e+02 6.214e+02, threshold=8.016e+02, percent-clipped=0.0 2026-09-24 06:16:32,530 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=194200.0, ans=0.125 2026-09-24 06:16:34,832 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:16:43,080 INFO [train.py:1192] (1/2) Epoch 61, batch 800, loss[loss=0.211, simple_loss=0.3297, pruned_loss=0.04618, over 24546.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3761, pruned_loss=0.07091, over 4748117.15 frames. ], batch size: 137, lr: 3.45e-03, grad_scale: 32.0 2026-09-24 06:16:44,202 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=194266.66666666666, ans=0.0 2026-09-24 06:16:49,559 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.49 vs. limit=22.5 2026-09-24 06:16:49,808 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=194300.0, ans=0.125 2026-09-24 06:16:50,324 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=194300.0, ans=0.2 2026-09-24 06:16:52,777 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=194333.33333333334, ans=0.025 2026-09-24 06:16:54,160 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=194333.33333333334, ans=0.025 2026-09-24 06:17:06,651 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=194400.0, ans=0.09899494936611666 2026-09-24 06:17:08,615 INFO [train.py:1192] (1/2) Epoch 61, batch 850, loss[loss=0.3131, simple_loss=0.4314, pruned_loss=0.09737, over 24526.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3762, pruned_loss=0.07123, over 4767079.74 frames. ], batch size: 204, lr: 3.45e-03, grad_scale: 32.0 2026-09-24 06:17:19,023 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.850e+02 3.451e+02 3.900e+02 4.481e+02 6.204e+02, threshold=7.800e+02, percent-clipped=0.0 2026-09-24 06:17:20,627 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten.whitening_limit, batch_count=194500.0, ans=15.0 2026-09-24 06:17:34,281 INFO [train.py:1192] (1/2) Epoch 61, batch 900, loss[loss=0.2019, simple_loss=0.3227, pruned_loss=0.04055, over 24589.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3765, pruned_loss=0.0711, over 4778670.49 frames. ], batch size: 137, lr: 3.45e-03, grad_scale: 32.0 2026-09-24 06:17:59,267 INFO [train.py:1192] (1/2) Epoch 61, batch 950, loss[loss=0.3602, simple_loss=0.4322, pruned_loss=0.1441, over 11510.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3751, pruned_loss=0.07179, over 4709755.22 frames. ], batch size: 333, lr: 3.44e-03, grad_scale: 32.0 2026-09-24 06:18:02,218 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=194766.66666666666, ans=0.0 2026-09-24 06:18:08,384 INFO [train.py:1192] (1/2) Epoch 62, batch 0, loss[loss=0.2331, simple_loss=0.3492, pruned_loss=0.05845, over 24590.00 frames. ], tot_loss[loss=0.2331, simple_loss=0.3492, pruned_loss=0.05845, over 24590.00 frames. ], batch size: 137, lr: 3.42e-03, grad_scale: 32.0 2026-09-24 06:18:08,384 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 06:18:20,016 INFO [train.py:1224] (1/2) Epoch 62, validation: loss=0.1718, simple_loss=0.29, pruned_loss=0.02679, over 2564189.00 frames. 2026-09-24 06:18:20,016 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 06:18:22,197 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.82 vs. limit=6.0 2026-09-24 06:18:25,909 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.504e+02 3.508e+02 3.896e+02 4.877e+02 6.832e+02, threshold=7.792e+02, percent-clipped=0.0 2026-09-24 06:18:29,126 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=194826.66666666666, ans=0.125 2026-09-24 06:18:33,448 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.65 vs. limit=15.0 2026-09-24 06:18:46,003 INFO [train.py:1192] (1/2) Epoch 62, batch 50, loss[loss=0.2124, simple_loss=0.3272, pruned_loss=0.04881, over 24303.00 frames. ], tot_loss[loss=0.2676, simple_loss=0.3837, pruned_loss=0.07581, over 1082348.64 frames. ], batch size: 125, lr: 3.41e-03, grad_scale: 32.0 2026-09-24 06:19:00,675 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=195060.0, ans=0.125 2026-09-24 06:19:10,907 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=195093.33333333334, ans=0.0 2026-09-24 06:19:11,752 INFO [train.py:1192] (1/2) Epoch 62, batch 100, loss[loss=0.2613, simple_loss=0.3721, pruned_loss=0.07525, over 24608.00 frames. ], tot_loss[loss=0.2708, simple_loss=0.3884, pruned_loss=0.07665, over 1916188.95 frames. ], batch size: 154, lr: 3.41e-03, grad_scale: 32.0 2026-09-24 06:19:17,832 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.792e+02 3.477e+02 3.857e+02 4.199e+02 8.261e+02, threshold=7.714e+02, percent-clipped=1.0 2026-09-24 06:19:23,037 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=195193.33333333334, ans=0.1 2026-09-24 06:19:25,862 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=195193.33333333334, ans=0.2 2026-09-24 06:19:31,950 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=195260.0, ans=0.0 2026-09-24 06:19:36,620 INFO [train.py:1192] (1/2) Epoch 62, batch 150, loss[loss=0.2057, simple_loss=0.316, pruned_loss=0.04774, over 24282.00 frames. ], tot_loss[loss=0.263, simple_loss=0.3807, pruned_loss=0.07269, over 2561808.01 frames. ], batch size: 125, lr: 3.41e-03, grad_scale: 32.0 2026-09-24 06:19:48,157 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=195360.0, ans=0.0 2026-09-24 06:19:50,479 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=195360.0, ans=0.09899494936611666 2026-09-24 06:19:56,151 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.40 vs. limit=6.0 2026-09-24 06:19:57,191 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=195426.66666666666, ans=0.2 2026-09-24 06:19:58,168 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=195426.66666666666, ans=0.0 2026-09-24 06:20:00,226 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=10.12 vs. limit=12.0 2026-09-24 06:20:01,477 INFO [train.py:1192] (1/2) Epoch 62, batch 200, loss[loss=0.2832, simple_loss=0.4147, pruned_loss=0.07582, over 24205.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.3784, pruned_loss=0.07116, over 3060175.12 frames. ], batch size: 257, lr: 3.41e-03, grad_scale: 32.0 2026-09-24 06:20:01,998 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.57 vs. limit=15.0 2026-09-24 06:20:07,471 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.623e+02 3.457e+02 3.962e+02 4.499e+02 6.381e+02, threshold=7.924e+02, percent-clipped=0.0 2026-09-24 06:20:15,106 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=195526.66666666666, ans=0.125 2026-09-24 06:20:16,189 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=195526.66666666666, ans=0.1 2026-09-24 06:20:17,301 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=195560.0, ans=0.125 2026-09-24 06:20:27,548 INFO [train.py:1192] (1/2) Epoch 62, batch 250, loss[loss=0.2895, simple_loss=0.4175, pruned_loss=0.08071, over 24367.00 frames. ], tot_loss[loss=0.2605, simple_loss=0.378, pruned_loss=0.07146, over 3445548.16 frames. ], batch size: 225, lr: 3.41e-03, grad_scale: 32.0 2026-09-24 06:20:30,595 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=195626.66666666666, ans=0.07 2026-09-24 06:20:32,049 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=13.25 vs. limit=22.5 2026-09-24 06:20:34,971 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=195660.0, ans=0.125 2026-09-24 06:20:52,590 INFO [train.py:1192] (1/2) Epoch 62, batch 300, loss[loss=0.2945, simple_loss=0.4154, pruned_loss=0.08674, over 24498.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3776, pruned_loss=0.07129, over 3758080.82 frames. ], batch size: 204, lr: 3.41e-03, grad_scale: 32.0 2026-09-24 06:20:58,304 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.643e+02 3.333e+02 3.738e+02 4.237e+02 6.549e+02, threshold=7.477e+02, percent-clipped=0.0 2026-09-24 06:21:01,566 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.01 vs. limit=15.0 2026-09-24 06:21:04,607 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=195860.0, ans=0.125 2026-09-24 06:21:09,988 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=195893.33333333334, ans=0.125 2026-09-24 06:21:17,885 INFO [train.py:1192] (1/2) Epoch 62, batch 350, loss[loss=0.2244, simple_loss=0.3412, pruned_loss=0.05382, over 24561.00 frames. ], tot_loss[loss=0.2614, simple_loss=0.3787, pruned_loss=0.07199, over 3998948.11 frames. ], batch size: 137, lr: 3.41e-03, grad_scale: 32.0 2026-09-24 06:21:32,503 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.72 vs. limit=15.0 2026-09-24 06:21:36,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=196060.0, ans=0.125 2026-09-24 06:21:43,346 INFO [train.py:1192] (1/2) Epoch 62, batch 400, loss[loss=0.2619, simple_loss=0.3798, pruned_loss=0.07197, over 24564.00 frames. ], tot_loss[loss=0.2598, simple_loss=0.3772, pruned_loss=0.07121, over 4180530.08 frames. ], batch size: 170, lr: 3.40e-03, grad_scale: 32.0 2026-09-24 06:21:46,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=196126.66666666666, ans=0.025 2026-09-24 06:21:49,067 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.838e+02 3.389e+02 3.914e+02 4.523e+02 6.865e+02, threshold=7.829e+02, percent-clipped=0.0 2026-09-24 06:21:49,194 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=196160.0, ans=0.125 2026-09-24 06:21:56,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=196193.33333333334, ans=0.0 2026-09-24 06:22:03,422 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:22:06,216 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=196260.0, ans=0.0 2026-09-24 06:22:08,539 INFO [train.py:1192] (1/2) Epoch 62, batch 450, loss[loss=0.2773, simple_loss=0.3927, pruned_loss=0.08098, over 24638.00 frames. ], tot_loss[loss=0.2604, simple_loss=0.3775, pruned_loss=0.0716, over 4321076.48 frames. ], batch size: 175, lr: 3.40e-03, grad_scale: 32.0 2026-09-24 06:22:19,035 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=196360.0, ans=0.2 2026-09-24 06:22:22,301 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=196360.0, ans=0.125 2026-09-24 06:22:33,888 INFO [train.py:1192] (1/2) Epoch 62, batch 500, loss[loss=0.2929, simple_loss=0.4075, pruned_loss=0.08919, over 24492.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3762, pruned_loss=0.07101, over 4438710.03 frames. ], batch size: 218, lr: 3.40e-03, grad_scale: 32.0 2026-09-24 06:22:34,644 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.43 vs. limit=8.0 2026-09-24 06:22:40,093 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.696e+02 3.424e+02 3.778e+02 4.258e+02 6.180e+02, threshold=7.556e+02, percent-clipped=0.0 2026-09-24 06:22:42,123 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=196493.33333333334, ans=0.125 2026-09-24 06:22:45,834 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=196526.66666666666, ans=0.125 2026-09-24 06:22:59,923 INFO [train.py:1192] (1/2) Epoch 62, batch 550, loss[loss=0.2878, simple_loss=0.4136, pruned_loss=0.08098, over 24229.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.377, pruned_loss=0.07139, over 4523260.63 frames. ], batch size: 257, lr: 3.40e-03, grad_scale: 32.0 2026-09-24 06:23:04,771 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=196660.0, ans=0.125 2026-09-24 06:23:09,629 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=196693.33333333334, ans=0.2 2026-09-24 06:23:19,689 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=196760.0, ans=0.125 2026-09-24 06:23:23,000 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=196760.0, ans=0.1 2026-09-24 06:23:25,012 INFO [train.py:1192] (1/2) Epoch 62, batch 600, loss[loss=0.2673, simple_loss=0.3961, pruned_loss=0.06925, over 24305.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3769, pruned_loss=0.07089, over 4590313.17 frames. ], batch size: 234, lr: 3.40e-03, grad_scale: 32.0 2026-09-24 06:23:26,646 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=196793.33333333334, ans=0.125 2026-09-24 06:23:30,926 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.705e+02 3.299e+02 3.732e+02 4.086e+02 5.579e+02, threshold=7.464e+02, percent-clipped=0.0 2026-09-24 06:23:31,010 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=196826.66666666666, ans=0.125 2026-09-24 06:23:33,652 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=7.10 vs. limit=15.0 2026-09-24 06:23:50,018 INFO [train.py:1192] (1/2) Epoch 62, batch 650, loss[loss=0.2335, simple_loss=0.3548, pruned_loss=0.05609, over 24599.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3761, pruned_loss=0.0703, over 4654545.83 frames. ], batch size: 154, lr: 3.40e-03, grad_scale: 32.0 2026-09-24 06:23:50,125 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=196960.0, ans=0.1 2026-09-24 06:23:58,990 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=196993.33333333334, ans=0.0 2026-09-24 06:24:11,098 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.68 vs. limit=15.0 2026-09-24 06:24:15,511 INFO [train.py:1192] (1/2) Epoch 62, batch 700, loss[loss=0.2495, simple_loss=0.364, pruned_loss=0.06744, over 24561.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3777, pruned_loss=0.07118, over 4689978.38 frames. ], batch size: 158, lr: 3.40e-03, grad_scale: 32.0 2026-09-24 06:24:16,049 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=197126.66666666666, ans=0.125 2026-09-24 06:24:19,321 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=197126.66666666666, ans=0.2 2026-09-24 06:24:19,332 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=197126.66666666666, ans=0.125 2026-09-24 06:24:21,329 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.33 vs. limit=15.0 2026-09-24 06:24:21,540 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.887e+02 3.370e+02 3.867e+02 4.370e+02 6.944e+02, threshold=7.734e+02, percent-clipped=0.0 2026-09-24 06:24:24,340 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=197160.0, ans=0.0 2026-09-24 06:24:24,967 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.20 vs. limit=12.0 2026-09-24 06:24:27,481 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=197193.33333333334, ans=0.2 2026-09-24 06:24:40,635 INFO [train.py:1192] (1/2) Epoch 62, batch 750, loss[loss=0.2722, simple_loss=0.3957, pruned_loss=0.07437, over 24624.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3766, pruned_loss=0.07071, over 4726143.36 frames. ], batch size: 175, lr: 3.39e-03, grad_scale: 32.0 2026-09-24 06:24:50,068 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=197326.66666666666, ans=0.0 2026-09-24 06:25:04,897 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=197426.66666666666, ans=0.125 2026-09-24 06:25:06,419 INFO [train.py:1192] (1/2) Epoch 62, batch 800, loss[loss=0.2224, simple_loss=0.3348, pruned_loss=0.05496, over 24550.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3763, pruned_loss=0.07075, over 4751842.33 frames. ], batch size: 137, lr: 3.39e-03, grad_scale: 32.0 2026-09-24 06:25:08,204 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.15 vs. limit=10.0 2026-09-24 06:25:08,727 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.73 vs. limit=15.0 2026-09-24 06:25:12,027 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=197493.33333333334, ans=0.2 2026-09-24 06:25:12,833 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.689e+02 3.485e+02 3.958e+02 4.384e+02 6.565e+02, threshold=7.916e+02, percent-clipped=0.0 2026-09-24 06:25:14,957 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=197493.33333333334, ans=0.125 2026-09-24 06:25:17,566 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=11.14 vs. limit=22.5 2026-09-24 06:25:31,374 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=197626.66666666666, ans=0.04949747468305833 2026-09-24 06:25:31,745 INFO [train.py:1192] (1/2) Epoch 62, batch 850, loss[loss=0.2723, simple_loss=0.4, pruned_loss=0.07231, over 24551.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3757, pruned_loss=0.07054, over 4770380.84 frames. ], batch size: 204, lr: 3.39e-03, grad_scale: 32.0 2026-09-24 06:25:32,852 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=197626.66666666666, ans=0.1 2026-09-24 06:25:39,809 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=197660.0, ans=0.125 2026-09-24 06:25:52,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=197760.0, ans=0.1 2026-09-24 06:25:53,573 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:25:54,419 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=197760.0, ans=0.2 2026-09-24 06:25:57,804 INFO [train.py:1192] (1/2) Epoch 62, batch 900, loss[loss=0.2096, simple_loss=0.3297, pruned_loss=0.04475, over 24542.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.3761, pruned_loss=0.07071, over 4781174.12 frames. ], batch size: 137, lr: 3.39e-03, grad_scale: 32.0 2026-09-24 06:25:58,255 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=197793.33333333334, ans=0.025 2026-09-24 06:26:01,361 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=13.05 vs. limit=15.0 2026-09-24 06:26:04,314 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.853e+02 3.460e+02 4.122e+02 4.592e+02 8.345e+02, threshold=8.244e+02, percent-clipped=1.0 2026-09-24 06:26:09,115 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=197860.0, ans=0.0 2026-09-24 06:26:19,465 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten.whitening_limit, batch_count=197926.66666666666, ans=15.0 2026-09-24 06:26:23,048 INFO [train.py:1192] (1/2) Epoch 62, batch 950, loss[loss=0.4066, simple_loss=0.469, pruned_loss=0.1722, over 10746.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.3747, pruned_loss=0.07143, over 4714028.29 frames. ], batch size: 333, lr: 3.39e-03, grad_scale: 32.0 2026-09-24 06:26:33,329 INFO [train.py:1192] (1/2) Epoch 63, batch 0, loss[loss=0.2053, simple_loss=0.3303, pruned_loss=0.04016, over 24558.00 frames. ], tot_loss[loss=0.2053, simple_loss=0.3303, pruned_loss=0.04016, over 24558.00 frames. ], batch size: 137, lr: 3.36e-03, grad_scale: 32.0 2026-09-24 06:26:33,329 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 06:26:44,985 INFO [train.py:1224] (1/2) Epoch 63, validation: loss=0.1706, simple_loss=0.2889, pruned_loss=0.02612, over 2564189.00 frames. 2026-09-24 06:26:44,985 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 06:26:47,921 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=197986.66666666666, ans=0.07 2026-09-24 06:26:55,444 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=198053.33333333334, ans=0.125 2026-09-24 06:27:00,852 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=198086.66666666666, ans=0.1 2026-09-24 06:27:03,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=198086.66666666666, ans=0.2 2026-09-24 06:27:08,587 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=198120.0, ans=0.2 2026-09-24 06:27:10,463 INFO [train.py:1192] (1/2) Epoch 63, batch 50, loss[loss=0.1943, simple_loss=0.3116, pruned_loss=0.0385, over 24303.00 frames. ], tot_loss[loss=0.2595, simple_loss=0.3777, pruned_loss=0.07067, over 1083394.90 frames. ], batch size: 125, lr: 3.36e-03, grad_scale: 32.0 2026-09-24 06:27:12,233 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.726e+02 3.514e+02 4.081e+02 4.504e+02 6.153e+02, threshold=8.162e+02, percent-clipped=0.0 2026-09-24 06:27:13,254 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=198153.33333333334, ans=0.1 2026-09-24 06:27:14,265 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.out_combiner.scale_min, batch_count=198153.33333333334, ans=0.2 2026-09-24 06:27:14,703 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=198186.66666666666, ans=0.125 2026-09-24 06:27:24,881 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=198220.0, ans=0.125 2026-09-24 06:27:31,891 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.46 vs. limit=15.0 2026-09-24 06:27:35,848 INFO [train.py:1192] (1/2) Epoch 63, batch 100, loss[loss=0.241, simple_loss=0.3596, pruned_loss=0.06121, over 24622.00 frames. ], tot_loss[loss=0.2619, simple_loss=0.382, pruned_loss=0.07097, over 1915574.04 frames. ], batch size: 154, lr: 3.36e-03, grad_scale: 32.0 2026-09-24 06:28:01,247 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=198486.66666666666, ans=0.125 2026-09-24 06:28:01,703 INFO [train.py:1192] (1/2) Epoch 63, batch 150, loss[loss=0.2289, simple_loss=0.3388, pruned_loss=0.05948, over 24212.00 frames. ], tot_loss[loss=0.2599, simple_loss=0.3785, pruned_loss=0.0707, over 2560891.46 frames. ], batch size: 125, lr: 3.36e-03, grad_scale: 32.0 2026-09-24 06:28:03,634 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.814e+02 3.458e+02 3.872e+02 4.270e+02 5.583e+02, threshold=7.744e+02, percent-clipped=0.0 2026-09-24 06:28:04,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=198486.66666666666, ans=0.0 2026-09-24 06:28:16,403 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.54 vs. limit=22.5 2026-09-24 06:28:19,932 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=198586.66666666666, ans=0.2 2026-09-24 06:28:26,026 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=198620.0, ans=0.125 2026-09-24 06:28:26,045 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=198620.0, ans=0.125 2026-09-24 06:28:27,414 INFO [train.py:1192] (1/2) Epoch 63, batch 200, loss[loss=0.324, simple_loss=0.4369, pruned_loss=0.1056, over 24229.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3769, pruned_loss=0.07048, over 3059574.36 frames. ], batch size: 257, lr: 3.36e-03, grad_scale: 32.0 2026-09-24 06:28:31,619 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=198653.33333333334, ans=0.0 2026-09-24 06:28:37,659 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=198720.0, ans=0.1 2026-09-24 06:28:38,215 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=198720.0, ans=0.125 2026-09-24 06:28:51,990 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.23 vs. limit=15.0 2026-09-24 06:28:52,420 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=198820.0, ans=0.2 2026-09-24 06:28:52,838 INFO [train.py:1192] (1/2) Epoch 63, batch 250, loss[loss=0.2958, simple_loss=0.4189, pruned_loss=0.08635, over 24362.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3761, pruned_loss=0.07033, over 3443848.83 frames. ], batch size: 225, lr: 3.35e-03, grad_scale: 16.0 2026-09-24 06:28:55,580 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.737e+02 3.490e+02 3.926e+02 4.577e+02 6.260e+02, threshold=7.853e+02, percent-clipped=0.0 2026-09-24 06:28:56,402 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.22 vs. limit=15.0 2026-09-24 06:29:05,031 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=198886.66666666666, ans=0.2 2026-09-24 06:29:06,874 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=198886.66666666666, ans=0.1 2026-09-24 06:29:09,216 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=198920.0, ans=0.95 2026-09-24 06:29:15,610 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=198953.33333333334, ans=0.125 2026-09-24 06:29:18,058 INFO [train.py:1192] (1/2) Epoch 63, batch 300, loss[loss=0.2762, simple_loss=0.4035, pruned_loss=0.07442, over 24551.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3755, pruned_loss=0.0699, over 3756649.27 frames. ], batch size: 204, lr: 3.35e-03, grad_scale: 16.0 2026-09-24 06:29:19,510 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=198986.66666666666, ans=0.0 2026-09-24 06:29:36,076 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=199086.66666666666, ans=0.125 2026-09-24 06:29:43,382 INFO [train.py:1192] (1/2) Epoch 63, batch 350, loss[loss=0.2226, simple_loss=0.335, pruned_loss=0.05509, over 24573.00 frames. ], tot_loss[loss=0.2585, simple_loss=0.3765, pruned_loss=0.07027, over 3997441.61 frames. ], batch size: 137, lr: 3.35e-03, grad_scale: 16.0 2026-09-24 06:29:45,836 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.672e+02 3.255e+02 3.680e+02 4.171e+02 6.940e+02, threshold=7.361e+02, percent-clipped=0.0 2026-09-24 06:29:51,115 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=15.47 vs. limit=15.0 2026-09-24 06:29:51,285 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2.whitening_limit, batch_count=199186.66666666666, ans=15.0 2026-09-24 06:30:01,347 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=199253.33333333334, ans=0.0 2026-09-24 06:30:08,945 INFO [train.py:1192] (1/2) Epoch 63, batch 400, loss[loss=0.2519, simple_loss=0.3744, pruned_loss=0.06465, over 24573.00 frames. ], tot_loss[loss=0.2579, simple_loss=0.3759, pruned_loss=0.07, over 4179319.92 frames. ], batch size: 170, lr: 3.35e-03, grad_scale: 32.0 2026-09-24 06:30:32,007 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=199453.33333333334, ans=0.2 2026-09-24 06:30:34,533 INFO [train.py:1192] (1/2) Epoch 63, batch 450, loss[loss=0.2775, simple_loss=0.3964, pruned_loss=0.0793, over 24640.00 frames. ], tot_loss[loss=0.258, simple_loss=0.376, pruned_loss=0.07, over 4317400.23 frames. ], batch size: 175, lr: 3.35e-03, grad_scale: 32.0 2026-09-24 06:30:36,654 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.38 vs. limit=6.0 2026-09-24 06:30:37,032 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.697e+02 3.269e+02 3.844e+02 4.496e+02 6.455e+02, threshold=7.689e+02, percent-clipped=0.0 2026-09-24 06:30:47,550 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=199553.33333333334, ans=0.0 2026-09-24 06:30:59,796 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=199653.33333333334, ans=0.2 2026-09-24 06:31:00,203 INFO [train.py:1192] (1/2) Epoch 63, batch 500, loss[loss=0.2828, simple_loss=0.4105, pruned_loss=0.07759, over 24525.00 frames. ], tot_loss[loss=0.2567, simple_loss=0.3747, pruned_loss=0.06937, over 4435212.88 frames. ], batch size: 218, lr: 3.35e-03, grad_scale: 32.0 2026-09-24 06:31:10,290 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=199720.0, ans=0.125 2026-09-24 06:31:12,507 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass_mid.scale_min, batch_count=199720.0, ans=0.2 2026-09-24 06:31:22,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=199786.66666666666, ans=0.125 2026-09-24 06:31:24,472 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=199786.66666666666, ans=0.125 2026-09-24 06:31:25,650 INFO [train.py:1192] (1/2) Epoch 63, batch 550, loss[loss=0.2688, simple_loss=0.3941, pruned_loss=0.07171, over 24269.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3751, pruned_loss=0.06949, over 4520985.31 frames. ], batch size: 257, lr: 3.35e-03, grad_scale: 32.0 2026-09-24 06:31:27,959 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.555e+02 3.292e+02 3.652e+02 4.147e+02 6.169e+02, threshold=7.304e+02, percent-clipped=0.0 2026-09-24 06:31:34,102 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=199853.33333333334, ans=0.1 2026-09-24 06:31:50,749 INFO [train.py:1192] (1/2) Epoch 63, batch 600, loss[loss=0.2895, simple_loss=0.4152, pruned_loss=0.0819, over 24350.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3754, pruned_loss=0.06927, over 4587006.02 frames. ], batch size: 234, lr: 3.34e-03, grad_scale: 32.0 2026-09-24 06:31:52,725 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=199986.66666666666, ans=0.0 2026-09-24 06:31:53,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.max_positive, batch_count=199986.66666666666, ans=0.95 2026-09-24 06:31:59,936 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=200020.0, ans=0.0 2026-09-24 06:32:00,898 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=200053.33333333334, ans=0.1 2026-09-24 06:32:14,960 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=200120.0, ans=0.125 2026-09-24 06:32:15,430 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=200120.0, ans=0.0 2026-09-24 06:32:15,459 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=200120.0, ans=0.0 2026-09-24 06:32:16,188 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.47 vs. limit=22.5 2026-09-24 06:32:16,628 INFO [train.py:1192] (1/2) Epoch 63, batch 650, loss[loss=0.2263, simple_loss=0.352, pruned_loss=0.0503, over 24587.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3749, pruned_loss=0.06895, over 4652492.57 frames. ], batch size: 154, lr: 3.34e-03, grad_scale: 32.0 2026-09-24 06:32:19,332 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.689e+02 3.268e+02 3.715e+02 4.211e+02 6.195e+02, threshold=7.429e+02, percent-clipped=0.0 2026-09-24 06:32:19,445 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_positive, batch_count=200153.33333333334, ans=0.05 2026-09-24 06:32:19,457 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=200153.33333333334, ans=0.125 2026-09-24 06:32:22,285 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=200186.66666666666, ans=0.2 2026-09-24 06:32:25,013 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=200186.66666666666, ans=0.1 2026-09-24 06:32:32,092 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=200253.33333333334, ans=0.125 2026-09-24 06:32:38,212 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff2_skip_rate, batch_count=200286.66666666666, ans=0.0 2026-09-24 06:32:41,839 INFO [train.py:1192] (1/2) Epoch 63, batch 700, loss[loss=0.2325, simple_loss=0.3572, pruned_loss=0.05393, over 24555.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3758, pruned_loss=0.06913, over 4688700.40 frames. ], batch size: 158, lr: 3.34e-03, grad_scale: 32.0 2026-09-24 06:32:52,679 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=200386.66666666666, ans=0.0 2026-09-24 06:32:57,689 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.69 vs. limit=10.0 2026-09-24 06:32:58,984 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=200420.0, ans=0.125 2026-09-24 06:33:06,706 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=200486.66666666666, ans=0.125 2026-09-24 06:33:07,093 INFO [train.py:1192] (1/2) Epoch 63, batch 750, loss[loss=0.2492, simple_loss=0.369, pruned_loss=0.06467, over 24634.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3743, pruned_loss=0.06852, over 4725591.41 frames. ], batch size: 175, lr: 3.34e-03, grad_scale: 32.0 2026-09-24 06:33:09,912 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.690e+02 3.492e+02 3.957e+02 4.555e+02 6.218e+02, threshold=7.914e+02, percent-clipped=0.0 2026-09-24 06:33:12,151 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.69 vs. limit=15.0 2026-09-24 06:33:12,492 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=200520.0, ans=0.025 2026-09-24 06:33:12,518 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=200520.0, ans=0.0 2026-09-24 06:33:15,980 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=9.64 vs. limit=15.0 2026-09-24 06:33:22,611 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=200586.66666666666, ans=0.0 2026-09-24 06:33:27,055 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=200586.66666666666, ans=0.1 2026-09-24 06:33:32,821 INFO [train.py:1192] (1/2) Epoch 63, batch 800, loss[loss=0.2188, simple_loss=0.3354, pruned_loss=0.05115, over 24536.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3747, pruned_loss=0.06896, over 4752404.83 frames. ], batch size: 137, lr: 3.34e-03, grad_scale: 32.0 2026-09-24 06:33:53,224 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=200786.66666666666, ans=0.125 2026-09-24 06:33:58,365 INFO [train.py:1192] (1/2) Epoch 63, batch 850, loss[loss=0.2763, simple_loss=0.4013, pruned_loss=0.07568, over 24532.00 frames. ], tot_loss[loss=0.2562, simple_loss=0.3743, pruned_loss=0.06899, over 4771021.63 frames. ], batch size: 204, lr: 3.34e-03, grad_scale: 32.0 2026-09-24 06:33:58,921 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=200820.0, ans=0.125 2026-09-24 06:34:00,914 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.543e+02 3.497e+02 3.836e+02 4.377e+02 6.099e+02, threshold=7.673e+02, percent-clipped=0.0 2026-09-24 06:34:04,192 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=200853.33333333334, ans=0.0 2026-09-24 06:34:09,711 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.14 vs. limit=15.0 2026-09-24 06:34:20,792 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=200953.33333333334, ans=0.1 2026-09-24 06:34:24,351 INFO [train.py:1192] (1/2) Epoch 63, batch 900, loss[loss=0.2107, simple_loss=0.3298, pruned_loss=0.04585, over 24553.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3751, pruned_loss=0.06954, over 4781751.81 frames. ], batch size: 137, lr: 3.34e-03, grad_scale: 32.0 2026-09-24 06:34:30,778 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=201020.0, ans=0.125 2026-09-24 06:34:35,250 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=201053.33333333334, ans=0.125 2026-09-24 06:34:36,379 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.05 vs. limit=15.0 2026-09-24 06:34:46,377 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=201120.0, ans=0.125 2026-09-24 06:34:49,489 INFO [train.py:1192] (1/2) Epoch 63, batch 950, loss[loss=0.3665, simple_loss=0.4278, pruned_loss=0.1526, over 11480.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3738, pruned_loss=0.07016, over 4713107.13 frames. ], batch size: 333, lr: 3.33e-03, grad_scale: 32.0 2026-09-24 06:34:51,756 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.689e+02 3.427e+02 4.031e+02 4.632e+02 6.244e+02, threshold=8.063e+02, percent-clipped=0.0 2026-09-24 06:34:58,215 INFO [train.py:1192] (1/2) Epoch 64, batch 0, loss[loss=0.1997, simple_loss=0.3259, pruned_loss=0.03674, over 24555.00 frames. ], tot_loss[loss=0.1997, simple_loss=0.3259, pruned_loss=0.03674, over 24555.00 frames. ], batch size: 137, lr: 3.31e-03, grad_scale: 32.0 2026-09-24 06:34:58,215 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 06:35:09,799 INFO [train.py:1224] (1/2) Epoch 64, validation: loss=0.1703, simple_loss=0.2888, pruned_loss=0.02589, over 2564189.00 frames. 2026-09-24 06:35:09,800 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 06:35:22,359 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=201246.66666666666, ans=0.1 2026-09-24 06:35:22,844 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:35:23,801 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:35:32,449 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.97 vs. limit=22.5 2026-09-24 06:35:32,760 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=201313.33333333334, ans=0.125 2026-09-24 06:35:35,566 INFO [train.py:1192] (1/2) Epoch 64, batch 50, loss[loss=0.1986, simple_loss=0.3159, pruned_loss=0.04064, over 24210.00 frames. ], tot_loss[loss=0.2648, simple_loss=0.3817, pruned_loss=0.07401, over 1081140.12 frames. ], batch size: 125, lr: 3.31e-03, grad_scale: 32.0 2026-09-24 06:35:37,034 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=201346.66666666666, ans=0.0 2026-09-24 06:35:42,507 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=201380.0, ans=0.0 2026-09-24 06:35:56,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=201480.0, ans=0.0 2026-09-24 06:35:59,807 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.810e+02 3.494e+02 3.954e+02 4.353e+02 6.086e+02, threshold=7.909e+02, percent-clipped=0.0 2026-09-24 06:36:01,240 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=9.01 vs. limit=15.0 2026-09-24 06:36:01,518 INFO [train.py:1192] (1/2) Epoch 64, batch 100, loss[loss=0.2549, simple_loss=0.3675, pruned_loss=0.07118, over 24609.00 frames. ], tot_loss[loss=0.2662, simple_loss=0.3847, pruned_loss=0.07389, over 1916197.42 frames. ], batch size: 154, lr: 3.31e-03, grad_scale: 32.0 2026-09-24 06:36:09,502 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.10 vs. limit=22.5 2026-09-24 06:36:14,781 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.89 vs. limit=15.0 2026-09-24 06:36:26,888 INFO [train.py:1192] (1/2) Epoch 64, batch 150, loss[loss=0.222, simple_loss=0.3311, pruned_loss=0.05643, over 24284.00 frames. ], tot_loss[loss=0.2611, simple_loss=0.3789, pruned_loss=0.07163, over 2561454.35 frames. ], batch size: 125, lr: 3.30e-03, grad_scale: 32.0 2026-09-24 06:36:29,296 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=201680.0, ans=0.0 2026-09-24 06:36:31,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=201713.33333333334, ans=0.125 2026-09-24 06:36:39,626 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=201746.66666666666, ans=0.125 2026-09-24 06:36:39,920 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.76 vs. limit=15.0 2026-09-24 06:36:40,436 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.35 vs. limit=15.0 2026-09-24 06:36:50,961 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.955e+02 3.309e+02 3.704e+02 4.199e+02 6.164e+02, threshold=7.408e+02, percent-clipped=0.0 2026-09-24 06:36:52,519 INFO [train.py:1192] (1/2) Epoch 64, batch 200, loss[loss=0.2949, simple_loss=0.4211, pruned_loss=0.0844, over 24233.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3772, pruned_loss=0.07042, over 3058758.70 frames. ], batch size: 257, lr: 3.30e-03, grad_scale: 32.0 2026-09-24 06:36:58,433 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=201880.0, ans=0.5 2026-09-24 06:36:58,440 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=201880.0, ans=0.0 2026-09-24 06:36:58,593 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.11 vs. limit=15.0 2026-09-24 06:37:17,625 INFO [train.py:1192] (1/2) Epoch 64, batch 250, loss[loss=0.2808, simple_loss=0.4046, pruned_loss=0.07849, over 24401.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3763, pruned_loss=0.07002, over 3442587.34 frames. ], batch size: 225, lr: 3.30e-03, grad_scale: 32.0 2026-09-24 06:37:26,825 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=9.66 vs. limit=15.0 2026-09-24 06:37:35,578 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.28 vs. limit=15.0 2026-09-24 06:37:40,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.min_positive, batch_count=202146.66666666666, ans=0.05 2026-09-24 06:37:41,608 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.474e+02 3.346e+02 3.765e+02 4.247e+02 6.018e+02, threshold=7.530e+02, percent-clipped=0.0 2026-09-24 06:37:43,182 INFO [train.py:1192] (1/2) Epoch 64, batch 300, loss[loss=0.2859, simple_loss=0.4059, pruned_loss=0.08296, over 24514.00 frames. ], tot_loss[loss=0.2584, simple_loss=0.3764, pruned_loss=0.07022, over 3755891.44 frames. ], batch size: 204, lr: 3.30e-03, grad_scale: 32.0 2026-09-24 06:37:52,412 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=202213.33333333334, ans=0.125 2026-09-24 06:37:55,429 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=10.86 vs. limit=22.5 2026-09-24 06:37:55,834 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.68 vs. limit=6.0 2026-09-24 06:37:58,124 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=202280.0, ans=0.1 2026-09-24 06:38:00,766 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=202280.0, ans=0.1 2026-09-24 06:38:06,601 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=6.07 vs. limit=15.0 2026-09-24 06:38:08,215 INFO [train.py:1192] (1/2) Epoch 64, batch 350, loss[loss=0.224, simple_loss=0.338, pruned_loss=0.05494, over 24576.00 frames. ], tot_loss[loss=0.259, simple_loss=0.3771, pruned_loss=0.07046, over 3997352.80 frames. ], batch size: 137, lr: 3.30e-03, grad_scale: 32.0 2026-09-24 06:38:30,729 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.45 vs. limit=15.0 2026-09-24 06:38:31,788 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.529e+02 3.427e+02 3.829e+02 4.368e+02 6.561e+02, threshold=7.657e+02, percent-clipped=0.0 2026-09-24 06:38:33,357 INFO [train.py:1192] (1/2) Epoch 64, batch 400, loss[loss=0.2706, simple_loss=0.383, pruned_loss=0.07913, over 24582.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3761, pruned_loss=0.0701, over 4179822.25 frames. ], batch size: 170, lr: 3.30e-03, grad_scale: 32.0 2026-09-24 06:38:34,752 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=202513.33333333334, ans=0.125 2026-09-24 06:38:35,261 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=202513.33333333334, ans=0.025 2026-09-24 06:38:36,729 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=202513.33333333334, ans=0.125 2026-09-24 06:38:48,948 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module1.whiten, num_groups=1, num_channels=192, metric=7.91 vs. limit=15.0 2026-09-24 06:38:50,636 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.prob, batch_count=202613.33333333334, ans=0.125 2026-09-24 06:38:53,806 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=202646.66666666666, ans=0.0 2026-09-24 06:38:59,184 INFO [train.py:1192] (1/2) Epoch 64, batch 450, loss[loss=0.2596, simple_loss=0.3787, pruned_loss=0.07023, over 24628.00 frames. ], tot_loss[loss=0.2592, simple_loss=0.3768, pruned_loss=0.07077, over 4320427.10 frames. ], batch size: 175, lr: 3.30e-03, grad_scale: 32.0 2026-09-24 06:39:14,851 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=202780.0, ans=0.1 2026-09-24 06:39:23,158 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.645e+02 3.315e+02 3.849e+02 4.427e+02 7.168e+02, threshold=7.697e+02, percent-clipped=0.0 2026-09-24 06:39:24,690 INFO [train.py:1192] (1/2) Epoch 64, batch 500, loss[loss=0.2775, simple_loss=0.4026, pruned_loss=0.07619, over 24527.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3753, pruned_loss=0.07008, over 4438033.27 frames. ], batch size: 218, lr: 3.29e-03, grad_scale: 32.0 2026-09-24 06:39:29,507 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.57 vs. limit=15.0 2026-09-24 06:39:33,963 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=202880.0, ans=0.125 2026-09-24 06:39:45,904 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.13 vs. limit=15.0 2026-09-24 06:39:47,856 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_abs, batch_count=202980.0, ans=0.5 2026-09-24 06:39:47,870 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=202980.0, ans=0.0 2026-09-24 06:39:50,387 INFO [train.py:1192] (1/2) Epoch 64, batch 550, loss[loss=0.3252, simple_loss=0.4335, pruned_loss=0.1084, over 24243.00 frames. ], tot_loss[loss=0.2578, simple_loss=0.3754, pruned_loss=0.0701, over 4522840.12 frames. ], batch size: 257, lr: 3.29e-03, grad_scale: 32.0 2026-09-24 06:39:51,503 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=203013.33333333334, ans=0.125 2026-09-24 06:40:11,293 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=203146.66666666666, ans=0.0 2026-09-24 06:40:14,232 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.612e+02 3.344e+02 3.554e+02 4.056e+02 6.649e+02, threshold=7.108e+02, percent-clipped=0.0 2026-09-24 06:40:16,062 INFO [train.py:1192] (1/2) Epoch 64, batch 600, loss[loss=0.2553, simple_loss=0.3867, pruned_loss=0.06197, over 24334.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3762, pruned_loss=0.07002, over 4589497.71 frames. ], batch size: 234, lr: 3.29e-03, grad_scale: 32.0 2026-09-24 06:40:23,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=203213.33333333334, ans=0.125 2026-09-24 06:40:37,132 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=203313.33333333334, ans=0.125 2026-09-24 06:40:41,346 INFO [train.py:1192] (1/2) Epoch 64, batch 650, loss[loss=0.2564, simple_loss=0.3625, pruned_loss=0.07516, over 24589.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3749, pruned_loss=0.06912, over 4654428.77 frames. ], batch size: 154, lr: 3.29e-03, grad_scale: 32.0 2026-09-24 06:40:46,285 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=203380.0, ans=0.025 2026-09-24 06:40:46,717 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=203380.0, ans=0.125 2026-09-24 06:41:01,366 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=203480.0, ans=0.0 2026-09-24 06:41:01,795 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=203480.0, ans=0.125 2026-09-24 06:41:03,313 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=203480.0, ans=0.125 2026-09-24 06:41:05,046 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.666e+02 3.248e+02 3.709e+02 4.337e+02 6.887e+02, threshold=7.418e+02, percent-clipped=0.0 2026-09-24 06:41:06,436 INFO [train.py:1192] (1/2) Epoch 64, batch 700, loss[loss=0.2509, simple_loss=0.3681, pruned_loss=0.06685, over 24557.00 frames. ], tot_loss[loss=0.2571, simple_loss=0.3757, pruned_loss=0.06926, over 4689277.81 frames. ], batch size: 158, lr: 3.29e-03, grad_scale: 32.0 2026-09-24 06:41:14,865 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=203546.66666666666, ans=0.1 2026-09-24 06:41:21,774 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=203580.0, ans=0.0 2026-09-24 06:41:26,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=203613.33333333334, ans=0.125 2026-09-24 06:41:28,094 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=203646.66666666666, ans=0.0 2026-09-24 06:41:32,390 INFO [train.py:1192] (1/2) Epoch 64, batch 750, loss[loss=0.2372, simple_loss=0.3642, pruned_loss=0.05504, over 24647.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3754, pruned_loss=0.06947, over 4725865.77 frames. ], batch size: 175, lr: 3.29e-03, grad_scale: 32.0 2026-09-24 06:41:35,858 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=203680.0, ans=0.0 2026-09-24 06:41:36,420 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=203680.0, ans=0.2 2026-09-24 06:41:42,751 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=203746.66666666666, ans=0.125 2026-09-24 06:41:49,586 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:41:52,079 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=203780.0, ans=0.2 2026-09-24 06:41:54,878 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=203813.33333333334, ans=0.125 2026-09-24 06:41:56,202 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.674e+02 3.434e+02 4.021e+02 4.647e+02 7.395e+02, threshold=8.043e+02, percent-clipped=0.0 2026-09-24 06:41:58,034 INFO [train.py:1192] (1/2) Epoch 64, batch 800, loss[loss=0.2042, simple_loss=0.3219, pruned_loss=0.04321, over 24546.00 frames. ], tot_loss[loss=0.2576, simple_loss=0.3757, pruned_loss=0.06982, over 4752002.37 frames. ], batch size: 137, lr: 3.29e-03, grad_scale: 32.0 2026-09-24 06:42:04,580 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.60 vs. limit=12.0 2026-09-24 06:42:05,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=203880.0, ans=0.125 2026-09-24 06:42:17,130 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=203946.66666666666, ans=0.125 2026-09-24 06:42:23,705 INFO [train.py:1192] (1/2) Epoch 64, batch 850, loss[loss=0.2829, simple_loss=0.4057, pruned_loss=0.08009, over 24542.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3752, pruned_loss=0.06962, over 4770233.27 frames. ], batch size: 204, lr: 3.29e-03, grad_scale: 32.0 2026-09-24 06:42:24,707 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=204013.33333333334, ans=0.125 2026-09-24 06:42:25,184 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=204013.33333333334, ans=0.1 2026-09-24 06:42:37,959 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=204113.33333333334, ans=0.125 2026-09-24 06:42:43,073 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=204146.66666666666, ans=0.125 2026-09-24 06:42:47,168 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.581e+02 3.618e+02 4.019e+02 4.678e+02 6.392e+02, threshold=8.038e+02, percent-clipped=0.0 2026-09-24 06:42:48,204 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=204180.0, ans=0.1 2026-09-24 06:42:48,320 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.72 vs. limit=15.0 2026-09-24 06:42:48,523 INFO [train.py:1192] (1/2) Epoch 64, batch 900, loss[loss=0.2161, simple_loss=0.3349, pruned_loss=0.04863, over 24555.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3752, pruned_loss=0.06962, over 4780665.74 frames. ], batch size: 137, lr: 3.28e-03, grad_scale: 32.0 2026-09-24 06:42:53,403 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.27 vs. limit=22.5 2026-09-24 06:42:55,171 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:42:59,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=204246.66666666666, ans=0.2 2026-09-24 06:42:59,869 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=204246.66666666666, ans=0.125 2026-09-24 06:43:00,751 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=204246.66666666666, ans=0.125 2026-09-24 06:43:11,556 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=204313.33333333334, ans=0.125 2026-09-24 06:43:13,762 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=11.61 vs. limit=15.0 2026-09-24 06:43:13,901 INFO [train.py:1192] (1/2) Epoch 64, batch 950, loss[loss=0.3668, simple_loss=0.4288, pruned_loss=0.1524, over 11241.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.375, pruned_loss=0.07131, over 4706542.60 frames. ], batch size: 333, lr: 3.28e-03, grad_scale: 32.0 2026-09-24 06:43:15,088 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=23.62 vs. limit=22.5 2026-09-24 06:43:23,835 INFO [train.py:1192] (1/2) Epoch 65, batch 0, loss[loss=0.2053, simple_loss=0.3301, pruned_loss=0.04028, over 24574.00 frames. ], tot_loss[loss=0.2053, simple_loss=0.3301, pruned_loss=0.04028, over 24574.00 frames. ], batch size: 137, lr: 3.26e-03, grad_scale: 32.0 2026-09-24 06:43:23,835 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 06:43:25,181 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([4.7860, 4.2871, 4.2786, 4.6442], device='cuda:1') 2026-09-24 06:43:35,371 INFO [train.py:1224] (1/2) Epoch 65, validation: loss=0.1694, simple_loss=0.2879, pruned_loss=0.02545, over 2564189.00 frames. 2026-09-24 06:43:35,371 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 06:43:35,464 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=204373.33333333334, ans=0.125 2026-09-24 06:43:36,770 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=204373.33333333334, ans=0.125 2026-09-24 06:43:41,881 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=8.71 vs. limit=22.5 2026-09-24 06:43:55,232 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.690e+02 3.480e+02 4.057e+02 4.389e+02 1.010e+03, threshold=8.114e+02, percent-clipped=1.0 2026-09-24 06:44:00,885 INFO [train.py:1192] (1/2) Epoch 65, batch 50, loss[loss=0.216, simple_loss=0.3324, pruned_loss=0.04978, over 24290.00 frames. ], tot_loss[loss=0.262, simple_loss=0.3806, pruned_loss=0.07169, over 1083059.86 frames. ], batch size: 125, lr: 3.26e-03, grad_scale: 32.0 2026-09-24 06:44:13,111 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=204606.66666666666, ans=0.2 2026-09-24 06:44:23,598 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=204673.33333333334, ans=0.2 2026-09-24 06:44:24,022 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=204673.33333333334, ans=0.0 2026-09-24 06:44:26,486 INFO [train.py:1192] (1/2) Epoch 65, batch 100, loss[loss=0.2547, simple_loss=0.3671, pruned_loss=0.07118, over 24584.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3834, pruned_loss=0.07214, over 1915398.59 frames. ], batch size: 154, lr: 3.25e-03, grad_scale: 32.0 2026-09-24 06:44:42,566 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=204806.66666666666, ans=0.0 2026-09-24 06:44:42,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=204806.66666666666, ans=0.0 2026-09-24 06:44:45,953 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.589e+02 3.249e+02 3.693e+02 4.114e+02 5.857e+02, threshold=7.386e+02, percent-clipped=0.0 2026-09-24 06:44:52,172 INFO [train.py:1192] (1/2) Epoch 65, batch 150, loss[loss=0.203, simple_loss=0.321, pruned_loss=0.04251, over 24322.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.378, pruned_loss=0.07007, over 2561421.04 frames. ], batch size: 125, lr: 3.25e-03, grad_scale: 32.0 2026-09-24 06:45:01,585 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=204940.0, ans=0.125 2026-09-24 06:45:11,287 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=205006.66666666666, ans=0.1 2026-09-24 06:45:14,066 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.04 vs. limit=15.0 2026-09-24 06:45:17,146 INFO [train.py:1192] (1/2) Epoch 65, batch 200, loss[loss=0.2774, simple_loss=0.4064, pruned_loss=0.07424, over 24203.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3756, pruned_loss=0.06856, over 3059974.22 frames. ], batch size: 257, lr: 3.25e-03, grad_scale: 32.0 2026-09-24 06:45:17,707 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=205040.0, ans=0.0 2026-09-24 06:45:17,738 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=205040.0, ans=0.0 2026-09-24 06:45:19,159 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=205040.0, ans=0.0 2026-09-24 06:45:23,312 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.13 vs. limit=10.0 2026-09-24 06:45:24,088 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=205073.33333333334, ans=0.0 2026-09-24 06:45:34,529 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.89 vs. limit=15.0 2026-09-24 06:45:34,963 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=205140.0, ans=0.125 2026-09-24 06:45:36,742 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.542e+02 3.387e+02 3.702e+02 4.365e+02 5.971e+02, threshold=7.404e+02, percent-clipped=0.0 2026-09-24 06:45:38,626 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=205173.33333333334, ans=0.1 2026-09-24 06:45:42,846 INFO [train.py:1192] (1/2) Epoch 65, batch 250, loss[loss=0.2831, simple_loss=0.4026, pruned_loss=0.08186, over 24379.00 frames. ], tot_loss[loss=0.2573, simple_loss=0.3757, pruned_loss=0.0695, over 3442921.50 frames. ], batch size: 225, lr: 3.25e-03, grad_scale: 32.0 2026-09-24 06:45:59,061 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=205306.66666666666, ans=0.025 2026-09-24 06:46:06,743 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.74 vs. limit=15.0 2026-09-24 06:46:07,907 INFO [train.py:1192] (1/2) Epoch 65, batch 300, loss[loss=0.2562, simple_loss=0.3793, pruned_loss=0.06654, over 24536.00 frames. ], tot_loss[loss=0.2568, simple_loss=0.3749, pruned_loss=0.06937, over 3755905.44 frames. ], batch size: 204, lr: 3.25e-03, grad_scale: 32.0 2026-09-24 06:46:17,068 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=205406.66666666666, ans=0.125 2026-09-24 06:46:18,947 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=205440.0, ans=0.07 2026-09-24 06:46:21,699 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=205440.0, ans=0.0 2026-09-24 06:46:22,218 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=205440.0, ans=0.2 2026-09-24 06:46:26,677 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=205473.33333333334, ans=0.1 2026-09-24 06:46:27,239 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:46:27,621 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.897e+02 3.396e+02 3.848e+02 4.457e+02 6.052e+02, threshold=7.696e+02, percent-clipped=0.0 2026-09-24 06:46:33,104 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=205540.0, ans=0.1 2026-09-24 06:46:33,287 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.05 vs. limit=22.5 2026-09-24 06:46:33,595 INFO [train.py:1192] (1/2) Epoch 65, batch 350, loss[loss=0.2209, simple_loss=0.335, pruned_loss=0.05338, over 24559.00 frames. ], tot_loss[loss=0.258, simple_loss=0.3762, pruned_loss=0.06992, over 3993133.39 frames. ], batch size: 137, lr: 3.25e-03, grad_scale: 64.0 2026-09-24 06:46:37,846 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=205540.0, ans=0.0 2026-09-24 06:46:43,365 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=205606.66666666666, ans=0.125 2026-09-24 06:46:45,363 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.whiten.whitening_limit, batch_count=205606.66666666666, ans=12.0 2026-09-24 06:46:49,565 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=205640.0, ans=0.09899494936611666 2026-09-24 06:46:57,320 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=205673.33333333334, ans=0.0 2026-09-24 06:46:58,630 INFO [train.py:1192] (1/2) Epoch 65, batch 400, loss[loss=0.2841, simple_loss=0.3987, pruned_loss=0.08472, over 24574.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3754, pruned_loss=0.06948, over 4179879.75 frames. ], batch size: 170, lr: 3.25e-03, grad_scale: 64.0 2026-09-24 06:47:05,471 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.43 vs. limit=22.5 2026-09-24 06:47:18,815 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.618e+02 3.420e+02 3.842e+02 4.354e+02 6.399e+02, threshold=7.683e+02, percent-clipped=0.0 2026-09-24 06:47:24,758 INFO [train.py:1192] (1/2) Epoch 65, batch 450, loss[loss=0.2752, simple_loss=0.3971, pruned_loss=0.07662, over 24624.00 frames. ], tot_loss[loss=0.258, simple_loss=0.3758, pruned_loss=0.07004, over 4319887.48 frames. ], batch size: 175, lr: 3.25e-03, grad_scale: 64.0 2026-09-24 06:47:25,383 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.21 vs. limit=22.5 2026-09-24 06:47:31,300 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=205906.66666666666, ans=0.2 2026-09-24 06:47:34,673 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.24 vs. limit=15.0 2026-09-24 06:47:36,429 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=205940.0, ans=0.125 2026-09-24 06:47:50,111 INFO [train.py:1192] (1/2) Epoch 65, batch 500, loss[loss=0.251, simple_loss=0.3849, pruned_loss=0.05858, over 24519.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3741, pruned_loss=0.06926, over 4437723.84 frames. ], batch size: 218, lr: 3.24e-03, grad_scale: 64.0 2026-09-24 06:47:54,078 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=206040.0, ans=0.2 2026-09-24 06:47:54,685 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.65 vs. limit=10.0 2026-09-24 06:47:55,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=206073.33333333334, ans=0.125 2026-09-24 06:48:00,746 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=206106.66666666666, ans=0.04949747468305833 2026-09-24 06:48:09,453 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.664e+02 3.408e+02 3.757e+02 4.234e+02 7.032e+02, threshold=7.514e+02, percent-clipped=0.0 2026-09-24 06:48:15,447 INFO [train.py:1192] (1/2) Epoch 65, batch 550, loss[loss=0.2961, simple_loss=0.419, pruned_loss=0.08658, over 24248.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3754, pruned_loss=0.06997, over 4522859.55 frames. ], batch size: 257, lr: 3.24e-03, grad_scale: 64.0 2026-09-24 06:48:18,670 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=7.26 vs. limit=15.0 2026-09-24 06:48:21,021 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=206240.0, ans=0.2 2026-09-24 06:48:24,268 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=206240.0, ans=0.025 2026-09-24 06:48:30,873 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=206306.66666666666, ans=0.1 2026-09-24 06:48:31,403 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=206306.66666666666, ans=0.125 2026-09-24 06:48:41,085 INFO [train.py:1192] (1/2) Epoch 65, batch 600, loss[loss=0.2936, simple_loss=0.4193, pruned_loss=0.084, over 24328.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3768, pruned_loss=0.0709, over 4588318.04 frames. ], batch size: 234, lr: 3.24e-03, grad_scale: 32.0 2026-09-24 06:48:44,600 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=206373.33333333334, ans=0.1 2026-09-24 06:48:52,288 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=206440.0, ans=0.125 2026-09-24 06:48:57,280 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:49:01,404 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.814e+02 3.398e+02 3.684e+02 4.322e+02 6.606e+02, threshold=7.368e+02, percent-clipped=0.0 2026-09-24 06:49:06,102 INFO [train.py:1192] (1/2) Epoch 65, batch 650, loss[loss=0.2752, simple_loss=0.3819, pruned_loss=0.08425, over 24626.00 frames. ], tot_loss[loss=0.2579, simple_loss=0.3757, pruned_loss=0.07007, over 4653253.23 frames. ], batch size: 154, lr: 3.24e-03, grad_scale: 32.0 2026-09-24 06:49:17,199 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=206606.66666666666, ans=0.0 2026-09-24 06:49:31,600 INFO [train.py:1192] (1/2) Epoch 65, batch 700, loss[loss=0.2343, simple_loss=0.359, pruned_loss=0.05482, over 24553.00 frames. ], tot_loss[loss=0.2589, simple_loss=0.3769, pruned_loss=0.07039, over 4688472.76 frames. ], batch size: 158, lr: 3.24e-03, grad_scale: 32.0 2026-09-24 06:49:34,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=206706.66666666666, ans=0.1 2026-09-24 06:49:36,954 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=206740.0, ans=0.125 2026-09-24 06:49:45,617 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.2.prob, batch_count=206773.33333333334, ans=0.125 2026-09-24 06:49:52,782 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.587e+02 3.473e+02 3.863e+02 4.424e+02 6.086e+02, threshold=7.726e+02, percent-clipped=0.0 2026-09-24 06:49:52,916 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=206840.0, ans=0.125 2026-09-24 06:49:57,494 INFO [train.py:1192] (1/2) Epoch 65, batch 750, loss[loss=0.245, simple_loss=0.3679, pruned_loss=0.06104, over 24642.00 frames. ], tot_loss[loss=0.2588, simple_loss=0.3765, pruned_loss=0.0706, over 4725220.89 frames. ], batch size: 175, lr: 3.24e-03, grad_scale: 32.0 2026-09-24 06:49:58,949 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=206873.33333333334, ans=0.2 2026-09-24 06:50:07,489 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=206940.0, ans=0.125 2026-09-24 06:50:08,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=206940.0, ans=0.0 2026-09-24 06:50:13,708 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=206973.33333333334, ans=0.125 2026-09-24 06:50:17,043 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=206973.33333333334, ans=0.1 2026-09-24 06:50:21,492 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=207006.66666666666, ans=0.125 2026-09-24 06:50:22,735 INFO [train.py:1192] (1/2) Epoch 65, batch 800, loss[loss=0.2112, simple_loss=0.3276, pruned_loss=0.04742, over 24539.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3753, pruned_loss=0.06986, over 4751088.71 frames. ], batch size: 137, lr: 3.24e-03, grad_scale: 32.0 2026-09-24 06:50:27,544 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=207073.33333333334, ans=0.0 2026-09-24 06:50:31,960 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.37 vs. limit=22.5 2026-09-24 06:50:43,051 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.588e+02 3.517e+02 3.795e+02 4.231e+02 7.044e+02, threshold=7.591e+02, percent-clipped=0.0 2026-09-24 06:50:47,005 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=207173.33333333334, ans=0.0 2026-09-24 06:50:47,911 INFO [train.py:1192] (1/2) Epoch 65, batch 850, loss[loss=0.286, simple_loss=0.4132, pruned_loss=0.07943, over 24537.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3743, pruned_loss=0.06921, over 4770271.77 frames. ], batch size: 204, lr: 3.24e-03, grad_scale: 32.0 2026-09-24 06:51:01,989 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=207273.33333333334, ans=10.0 2026-09-24 06:51:06,593 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=207306.66666666666, ans=0.0 2026-09-24 06:51:07,537 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=207306.66666666666, ans=0.125 2026-09-24 06:51:13,800 INFO [train.py:1192] (1/2) Epoch 65, batch 900, loss[loss=0.2103, simple_loss=0.3301, pruned_loss=0.04522, over 24531.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3748, pruned_loss=0.06956, over 4780750.36 frames. ], batch size: 137, lr: 3.23e-03, grad_scale: 32.0 2026-09-24 06:51:33,390 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.511e+02 3.421e+02 4.215e+02 4.846e+02 8.093e+02, threshold=8.430e+02, percent-clipped=1.0 2026-09-24 06:51:38,709 INFO [train.py:1192] (1/2) Epoch 65, batch 950, loss[loss=0.3475, simple_loss=0.4203, pruned_loss=0.1374, over 11432.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3737, pruned_loss=0.07057, over 4714232.73 frames. ], batch size: 333, lr: 3.23e-03, grad_scale: 32.0 2026-09-24 06:51:50,075 INFO [train.py:1192] (1/2) Epoch 66, batch 0, loss[loss=0.2141, simple_loss=0.3375, pruned_loss=0.04531, over 24556.00 frames. ], tot_loss[loss=0.2141, simple_loss=0.3375, pruned_loss=0.04531, over 24556.00 frames. ], batch size: 137, lr: 3.21e-03, grad_scale: 32.0 2026-09-24 06:51:50,075 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 06:51:56,056 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8228, 2.2827, 3.0310, 1.4945], device='cuda:1') 2026-09-24 06:52:01,616 INFO [train.py:1224] (1/2) Epoch 66, validation: loss=0.1703, simple_loss=0.2885, pruned_loss=0.02602, over 2564189.00 frames. 2026-09-24 06:52:01,616 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 06:52:02,825 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=4.44 vs. limit=15.0 2026-09-24 06:52:14,594 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=207633.33333333334, ans=0.125 2026-09-24 06:52:23,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=207700.0, ans=0.125 2026-09-24 06:52:27,748 INFO [train.py:1192] (1/2) Epoch 66, batch 50, loss[loss=0.2068, simple_loss=0.3145, pruned_loss=0.04954, over 24227.00 frames. ], tot_loss[loss=0.2634, simple_loss=0.3812, pruned_loss=0.07285, over 1082213.20 frames. ], batch size: 125, lr: 3.21e-03, grad_scale: 32.0 2026-09-24 06:52:28,771 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=207733.33333333334, ans=0.125 2026-09-24 06:52:32,348 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=10.14 vs. limit=15.0 2026-09-24 06:52:38,171 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=10.00 vs. limit=15.0 2026-09-24 06:52:38,520 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:52:43,634 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.716e+02 3.550e+02 3.980e+02 4.670e+02 6.089e+02, threshold=7.959e+02, percent-clipped=0.0 2026-09-24 06:52:52,243 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.32 vs. limit=15.0 2026-09-24 06:52:53,539 INFO [train.py:1192] (1/2) Epoch 66, batch 100, loss[loss=0.2806, simple_loss=0.3847, pruned_loss=0.08818, over 24603.00 frames. ], tot_loss[loss=0.2666, simple_loss=0.3849, pruned_loss=0.07411, over 1916406.01 frames. ], batch size: 154, lr: 3.20e-03, grad_scale: 32.0 2026-09-24 06:52:53,630 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=207900.0, ans=0.1 2026-09-24 06:52:56,536 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.84 vs. limit=15.0 2026-09-24 06:52:58,717 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=207933.33333333334, ans=0.125 2026-09-24 06:53:16,296 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=208033.33333333334, ans=0.0 2026-09-24 06:53:19,162 INFO [train.py:1192] (1/2) Epoch 66, batch 150, loss[loss=0.2195, simple_loss=0.3246, pruned_loss=0.05718, over 24230.00 frames. ], tot_loss[loss=0.2628, simple_loss=0.3806, pruned_loss=0.07254, over 2561210.01 frames. ], batch size: 125, lr: 3.20e-03, grad_scale: 32.0 2026-09-24 06:53:27,707 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=208100.0, ans=0.1 2026-09-24 06:53:35,706 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.697e+02 3.405e+02 3.768e+02 4.274e+02 6.558e+02, threshold=7.536e+02, percent-clipped=0.0 2026-09-24 06:53:39,509 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=208200.0, ans=0.1 2026-09-24 06:53:44,413 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=208233.33333333334, ans=0.1 2026-09-24 06:53:44,824 INFO [train.py:1192] (1/2) Epoch 66, batch 200, loss[loss=0.3174, simple_loss=0.4368, pruned_loss=0.099, over 24195.00 frames. ], tot_loss[loss=0.2601, simple_loss=0.3782, pruned_loss=0.07101, over 3060734.10 frames. ], batch size: 257, lr: 3.20e-03, grad_scale: 32.0 2026-09-24 06:53:58,134 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=7.41 vs. limit=15.0 2026-09-24 06:54:01,598 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.prob, batch_count=208333.33333333334, ans=0.125 2026-09-24 06:54:10,844 INFO [train.py:1192] (1/2) Epoch 66, batch 250, loss[loss=0.3061, simple_loss=0.4238, pruned_loss=0.09418, over 24375.00 frames. ], tot_loss[loss=0.2603, simple_loss=0.3779, pruned_loss=0.07138, over 3444956.25 frames. ], batch size: 225, lr: 3.20e-03, grad_scale: 32.0 2026-09-24 06:54:10,936 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=208400.0, ans=0.125 2026-09-24 06:54:16,843 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=208433.33333333334, ans=0.0 2026-09-24 06:54:26,503 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.729e+02 3.440e+02 3.866e+02 4.640e+02 6.209e+02, threshold=7.733e+02, percent-clipped=0.0 2026-09-24 06:54:35,695 INFO [train.py:1192] (1/2) Epoch 66, batch 300, loss[loss=0.2852, simple_loss=0.4088, pruned_loss=0.08083, over 24594.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3761, pruned_loss=0.07008, over 3757859.47 frames. ], batch size: 204, lr: 3.20e-03, grad_scale: 32.0 2026-09-24 06:55:01,036 INFO [train.py:1192] (1/2) Epoch 66, batch 350, loss[loss=0.2223, simple_loss=0.3316, pruned_loss=0.05644, over 24578.00 frames. ], tot_loss[loss=0.2582, simple_loss=0.3766, pruned_loss=0.06987, over 3998929.71 frames. ], batch size: 137, lr: 3.20e-03, grad_scale: 32.0 2026-09-24 06:55:03,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=208733.33333333334, ans=0.125 2026-09-24 06:55:03,457 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=208733.33333333334, ans=0.1 2026-09-24 06:55:08,552 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=208766.66666666666, ans=0.07 2026-09-24 06:55:09,865 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=208766.66666666666, ans=0.125 2026-09-24 06:55:16,964 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.757e+02 3.335e+02 3.673e+02 4.164e+02 6.012e+02, threshold=7.346e+02, percent-clipped=0.0 2026-09-24 06:55:19,968 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=208833.33333333334, ans=0.95 2026-09-24 06:55:26,228 INFO [train.py:1192] (1/2) Epoch 66, batch 400, loss[loss=0.2747, simple_loss=0.3925, pruned_loss=0.07848, over 24564.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3751, pruned_loss=0.06902, over 4180827.97 frames. ], batch size: 170, lr: 3.20e-03, grad_scale: 32.0 2026-09-24 06:55:37,378 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=208966.66666666666, ans=0.0 2026-09-24 06:55:39,873 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=208966.66666666666, ans=0.125 2026-09-24 06:55:45,770 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=209033.33333333334, ans=0.125 2026-09-24 06:55:51,368 INFO [train.py:1192] (1/2) Epoch 66, batch 450, loss[loss=0.2845, simple_loss=0.4035, pruned_loss=0.08269, over 24634.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3756, pruned_loss=0.06942, over 4319773.49 frames. ], batch size: 175, lr: 3.20e-03, grad_scale: 32.0 2026-09-24 06:55:56,100 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=209100.0, ans=0.125 2026-09-24 06:55:58,906 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.prob, batch_count=209100.0, ans=0.125 2026-09-24 06:55:58,910 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=209100.0, ans=0.1 2026-09-24 06:56:07,235 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.474e+02 3.290e+02 3.686e+02 4.229e+02 6.187e+02, threshold=7.373e+02, percent-clipped=0.0 2026-09-24 06:56:16,537 INFO [train.py:1192] (1/2) Epoch 66, batch 500, loss[loss=0.2504, simple_loss=0.3861, pruned_loss=0.05731, over 24492.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3738, pruned_loss=0.06868, over 4436762.34 frames. ], batch size: 218, lr: 3.19e-03, grad_scale: 32.0 2026-09-24 06:56:30,444 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=209300.0, ans=0.1 2026-09-24 06:56:39,975 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=209366.66666666666, ans=0.125 2026-09-24 06:56:41,924 INFO [train.py:1192] (1/2) Epoch 66, batch 550, loss[loss=0.279, simple_loss=0.4029, pruned_loss=0.07754, over 24304.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3741, pruned_loss=0.06884, over 4521888.67 frames. ], batch size: 257, lr: 3.19e-03, grad_scale: 32.0 2026-09-24 06:56:57,892 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.736e+02 3.320e+02 3.729e+02 3.919e+02 5.850e+02, threshold=7.458e+02, percent-clipped=0.0 2026-09-24 06:57:05,531 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.56 vs. limit=15.0 2026-09-24 06:57:06,924 INFO [train.py:1192] (1/2) Epoch 66, batch 600, loss[loss=0.2523, simple_loss=0.3858, pruned_loss=0.05942, over 24327.00 frames. ], tot_loss[loss=0.256, simple_loss=0.3745, pruned_loss=0.06874, over 4589086.61 frames. ], batch size: 234, lr: 3.19e-03, grad_scale: 32.0 2026-09-24 06:57:13,798 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=209600.0, ans=0.1 2026-09-24 06:57:14,342 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=209600.0, ans=0.1 2026-09-24 06:57:14,972 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.91 vs. limit=6.0 2026-09-24 06:57:18,150 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=8.08 vs. limit=15.0 2026-09-24 06:57:19,751 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=209633.33333333334, ans=0.1 2026-09-24 06:57:31,051 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=209700.0, ans=0.125 2026-09-24 06:57:31,970 INFO [train.py:1192] (1/2) Epoch 66, batch 650, loss[loss=0.2581, simple_loss=0.3691, pruned_loss=0.07357, over 24608.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3734, pruned_loss=0.06796, over 4653585.06 frames. ], batch size: 154, lr: 3.19e-03, grad_scale: 32.0 2026-09-24 06:57:32,059 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:57:40,990 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff2_skip_rate, batch_count=209766.66666666666, ans=0.0 2026-09-24 06:57:46,200 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=209800.0, ans=0.125 2026-09-24 06:57:46,996 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=209833.33333333334, ans=0.125 2026-09-24 06:57:48,343 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.742e+02 3.431e+02 3.744e+02 4.216e+02 6.535e+02, threshold=7.489e+02, percent-clipped=0.0 2026-09-24 06:57:50,139 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=209833.33333333334, ans=0.125 2026-09-24 06:57:57,160 INFO [train.py:1192] (1/2) Epoch 66, batch 700, loss[loss=0.251, simple_loss=0.3673, pruned_loss=0.06733, over 24551.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3747, pruned_loss=0.06842, over 4688879.00 frames. ], batch size: 158, lr: 3.19e-03, grad_scale: 32.0 2026-09-24 06:58:00,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=209900.0, ans=0.035 2026-09-24 06:58:04,246 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:58:04,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=209933.33333333334, ans=0.0 2026-09-24 06:58:10,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=209966.66666666666, ans=0.07 2026-09-24 06:58:12,097 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.54 vs. limit=6.0 2026-09-24 06:58:23,016 INFO [train.py:1192] (1/2) Epoch 66, batch 750, loss[loss=0.2726, simple_loss=0.3865, pruned_loss=0.07929, over 24610.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3749, pruned_loss=0.06917, over 4725672.61 frames. ], batch size: 175, lr: 3.19e-03, grad_scale: 32.0 2026-09-24 06:58:39,816 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 3.007e+02 3.496e+02 4.142e+02 4.613e+02 7.327e+02, threshold=8.284e+02, percent-clipped=0.0 2026-09-24 06:58:46,617 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=210200.0, ans=0.0 2026-09-24 06:58:48,446 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=210233.33333333334, ans=0.07 2026-09-24 06:58:48,822 INFO [train.py:1192] (1/2) Epoch 66, batch 800, loss[loss=0.2235, simple_loss=0.3366, pruned_loss=0.05526, over 24532.00 frames. ], tot_loss[loss=0.2564, simple_loss=0.3747, pruned_loss=0.06906, over 4752161.14 frames. ], batch size: 137, lr: 3.19e-03, grad_scale: 32.0 2026-09-24 06:58:53,571 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=210266.66666666666, ans=0.2 2026-09-24 06:58:53,581 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=210266.66666666666, ans=0.125 2026-09-24 06:59:13,475 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.40 vs. limit=15.0 2026-09-24 06:59:14,283 INFO [train.py:1192] (1/2) Epoch 66, batch 850, loss[loss=0.2849, simple_loss=0.4063, pruned_loss=0.08172, over 24559.00 frames. ], tot_loss[loss=0.2563, simple_loss=0.3745, pruned_loss=0.0691, over 4770523.92 frames. ], batch size: 204, lr: 3.19e-03, grad_scale: 32.0 2026-09-24 06:59:19,886 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=210433.33333333334, ans=0.125 2026-09-24 06:59:21,788 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=210433.33333333334, ans=0.125 2026-09-24 06:59:22,310 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=210433.33333333334, ans=10.0 2026-09-24 06:59:24,691 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=210466.66666666666, ans=0.125 2026-09-24 06:59:30,796 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.620e+02 3.507e+02 3.896e+02 4.569e+02 6.621e+02, threshold=7.791e+02, percent-clipped=0.0 2026-09-24 06:59:40,242 INFO [train.py:1192] (1/2) Epoch 66, batch 900, loss[loss=0.2054, simple_loss=0.323, pruned_loss=0.04393, over 24550.00 frames. ], tot_loss[loss=0.2574, simple_loss=0.3753, pruned_loss=0.06976, over 4780917.30 frames. ], batch size: 137, lr: 3.18e-03, grad_scale: 32.0 2026-09-24 06:59:43,580 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.skip_rate, batch_count=210566.66666666666, ans=0.035 2026-09-24 06:59:44,095 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=210566.66666666666, ans=0.0 2026-09-24 06:59:44,502 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=210566.66666666666, ans=0.025 2026-09-24 06:59:45,455 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.96 vs. limit=15.0 2026-09-24 06:59:48,239 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 06:59:49,744 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.23 vs. limit=15.0 2026-09-24 06:59:56,583 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=210666.66666666666, ans=0.125 2026-09-24 06:59:56,601 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=210666.66666666666, ans=0.025 2026-09-24 07:00:05,314 INFO [train.py:1192] (1/2) Epoch 66, batch 950, loss[loss=0.3628, simple_loss=0.4358, pruned_loss=0.1449, over 11545.00 frames. ], tot_loss[loss=0.2576, simple_loss=0.3742, pruned_loss=0.07046, over 4714678.05 frames. ], batch size: 333, lr: 3.18e-03, grad_scale: 16.0 2026-09-24 07:00:15,349 INFO [train.py:1192] (1/2) Epoch 67, batch 0, loss[loss=0.2106, simple_loss=0.3335, pruned_loss=0.04382, over 24551.00 frames. ], tot_loss[loss=0.2106, simple_loss=0.3335, pruned_loss=0.04382, over 24551.00 frames. ], batch size: 137, lr: 3.16e-03, grad_scale: 32.0 2026-09-24 07:00:15,349 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 07:00:26,986 INFO [train.py:1224] (1/2) Epoch 67, validation: loss=0.1709, simple_loss=0.2889, pruned_loss=0.02645, over 2564189.00 frames. 2026-09-24 07:00:26,986 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 07:00:27,100 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:00:27,942 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=210760.0, ans=0.2 2026-09-24 07:00:32,239 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=210793.33333333334, ans=0.0 2026-09-24 07:00:39,935 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.612e+02 3.476e+02 4.064e+02 4.784e+02 8.984e+02, threshold=8.128e+02, percent-clipped=1.0 2026-09-24 07:00:42,549 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=210860.0, ans=0.025 2026-09-24 07:00:52,611 INFO [train.py:1192] (1/2) Epoch 67, batch 50, loss[loss=0.2194, simple_loss=0.3302, pruned_loss=0.05428, over 24286.00 frames. ], tot_loss[loss=0.2597, simple_loss=0.3782, pruned_loss=0.07053, over 1080758.72 frames. ], batch size: 125, lr: 3.16e-03, grad_scale: 32.0 2026-09-24 07:00:55,109 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=210926.66666666666, ans=0.125 2026-09-24 07:01:02,911 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=210993.33333333334, ans=0.1 2026-09-24 07:01:08,751 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.min_abs, batch_count=211026.66666666666, ans=0.5 2026-09-24 07:01:11,891 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=211026.66666666666, ans=0.0 2026-09-24 07:01:15,428 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=3.80 vs. limit=10.0 2026-09-24 07:01:17,681 INFO [train.py:1192] (1/2) Epoch 67, batch 100, loss[loss=0.2465, simple_loss=0.3603, pruned_loss=0.06631, over 24602.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3821, pruned_loss=0.07128, over 1915639.82 frames. ], batch size: 154, lr: 3.16e-03, grad_scale: 32.0 2026-09-24 07:01:19,345 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.40 vs. limit=6.0 2026-09-24 07:01:25,007 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=211126.66666666666, ans=0.0 2026-09-24 07:01:25,019 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=211126.66666666666, ans=0.125 2026-09-24 07:01:25,984 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=211126.66666666666, ans=0.0 2026-09-24 07:01:30,025 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.574e+02 3.541e+02 3.935e+02 4.334e+02 7.130e+02, threshold=7.871e+02, percent-clipped=0.0 2026-09-24 07:01:37,086 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=211193.33333333334, ans=0.1 2026-09-24 07:01:37,607 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=211226.66666666666, ans=0.125 2026-09-24 07:01:43,176 INFO [train.py:1192] (1/2) Epoch 67, batch 150, loss[loss=0.2151, simple_loss=0.327, pruned_loss=0.0516, over 24283.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3768, pruned_loss=0.06912, over 2561035.04 frames. ], batch size: 125, lr: 3.16e-03, grad_scale: 32.0 2026-09-24 07:01:48,743 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=5.59 vs. limit=15.0 2026-09-24 07:01:56,569 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=211326.66666666666, ans=0.125 2026-09-24 07:01:58,949 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=211360.0, ans=0.2 2026-09-24 07:02:08,369 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=211393.33333333334, ans=0.025 2026-09-24 07:02:09,270 INFO [train.py:1192] (1/2) Epoch 67, batch 200, loss[loss=0.3159, simple_loss=0.434, pruned_loss=0.09885, over 24194.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3751, pruned_loss=0.06838, over 3059813.48 frames. ], batch size: 257, lr: 3.15e-03, grad_scale: 32.0 2026-09-24 07:02:17,375 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=211460.0, ans=0.125 2026-09-24 07:02:22,142 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.513e+02 3.399e+02 3.723e+02 4.270e+02 5.747e+02, threshold=7.446e+02, percent-clipped=0.0 2026-09-24 07:02:28,520 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=211526.66666666666, ans=0.125 2026-09-24 07:02:28,982 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=211526.66666666666, ans=0.1 2026-09-24 07:02:35,315 INFO [train.py:1192] (1/2) Epoch 67, batch 250, loss[loss=0.266, simple_loss=0.3958, pruned_loss=0.06814, over 24384.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3748, pruned_loss=0.06848, over 3444172.79 frames. ], batch size: 225, lr: 3.15e-03, grad_scale: 32.0 2026-09-24 07:02:39,664 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.74 vs. limit=15.0 2026-09-24 07:02:40,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=211626.66666666666, ans=0.1 2026-09-24 07:02:40,587 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=211626.66666666666, ans=0.125 2026-09-24 07:02:43,085 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=211626.66666666666, ans=0.0 2026-09-24 07:02:57,489 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=211726.66666666666, ans=0.0 2026-09-24 07:02:57,526 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=211726.66666666666, ans=0.025 2026-09-24 07:03:00,411 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=211726.66666666666, ans=0.0 2026-09-24 07:03:01,166 INFO [train.py:1192] (1/2) Epoch 67, batch 300, loss[loss=0.2632, simple_loss=0.3906, pruned_loss=0.0679, over 24550.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3742, pruned_loss=0.06861, over 3754044.60 frames. ], batch size: 204, lr: 3.15e-03, grad_scale: 32.0 2026-09-24 07:03:02,283 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=211760.0, ans=0.2 2026-09-24 07:03:04,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=211760.0, ans=0.2 2026-09-24 07:03:13,841 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.648e+02 3.406e+02 3.799e+02 4.131e+02 6.192e+02, threshold=7.597e+02, percent-clipped=0.0 2026-09-24 07:03:14,900 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=211826.66666666666, ans=0.1 2026-09-24 07:03:26,873 INFO [train.py:1192] (1/2) Epoch 67, batch 350, loss[loss=0.2315, simple_loss=0.3413, pruned_loss=0.06086, over 24573.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3752, pruned_loss=0.06904, over 3992191.10 frames. ], batch size: 137, lr: 3.15e-03, grad_scale: 32.0 2026-09-24 07:03:29,991 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=211926.66666666666, ans=0.125 2026-09-24 07:03:32,978 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=211960.0, ans=0.125 2026-09-24 07:03:36,221 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=211960.0, ans=0.025 2026-09-24 07:03:38,524 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=211993.33333333334, ans=0.125 2026-09-24 07:03:52,995 INFO [train.py:1192] (1/2) Epoch 67, batch 400, loss[loss=0.2751, simple_loss=0.3907, pruned_loss=0.07974, over 24585.00 frames. ], tot_loss[loss=0.2562, simple_loss=0.3747, pruned_loss=0.06889, over 4176276.05 frames. ], batch size: 170, lr: 3.15e-03, grad_scale: 32.0 2026-09-24 07:04:03,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=212160.0, ans=0.0 2026-09-24 07:04:06,105 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.632e+02 3.467e+02 3.832e+02 4.396e+02 7.099e+02, threshold=7.664e+02, percent-clipped=0.0 2026-09-24 07:04:18,260 INFO [train.py:1192] (1/2) Epoch 67, batch 450, loss[loss=0.2305, simple_loss=0.3588, pruned_loss=0.05112, over 24638.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.375, pruned_loss=0.06908, over 4317642.97 frames. ], batch size: 175, lr: 3.15e-03, grad_scale: 32.0 2026-09-24 07:04:23,804 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=212293.33333333334, ans=0.1 2026-09-24 07:04:44,113 INFO [train.py:1192] (1/2) Epoch 67, batch 500, loss[loss=0.2839, simple_loss=0.4093, pruned_loss=0.07922, over 24492.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3735, pruned_loss=0.06853, over 4436090.15 frames. ], batch size: 218, lr: 3.15e-03, grad_scale: 32.0 2026-09-24 07:04:46,011 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.min_positive, batch_count=212426.66666666666, ans=0.05 2026-09-24 07:04:51,894 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=212460.0, ans=0.0 2026-09-24 07:04:52,413 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=212460.0, ans=0.025 2026-09-24 07:04:56,829 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.703e+02 3.364e+02 3.669e+02 4.236e+02 6.673e+02, threshold=7.338e+02, percent-clipped=0.0 2026-09-24 07:05:02,463 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=212526.66666666666, ans=0.125 2026-09-24 07:05:02,470 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=212526.66666666666, ans=0.025 2026-09-24 07:05:09,565 INFO [train.py:1192] (1/2) Epoch 67, batch 550, loss[loss=0.2699, simple_loss=0.3985, pruned_loss=0.07058, over 24274.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3741, pruned_loss=0.06866, over 4521196.24 frames. ], batch size: 257, lr: 3.15e-03, grad_scale: 32.0 2026-09-24 07:05:11,132 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=212593.33333333334, ans=0.125 2026-09-24 07:05:13,460 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=212593.33333333334, ans=0.0 2026-09-24 07:05:14,900 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=212626.66666666666, ans=0.125 2026-09-24 07:05:28,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=212693.33333333334, ans=0.125 2026-09-24 07:05:30,556 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=212726.66666666666, ans=0.0 2026-09-24 07:05:32,110 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=212726.66666666666, ans=0.125 2026-09-24 07:05:35,537 INFO [train.py:1192] (1/2) Epoch 67, batch 600, loss[loss=0.2535, simple_loss=0.3833, pruned_loss=0.06183, over 24302.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3751, pruned_loss=0.06907, over 4587884.35 frames. ], batch size: 234, lr: 3.14e-03, grad_scale: 32.0 2026-09-24 07:05:36,900 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=212760.0, ans=0.125 2026-09-24 07:05:40,247 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=212793.33333333334, ans=0.2 2026-09-24 07:05:42,852 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=212793.33333333334, ans=0.0 2026-09-24 07:05:44,460 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=212793.33333333334, ans=0.2 2026-09-24 07:05:48,145 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=212826.66666666666, ans=0.0 2026-09-24 07:05:48,924 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.724e+02 3.420e+02 3.759e+02 4.319e+02 7.115e+02, threshold=7.517e+02, percent-clipped=0.0 2026-09-24 07:05:51,136 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.41 vs. limit=15.0 2026-09-24 07:06:01,452 INFO [train.py:1192] (1/2) Epoch 67, batch 650, loss[loss=0.2586, simple_loss=0.3763, pruned_loss=0.07042, over 24584.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3737, pruned_loss=0.06818, over 4653252.83 frames. ], batch size: 154, lr: 3.14e-03, grad_scale: 32.0 2026-09-24 07:06:14,517 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.41 vs. limit=15.0 2026-09-24 07:06:16,615 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=213026.66666666666, ans=0.04949747468305833 2026-09-24 07:06:19,535 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.skip_rate, batch_count=213026.66666666666, ans=0.04949747468305833 2026-09-24 07:06:26,440 INFO [train.py:1192] (1/2) Epoch 67, batch 700, loss[loss=0.2409, simple_loss=0.3608, pruned_loss=0.06049, over 24559.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.3744, pruned_loss=0.06822, over 4688842.87 frames. ], batch size: 158, lr: 3.14e-03, grad_scale: 32.0 2026-09-24 07:06:28,718 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=213093.33333333334, ans=0.2 2026-09-24 07:06:29,300 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=213093.33333333334, ans=0.125 2026-09-24 07:06:34,174 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=213126.66666666666, ans=0.125 2026-09-24 07:06:39,314 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.576e+02 3.459e+02 3.714e+02 4.345e+02 6.399e+02, threshold=7.428e+02, percent-clipped=0.0 2026-09-24 07:06:52,193 INFO [train.py:1192] (1/2) Epoch 67, batch 750, loss[loss=0.2555, simple_loss=0.3809, pruned_loss=0.06504, over 24632.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3735, pruned_loss=0.06807, over 4722577.11 frames. ], batch size: 175, lr: 3.14e-03, grad_scale: 32.0 2026-09-24 07:06:56,397 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.67 vs. limit=15.0 2026-09-24 07:06:56,738 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=213260.0, ans=0.1 2026-09-24 07:06:59,265 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=213293.33333333334, ans=0.125 2026-09-24 07:07:07,214 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=213326.66666666666, ans=0.125 2026-09-24 07:07:17,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=213426.66666666666, ans=0.125 2026-09-24 07:07:18,307 INFO [train.py:1192] (1/2) Epoch 67, batch 800, loss[loss=0.2016, simple_loss=0.3233, pruned_loss=0.03997, over 24539.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3732, pruned_loss=0.06776, over 4749060.56 frames. ], batch size: 137, lr: 3.14e-03, grad_scale: 32.0 2026-09-24 07:07:30,583 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:07:31,425 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.871e+02 3.614e+02 4.001e+02 4.536e+02 6.366e+02, threshold=8.002e+02, percent-clipped=0.0 2026-09-24 07:07:33,881 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=213526.66666666666, ans=0.125 2026-09-24 07:07:37,396 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=213526.66666666666, ans=0.0 2026-09-24 07:07:37,415 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=213526.66666666666, ans=0.07 2026-09-24 07:07:40,248 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=213560.0, ans=0.1 2026-09-24 07:07:43,985 INFO [train.py:1192] (1/2) Epoch 67, batch 850, loss[loss=0.2839, simple_loss=0.4118, pruned_loss=0.07798, over 24521.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3728, pruned_loss=0.06761, over 4767891.41 frames. ], batch size: 204, lr: 3.14e-03, grad_scale: 32.0 2026-09-24 07:07:52,690 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=213626.66666666666, ans=0.125 2026-09-24 07:07:59,251 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=4.45 vs. limit=12.0 2026-09-24 07:08:07,867 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=13.00 vs. limit=22.5 2026-09-24 07:08:09,558 INFO [train.py:1192] (1/2) Epoch 67, batch 900, loss[loss=0.2044, simple_loss=0.3275, pruned_loss=0.04058, over 24565.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3732, pruned_loss=0.06761, over 4778682.29 frames. ], batch size: 137, lr: 3.14e-03, grad_scale: 32.0 2026-09-24 07:08:10,136 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=213760.0, ans=0.0 2026-09-24 07:08:16,488 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=213793.33333333334, ans=0.125 2026-09-24 07:08:21,022 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.const_attention_rate, batch_count=213826.66666666666, ans=0.025 2026-09-24 07:08:22,420 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.594e+02 3.509e+02 4.010e+02 4.495e+02 6.260e+02, threshold=8.019e+02, percent-clipped=0.0 2026-09-24 07:08:31,976 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=213893.33333333334, ans=0.0 2026-09-24 07:08:35,373 INFO [train.py:1192] (1/2) Epoch 67, batch 950, loss[loss=0.3295, simple_loss=0.4036, pruned_loss=0.1277, over 11098.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3723, pruned_loss=0.06913, over 4714368.13 frames. ], batch size: 333, lr: 3.14e-03, grad_scale: 32.0 2026-09-24 07:08:37,342 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer_na.min_abs, batch_count=213926.66666666666, ans=0.02 2026-09-24 07:08:38,307 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=213926.66666666666, ans=0.125 2026-09-24 07:08:44,514 INFO [train.py:1192] (1/2) Epoch 68, batch 0, loss[loss=0.2079, simple_loss=0.3295, pruned_loss=0.04315, over 24579.00 frames. ], tot_loss[loss=0.2079, simple_loss=0.3295, pruned_loss=0.04315, over 24579.00 frames. ], batch size: 137, lr: 3.11e-03, grad_scale: 32.0 2026-09-24 07:08:44,514 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 07:08:47,769 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.7944, 2.2200, 3.1731, 1.5368], device='cuda:1') 2026-09-24 07:08:48,090 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.7713, 2.6665, 2.1562, 3.1725], device='cuda:1') 2026-09-24 07:08:56,320 INFO [train.py:1224] (1/2) Epoch 68, validation: loss=0.17, simple_loss=0.2884, pruned_loss=0.0258, over 2564189.00 frames. 2026-09-24 07:08:56,321 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 07:08:56,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=213953.33333333334, ans=10.0 2026-09-24 07:08:57,449 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.30 vs. limit=15.0 2026-09-24 07:09:00,891 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=213953.33333333334, ans=0.125 2026-09-24 07:09:09,491 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.15 vs. limit=6.0 2026-09-24 07:09:12,435 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=214053.33333333334, ans=0.1 2026-09-24 07:09:13,755 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=214053.33333333334, ans=0.125 2026-09-24 07:09:18,089 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.prob, batch_count=214086.66666666666, ans=0.125 2026-09-24 07:09:22,136 INFO [train.py:1192] (1/2) Epoch 68, batch 50, loss[loss=0.2072, simple_loss=0.323, pruned_loss=0.04572, over 24242.00 frames. ], tot_loss[loss=0.2612, simple_loss=0.38, pruned_loss=0.07118, over 1081644.00 frames. ], batch size: 125, lr: 3.11e-03, grad_scale: 32.0 2026-09-24 07:09:28,754 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=214153.33333333334, ans=0.5 2026-09-24 07:09:30,324 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.745e+02 3.640e+02 4.106e+02 4.924e+02 7.084e+02, threshold=8.211e+02, percent-clipped=0.0 2026-09-24 07:09:33,631 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=214186.66666666666, ans=0.125 2026-09-24 07:09:38,096 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.prob, batch_count=214220.0, ans=0.125 2026-09-24 07:09:44,539 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.35 vs. limit=15.0 2026-09-24 07:09:46,862 INFO [train.py:1192] (1/2) Epoch 68, batch 100, loss[loss=0.2409, simple_loss=0.3629, pruned_loss=0.05941, over 24587.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3828, pruned_loss=0.0709, over 1916470.66 frames. ], batch size: 154, lr: 3.11e-03, grad_scale: 32.0 2026-09-24 07:09:50,446 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.09 vs. limit=15.0 2026-09-24 07:10:02,900 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=214386.66666666666, ans=0.0 2026-09-24 07:10:05,872 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=214386.66666666666, ans=0.125 2026-09-24 07:10:08,332 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=214420.0, ans=0.2 2026-09-24 07:10:08,741 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=214420.0, ans=0.0 2026-09-24 07:10:12,471 INFO [train.py:1192] (1/2) Epoch 68, batch 150, loss[loss=0.2106, simple_loss=0.327, pruned_loss=0.04705, over 24314.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3787, pruned_loss=0.0699, over 2561430.51 frames. ], batch size: 125, lr: 3.11e-03, grad_scale: 32.0 2026-09-24 07:10:13,106 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=214453.33333333334, ans=0.2 2026-09-24 07:10:17,575 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.61 vs. limit=15.0 2026-09-24 07:10:21,413 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.370e+02 3.475e+02 3.740e+02 4.253e+02 5.776e+02, threshold=7.480e+02, percent-clipped=0.0 2026-09-24 07:10:30,850 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=214553.33333333334, ans=0.1 2026-09-24 07:10:38,400 INFO [train.py:1192] (1/2) Epoch 68, batch 200, loss[loss=0.2825, simple_loss=0.4097, pruned_loss=0.07759, over 24239.00 frames. ], tot_loss[loss=0.2562, simple_loss=0.3756, pruned_loss=0.06837, over 3060501.37 frames. ], batch size: 257, lr: 3.11e-03, grad_scale: 32.0 2026-09-24 07:11:00,814 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=8.87 vs. limit=22.5 2026-09-24 07:11:03,806 INFO [train.py:1192] (1/2) Epoch 68, batch 250, loss[loss=0.271, simple_loss=0.3994, pruned_loss=0.07133, over 24382.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3749, pruned_loss=0.06864, over 3443228.35 frames. ], batch size: 225, lr: 3.11e-03, grad_scale: 32.0 2026-09-24 07:11:06,826 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=214786.66666666666, ans=0.0 2026-09-24 07:11:12,904 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.745e+02 3.398e+02 3.956e+02 4.810e+02 6.309e+02, threshold=7.912e+02, percent-clipped=0.0 2026-09-24 07:11:14,050 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=214853.33333333334, ans=0.2 2026-09-24 07:11:14,518 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=214853.33333333334, ans=0.1 2026-09-24 07:11:23,015 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=214886.66666666666, ans=0.1 2026-09-24 07:11:24,017 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=214886.66666666666, ans=0.0 2026-09-24 07:11:30,098 INFO [train.py:1192] (1/2) Epoch 68, batch 300, loss[loss=0.2876, simple_loss=0.4133, pruned_loss=0.08099, over 24550.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3746, pruned_loss=0.06845, over 3754195.39 frames. ], batch size: 204, lr: 3.11e-03, grad_scale: 32.0 2026-09-24 07:11:30,643 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=214953.33333333334, ans=0.0 2026-09-24 07:11:44,445 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=215020.0, ans=0.025 2026-09-24 07:11:45,007 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=215020.0, ans=0.2 2026-09-24 07:11:45,543 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer1.prob, batch_count=215053.33333333334, ans=0.125 2026-09-24 07:11:55,921 INFO [train.py:1192] (1/2) Epoch 68, batch 350, loss[loss=0.197, simple_loss=0.3172, pruned_loss=0.03835, over 24587.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3752, pruned_loss=0.06849, over 3995340.30 frames. ], batch size: 137, lr: 3.10e-03, grad_scale: 32.0 2026-09-24 07:12:04,078 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.822e+02 3.399e+02 3.724e+02 4.282e+02 5.845e+02, threshold=7.447e+02, percent-clipped=0.0 2026-09-24 07:12:07,275 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=10.22 vs. limit=15.0 2026-09-24 07:12:08,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=215186.66666666666, ans=0.07 2026-09-24 07:12:10,001 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=215186.66666666666, ans=0.125 2026-09-24 07:12:14,681 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=5.89 vs. limit=15.0 2026-09-24 07:12:16,945 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=215253.33333333334, ans=0.125 2026-09-24 07:12:18,418 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=215253.33333333334, ans=0.025 2026-09-24 07:12:21,559 INFO [train.py:1192] (1/2) Epoch 68, batch 400, loss[loss=0.2493, simple_loss=0.3777, pruned_loss=0.06048, over 24578.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.374, pruned_loss=0.0678, over 4180441.49 frames. ], batch size: 170, lr: 3.10e-03, grad_scale: 32.0 2026-09-24 07:12:31,946 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=215353.33333333334, ans=0.1 2026-09-24 07:12:40,143 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=215386.66666666666, ans=0.1 2026-09-24 07:12:47,196 INFO [train.py:1192] (1/2) Epoch 68, batch 450, loss[loss=0.2609, simple_loss=0.3847, pruned_loss=0.06856, over 24631.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.3744, pruned_loss=0.06826, over 4320768.44 frames. ], batch size: 175, lr: 3.10e-03, grad_scale: 32.0 2026-09-24 07:12:55,952 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.825e+02 3.489e+02 3.928e+02 4.300e+02 6.364e+02, threshold=7.857e+02, percent-clipped=0.0 2026-09-24 07:13:02,261 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=215553.33333333334, ans=0.0 2026-09-24 07:13:04,617 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=215553.33333333334, ans=0.0 2026-09-24 07:13:09,126 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=215586.66666666666, ans=10.0 2026-09-24 07:13:12,698 INFO [train.py:1192] (1/2) Epoch 68, batch 500, loss[loss=0.2745, simple_loss=0.4049, pruned_loss=0.07205, over 24519.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.373, pruned_loss=0.06774, over 4438382.29 frames. ], batch size: 218, lr: 3.10e-03, grad_scale: 32.0 2026-09-24 07:13:18,617 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=215653.33333333334, ans=0.0 2026-09-24 07:13:21,080 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=9.74 vs. limit=15.0 2026-09-24 07:13:24,037 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=215686.66666666666, ans=0.125 2026-09-24 07:13:27,754 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=215720.0, ans=0.1 2026-09-24 07:13:34,146 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=215753.33333333334, ans=0.025 2026-09-24 07:13:38,728 INFO [train.py:1192] (1/2) Epoch 68, batch 550, loss[loss=0.3032, simple_loss=0.4244, pruned_loss=0.09096, over 24255.00 frames. ], tot_loss[loss=0.2545, simple_loss=0.3732, pruned_loss=0.0679, over 4523914.37 frames. ], batch size: 257, lr: 3.10e-03, grad_scale: 32.0 2026-09-24 07:13:39,319 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=215786.66666666666, ans=0.025 2026-09-24 07:13:46,470 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=215820.0, ans=0.125 2026-09-24 07:13:47,376 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.517e+02 3.330e+02 3.662e+02 4.043e+02 5.524e+02, threshold=7.324e+02, percent-clipped=0.0 2026-09-24 07:13:51,184 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=215853.33333333334, ans=0.1 2026-09-24 07:14:04,510 INFO [train.py:1192] (1/2) Epoch 68, batch 600, loss[loss=0.2969, simple_loss=0.4167, pruned_loss=0.08861, over 24322.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.3741, pruned_loss=0.06842, over 4589971.18 frames. ], batch size: 234, lr: 3.10e-03, grad_scale: 32.0 2026-09-24 07:14:13,170 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:14:15,553 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=216020.0, ans=0.1 2026-09-24 07:14:29,990 INFO [train.py:1192] (1/2) Epoch 68, batch 650, loss[loss=0.247, simple_loss=0.3656, pruned_loss=0.06414, over 24624.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3725, pruned_loss=0.06731, over 4654479.84 frames. ], batch size: 154, lr: 3.10e-03, grad_scale: 32.0 2026-09-24 07:14:31,568 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=216120.0, ans=0.125 2026-09-24 07:14:38,831 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.864e+02 3.384e+02 3.669e+02 4.044e+02 6.181e+02, threshold=7.338e+02, percent-clipped=0.0 2026-09-24 07:14:39,905 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=1.044e-02 2026-09-24 07:14:40,346 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=216186.66666666666, ans=0.05 2026-09-24 07:14:42,735 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=216186.66666666666, ans=0.025 2026-09-24 07:14:47,258 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.63 vs. limit=10.0 2026-09-24 07:14:49,185 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=216220.0, ans=0.2 2026-09-24 07:14:49,272 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.02 vs. limit=22.5 2026-09-24 07:14:50,538 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=216253.33333333334, ans=0.1 2026-09-24 07:14:51,006 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=216253.33333333334, ans=0.125 2026-09-24 07:14:53,552 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=216253.33333333334, ans=0.1 2026-09-24 07:14:53,995 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass_mid.scale_min, batch_count=216253.33333333334, ans=0.2 2026-09-24 07:14:55,706 INFO [train.py:1192] (1/2) Epoch 68, batch 700, loss[loss=0.2471, simple_loss=0.3662, pruned_loss=0.064, over 24557.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3738, pruned_loss=0.06783, over 4691099.55 frames. ], batch size: 158, lr: 3.10e-03, grad_scale: 32.0 2026-09-24 07:15:00,574 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=216320.0, ans=0.2 2026-09-24 07:15:03,613 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=216320.0, ans=0.125 2026-09-24 07:15:11,678 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.const_attention_rate, batch_count=216386.66666666666, ans=0.025 2026-09-24 07:15:13,133 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=216386.66666666666, ans=0.0 2026-09-24 07:15:17,618 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.out_balancer.prob, batch_count=216420.0, ans=0.125 2026-09-24 07:15:21,786 INFO [train.py:1192] (1/2) Epoch 68, batch 750, loss[loss=0.2622, simple_loss=0.3803, pruned_loss=0.07205, over 24625.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3731, pruned_loss=0.06768, over 4727120.77 frames. ], batch size: 175, lr: 3.09e-03, grad_scale: 32.0 2026-09-24 07:15:22,727 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=216453.33333333334, ans=0.125 2026-09-24 07:15:30,336 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.632e+02 3.551e+02 4.134e+02 4.479e+02 7.371e+02, threshold=8.267e+02, percent-clipped=1.0 2026-09-24 07:15:35,083 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=216520.0, ans=0.125 2026-09-24 07:15:39,703 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=216553.33333333334, ans=0.1 2026-09-24 07:15:45,126 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=216586.66666666666, ans=0.1 2026-09-24 07:15:46,846 INFO [train.py:1192] (1/2) Epoch 68, batch 800, loss[loss=0.2399, simple_loss=0.3534, pruned_loss=0.06322, over 24554.00 frames. ], tot_loss[loss=0.2537, simple_loss=0.3726, pruned_loss=0.06737, over 4753463.11 frames. ], batch size: 137, lr: 3.09e-03, grad_scale: 32.0 2026-09-24 07:15:49,904 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=216620.0, ans=0.1 2026-09-24 07:15:51,129 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=216620.0, ans=0.025 2026-09-24 07:15:52,221 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.19 vs. limit=10.0 2026-09-24 07:16:00,003 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=216686.66666666666, ans=0.125 2026-09-24 07:16:00,026 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff3_skip_rate, batch_count=216686.66666666666, ans=0.0 2026-09-24 07:16:00,970 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=216686.66666666666, ans=0.0 2026-09-24 07:16:00,998 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:16:07,893 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=216753.33333333334, ans=0.0 2026-09-24 07:16:12,511 INFO [train.py:1192] (1/2) Epoch 68, batch 850, loss[loss=0.2796, simple_loss=0.3986, pruned_loss=0.08026, over 24528.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3721, pruned_loss=0.06715, over 4771773.86 frames. ], batch size: 204, lr: 3.09e-03, grad_scale: 32.0 2026-09-24 07:16:21,233 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.733e+02 3.458e+02 3.987e+02 4.457e+02 5.664e+02, threshold=7.974e+02, percent-clipped=0.0 2026-09-24 07:16:21,861 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=216820.0, ans=0.125 2026-09-24 07:16:24,637 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=216853.33333333334, ans=0.125 2026-09-24 07:16:34,713 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=216920.0, ans=0.5 2026-09-24 07:16:38,118 INFO [train.py:1192] (1/2) Epoch 68, batch 900, loss[loss=0.2031, simple_loss=0.3251, pruned_loss=0.04049, over 24564.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.3729, pruned_loss=0.0678, over 4782720.05 frames. ], batch size: 137, lr: 3.09e-03, grad_scale: 32.0 2026-09-24 07:16:59,048 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=217086.66666666666, ans=0.0 2026-09-24 07:17:02,733 INFO [train.py:1192] (1/2) Epoch 68, batch 950, loss[loss=0.3799, simple_loss=0.4436, pruned_loss=0.1581, over 11553.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3717, pruned_loss=0.06856, over 4711285.25 frames. ], batch size: 333, lr: 3.09e-03, grad_scale: 32.0 2026-09-24 07:17:40,884 INFO [train.py:1192] (1/2) Epoch 69, batch 0, loss[loss=0.2101, simple_loss=0.3314, pruned_loss=0.04439, over 24577.00 frames. ], tot_loss[loss=0.2101, simple_loss=0.3314, pruned_loss=0.04439, over 24577.00 frames. ], batch size: 137, lr: 3.07e-03, grad_scale: 32.0 2026-09-24 07:17:40,884 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 07:17:52,461 INFO [train.py:1224] (1/2) Epoch 69, validation: loss=0.1691, simple_loss=0.2872, pruned_loss=0.02547, over 2564189.00 frames. 2026-09-24 07:17:52,461 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 07:17:56,833 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.733e+02 3.531e+02 4.038e+02 4.647e+02 6.528e+02, threshold=8.076e+02, percent-clipped=0.0 2026-09-24 07:17:58,403 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.attention_skip_rate, batch_count=217180.0, ans=0.0 2026-09-24 07:18:06,937 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=217246.66666666666, ans=0.125 2026-09-24 07:18:09,720 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=217246.66666666666, ans=0.125 2026-09-24 07:18:16,106 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=217280.0, ans=0.125 2026-09-24 07:18:17,522 INFO [train.py:1192] (1/2) Epoch 69, batch 50, loss[loss=0.2323, simple_loss=0.3402, pruned_loss=0.06225, over 24294.00 frames. ], tot_loss[loss=0.2587, simple_loss=0.3779, pruned_loss=0.06975, over 1081951.21 frames. ], batch size: 125, lr: 3.07e-03, grad_scale: 32.0 2026-09-24 07:18:28,442 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=217380.0, ans=0.1 2026-09-24 07:18:34,469 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=217413.33333333334, ans=0.0 2026-09-24 07:18:43,403 INFO [train.py:1192] (1/2) Epoch 69, batch 100, loss[loss=0.2575, simple_loss=0.3641, pruned_loss=0.07548, over 24597.00 frames. ], tot_loss[loss=0.2623, simple_loss=0.3822, pruned_loss=0.07116, over 1915691.95 frames. ], batch size: 154, lr: 3.06e-03, grad_scale: 64.0 2026-09-24 07:18:47,696 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.741e+02 3.588e+02 3.935e+02 4.341e+02 5.984e+02, threshold=7.870e+02, percent-clipped=0.0 2026-09-24 07:18:53,607 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=217546.66666666666, ans=0.125 2026-09-24 07:19:01,296 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=217580.0, ans=0.0 2026-09-24 07:19:05,756 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=217613.33333333334, ans=0.1 2026-09-24 07:19:08,937 INFO [train.py:1192] (1/2) Epoch 69, batch 150, loss[loss=0.2079, simple_loss=0.3223, pruned_loss=0.04677, over 24239.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.377, pruned_loss=0.06919, over 2560964.50 frames. ], batch size: 125, lr: 3.06e-03, grad_scale: 64.0 2026-09-24 07:19:11,536 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.89 vs. limit=22.5 2026-09-24 07:19:34,964 INFO [train.py:1192] (1/2) Epoch 69, batch 200, loss[loss=0.3056, simple_loss=0.4285, pruned_loss=0.0914, over 24218.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3744, pruned_loss=0.06782, over 3061409.21 frames. ], batch size: 257, lr: 3.06e-03, grad_scale: 64.0 2026-09-24 07:19:39,199 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.799e+02 3.352e+02 3.841e+02 4.410e+02 6.405e+02, threshold=7.682e+02, percent-clipped=0.0 2026-09-24 07:19:39,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=217846.66666666666, ans=0.0 2026-09-24 07:19:45,525 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=217880.0, ans=0.0 2026-09-24 07:19:49,028 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.const_attention_rate, batch_count=217880.0, ans=0.025 2026-09-24 07:19:52,959 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=217913.33333333334, ans=0.125 2026-09-24 07:19:53,404 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=217913.33333333334, ans=0.2 2026-09-24 07:20:00,536 INFO [train.py:1192] (1/2) Epoch 69, batch 250, loss[loss=0.2744, simple_loss=0.403, pruned_loss=0.07288, over 24390.00 frames. ], tot_loss[loss=0.2548, simple_loss=0.3739, pruned_loss=0.06787, over 3445862.38 frames. ], batch size: 225, lr: 3.06e-03, grad_scale: 32.0 2026-09-24 07:20:00,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=217980.0, ans=0.0 2026-09-24 07:20:07,590 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=218013.33333333334, ans=0.125 2026-09-24 07:20:10,779 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=218046.66666666666, ans=0.2 2026-09-24 07:20:12,639 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=218046.66666666666, ans=0.025 2026-09-24 07:20:13,167 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.skip_rate, batch_count=218046.66666666666, ans=0.07 2026-09-24 07:20:14,137 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.41 vs. limit=22.5 2026-09-24 07:20:26,401 INFO [train.py:1192] (1/2) Epoch 69, batch 300, loss[loss=0.2863, simple_loss=0.4107, pruned_loss=0.081, over 24565.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3727, pruned_loss=0.06716, over 3759269.96 frames. ], batch size: 204, lr: 3.06e-03, grad_scale: 32.0 2026-09-24 07:20:31,668 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.606e+02 3.510e+02 4.031e+02 4.663e+02 6.511e+02, threshold=8.061e+02, percent-clipped=0.0 2026-09-24 07:20:32,758 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.28 vs. limit=15.0 2026-09-24 07:20:37,387 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=218213.33333333334, ans=0.0 2026-09-24 07:20:43,169 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=218246.66666666666, ans=0.1 2026-09-24 07:20:50,035 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=9.32 vs. limit=12.0 2026-09-24 07:20:52,928 INFO [train.py:1192] (1/2) Epoch 69, batch 350, loss[loss=0.2297, simple_loss=0.3388, pruned_loss=0.06036, over 24586.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.3742, pruned_loss=0.06809, over 3999351.84 frames. ], batch size: 137, lr: 3.06e-03, grad_scale: 32.0 2026-09-24 07:20:53,984 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=218313.33333333334, ans=0.125 2026-09-24 07:20:54,026 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:21:01,654 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=218346.66666666666, ans=0.2 2026-09-24 07:21:06,530 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=6.20 vs. limit=15.0 2026-09-24 07:21:09,261 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.74 vs. limit=8.0 2026-09-24 07:21:10,408 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=218413.33333333334, ans=0.1 2026-09-24 07:21:10,420 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=218413.33333333334, ans=0.2 2026-09-24 07:21:18,286 INFO [train.py:1192] (1/2) Epoch 69, batch 400, loss[loss=0.2413, simple_loss=0.3667, pruned_loss=0.05794, over 24560.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3733, pruned_loss=0.06751, over 4184955.14 frames. ], batch size: 170, lr: 3.06e-03, grad_scale: 32.0 2026-09-24 07:21:23,428 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.919e+02 3.473e+02 3.848e+02 4.402e+02 6.832e+02, threshold=7.696e+02, percent-clipped=0.0 2026-09-24 07:21:26,013 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=218513.33333333334, ans=0.0 2026-09-24 07:21:44,374 INFO [train.py:1192] (1/2) Epoch 69, batch 450, loss[loss=0.2735, simple_loss=0.396, pruned_loss=0.0755, over 24620.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3744, pruned_loss=0.06847, over 4324015.15 frames. ], batch size: 175, lr: 3.06e-03, grad_scale: 16.0 2026-09-24 07:21:44,832 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass.scale_min, batch_count=218646.66666666666, ans=0.2 2026-09-24 07:21:45,310 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=218646.66666666666, ans=0.125 2026-09-24 07:21:45,821 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=218646.66666666666, ans=0.0 2026-09-24 07:21:51,069 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:21:59,023 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=218713.33333333334, ans=0.0 2026-09-24 07:22:10,551 INFO [train.py:1192] (1/2) Epoch 69, batch 500, loss[loss=0.2922, simple_loss=0.4128, pruned_loss=0.08576, over 24487.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.3729, pruned_loss=0.06785, over 4440382.39 frames. ], batch size: 218, lr: 3.06e-03, grad_scale: 16.0 2026-09-24 07:22:11,689 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.98 vs. limit=12.0 2026-09-24 07:22:16,003 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.776e+02 3.243e+02 3.613e+02 4.164e+02 1.083e+03, threshold=7.226e+02, percent-clipped=1.0 2026-09-24 07:22:16,104 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=218846.66666666666, ans=0.2 2026-09-24 07:22:18,543 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=218846.66666666666, ans=0.125 2026-09-24 07:22:33,092 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=218946.66666666666, ans=0.1 2026-09-24 07:22:36,592 INFO [train.py:1192] (1/2) Epoch 69, batch 550, loss[loss=0.2829, simple_loss=0.4075, pruned_loss=0.0791, over 24313.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3735, pruned_loss=0.06819, over 4524338.32 frames. ], batch size: 257, lr: 3.05e-03, grad_scale: 16.0 2026-09-24 07:22:45,336 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=219013.33333333334, ans=0.0 2026-09-24 07:22:48,209 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=219046.66666666666, ans=0.2 2026-09-24 07:22:54,229 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=219080.0, ans=0.125 2026-09-24 07:22:58,491 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer2.prob, batch_count=219113.33333333334, ans=0.125 2026-09-24 07:23:02,641 INFO [train.py:1192] (1/2) Epoch 69, batch 600, loss[loss=0.2751, simple_loss=0.4063, pruned_loss=0.07192, over 24327.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3744, pruned_loss=0.06845, over 4590087.19 frames. ], batch size: 234, lr: 3.05e-03, grad_scale: 16.0 2026-09-24 07:23:07,411 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff2_skip_rate, batch_count=219180.0, ans=0.0 2026-09-24 07:23:08,221 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.677e+02 3.268e+02 3.901e+02 4.607e+02 6.489e+02, threshold=7.802e+02, percent-clipped=0.0 2026-09-24 07:23:09,950 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=219180.0, ans=0.1 2026-09-24 07:23:10,418 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=219180.0, ans=0.125 2026-09-24 07:23:18,180 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=219246.66666666666, ans=0.125 2026-09-24 07:23:22,069 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=219246.66666666666, ans=0.125 2026-09-24 07:23:24,083 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=219280.0, ans=0.125 2026-09-24 07:23:27,053 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=5.45 vs. limit=12.0 2026-09-24 07:23:27,820 INFO [train.py:1192] (1/2) Epoch 69, batch 650, loss[loss=0.2644, simple_loss=0.3786, pruned_loss=0.07511, over 24587.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3731, pruned_loss=0.06766, over 4654819.55 frames. ], batch size: 154, lr: 3.05e-03, grad_scale: 16.0 2026-09-24 07:23:48,931 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=219446.66666666666, ans=0.125 2026-09-24 07:23:53,974 INFO [train.py:1192] (1/2) Epoch 69, batch 700, loss[loss=0.2684, simple_loss=0.38, pruned_loss=0.07841, over 24549.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3747, pruned_loss=0.06834, over 4690341.72 frames. ], batch size: 158, lr: 3.05e-03, grad_scale: 16.0 2026-09-24 07:23:59,796 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.624e+02 3.327e+02 3.649e+02 4.152e+02 7.303e+02, threshold=7.297e+02, percent-clipped=0.0 2026-09-24 07:24:05,362 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass_mid.scale_min, batch_count=219546.66666666666, ans=0.2 2026-09-24 07:24:05,603 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=11.06 vs. limit=22.5 2026-09-24 07:24:19,851 INFO [train.py:1192] (1/2) Epoch 69, batch 750, loss[loss=0.2725, simple_loss=0.3947, pruned_loss=0.07513, over 24629.00 frames. ], tot_loss[loss=0.2551, simple_loss=0.3739, pruned_loss=0.06821, over 4726719.95 frames. ], batch size: 175, lr: 3.05e-03, grad_scale: 16.0 2026-09-24 07:24:28,477 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:24:34,523 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=8.62 vs. limit=22.5 2026-09-24 07:24:40,052 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.97 vs. limit=15.0 2026-09-24 07:24:40,534 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.77 vs. limit=10.0 2026-09-24 07:24:45,599 INFO [train.py:1192] (1/2) Epoch 69, batch 800, loss[loss=0.238, simple_loss=0.3475, pruned_loss=0.06421, over 24521.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.3732, pruned_loss=0.06773, over 4752651.13 frames. ], batch size: 137, lr: 3.05e-03, grad_scale: 32.0 2026-09-24 07:24:45,670 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=219813.33333333334, ans=0.0 2026-09-24 07:24:49,595 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.skip_rate, batch_count=219813.33333333334, ans=0.04949747468305833 2026-09-24 07:24:50,958 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=16.90 vs. limit=22.5 2026-09-24 07:24:51,633 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.655e+02 3.384e+02 3.857e+02 4.281e+02 7.154e+02, threshold=7.715e+02, percent-clipped=0.0 2026-09-24 07:25:11,172 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.94 vs. limit=15.0 2026-09-24 07:25:11,908 INFO [train.py:1192] (1/2) Epoch 69, batch 850, loss[loss=0.2848, simple_loss=0.4136, pruned_loss=0.078, over 24514.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3731, pruned_loss=0.0678, over 4770438.40 frames. ], batch size: 204, lr: 3.05e-03, grad_scale: 32.0 2026-09-24 07:25:18,414 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.01 vs. limit=22.5 2026-09-24 07:25:18,797 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=220013.33333333334, ans=0.125 2026-09-24 07:25:19,201 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=220013.33333333334, ans=0.125 2026-09-24 07:25:23,744 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=220046.66666666666, ans=0.2 2026-09-24 07:25:27,512 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=220080.0, ans=0.125 2026-09-24 07:25:38,144 INFO [train.py:1192] (1/2) Epoch 69, batch 900, loss[loss=0.2014, simple_loss=0.3224, pruned_loss=0.04023, over 24582.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3735, pruned_loss=0.06818, over 4780865.53 frames. ], batch size: 137, lr: 3.05e-03, grad_scale: 32.0 2026-09-24 07:25:43,774 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.692e+02 3.583e+02 4.252e+02 4.779e+02 6.888e+02, threshold=8.504e+02, percent-clipped=0.0 2026-09-24 07:25:50,047 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=220213.33333333334, ans=0.125 2026-09-24 07:25:53,155 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=220246.66666666666, ans=0.04949747468305833 2026-09-24 07:26:03,477 INFO [train.py:1192] (1/2) Epoch 69, batch 950, loss[loss=0.3156, simple_loss=0.3921, pruned_loss=0.1195, over 11436.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3729, pruned_loss=0.06934, over 4710678.87 frames. ], batch size: 333, lr: 3.05e-03, grad_scale: 32.0 2026-09-24 07:26:13,677 INFO [train.py:1192] (1/2) Epoch 70, batch 0, loss[loss=0.216, simple_loss=0.3348, pruned_loss=0.04863, over 24577.00 frames. ], tot_loss[loss=0.216, simple_loss=0.3348, pruned_loss=0.04863, over 24577.00 frames. ], batch size: 137, lr: 3.02e-03, grad_scale: 32.0 2026-09-24 07:26:13,680 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 07:26:20,896 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.5944, 2.4693, 1.9806, 3.0509], device='cuda:1') 2026-09-24 07:26:25,447 INFO [train.py:1224] (1/2) Epoch 70, validation: loss=0.1687, simple_loss=0.2872, pruned_loss=0.02509, over 2564189.00 frames. 2026-09-24 07:26:25,447 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 07:26:26,490 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=220340.0, ans=0.125 2026-09-24 07:26:29,651 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.max_abs, batch_count=220340.0, ans=10.0 2026-09-24 07:26:38,505 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=220406.66666666666, ans=0.1 2026-09-24 07:26:43,392 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=220440.0, ans=0.125 2026-09-24 07:26:46,507 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=220473.33333333334, ans=0.035 2026-09-24 07:26:46,544 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=220473.33333333334, ans=0.125 2026-09-24 07:26:51,172 INFO [train.py:1192] (1/2) Epoch 70, batch 50, loss[loss=0.2119, simple_loss=0.3231, pruned_loss=0.05038, over 24276.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3799, pruned_loss=0.07136, over 1081871.29 frames. ], batch size: 125, lr: 3.02e-03, grad_scale: 32.0 2026-09-24 07:26:52,421 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.735e+02 3.450e+02 4.028e+02 5.068e+02 6.444e+02, threshold=8.056e+02, percent-clipped=0.0 2026-09-24 07:27:03,384 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer2.prob, batch_count=220573.33333333334, ans=0.125 2026-09-24 07:27:06,740 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=220606.66666666666, ans=0.0 2026-09-24 07:27:13,376 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer2.prob, batch_count=220640.0, ans=0.125 2026-09-24 07:27:16,697 INFO [train.py:1192] (1/2) Epoch 70, batch 100, loss[loss=0.2537, simple_loss=0.3649, pruned_loss=0.0713, over 24632.00 frames. ], tot_loss[loss=0.2618, simple_loss=0.3824, pruned_loss=0.07062, over 1915652.14 frames. ], batch size: 154, lr: 3.02e-03, grad_scale: 32.0 2026-09-24 07:27:26,831 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=220740.0, ans=0.125 2026-09-24 07:27:30,912 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=220740.0, ans=0.125 2026-09-24 07:27:36,619 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=220773.33333333334, ans=0.0 2026-09-24 07:27:42,707 INFO [train.py:1192] (1/2) Epoch 70, batch 150, loss[loss=0.2136, simple_loss=0.3247, pruned_loss=0.05129, over 24260.00 frames. ], tot_loss[loss=0.258, simple_loss=0.3772, pruned_loss=0.06941, over 2561083.51 frames. ], batch size: 125, lr: 3.02e-03, grad_scale: 32.0 2026-09-24 07:27:44,616 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.678e+02 3.486e+02 3.795e+02 4.140e+02 5.567e+02, threshold=7.590e+02, percent-clipped=0.0 2026-09-24 07:27:47,675 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=220873.33333333334, ans=0.04949747468305833 2026-09-24 07:27:50,941 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=220873.33333333334, ans=0.125 2026-09-24 07:27:51,947 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=220873.33333333334, ans=0.2 2026-09-24 07:27:55,926 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=220906.66666666666, ans=0.025 2026-09-24 07:28:00,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=220940.0, ans=0.125 2026-09-24 07:28:04,991 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:28:07,913 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=220973.33333333334, ans=0.025 2026-09-24 07:28:08,732 INFO [train.py:1192] (1/2) Epoch 70, batch 200, loss[loss=0.2895, simple_loss=0.4126, pruned_loss=0.08318, over 24197.00 frames. ], tot_loss[loss=0.2565, simple_loss=0.3755, pruned_loss=0.06878, over 3059738.58 frames. ], batch size: 257, lr: 3.02e-03, grad_scale: 32.0 2026-09-24 07:28:15,253 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=221040.0, ans=0.2 2026-09-24 07:28:16,091 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=221040.0, ans=0.125 2026-09-24 07:28:24,830 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=10.65 vs. limit=22.5 2026-09-24 07:28:26,176 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=221106.66666666666, ans=0.0 2026-09-24 07:28:34,549 INFO [train.py:1192] (1/2) Epoch 70, batch 250, loss[loss=0.2824, simple_loss=0.4051, pruned_loss=0.07984, over 24387.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3748, pruned_loss=0.06875, over 3443335.19 frames. ], batch size: 225, lr: 3.02e-03, grad_scale: 32.0 2026-09-24 07:28:36,014 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.660e+02 3.395e+02 3.896e+02 4.437e+02 8.047e+02, threshold=7.793e+02, percent-clipped=1.0 2026-09-24 07:28:44,567 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=4.20 vs. limit=6.0 2026-09-24 07:28:57,076 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=221306.66666666666, ans=0.125 2026-09-24 07:29:00,012 INFO [train.py:1192] (1/2) Epoch 70, batch 300, loss[loss=0.3065, simple_loss=0.4234, pruned_loss=0.09478, over 24541.00 frames. ], tot_loss[loss=0.2562, simple_loss=0.3747, pruned_loss=0.06886, over 3758299.48 frames. ], batch size: 204, lr: 3.02e-03, grad_scale: 32.0 2026-09-24 07:29:13,317 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=221406.66666666666, ans=0.025 2026-09-24 07:29:15,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=221440.0, ans=0.125 2026-09-24 07:29:18,299 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.67 vs. limit=15.0 2026-09-24 07:29:24,352 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=221473.33333333334, ans=0.2 2026-09-24 07:29:26,351 INFO [train.py:1192] (1/2) Epoch 70, batch 350, loss[loss=0.2287, simple_loss=0.3433, pruned_loss=0.05704, over 24583.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3756, pruned_loss=0.06921, over 3998663.24 frames. ], batch size: 137, lr: 3.02e-03, grad_scale: 32.0 2026-09-24 07:29:27,842 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.642e+02 3.393e+02 3.774e+02 4.172e+02 6.242e+02, threshold=7.549e+02, percent-clipped=0.0 2026-09-24 07:29:47,828 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.27 vs. limit=10.0 2026-09-24 07:29:48,082 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=221640.0, ans=0.125 2026-09-24 07:29:49,829 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.37 vs. limit=22.5 2026-09-24 07:29:51,881 INFO [train.py:1192] (1/2) Epoch 70, batch 400, loss[loss=0.2944, simple_loss=0.4028, pruned_loss=0.09304, over 24561.00 frames. ], tot_loss[loss=0.2552, simple_loss=0.374, pruned_loss=0.06822, over 4183592.67 frames. ], batch size: 170, lr: 3.01e-03, grad_scale: 32.0 2026-09-24 07:30:17,944 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.whiten1.whitening_limit, batch_count=221840.0, ans=10.0 2026-09-24 07:30:18,062 INFO [train.py:1192] (1/2) Epoch 70, batch 450, loss[loss=0.2896, simple_loss=0.4001, pruned_loss=0.08954, over 24626.00 frames. ], tot_loss[loss=0.2562, simple_loss=0.3747, pruned_loss=0.06881, over 4323294.05 frames. ], batch size: 175, lr: 3.01e-03, grad_scale: 32.0 2026-09-24 07:30:19,415 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.680e+02 3.391e+02 3.831e+02 4.554e+02 6.756e+02, threshold=7.662e+02, percent-clipped=0.0 2026-09-24 07:30:23,484 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.44 vs. limit=15.0 2026-09-24 07:30:36,037 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=221940.0, ans=0.125 2026-09-24 07:30:43,624 INFO [train.py:1192] (1/2) Epoch 70, batch 500, loss[loss=0.2827, simple_loss=0.4066, pruned_loss=0.07937, over 24510.00 frames. ], tot_loss[loss=0.2549, simple_loss=0.3732, pruned_loss=0.06825, over 4439849.74 frames. ], batch size: 218, lr: 3.01e-03, grad_scale: 32.0 2026-09-24 07:30:43,797 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.00 vs. limit=15.0 2026-09-24 07:30:46,678 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=222006.66666666666, ans=0.0 2026-09-24 07:30:48,210 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=222006.66666666666, ans=0.0 2026-09-24 07:31:01,262 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=222106.66666666666, ans=0.125 2026-09-24 07:31:04,923 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=222140.0, ans=0.125 2026-09-24 07:31:09,730 INFO [train.py:1192] (1/2) Epoch 70, batch 550, loss[loss=0.3049, simple_loss=0.4249, pruned_loss=0.09241, over 24290.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.3739, pruned_loss=0.06851, over 4524331.12 frames. ], batch size: 257, lr: 3.01e-03, grad_scale: 32.0 2026-09-24 07:31:11,275 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.566e+02 3.420e+02 3.675e+02 4.165e+02 5.906e+02, threshold=7.350e+02, percent-clipped=0.0 2026-09-24 07:31:28,222 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=222273.33333333334, ans=0.125 2026-09-24 07:31:35,476 INFO [train.py:1192] (1/2) Epoch 70, batch 600, loss[loss=0.3015, simple_loss=0.4223, pruned_loss=0.09034, over 24333.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3743, pruned_loss=0.06847, over 4590036.47 frames. ], batch size: 234, lr: 3.01e-03, grad_scale: 32.0 2026-09-24 07:31:37,521 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=222340.0, ans=0.1 2026-09-24 07:31:43,689 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=222373.33333333334, ans=0.1 2026-09-24 07:31:44,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.whiten.whitening_limit, batch_count=222373.33333333334, ans=15.0 2026-09-24 07:31:48,048 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=222406.66666666666, ans=0.125 2026-09-24 07:32:00,848 INFO [train.py:1192] (1/2) Epoch 70, batch 650, loss[loss=0.2712, simple_loss=0.3806, pruned_loss=0.08093, over 24591.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3733, pruned_loss=0.06776, over 4654769.17 frames. ], batch size: 154, lr: 3.01e-03, grad_scale: 32.0 2026-09-24 07:32:02,769 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.728e+02 3.370e+02 3.663e+02 4.136e+02 7.034e+02, threshold=7.327e+02, percent-clipped=0.0 2026-09-24 07:32:02,966 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.09 vs. limit=10.0 2026-09-24 07:32:24,258 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=222640.0, ans=0.0 2026-09-24 07:32:26,816 INFO [train.py:1192] (1/2) Epoch 70, batch 700, loss[loss=0.2751, simple_loss=0.3847, pruned_loss=0.08272, over 24555.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3744, pruned_loss=0.06813, over 4689842.40 frames. ], batch size: 158, lr: 3.01e-03, grad_scale: 32.0 2026-09-24 07:32:36,880 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:32:41,037 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:32:42,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=222773.33333333334, ans=0.125 2026-09-24 07:32:43,301 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=222773.33333333334, ans=0.1 2026-09-24 07:32:53,105 INFO [train.py:1192] (1/2) Epoch 70, batch 750, loss[loss=0.2694, simple_loss=0.3872, pruned_loss=0.07586, over 24637.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3735, pruned_loss=0.06797, over 4726389.38 frames. ], batch size: 175, lr: 3.01e-03, grad_scale: 32.0 2026-09-24 07:32:54,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.prob, batch_count=222840.0, ans=0.125 2026-09-24 07:32:54,668 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.865e+02 3.519e+02 3.933e+02 4.610e+02 7.367e+02, threshold=7.866e+02, percent-clipped=1.0 2026-09-24 07:32:59,051 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.67 vs. limit=15.0 2026-09-24 07:33:10,842 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten.whitening_limit, batch_count=222940.0, ans=15.0 2026-09-24 07:33:15,299 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=222973.33333333334, ans=0.125 2026-09-24 07:33:19,278 INFO [train.py:1192] (1/2) Epoch 70, batch 800, loss[loss=0.2149, simple_loss=0.3317, pruned_loss=0.04902, over 24577.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3734, pruned_loss=0.06791, over 4752767.43 frames. ], batch size: 137, lr: 3.00e-03, grad_scale: 32.0 2026-09-24 07:33:27,787 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=223040.0, ans=0.125 2026-09-24 07:33:29,460 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.50 vs. limit=15.0 2026-09-24 07:33:32,412 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer1.min_positive, batch_count=223073.33333333334, ans=0.025 2026-09-24 07:33:43,145 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=223140.0, ans=0.125 2026-09-24 07:33:44,865 INFO [train.py:1192] (1/2) Epoch 70, batch 850, loss[loss=0.2673, simple_loss=0.394, pruned_loss=0.07028, over 24554.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3727, pruned_loss=0.06749, over 4770871.13 frames. ], batch size: 204, lr: 3.00e-03, grad_scale: 32.0 2026-09-24 07:33:46,671 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.632e+02 3.514e+02 3.965e+02 4.665e+02 6.328e+02, threshold=7.930e+02, percent-clipped=0.0 2026-09-24 07:33:54,565 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=223206.66666666666, ans=0.0 2026-09-24 07:33:56,457 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=223240.0, ans=0.1 2026-09-24 07:33:59,605 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.max_abs, batch_count=223240.0, ans=10.0 2026-09-24 07:33:59,629 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=223240.0, ans=0.09899494936611666 2026-09-24 07:34:01,174 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=223273.33333333334, ans=0.2 2026-09-24 07:34:01,778 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=223273.33333333334, ans=0.125 2026-09-24 07:34:11,226 INFO [train.py:1192] (1/2) Epoch 70, batch 900, loss[loss=0.2042, simple_loss=0.3251, pruned_loss=0.0416, over 24541.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3734, pruned_loss=0.06803, over 4780898.53 frames. ], batch size: 137, lr: 3.00e-03, grad_scale: 32.0 2026-09-24 07:34:15,231 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=223340.0, ans=0.025 2026-09-24 07:34:27,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.min_positive, batch_count=223440.0, ans=0.025 2026-09-24 07:34:29,684 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.19 vs. limit=15.0 2026-09-24 07:34:36,432 INFO [train.py:1192] (1/2) Epoch 70, batch 950, loss[loss=0.3018, simple_loss=0.3842, pruned_loss=0.1097, over 11393.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.3727, pruned_loss=0.06907, over 4710249.88 frames. ], batch size: 334, lr: 3.00e-03, grad_scale: 32.0 2026-09-24 07:34:37,871 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.739e+02 3.481e+02 3.831e+02 4.616e+02 8.227e+02, threshold=7.662e+02, percent-clipped=1.0 2026-09-24 07:34:38,474 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=223506.66666666666, ans=0.125 2026-09-24 07:34:47,726 INFO [train.py:1192] (1/2) Epoch 71, batch 0, loss[loss=0.2104, simple_loss=0.3336, pruned_loss=0.04359, over 24552.00 frames. ], tot_loss[loss=0.2104, simple_loss=0.3336, pruned_loss=0.04359, over 24552.00 frames. ], batch size: 137, lr: 2.98e-03, grad_scale: 32.0 2026-09-24 07:34:47,726 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 07:34:52,335 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([3.8727, 3.0169, 2.5170, 1.9992], device='cuda:1') 2026-09-24 07:34:57,108 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.7417, 4.1776, 4.4674, 4.2095], device='cuda:1') 2026-09-24 07:34:59,539 INFO [train.py:1224] (1/2) Epoch 71, validation: loss=0.1691, simple_loss=0.2875, pruned_loss=0.02536, over 2564189.00 frames. 2026-09-24 07:34:59,539 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 07:35:01,059 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=223533.33333333334, ans=0.1 2026-09-24 07:35:01,505 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass_mid.scale_min, batch_count=223533.33333333334, ans=0.2 2026-09-24 07:35:06,761 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=223566.66666666666, ans=0.1 2026-09-24 07:35:07,683 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=223566.66666666666, ans=0.0 2026-09-24 07:35:08,150 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=223566.66666666666, ans=0.125 2026-09-24 07:35:08,304 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.58 vs. limit=15.0 2026-09-24 07:35:15,081 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=223633.33333333334, ans=0.125 2026-09-24 07:35:24,957 INFO [train.py:1192] (1/2) Epoch 71, batch 50, loss[loss=0.2117, simple_loss=0.3266, pruned_loss=0.04834, over 24230.00 frames. ], tot_loss[loss=0.2613, simple_loss=0.3792, pruned_loss=0.07167, over 1081816.65 frames. ], batch size: 125, lr: 2.98e-03, grad_scale: 32.0 2026-09-24 07:35:37,590 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=223766.66666666666, ans=0.125 2026-09-24 07:35:43,000 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=223800.0, ans=0.125 2026-09-24 07:35:43,380 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=223800.0, ans=0.2 2026-09-24 07:35:47,667 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=223833.33333333334, ans=0.0 2026-09-24 07:35:48,599 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.629e+02 3.501e+02 3.879e+02 4.453e+02 6.125e+02, threshold=7.759e+02, percent-clipped=0.0 2026-09-24 07:35:50,163 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=223833.33333333334, ans=0.0 2026-09-24 07:35:51,055 INFO [train.py:1192] (1/2) Epoch 71, batch 100, loss[loss=0.2617, simple_loss=0.3721, pruned_loss=0.07564, over 24644.00 frames. ], tot_loss[loss=0.2638, simple_loss=0.3832, pruned_loss=0.07223, over 1916708.00 frames. ], batch size: 154, lr: 2.98e-03, grad_scale: 32.0 2026-09-24 07:36:01,079 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=223933.33333333334, ans=0.125 2026-09-24 07:36:08,710 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=223966.66666666666, ans=0.0 2026-09-24 07:36:12,949 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=224000.0, ans=0.125 2026-09-24 07:36:16,649 INFO [train.py:1192] (1/2) Epoch 71, batch 150, loss[loss=0.2073, simple_loss=0.3229, pruned_loss=0.04587, over 24288.00 frames. ], tot_loss[loss=0.2586, simple_loss=0.3775, pruned_loss=0.06979, over 2561943.57 frames. ], batch size: 125, lr: 2.98e-03, grad_scale: 32.0 2026-09-24 07:36:24,448 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=224066.66666666666, ans=0.125 2026-09-24 07:36:29,859 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.72 vs. limit=15.0 2026-09-24 07:36:31,024 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=224100.0, ans=0.1 2026-09-24 07:36:38,167 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=224166.66666666666, ans=0.125 2026-09-24 07:36:40,097 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.787e+02 3.394e+02 3.776e+02 4.264e+02 6.298e+02, threshold=7.553e+02, percent-clipped=0.0 2026-09-24 07:36:40,636 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=224166.66666666666, ans=0.1 2026-09-24 07:36:41,633 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.max_abs, batch_count=224166.66666666666, ans=10.0 2026-09-24 07:36:42,471 INFO [train.py:1192] (1/2) Epoch 71, batch 200, loss[loss=0.2644, simple_loss=0.4013, pruned_loss=0.06375, over 24220.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.375, pruned_loss=0.0686, over 3062325.18 frames. ], batch size: 257, lr: 2.98e-03, grad_scale: 32.0 2026-09-24 07:36:46,505 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.85 vs. limit=6.0 2026-09-24 07:36:50,393 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=224233.33333333334, ans=0.0 2026-09-24 07:36:52,131 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.attention_skip_rate, batch_count=224233.33333333334, ans=0.0 2026-09-24 07:36:56,334 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=224266.66666666666, ans=0.05 2026-09-24 07:36:57,292 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=224266.66666666666, ans=0.125 2026-09-24 07:37:02,914 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=224333.33333333334, ans=0.125 2026-09-24 07:37:09,169 INFO [train.py:1192] (1/2) Epoch 71, batch 250, loss[loss=0.2649, simple_loss=0.3959, pruned_loss=0.06692, over 24399.00 frames. ], tot_loss[loss=0.2562, simple_loss=0.3749, pruned_loss=0.06876, over 3445375.00 frames. ], batch size: 225, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:37:10,236 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.max_positive, batch_count=224366.66666666666, ans=0.95 2026-09-24 07:37:28,670 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.61 vs. limit=8.0 2026-09-24 07:37:30,397 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=224500.0, ans=0.125 2026-09-24 07:37:32,567 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.491e+02 3.620e+02 4.048e+02 4.670e+02 6.190e+02, threshold=8.096e+02, percent-clipped=0.0 2026-09-24 07:37:35,403 INFO [train.py:1192] (1/2) Epoch 71, batch 300, loss[loss=0.2791, simple_loss=0.4027, pruned_loss=0.0778, over 24555.00 frames. ], tot_loss[loss=0.256, simple_loss=0.3744, pruned_loss=0.06876, over 3758953.71 frames. ], batch size: 204, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:37:38,916 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=224533.33333333334, ans=0.0 2026-09-24 07:37:39,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=224566.66666666666, ans=0.2 2026-09-24 07:37:48,666 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.50 vs. limit=15.0 2026-09-24 07:37:49,499 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=224600.0, ans=0.125 2026-09-24 07:37:57,128 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.74 vs. limit=22.5 2026-09-24 07:38:01,653 INFO [train.py:1192] (1/2) Epoch 71, batch 350, loss[loss=0.2196, simple_loss=0.3341, pruned_loss=0.05259, over 24566.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3748, pruned_loss=0.06843, over 3999952.29 frames. ], batch size: 137, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:38:06,687 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=224733.33333333334, ans=0.125 2026-09-24 07:38:22,047 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=224833.33333333334, ans=0.0 2026-09-24 07:38:22,064 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=224833.33333333334, ans=0.025 2026-09-24 07:38:25,052 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.748e+02 3.403e+02 3.758e+02 4.281e+02 6.376e+02, threshold=7.516e+02, percent-clipped=0.0 2026-09-24 07:38:25,710 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=224833.33333333334, ans=0.0 2026-09-24 07:38:27,492 INFO [train.py:1192] (1/2) Epoch 71, batch 400, loss[loss=0.2449, simple_loss=0.3695, pruned_loss=0.06015, over 24572.00 frames. ], tot_loss[loss=0.2554, simple_loss=0.3742, pruned_loss=0.06829, over 4181828.17 frames. ], batch size: 170, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:38:31,493 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=224866.66666666666, ans=0.1 2026-09-24 07:38:35,160 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=224900.0, ans=0.2 2026-09-24 07:38:41,214 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=224933.33333333334, ans=0.125 2026-09-24 07:38:43,020 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=224966.66666666666, ans=0.1 2026-09-24 07:38:52,918 INFO [train.py:1192] (1/2) Epoch 71, batch 450, loss[loss=0.2599, simple_loss=0.3862, pruned_loss=0.06679, over 24609.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.3741, pruned_loss=0.06844, over 4320265.80 frames. ], batch size: 175, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:38:58,150 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=225066.66666666666, ans=0.125 2026-09-24 07:39:00,708 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=225066.66666666666, ans=0.125 2026-09-24 07:39:10,673 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.33 vs. limit=10.0 2026-09-24 07:39:15,518 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.538e+02 3.379e+02 3.840e+02 4.380e+02 6.758e+02, threshold=7.679e+02, percent-clipped=0.0 2026-09-24 07:39:18,491 INFO [train.py:1192] (1/2) Epoch 71, batch 500, loss[loss=0.304, simple_loss=0.4186, pruned_loss=0.09468, over 24515.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3729, pruned_loss=0.0682, over 4437800.98 frames. ], batch size: 218, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:39:25,906 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=225233.33333333334, ans=0.07 2026-09-24 07:39:27,813 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=225233.33333333334, ans=0.125 2026-09-24 07:39:36,640 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=225300.0, ans=0.125 2026-09-24 07:39:43,337 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=225333.33333333334, ans=0.0 2026-09-24 07:39:44,707 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=225366.66666666666, ans=0.1 2026-09-24 07:39:44,866 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.86 vs. limit=15.0 2026-09-24 07:39:45,111 INFO [train.py:1192] (1/2) Epoch 71, batch 550, loss[loss=0.3073, simple_loss=0.4253, pruned_loss=0.09471, over 24278.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3738, pruned_loss=0.06868, over 4522557.35 frames. ], batch size: 257, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:39:58,302 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.scale_min, batch_count=225433.33333333334, ans=0.2 2026-09-24 07:40:00,663 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.12 vs. limit=15.0 2026-09-24 07:40:06,176 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.35 vs. limit=15.0 2026-09-24 07:40:08,167 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.593e+02 3.323e+02 3.699e+02 4.232e+02 6.947e+02, threshold=7.397e+02, percent-clipped=0.0 2026-09-24 07:40:10,452 INFO [train.py:1192] (1/2) Epoch 71, batch 600, loss[loss=0.2708, simple_loss=0.4004, pruned_loss=0.07061, over 24320.00 frames. ], tot_loss[loss=0.2557, simple_loss=0.3745, pruned_loss=0.06842, over 4588510.08 frames. ], batch size: 234, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:40:13,106 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.75 vs. limit=15.0 2026-09-24 07:40:13,487 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:40:22,822 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=225600.0, ans=0.125 2026-09-24 07:40:28,851 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=225633.33333333334, ans=0.2 2026-09-24 07:40:32,641 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.59 vs. limit=15.0 2026-09-24 07:40:36,019 INFO [train.py:1192] (1/2) Epoch 71, batch 650, loss[loss=0.2186, simple_loss=0.3421, pruned_loss=0.04753, over 24604.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.373, pruned_loss=0.06743, over 4653341.93 frames. ], batch size: 154, lr: 2.97e-03, grad_scale: 32.0 2026-09-24 07:40:36,146 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:40:37,865 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=225700.0, ans=0.125 2026-09-24 07:40:40,283 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=225700.0, ans=10.0 2026-09-24 07:40:40,286 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=225700.0, ans=0.125 2026-09-24 07:40:40,298 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=225700.0, ans=0.0 2026-09-24 07:40:48,426 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=225766.66666666666, ans=0.0 2026-09-24 07:40:54,706 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=225800.0, ans=0.125 2026-09-24 07:40:57,283 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=6.68 vs. limit=15.0 2026-09-24 07:40:59,068 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=225833.33333333334, ans=0.125 2026-09-24 07:40:59,967 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.716e+02 3.479e+02 3.828e+02 4.312e+02 7.119e+02, threshold=7.656e+02, percent-clipped=0.0 2026-09-24 07:41:01,784 INFO [train.py:1192] (1/2) Epoch 71, batch 700, loss[loss=0.2329, simple_loss=0.3532, pruned_loss=0.05631, over 24559.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.374, pruned_loss=0.06767, over 4688536.74 frames. ], batch size: 158, lr: 2.96e-03, grad_scale: 32.0 2026-09-24 07:41:03,715 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=225866.66666666666, ans=0.125 2026-09-24 07:41:05,676 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=225866.66666666666, ans=0.125 2026-09-24 07:41:05,705 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=225866.66666666666, ans=0.125 2026-09-24 07:41:11,556 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=225900.0, ans=0.0 2026-09-24 07:41:14,098 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=225933.33333333334, ans=0.1 2026-09-24 07:41:17,784 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=225966.66666666666, ans=0.04949747468305833 2026-09-24 07:41:25,758 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.bypass.skip_rate, batch_count=226000.0, ans=0.09899494936611666 2026-09-24 07:41:27,539 INFO [train.py:1192] (1/2) Epoch 71, batch 750, loss[loss=0.2466, simple_loss=0.3718, pruned_loss=0.06073, over 24639.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3729, pruned_loss=0.06735, over 4722648.45 frames. ], batch size: 175, lr: 2.96e-03, grad_scale: 32.0 2026-09-24 07:41:28,121 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=226033.33333333334, ans=0.125 2026-09-24 07:41:28,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=226033.33333333334, ans=0.1 2026-09-24 07:41:32,430 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=226066.66666666666, ans=0.125 2026-09-24 07:41:42,173 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=226100.0, ans=0.125 2026-09-24 07:41:42,177 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer2.prob, batch_count=226100.0, ans=0.125 2026-09-24 07:41:50,498 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=226166.66666666666, ans=0.125 2026-09-24 07:41:50,983 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=226166.66666666666, ans=0.125 2026-09-24 07:41:51,394 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.692e+02 3.469e+02 3.978e+02 4.612e+02 7.251e+02, threshold=7.957e+02, percent-clipped=0.0 2026-09-24 07:41:53,392 INFO [train.py:1192] (1/2) Epoch 71, batch 800, loss[loss=0.2263, simple_loss=0.3436, pruned_loss=0.0545, over 24519.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3722, pruned_loss=0.06669, over 4748709.64 frames. ], batch size: 137, lr: 2.96e-03, grad_scale: 32.0 2026-09-24 07:42:07,155 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:42:12,579 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=226300.0, ans=0.0 2026-09-24 07:42:19,353 INFO [train.py:1192] (1/2) Epoch 71, batch 850, loss[loss=0.2555, simple_loss=0.3907, pruned_loss=0.06019, over 24553.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3714, pruned_loss=0.06619, over 4769341.74 frames. ], batch size: 204, lr: 2.96e-03, grad_scale: 32.0 2026-09-24 07:42:28,194 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.30 vs. limit=22.5 2026-09-24 07:42:32,302 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.53 vs. limit=15.0 2026-09-24 07:42:42,964 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.466e+02 3.586e+02 3.961e+02 4.460e+02 5.915e+02, threshold=7.923e+02, percent-clipped=0.0 2026-09-24 07:42:43,576 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:42:44,477 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=226533.33333333334, ans=0.1 2026-09-24 07:42:44,813 INFO [train.py:1192] (1/2) Epoch 71, batch 900, loss[loss=0.2166, simple_loss=0.3353, pruned_loss=0.04892, over 24568.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3718, pruned_loss=0.06637, over 4779753.15 frames. ], batch size: 137, lr: 2.96e-03, grad_scale: 32.0 2026-09-24 07:43:10,065 INFO [train.py:1192] (1/2) Epoch 71, batch 950, loss[loss=0.3534, simple_loss=0.4252, pruned_loss=0.1408, over 11298.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3707, pruned_loss=0.06731, over 4715834.88 frames. ], batch size: 334, lr: 2.96e-03, grad_scale: 32.0 2026-09-24 07:43:12,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=226700.0, ans=0.125 2026-09-24 07:43:19,962 INFO [train.py:1192] (1/2) Epoch 72, batch 0, loss[loss=0.2108, simple_loss=0.3341, pruned_loss=0.04371, over 24578.00 frames. ], tot_loss[loss=0.2108, simple_loss=0.3341, pruned_loss=0.04371, over 24578.00 frames. ], batch size: 137, lr: 2.94e-03, grad_scale: 32.0 2026-09-24 07:43:19,963 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 07:43:22,387 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.9529, 1.6939, 2.8247, 1.8113], device='cuda:1') 2026-09-24 07:43:31,727 INFO [train.py:1224] (1/2) Epoch 72, validation: loss=0.169, simple_loss=0.2872, pruned_loss=0.02538, over 2564189.00 frames. 2026-09-24 07:43:31,728 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 07:43:36,114 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=226726.66666666666, ans=0.125 2026-09-24 07:43:39,742 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.92 vs. limit=15.0 2026-09-24 07:43:40,074 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=226760.0, ans=0.125 2026-09-24 07:43:43,942 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=226793.33333333334, ans=0.125 2026-09-24 07:43:49,611 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=226826.66666666666, ans=0.0 2026-09-24 07:43:51,424 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.743e+02 3.499e+02 3.913e+02 4.779e+02 6.698e+02, threshold=7.827e+02, percent-clipped=0.0 2026-09-24 07:43:52,499 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=226860.0, ans=0.0 2026-09-24 07:43:55,344 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.13 vs. limit=15.0 2026-09-24 07:43:57,810 INFO [train.py:1192] (1/2) Epoch 72, batch 50, loss[loss=0.2096, simple_loss=0.3189, pruned_loss=0.05015, over 24252.00 frames. ], tot_loss[loss=0.2572, simple_loss=0.3766, pruned_loss=0.06886, over 1082627.20 frames. ], batch size: 125, lr: 2.94e-03, grad_scale: 32.0 2026-09-24 07:44:06,298 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn1.whiten, num_groups=1, num_channels=192, metric=10.95 vs. limit=22.5 2026-09-24 07:44:08,017 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=226960.0, ans=0.0 2026-09-24 07:44:10,802 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.20 vs. limit=15.0 2026-09-24 07:44:23,436 INFO [train.py:1192] (1/2) Epoch 72, batch 100, loss[loss=0.234, simple_loss=0.3525, pruned_loss=0.05779, over 24590.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3814, pruned_loss=0.06988, over 1916697.03 frames. ], batch size: 154, lr: 2.94e-03, grad_scale: 32.0 2026-09-24 07:44:41,127 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=227160.0, ans=0.1 2026-09-24 07:44:42,186 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=227160.0, ans=0.025 2026-09-24 07:44:43,111 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.867e+02 3.517e+02 3.844e+02 4.529e+02 6.217e+02, threshold=7.689e+02, percent-clipped=0.0 2026-09-24 07:44:46,836 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=227193.33333333334, ans=0.2 2026-09-24 07:44:49,366 INFO [train.py:1192] (1/2) Epoch 72, batch 150, loss[loss=0.1999, simple_loss=0.3177, pruned_loss=0.04112, over 24274.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3765, pruned_loss=0.06828, over 2561721.40 frames. ], batch size: 125, lr: 2.94e-03, grad_scale: 32.0 2026-09-24 07:44:50,525 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=227226.66666666666, ans=0.025 2026-09-24 07:44:53,110 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=227226.66666666666, ans=0.1 2026-09-24 07:44:55,443 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_skip_rate, batch_count=227260.0, ans=0.0 2026-09-24 07:44:56,562 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.75 vs. limit=15.0 2026-09-24 07:44:58,304 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer2.prob, batch_count=227260.0, ans=0.125 2026-09-24 07:44:59,665 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=227293.33333333334, ans=0.125 2026-09-24 07:45:06,300 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=227326.66666666666, ans=0.125 2026-09-24 07:45:15,647 INFO [train.py:1192] (1/2) Epoch 72, batch 200, loss[loss=0.271, simple_loss=0.4068, pruned_loss=0.06758, over 24235.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3746, pruned_loss=0.06733, over 3060553.60 frames. ], batch size: 257, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:45:21,229 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=227426.66666666666, ans=0.125 2026-09-24 07:45:25,704 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.33 vs. limit=10.0 2026-09-24 07:45:26,107 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=227460.0, ans=0.0 2026-09-24 07:45:34,969 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.93 vs. limit=8.0 2026-09-24 07:45:35,044 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.856e+02 3.499e+02 3.903e+02 4.417e+02 8.811e+02, threshold=7.805e+02, percent-clipped=1.0 2026-09-24 07:45:36,742 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.43 vs. limit=6.0 2026-09-24 07:45:41,419 INFO [train.py:1192] (1/2) Epoch 72, batch 250, loss[loss=0.2925, simple_loss=0.4111, pruned_loss=0.08693, over 24399.00 frames. ], tot_loss[loss=0.255, simple_loss=0.3746, pruned_loss=0.06769, over 3444034.51 frames. ], batch size: 225, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:45:45,719 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=227560.0, ans=0.0 2026-09-24 07:46:04,837 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=227693.33333333334, ans=0.125 2026-09-24 07:46:06,545 INFO [train.py:1192] (1/2) Epoch 72, batch 300, loss[loss=0.2676, simple_loss=0.3976, pruned_loss=0.06881, over 24544.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.373, pruned_loss=0.06683, over 3758319.61 frames. ], batch size: 204, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:46:09,994 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=227726.66666666666, ans=0.0 2026-09-24 07:46:20,455 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=227793.33333333334, ans=0.0 2026-09-24 07:46:26,608 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.613e+02 3.439e+02 3.792e+02 4.297e+02 6.948e+02, threshold=7.583e+02, percent-clipped=0.0 2026-09-24 07:46:29,861 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:46:31,833 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.89 vs. limit=12.0 2026-09-24 07:46:33,070 INFO [train.py:1192] (1/2) Epoch 72, batch 350, loss[loss=0.2248, simple_loss=0.3362, pruned_loss=0.05671, over 24561.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3747, pruned_loss=0.06797, over 3999622.88 frames. ], batch size: 137, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:46:34,275 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=227893.33333333334, ans=0.0 2026-09-24 07:46:36,190 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=227893.33333333334, ans=0.0 2026-09-24 07:46:37,776 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=227926.66666666666, ans=0.125 2026-09-24 07:46:48,412 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=227993.33333333334, ans=0.07 2026-09-24 07:46:59,149 INFO [train.py:1192] (1/2) Epoch 72, batch 400, loss[loss=0.273, simple_loss=0.3882, pruned_loss=0.07893, over 24570.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3735, pruned_loss=0.0674, over 4184549.56 frames. ], batch size: 170, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:47:00,750 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=228060.0, ans=0.125 2026-09-24 07:47:04,577 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.prob, batch_count=228093.33333333334, ans=0.125 2026-09-24 07:47:06,940 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=19.04 vs. limit=22.5 2026-09-24 07:47:15,947 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=228160.0, ans=15.0 2026-09-24 07:47:16,833 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=228160.0, ans=0.125 2026-09-24 07:47:16,848 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=228160.0, ans=0.125 2026-09-24 07:47:18,231 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.698e+02 3.303e+02 3.749e+02 4.406e+02 6.679e+02, threshold=7.497e+02, percent-clipped=0.0 2026-09-24 07:47:24,747 INFO [train.py:1192] (1/2) Epoch 72, batch 450, loss[loss=0.2477, simple_loss=0.3731, pruned_loss=0.06113, over 24619.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3735, pruned_loss=0.06738, over 4323909.27 frames. ], batch size: 175, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:47:47,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.attention_skip_rate, batch_count=228360.0, ans=0.0 2026-09-24 07:47:49,945 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.01 vs. limit=15.0 2026-09-24 07:47:50,213 INFO [train.py:1192] (1/2) Epoch 72, batch 500, loss[loss=0.2945, simple_loss=0.413, pruned_loss=0.08799, over 24536.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3721, pruned_loss=0.06703, over 4440690.23 frames. ], batch size: 218, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:47:50,320 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=228393.33333333334, ans=0.125 2026-09-24 07:47:52,752 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=228393.33333333334, ans=0.125 2026-09-24 07:47:53,655 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=4.53 vs. limit=15.0 2026-09-24 07:48:05,149 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.31 vs. limit=15.0 2026-09-24 07:48:06,408 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=228493.33333333334, ans=0.0 2026-09-24 07:48:10,152 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.588e+02 3.380e+02 3.944e+02 4.318e+02 5.448e+02, threshold=7.889e+02, percent-clipped=0.0 2026-09-24 07:48:10,289 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=228493.33333333334, ans=0.125 2026-09-24 07:48:12,226 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=228526.66666666666, ans=0.0 2026-09-24 07:48:13,715 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=228526.66666666666, ans=0.0 2026-09-24 07:48:15,704 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=228560.0, ans=0.125 2026-09-24 07:48:16,316 INFO [train.py:1192] (1/2) Epoch 72, batch 550, loss[loss=0.2594, simple_loss=0.3958, pruned_loss=0.06151, over 24271.00 frames. ], tot_loss[loss=0.2539, simple_loss=0.373, pruned_loss=0.06745, over 4524490.98 frames. ], batch size: 257, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:48:21,400 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.37 vs. limit=15.0 2026-09-24 07:48:27,796 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.74 vs. limit=15.0 2026-09-24 07:48:42,328 INFO [train.py:1192] (1/2) Epoch 72, batch 600, loss[loss=0.2524, simple_loss=0.3859, pruned_loss=0.05941, over 24328.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3733, pruned_loss=0.06737, over 4591564.15 frames. ], batch size: 234, lr: 2.93e-03, grad_scale: 32.0 2026-09-24 07:48:58,349 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer2.prob, batch_count=228826.66666666666, ans=0.125 2026-09-24 07:48:59,374 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=8.43 vs. limit=15.0 2026-09-24 07:49:01,645 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.785e+02 3.329e+02 3.709e+02 4.336e+02 7.314e+02, threshold=7.418e+02, percent-clipped=0.0 2026-09-24 07:49:02,792 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=228860.0, ans=0.2 2026-09-24 07:49:07,598 INFO [train.py:1192] (1/2) Epoch 72, batch 650, loss[loss=0.2615, simple_loss=0.3709, pruned_loss=0.07606, over 24634.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3725, pruned_loss=0.06714, over 4656097.56 frames. ], batch size: 154, lr: 2.92e-03, grad_scale: 32.0 2026-09-24 07:49:14,884 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=228926.66666666666, ans=0.125 2026-09-24 07:49:16,709 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=228926.66666666666, ans=0.125 2026-09-24 07:49:22,060 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=228960.0, ans=0.125 2026-09-24 07:49:33,606 INFO [train.py:1192] (1/2) Epoch 72, batch 700, loss[loss=0.2373, simple_loss=0.3589, pruned_loss=0.05782, over 24551.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3734, pruned_loss=0.0675, over 4691599.55 frames. ], batch size: 158, lr: 2.92e-03, grad_scale: 32.0 2026-09-24 07:49:41,224 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=229093.33333333334, ans=0.125 2026-09-24 07:49:46,262 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.84 vs. limit=15.0 2026-09-24 07:49:46,562 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=229126.66666666666, ans=0.125 2026-09-24 07:49:53,195 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.792e+02 3.430e+02 3.933e+02 4.419e+02 6.993e+02, threshold=7.865e+02, percent-clipped=0.0 2026-09-24 07:49:59,598 INFO [train.py:1192] (1/2) Epoch 72, batch 750, loss[loss=0.262, simple_loss=0.3779, pruned_loss=0.07305, over 24646.00 frames. ], tot_loss[loss=0.254, simple_loss=0.3729, pruned_loss=0.06757, over 4727485.63 frames. ], batch size: 175, lr: 2.92e-03, grad_scale: 32.0 2026-09-24 07:50:09,904 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.balancer1.prob, batch_count=229293.33333333334, ans=0.125 2026-09-24 07:50:25,267 INFO [train.py:1192] (1/2) Epoch 72, batch 800, loss[loss=0.1955, simple_loss=0.3154, pruned_loss=0.03778, over 24553.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3727, pruned_loss=0.06749, over 4752817.07 frames. ], batch size: 137, lr: 2.92e-03, grad_scale: 32.0 2026-09-24 07:50:35,648 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=229460.0, ans=0.95 2026-09-24 07:50:36,135 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=229460.0, ans=0.0 2026-09-24 07:50:38,644 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=229460.0, ans=0.125 2026-09-24 07:50:40,093 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=229493.33333333334, ans=0.1 2026-09-24 07:50:44,704 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.710e+02 3.454e+02 3.890e+02 4.486e+02 6.628e+02, threshold=7.780e+02, percent-clipped=0.0 2026-09-24 07:50:47,642 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=229526.66666666666, ans=0.0 2026-09-24 07:50:51,052 INFO [train.py:1192] (1/2) Epoch 72, batch 850, loss[loss=0.2621, simple_loss=0.3891, pruned_loss=0.06756, over 24560.00 frames. ], tot_loss[loss=0.253, simple_loss=0.3721, pruned_loss=0.06691, over 4771492.20 frames. ], batch size: 204, lr: 2.92e-03, grad_scale: 32.0 2026-09-24 07:50:52,716 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:51:16,641 INFO [train.py:1192] (1/2) Epoch 72, batch 900, loss[loss=0.2064, simple_loss=0.3261, pruned_loss=0.04338, over 24539.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3722, pruned_loss=0.06708, over 4781683.79 frames. ], batch size: 137, lr: 2.92e-03, grad_scale: 32.0 2026-09-24 07:51:24,953 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=229760.0, ans=0.1 2026-09-24 07:51:26,399 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward3.hidden_balancer.prob, batch_count=229760.0, ans=0.125 2026-09-24 07:51:36,060 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.684e+02 3.390e+02 3.920e+02 4.670e+02 6.022e+02, threshold=7.840e+02, percent-clipped=0.0 2026-09-24 07:51:41,281 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.58 vs. limit=22.5 2026-09-24 07:51:42,075 INFO [train.py:1192] (1/2) Epoch 72, batch 950, loss[loss=0.3529, simple_loss=0.4169, pruned_loss=0.1445, over 11085.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3709, pruned_loss=0.06789, over 4712710.22 frames. ], batch size: 334, lr: 2.92e-03, grad_scale: 16.0 2026-09-24 07:51:52,248 INFO [train.py:1192] (1/2) Epoch 73, batch 0, loss[loss=0.2219, simple_loss=0.3397, pruned_loss=0.05205, over 24556.00 frames. ], tot_loss[loss=0.2219, simple_loss=0.3397, pruned_loss=0.05205, over 24556.00 frames. ], batch size: 137, lr: 2.90e-03, grad_scale: 32.0 2026-09-24 07:51:52,249 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 07:52:04,059 INFO [train.py:1224] (1/2) Epoch 73, validation: loss=0.1694, simple_loss=0.2874, pruned_loss=0.0257, over 2564189.00 frames. 2026-09-24 07:52:04,059 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 07:52:07,414 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.20 vs. limit=22.5 2026-09-24 07:52:15,205 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.prob, batch_count=229986.66666666666, ans=0.125 2026-09-24 07:52:16,119 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward3.hidden_balancer.prob, batch_count=229986.66666666666, ans=0.125 2026-09-24 07:52:27,528 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=3.58 vs. limit=12.0 2026-09-24 07:52:29,778 INFO [train.py:1192] (1/2) Epoch 73, batch 50, loss[loss=0.2139, simple_loss=0.3296, pruned_loss=0.04913, over 24244.00 frames. ], tot_loss[loss=0.2581, simple_loss=0.3774, pruned_loss=0.06941, over 1081440.55 frames. ], batch size: 125, lr: 2.90e-03, grad_scale: 32.0 2026-09-24 07:52:37,440 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn2.whiten, num_groups=1, num_channels=256, metric=8.28 vs. limit=22.5 2026-09-24 07:52:38,700 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=230120.0, ans=0.125 2026-09-24 07:52:45,190 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.833e+02 3.477e+02 3.944e+02 4.589e+02 6.689e+02, threshold=7.889e+02, percent-clipped=0.0 2026-09-24 07:52:47,798 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=230186.66666666666, ans=0.125 2026-09-24 07:52:54,217 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer2.prob, batch_count=230220.0, ans=0.125 2026-09-24 07:52:54,668 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer2.prob, batch_count=230253.33333333334, ans=0.125 2026-09-24 07:52:54,699 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=230253.33333333334, ans=0.0 2026-09-24 07:52:55,009 INFO [train.py:1192] (1/2) Epoch 73, batch 100, loss[loss=0.2502, simple_loss=0.3646, pruned_loss=0.06786, over 24640.00 frames. ], tot_loss[loss=0.26, simple_loss=0.3808, pruned_loss=0.06963, over 1916096.63 frames. ], batch size: 154, lr: 2.90e-03, grad_scale: 32.0 2026-09-24 07:52:57,211 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=9.87 vs. limit=12.0 2026-09-24 07:53:00,029 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=230286.66666666666, ans=0.2 2026-09-24 07:53:08,298 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=230320.0, ans=0.125 2026-09-24 07:53:12,162 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=3.80 vs. limit=12.0 2026-09-24 07:53:12,959 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.max_abs, batch_count=230353.33333333334, ans=10.0 2026-09-24 07:53:12,971 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=230353.33333333334, ans=0.125 2026-09-24 07:53:13,521 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=230353.33333333334, ans=0.0 2026-09-24 07:53:21,073 INFO [train.py:1192] (1/2) Epoch 73, batch 150, loss[loss=0.2214, simple_loss=0.3315, pruned_loss=0.05564, over 24261.00 frames. ], tot_loss[loss=0.2577, simple_loss=0.3771, pruned_loss=0.06915, over 2561283.86 frames. ], batch size: 125, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:53:24,210 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=230420.0, ans=0.125 2026-09-24 07:53:36,846 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.546e+02 3.340e+02 3.719e+02 4.243e+02 6.082e+02, threshold=7.438e+02, percent-clipped=0.0 2026-09-24 07:53:39,310 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.5.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:53:42,884 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=230553.33333333334, ans=0.2 2026-09-24 07:53:46,857 INFO [train.py:1192] (1/2) Epoch 73, batch 200, loss[loss=0.2814, simple_loss=0.4108, pruned_loss=0.07598, over 24216.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3754, pruned_loss=0.06819, over 3059643.50 frames. ], batch size: 257, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:53:51,154 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=230586.66666666666, ans=0.04949747468305833 2026-09-24 07:54:10,367 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=230720.0, ans=0.0 2026-09-24 07:54:12,661 INFO [train.py:1192] (1/2) Epoch 73, batch 250, loss[loss=0.2671, simple_loss=0.3974, pruned_loss=0.06846, over 24387.00 frames. ], tot_loss[loss=0.2551, simple_loss=0.3742, pruned_loss=0.06799, over 3445758.04 frames. ], batch size: 225, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:54:12,788 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=230753.33333333334, ans=0.04949747468305833 2026-09-24 07:54:15,768 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=230753.33333333334, ans=10.0 2026-09-24 07:54:18,021 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=230786.66666666666, ans=0.1 2026-09-24 07:54:19,429 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=230786.66666666666, ans=0.125 2026-09-24 07:54:22,629 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=230820.0, ans=0.125 2026-09-24 07:54:28,161 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=230853.33333333334, ans=0.1 2026-09-24 07:54:28,512 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.785e+02 3.488e+02 4.066e+02 4.516e+02 6.561e+02, threshold=8.133e+02, percent-clipped=0.0 2026-09-24 07:54:29,206 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=230853.33333333334, ans=0.07 2026-09-24 07:54:29,208 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=230853.33333333334, ans=0.125 2026-09-24 07:54:38,696 INFO [train.py:1192] (1/2) Epoch 73, batch 300, loss[loss=0.2758, simple_loss=0.4074, pruned_loss=0.07213, over 24560.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3737, pruned_loss=0.06759, over 3759644.50 frames. ], batch size: 204, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:54:44,954 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=230953.33333333334, ans=0.2 2026-09-24 07:54:48,886 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=230986.66666666666, ans=0.125 2026-09-24 07:55:04,727 INFO [train.py:1192] (1/2) Epoch 73, batch 350, loss[loss=0.2188, simple_loss=0.3345, pruned_loss=0.05156, over 24549.00 frames. ], tot_loss[loss=0.2556, simple_loss=0.3747, pruned_loss=0.06826, over 3999475.01 frames. ], batch size: 137, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:55:09,246 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=231086.66666666666, ans=0.0 2026-09-24 07:55:20,115 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.98 vs. limit=15.0 2026-09-24 07:55:20,945 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.837e+02 3.420e+02 3.800e+02 4.422e+02 6.792e+02, threshold=7.601e+02, percent-clipped=0.0 2026-09-24 07:55:30,543 INFO [train.py:1192] (1/2) Epoch 73, batch 400, loss[loss=0.2629, simple_loss=0.3814, pruned_loss=0.07213, over 24560.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3736, pruned_loss=0.06763, over 4181762.46 frames. ], batch size: 170, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:55:31,574 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=231253.33333333334, ans=0.125 2026-09-24 07:55:31,785 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=9.29 vs. limit=15.0 2026-09-24 07:55:32,012 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=231253.33333333334, ans=0.1 2026-09-24 07:55:40,976 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=231320.0, ans=0.1 2026-09-24 07:55:48,311 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=231353.33333333334, ans=0.07 2026-09-24 07:55:52,224 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=231386.66666666666, ans=0.1 2026-09-24 07:55:56,676 INFO [train.py:1192] (1/2) Epoch 73, batch 450, loss[loss=0.2801, simple_loss=0.4049, pruned_loss=0.07762, over 24640.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3738, pruned_loss=0.06776, over 4317816.61 frames. ], batch size: 175, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:56:02,108 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=231453.33333333334, ans=0.125 2026-09-24 07:56:11,197 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer2.prob, batch_count=231486.66666666666, ans=0.125 2026-09-24 07:56:12,492 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.564e+02 3.410e+02 3.808e+02 4.266e+02 5.624e+02, threshold=7.616e+02, percent-clipped=0.0 2026-09-24 07:56:20,975 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=9.65 vs. limit=15.0 2026-09-24 07:56:22,350 INFO [train.py:1192] (1/2) Epoch 73, batch 500, loss[loss=0.262, simple_loss=0.3946, pruned_loss=0.06469, over 24534.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3718, pruned_loss=0.06686, over 4436340.93 frames. ], batch size: 218, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:56:22,891 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.skip_rate, batch_count=231586.66666666666, ans=0.07 2026-09-24 07:56:48,158 INFO [train.py:1192] (1/2) Epoch 73, batch 550, loss[loss=0.2692, simple_loss=0.4034, pruned_loss=0.06744, over 24299.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3725, pruned_loss=0.06727, over 4522585.26 frames. ], batch size: 257, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:56:50,286 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=231753.33333333334, ans=0.0 2026-09-24 07:57:02,063 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=231820.0, ans=0.1 2026-09-24 07:57:03,946 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.688e+02 3.309e+02 3.713e+02 4.161e+02 6.571e+02, threshold=7.426e+02, percent-clipped=0.0 2026-09-24 07:57:08,408 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=231886.66666666666, ans=0.0 2026-09-24 07:57:13,999 INFO [train.py:1192] (1/2) Epoch 73, batch 600, loss[loss=0.2617, simple_loss=0.3933, pruned_loss=0.06504, over 24326.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.373, pruned_loss=0.06727, over 4588435.80 frames. ], batch size: 234, lr: 2.89e-03, grad_scale: 32.0 2026-09-24 07:57:14,572 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=231920.0, ans=0.0 2026-09-24 07:57:15,122 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.59 vs. limit=15.0 2026-09-24 07:57:17,383 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=231920.0, ans=0.1 2026-09-24 07:57:19,295 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=231953.33333333334, ans=0.0 2026-09-24 07:57:38,189 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.42 vs. limit=15.0 2026-09-24 07:57:40,363 INFO [train.py:1192] (1/2) Epoch 73, batch 650, loss[loss=0.2336, simple_loss=0.3505, pruned_loss=0.05835, over 24608.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3724, pruned_loss=0.06715, over 4653250.21 frames. ], batch size: 154, lr: 2.88e-03, grad_scale: 32.0 2026-09-24 07:57:46,059 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer2.prob, batch_count=232120.0, ans=0.125 2026-09-24 07:57:56,635 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.960e+02 3.399e+02 3.702e+02 4.237e+02 6.090e+02, threshold=7.404e+02, percent-clipped=0.0 2026-09-24 07:57:59,752 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.35 vs. limit=15.0 2026-09-24 07:58:05,961 INFO [train.py:1192] (1/2) Epoch 73, batch 700, loss[loss=0.2668, simple_loss=0.3784, pruned_loss=0.07757, over 24558.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3731, pruned_loss=0.06695, over 4689980.15 frames. ], batch size: 158, lr: 2.88e-03, grad_scale: 32.0 2026-09-24 07:58:10,634 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.whiten.whitening_limit, batch_count=232253.33333333334, ans=12.0 2026-09-24 07:58:15,138 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 07:58:32,090 INFO [train.py:1192] (1/2) Epoch 73, batch 750, loss[loss=0.2604, simple_loss=0.38, pruned_loss=0.07036, over 24622.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3725, pruned_loss=0.06708, over 4726658.86 frames. ], batch size: 175, lr: 2.88e-03, grad_scale: 32.0 2026-09-24 07:58:34,883 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.const_attention_rate, batch_count=232420.0, ans=0.025 2026-09-24 07:58:36,685 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=232420.0, ans=0.0 2026-09-24 07:58:38,137 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=4.54 vs. limit=15.0 2026-09-24 07:58:44,846 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.01 vs. limit=22.5 2026-09-24 07:58:46,196 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=232486.66666666666, ans=0.1 2026-09-24 07:58:48,340 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.899e+02 3.525e+02 4.022e+02 4.642e+02 6.227e+02, threshold=8.044e+02, percent-clipped=0.0 2026-09-24 07:58:55,249 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.min_positive, batch_count=232553.33333333334, ans=0.025 2026-09-24 07:58:57,953 INFO [train.py:1192] (1/2) Epoch 73, batch 800, loss[loss=0.2205, simple_loss=0.3328, pruned_loss=0.05407, over 24546.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3719, pruned_loss=0.06677, over 4752791.33 frames. ], batch size: 137, lr: 2.88e-03, grad_scale: 32.0 2026-09-24 07:59:05,105 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.16 vs. limit=15.0 2026-09-24 07:59:06,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=232620.0, ans=0.125 2026-09-24 07:59:10,845 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=232653.33333333334, ans=0.1 2026-09-24 07:59:12,928 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=232686.66666666666, ans=0.95 2026-09-24 07:59:16,624 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=232686.66666666666, ans=0.0 2026-09-24 07:59:22,274 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=232720.0, ans=0.2 2026-09-24 07:59:23,489 INFO [train.py:1192] (1/2) Epoch 73, batch 850, loss[loss=0.2719, simple_loss=0.3974, pruned_loss=0.0732, over 24539.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3712, pruned_loss=0.06645, over 4770915.89 frames. ], batch size: 204, lr: 2.88e-03, grad_scale: 32.0 2026-09-24 07:59:27,788 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.conv_module2.whiten, num_groups=1, num_channels=192, metric=7.03 vs. limit=15.0 2026-09-24 07:59:30,376 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff2_skip_rate, batch_count=232786.66666666666, ans=0.0 2026-09-24 07:59:30,873 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=232786.66666666666, ans=0.125 2026-09-24 07:59:39,301 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 3.019e+02 3.518e+02 3.918e+02 4.685e+02 6.562e+02, threshold=7.837e+02, percent-clipped=0.0 2026-09-24 07:59:43,168 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=232853.33333333334, ans=0.1 2026-09-24 07:59:44,876 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=232886.66666666666, ans=0.0 2026-09-24 07:59:45,379 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=232886.66666666666, ans=0.2 2026-09-24 07:59:49,227 INFO [train.py:1192] (1/2) Epoch 73, batch 900, loss[loss=0.2048, simple_loss=0.3259, pruned_loss=0.04181, over 24563.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.372, pruned_loss=0.06686, over 4781196.63 frames. ], batch size: 137, lr: 2.88e-03, grad_scale: 32.0 2026-09-24 07:59:51,348 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff2_skip_rate, batch_count=232920.0, ans=0.0 2026-09-24 08:00:02,456 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=232986.66666666666, ans=0.125 2026-09-24 08:00:14,509 INFO [train.py:1192] (1/2) Epoch 73, batch 950, loss[loss=0.3517, simple_loss=0.4211, pruned_loss=0.1411, over 11147.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3712, pruned_loss=0.06791, over 4708890.53 frames. ], batch size: 334, lr: 2.88e-03, grad_scale: 16.0 2026-09-24 08:00:17,317 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=21.12 vs. limit=22.5 2026-09-24 08:00:24,752 INFO [train.py:1192] (1/2) Epoch 74, batch 0, loss[loss=0.2149, simple_loss=0.3346, pruned_loss=0.04759, over 24576.00 frames. ], tot_loss[loss=0.2149, simple_loss=0.3346, pruned_loss=0.04759, over 24576.00 frames. ], batch size: 137, lr: 2.86e-03, grad_scale: 32.0 2026-09-24 08:00:24,752 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 08:00:29,141 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.1.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([4.9627, 4.3734, 4.6015, 4.3803], device='cuda:1') 2026-09-24 08:00:30,777 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.3.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8915, 2.9051, 3.3261, 2.7389, 2.5400, 3.2572, 1.9733, 2.6340], device='cuda:1') 2026-09-24 08:00:36,538 INFO [train.py:1224] (1/2) Epoch 74, validation: loss=0.1684, simple_loss=0.2869, pruned_loss=0.02499, over 2564189.00 frames. 2026-09-24 08:00:36,538 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 08:00:41,557 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward2.hidden_balancer.prob, batch_count=233146.66666666666, ans=0.125 2026-09-24 08:00:44,761 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=233146.66666666666, ans=0.0 2026-09-24 08:00:46,070 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.attention_skip_rate, batch_count=233180.0, ans=0.0 2026-09-24 08:00:46,897 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.min_positive, batch_count=233180.0, ans=0.025 2026-09-24 08:00:48,696 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.549e+02 3.454e+02 3.895e+02 4.574e+02 7.934e+02, threshold=7.790e+02, percent-clipped=1.0 2026-09-24 08:00:50,543 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=233180.0, ans=0.0 2026-09-24 08:00:50,550 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass_mid.scale_min, batch_count=233180.0, ans=0.2 2026-09-24 08:00:50,591 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=233180.0, ans=0.0 2026-09-24 08:00:55,344 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=5.09 vs. limit=15.0 2026-09-24 08:00:59,200 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=233246.66666666666, ans=0.125 2026-09-24 08:01:01,803 INFO [train.py:1192] (1/2) Epoch 74, batch 50, loss[loss=0.2064, simple_loss=0.3207, pruned_loss=0.04607, over 24261.00 frames. ], tot_loss[loss=0.2553, simple_loss=0.3746, pruned_loss=0.06801, over 1082658.16 frames. ], batch size: 125, lr: 2.86e-03, grad_scale: 32.0 2026-09-24 08:01:13,467 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=233346.66666666666, ans=0.125 2026-09-24 08:01:27,548 INFO [train.py:1192] (1/2) Epoch 74, batch 100, loss[loss=0.2503, simple_loss=0.363, pruned_loss=0.06876, over 24626.00 frames. ], tot_loss[loss=0.2594, simple_loss=0.3801, pruned_loss=0.06941, over 1916091.94 frames. ], batch size: 154, lr: 2.86e-03, grad_scale: 32.0 2026-09-24 08:01:38,479 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.scale_min, batch_count=233513.33333333334, ans=0.2 2026-09-24 08:01:39,792 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.764e+02 3.427e+02 3.827e+02 4.293e+02 5.928e+02, threshold=7.654e+02, percent-clipped=0.0 2026-09-24 08:01:50,391 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=233580.0, ans=0.07 2026-09-24 08:01:53,442 INFO [train.py:1192] (1/2) Epoch 74, batch 150, loss[loss=0.1987, simple_loss=0.3164, pruned_loss=0.04045, over 24263.00 frames. ], tot_loss[loss=0.2555, simple_loss=0.3752, pruned_loss=0.0679, over 2561079.34 frames. ], batch size: 125, lr: 2.86e-03, grad_scale: 32.0 2026-09-24 08:02:02,995 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:02:04,560 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=5.07 vs. limit=15.0 2026-09-24 08:02:05,275 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.min_positive, batch_count=233680.0, ans=0.05 2026-09-24 08:02:18,790 INFO [train.py:1192] (1/2) Epoch 74, batch 200, loss[loss=0.2995, simple_loss=0.423, pruned_loss=0.08803, over 24195.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3727, pruned_loss=0.06641, over 3060481.42 frames. ], batch size: 257, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:02:19,458 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=233780.0, ans=0.2 2026-09-24 08:02:19,923 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer2.prob, batch_count=233780.0, ans=0.125 2026-09-24 08:02:30,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.hidden_balancer.prob, batch_count=233846.66666666666, ans=0.125 2026-09-24 08:02:30,707 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.787e+02 3.340e+02 3.805e+02 4.289e+02 6.745e+02, threshold=7.609e+02, percent-clipped=0.0 2026-09-24 08:02:30,823 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=233846.66666666666, ans=0.0 2026-09-24 08:02:34,269 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=233880.0, ans=0.125 2026-09-24 08:02:37,658 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=233880.0, ans=0.125 2026-09-24 08:02:41,141 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.attention_skip_rate, batch_count=233913.33333333334, ans=0.0 2026-09-24 08:02:43,994 INFO [train.py:1192] (1/2) Epoch 74, batch 250, loss[loss=0.294, simple_loss=0.4171, pruned_loss=0.08545, over 24380.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.3724, pruned_loss=0.06638, over 3445443.16 frames. ], batch size: 225, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:02:47,581 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=4.88 vs. limit=15.0 2026-09-24 08:03:00,105 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=234046.66666666666, ans=0.0 2026-09-24 08:03:04,968 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=234080.0, ans=0.0 2026-09-24 08:03:10,273 INFO [train.py:1192] (1/2) Epoch 74, batch 300, loss[loss=0.2692, simple_loss=0.3875, pruned_loss=0.07547, over 24535.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3726, pruned_loss=0.06683, over 3759078.82 frames. ], batch size: 204, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:03:11,866 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=234113.33333333334, ans=0.1 2026-09-24 08:03:22,607 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.787e+02 3.564e+02 3.883e+02 4.568e+02 6.180e+02, threshold=7.765e+02, percent-clipped=0.0 2026-09-24 08:03:26,550 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=234213.33333333334, ans=0.125 2026-09-24 08:03:31,811 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=234246.66666666666, ans=0.125 2026-09-24 08:03:35,316 INFO [train.py:1192] (1/2) Epoch 74, batch 350, loss[loss=0.2171, simple_loss=0.3333, pruned_loss=0.05044, over 24578.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3731, pruned_loss=0.0669, over 3999480.04 frames. ], batch size: 137, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:03:39,113 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.min_positive, batch_count=234280.0, ans=0.05 2026-09-24 08:03:40,145 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=234313.33333333334, ans=0.125 2026-09-24 08:03:44,837 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=234313.33333333334, ans=0.125 2026-09-24 08:04:01,396 INFO [train.py:1192] (1/2) Epoch 74, batch 400, loss[loss=0.253, simple_loss=0.3709, pruned_loss=0.06758, over 24569.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3725, pruned_loss=0.0665, over 4185098.14 frames. ], batch size: 170, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:04:04,865 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer_ff2.min_abs, batch_count=234446.66666666666, ans=0.1 2026-09-24 08:04:10,421 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=234480.0, ans=0.125 2026-09-24 08:04:13,288 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.675e+02 3.316e+02 3.775e+02 4.302e+02 5.883e+02, threshold=7.551e+02, percent-clipped=0.0 2026-09-24 08:04:18,500 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.06 vs. limit=15.0 2026-09-24 08:04:21,328 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.95 vs. limit=6.0 2026-09-24 08:04:26,714 INFO [train.py:1192] (1/2) Epoch 74, batch 450, loss[loss=0.2526, simple_loss=0.3802, pruned_loss=0.06254, over 24627.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3727, pruned_loss=0.06646, over 4323801.72 frames. ], batch size: 175, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:04:30,861 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.50 vs. limit=15.0 2026-09-24 08:04:31,817 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.14 vs. limit=15.0 2026-09-24 08:04:32,187 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=234646.66666666666, ans=0.125 2026-09-24 08:04:38,764 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.13 vs. limit=6.0 2026-09-24 08:04:44,127 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.max_abs, batch_count=234713.33333333334, ans=10.0 2026-09-24 08:04:48,690 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=234746.66666666666, ans=0.1 2026-09-24 08:04:53,057 INFO [train.py:1192] (1/2) Epoch 74, batch 500, loss[loss=0.2419, simple_loss=0.3747, pruned_loss=0.05461, over 24509.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.3718, pruned_loss=0.06669, over 4440190.48 frames. ], batch size: 218, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:04:53,701 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.max_abs, batch_count=234780.0, ans=10.0 2026-09-24 08:05:05,585 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.703e+02 3.488e+02 3.882e+02 4.333e+02 6.074e+02, threshold=7.764e+02, percent-clipped=0.0 2026-09-24 08:05:17,103 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=234913.33333333334, ans=0.1 2026-09-24 08:05:19,764 INFO [train.py:1192] (1/2) Epoch 74, batch 550, loss[loss=0.2598, simple_loss=0.393, pruned_loss=0.06328, over 24308.00 frames. ], tot_loss[loss=0.2531, simple_loss=0.3725, pruned_loss=0.06682, over 4524559.02 frames. ], batch size: 257, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:05:26,269 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=1.88 vs. limit=6.0 2026-09-24 08:05:39,463 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.65 vs. limit=6.0 2026-09-24 08:05:45,467 INFO [train.py:1192] (1/2) Epoch 74, batch 600, loss[loss=0.2547, simple_loss=0.3839, pruned_loss=0.06278, over 24331.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3726, pruned_loss=0.06651, over 4590229.72 frames. ], batch size: 234, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:05:55,633 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.29 vs. limit=15.0 2026-09-24 08:05:57,771 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.632e+02 3.324e+02 3.672e+02 4.258e+02 7.730e+02, threshold=7.345e+02, percent-clipped=0.0 2026-09-24 08:06:06,461 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=235246.66666666666, ans=0.125 2026-09-24 08:06:08,173 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module2.whiten, num_groups=1, num_channels=192, metric=10.67 vs. limit=15.0 2026-09-24 08:06:11,285 INFO [train.py:1192] (1/2) Epoch 74, batch 650, loss[loss=0.2564, simple_loss=0.371, pruned_loss=0.07095, over 24602.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3719, pruned_loss=0.06634, over 4654979.41 frames. ], batch size: 154, lr: 2.85e-03, grad_scale: 32.0 2026-09-24 08:06:11,390 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:06:19,381 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=235313.33333333334, ans=0.125 2026-09-24 08:06:25,636 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=6.37 vs. limit=6.0 2026-09-24 08:06:29,189 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=235380.0, ans=0.1 2026-09-24 08:06:37,370 INFO [train.py:1192] (1/2) Epoch 74, batch 700, loss[loss=0.2545, simple_loss=0.3697, pruned_loss=0.06963, over 24543.00 frames. ], tot_loss[loss=0.2534, simple_loss=0.3732, pruned_loss=0.06684, over 4690421.23 frames. ], batch size: 158, lr: 2.84e-03, grad_scale: 32.0 2026-09-24 08:06:50,061 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.897e+02 3.449e+02 3.815e+02 4.155e+02 6.261e+02, threshold=7.629e+02, percent-clipped=0.0 2026-09-24 08:06:55,162 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=235546.66666666666, ans=0.0 2026-09-24 08:06:56,570 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=235546.66666666666, ans=0.0 2026-09-24 08:07:03,434 INFO [train.py:1192] (1/2) Epoch 74, batch 750, loss[loss=0.2507, simple_loss=0.3806, pruned_loss=0.06042, over 24626.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.3726, pruned_loss=0.06691, over 4726632.68 frames. ], batch size: 175, lr: 2.84e-03, grad_scale: 32.0 2026-09-24 08:07:04,052 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:07:05,926 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff3_skip_rate, batch_count=235613.33333333334, ans=0.0 2026-09-24 08:07:06,771 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=235613.33333333334, ans=0.125 2026-09-24 08:07:17,878 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=235680.0, ans=0.125 2026-09-24 08:07:28,763 INFO [train.py:1192] (1/2) Epoch 74, batch 800, loss[loss=0.2038, simple_loss=0.3214, pruned_loss=0.04308, over 24531.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3721, pruned_loss=0.06647, over 4752315.82 frames. ], batch size: 137, lr: 2.84e-03, grad_scale: 32.0 2026-09-24 08:07:41,908 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.661e+02 3.511e+02 3.915e+02 4.395e+02 7.015e+02, threshold=7.829e+02, percent-clipped=0.0 2026-09-24 08:07:54,993 INFO [train.py:1192] (1/2) Epoch 74, batch 850, loss[loss=0.2828, simple_loss=0.4037, pruned_loss=0.08089, over 24570.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3715, pruned_loss=0.06616, over 4770562.27 frames. ], batch size: 204, lr: 2.84e-03, grad_scale: 32.0 2026-09-24 08:08:04,223 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=235980.0, ans=0.125 2026-09-24 08:08:06,533 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=236013.33333333334, ans=0.125 2026-09-24 08:08:16,407 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.30 vs. limit=10.0 2026-09-24 08:08:21,586 INFO [train.py:1192] (1/2) Epoch 74, batch 900, loss[loss=0.2098, simple_loss=0.3248, pruned_loss=0.04742, over 24569.00 frames. ], tot_loss[loss=0.2522, simple_loss=0.3717, pruned_loss=0.06634, over 4781238.51 frames. ], batch size: 137, lr: 2.84e-03, grad_scale: 32.0 2026-09-24 08:08:29,212 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=236146.66666666666, ans=0.1 2026-09-24 08:08:29,647 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=236146.66666666666, ans=0.1 2026-09-24 08:08:32,119 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=236180.0, ans=0.125 2026-09-24 08:08:34,519 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.503e+02 3.414e+02 3.841e+02 4.540e+02 6.153e+02, threshold=7.681e+02, percent-clipped=0.0 2026-09-24 08:08:37,395 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:08:46,498 INFO [train.py:1192] (1/2) Epoch 74, batch 950, loss[loss=0.3158, simple_loss=0.3919, pruned_loss=0.1198, over 11343.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3701, pruned_loss=0.06676, over 4707980.60 frames. ], batch size: 333, lr: 2.84e-03, grad_scale: 32.0 2026-09-24 08:08:46,881 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=5.77 vs. limit=6.0 2026-09-24 08:08:58,646 INFO [train.py:1192] (1/2) Epoch 75, batch 0, loss[loss=0.2006, simple_loss=0.3251, pruned_loss=0.03805, over 24563.00 frames. ], tot_loss[loss=0.2006, simple_loss=0.3251, pruned_loss=0.03805, over 24563.00 frames. ], batch size: 137, lr: 2.82e-03, grad_scale: 32.0 2026-09-24 08:08:58,647 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 08:09:06,873 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.0.layers.1.self_attn_weights, attn_weights_entropy = tensor([5.2843, 4.7228, 4.7115, 5.1636], device='cuda:1') 2026-09-24 08:09:10,524 INFO [train.py:1224] (1/2) Epoch 75, validation: loss=0.1675, simple_loss=0.2858, pruned_loss=0.02463, over 2564189.00 frames. 2026-09-24 08:09:10,524 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 08:09:17,258 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.out_combiner.scale_min, batch_count=236340.0, ans=0.2 2026-09-24 08:09:22,376 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=236373.33333333334, ans=10.0 2026-09-24 08:09:25,612 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module1.balancer1.max_abs, batch_count=236406.66666666666, ans=10.0 2026-09-24 08:09:35,800 INFO [train.py:1192] (1/2) Epoch 75, batch 50, loss[loss=0.2064, simple_loss=0.3174, pruned_loss=0.04775, over 24238.00 frames. ], tot_loss[loss=0.2547, simple_loss=0.3748, pruned_loss=0.06726, over 1081391.56 frames. ], batch size: 125, lr: 2.82e-03, grad_scale: 32.0 2026-09-24 08:09:37,235 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=236473.33333333334, ans=0.125 2026-09-24 08:09:42,730 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=236506.66666666666, ans=0.1 2026-09-24 08:09:42,741 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass_mid.scale_min, batch_count=236506.66666666666, ans=0.2 2026-09-24 08:09:43,280 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.const_attention_rate, batch_count=236506.66666666666, ans=0.025 2026-09-24 08:09:44,422 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.860e+02 3.663e+02 4.192e+02 4.797e+02 6.594e+02, threshold=8.383e+02, percent-clipped=0.0 2026-09-24 08:09:44,960 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=236506.66666666666, ans=0.125 2026-09-24 08:09:59,017 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=236606.66666666666, ans=0.1 2026-09-24 08:10:00,839 INFO [train.py:1192] (1/2) Epoch 75, batch 100, loss[loss=0.2316, simple_loss=0.3517, pruned_loss=0.05577, over 24617.00 frames. ], tot_loss[loss=0.2575, simple_loss=0.3787, pruned_loss=0.0681, over 1916968.99 frames. ], batch size: 154, lr: 2.82e-03, grad_scale: 32.0 2026-09-24 08:10:06,165 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=236673.33333333334, ans=0.2 2026-09-24 08:10:09,198 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.prob, batch_count=236673.33333333334, ans=0.125 2026-09-24 08:10:13,156 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=236706.66666666666, ans=0.125 2026-09-24 08:10:16,597 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=236740.0, ans=0.0 2026-09-24 08:10:26,529 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=236806.66666666666, ans=0.0 2026-09-24 08:10:26,892 INFO [train.py:1192] (1/2) Epoch 75, batch 150, loss[loss=0.2013, simple_loss=0.3153, pruned_loss=0.04366, over 24298.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3743, pruned_loss=0.06704, over 2562064.49 frames. ], batch size: 125, lr: 2.82e-03, grad_scale: 32.0 2026-09-24 08:10:27,525 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.balancer1.prob, batch_count=236806.66666666666, ans=0.125 2026-09-24 08:10:30,814 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.ff2_skip_rate, batch_count=236806.66666666666, ans=0.0 2026-09-24 08:10:35,539 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:10:35,543 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=236840.0, ans=0.0 2026-09-24 08:10:35,889 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.735e+02 3.496e+02 3.861e+02 4.306e+02 6.041e+02, threshold=7.723e+02, percent-clipped=0.0 2026-09-24 08:10:40,001 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=236873.33333333334, ans=0.2 2026-09-24 08:10:53,327 INFO [train.py:1192] (1/2) Epoch 75, batch 200, loss[loss=0.2799, simple_loss=0.4131, pruned_loss=0.07336, over 24249.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3735, pruned_loss=0.06684, over 3061379.12 frames. ], batch size: 257, lr: 2.82e-03, grad_scale: 32.0 2026-09-24 08:11:07,425 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=237040.0, ans=0.0 2026-09-24 08:11:15,249 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=237106.66666666666, ans=0.04949747468305833 2026-09-24 08:11:18,734 INFO [train.py:1192] (1/2) Epoch 75, batch 250, loss[loss=0.2888, simple_loss=0.4152, pruned_loss=0.08118, over 24396.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3729, pruned_loss=0.06718, over 3444859.03 frames. ], batch size: 225, lr: 2.82e-03, grad_scale: 32.0 2026-09-24 08:11:27,134 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.670e+02 3.484e+02 3.917e+02 4.560e+02 6.379e+02, threshold=7.834e+02, percent-clipped=0.0 2026-09-24 08:11:38,411 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=237240.0, ans=0.125 2026-09-24 08:11:43,103 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=237273.33333333334, ans=0.0 2026-09-24 08:11:43,302 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.54 vs. limit=15.0 2026-09-24 08:11:45,062 INFO [train.py:1192] (1/2) Epoch 75, batch 300, loss[loss=0.2867, simple_loss=0.4135, pruned_loss=0.07999, over 24555.00 frames. ], tot_loss[loss=0.2533, simple_loss=0.3725, pruned_loss=0.06702, over 3758077.44 frames. ], batch size: 204, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:11:48,129 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.96 vs. limit=15.0 2026-09-24 08:11:52,609 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=9.45 vs. limit=15.0 2026-09-24 08:12:03,994 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer1.prob, batch_count=237406.66666666666, ans=0.125 2026-09-24 08:12:06,662 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=237440.0, ans=0.125 2026-09-24 08:12:09,646 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=237440.0, ans=0.2 2026-09-24 08:12:10,519 INFO [train.py:1192] (1/2) Epoch 75, batch 350, loss[loss=0.2108, simple_loss=0.3276, pruned_loss=0.04702, over 24556.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3735, pruned_loss=0.0674, over 3995000.62 frames. ], batch size: 137, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:12:10,614 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=237473.33333333334, ans=0.1 2026-09-24 08:12:11,685 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.93 vs. limit=15.0 2026-09-24 08:12:19,183 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.729e+02 3.346e+02 3.794e+02 4.475e+02 7.372e+02, threshold=7.587e+02, percent-clipped=0.0 2026-09-24 08:12:24,771 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=237540.0, ans=0.0 2026-09-24 08:12:29,245 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=237573.33333333334, ans=0.5 2026-09-24 08:12:35,724 INFO [train.py:1192] (1/2) Epoch 75, batch 400, loss[loss=0.2761, simple_loss=0.3856, pruned_loss=0.08332, over 24559.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3723, pruned_loss=0.06677, over 4177715.00 frames. ], batch size: 170, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:12:42,338 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=237673.33333333334, ans=0.2 2026-09-24 08:12:50,476 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=237740.0, ans=0.0 2026-09-24 08:12:53,559 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.skip_rate, batch_count=237740.0, ans=0.07 2026-09-24 08:13:01,226 INFO [train.py:1192] (1/2) Epoch 75, batch 450, loss[loss=0.2546, simple_loss=0.3778, pruned_loss=0.0657, over 24630.00 frames. ], tot_loss[loss=0.2535, simple_loss=0.3728, pruned_loss=0.06711, over 4317855.45 frames. ], batch size: 175, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:13:02,953 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=237806.66666666666, ans=0.125 2026-09-24 08:13:03,286 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.conv_module1.whiten, num_groups=1, num_channels=192, metric=10.96 vs. limit=15.0 2026-09-24 08:13:10,049 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.880e+02 3.483e+02 3.872e+02 4.424e+02 7.917e+02, threshold=7.745e+02, percent-clipped=1.0 2026-09-24 08:13:13,341 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.85 vs. limit=15.0 2026-09-24 08:13:21,310 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:13:23,154 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=237940.0, ans=0.0 2026-09-24 08:13:25,845 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=237940.0, ans=0.125 2026-09-24 08:13:27,557 INFO [train.py:1192] (1/2) Epoch 75, batch 500, loss[loss=0.2926, simple_loss=0.4156, pruned_loss=0.0848, over 24525.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3719, pruned_loss=0.06688, over 4436390.65 frames. ], batch size: 218, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:13:28,345 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.87 vs. limit=15.0 2026-09-24 08:13:36,238 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=238006.66666666666, ans=0.025 2026-09-24 08:13:37,642 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer2.min_abs, batch_count=238040.0, ans=0.5 2026-09-24 08:13:41,738 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=238040.0, ans=0.125 2026-09-24 08:13:50,270 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=238106.66666666666, ans=0.1 2026-09-24 08:13:53,903 INFO [train.py:1192] (1/2) Epoch 75, batch 550, loss[loss=0.2944, simple_loss=0.4189, pruned_loss=0.08499, over 24276.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3728, pruned_loss=0.06742, over 4521447.43 frames. ], batch size: 257, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:13:53,990 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=238140.0, ans=0.2 2026-09-24 08:13:55,033 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.17 vs. limit=15.0 2026-09-24 08:14:02,108 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.737e+02 3.424e+02 3.830e+02 4.349e+02 6.866e+02, threshold=7.661e+02, percent-clipped=0.0 2026-09-24 08:14:13,195 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=238273.33333333334, ans=0.0 2026-09-24 08:14:19,011 INFO [train.py:1192] (1/2) Epoch 75, batch 600, loss[loss=0.2935, simple_loss=0.418, pruned_loss=0.08445, over 24345.00 frames. ], tot_loss[loss=0.2543, simple_loss=0.3735, pruned_loss=0.06755, over 4588945.76 frames. ], batch size: 234, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:14:21,572 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=238306.66666666666, ans=0.125 2026-09-24 08:14:30,478 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=238373.33333333334, ans=0.2 2026-09-24 08:14:41,130 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer2.prob, batch_count=238440.0, ans=0.125 2026-09-24 08:14:43,902 INFO [train.py:1192] (1/2) Epoch 75, batch 650, loss[loss=0.2737, simple_loss=0.3767, pruned_loss=0.08531, over 24552.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.372, pruned_loss=0.0668, over 4653705.13 frames. ], batch size: 154, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:14:45,367 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=238473.33333333334, ans=0.1 2026-09-24 08:14:48,823 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=238506.66666666666, ans=0.125 2026-09-24 08:14:52,140 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.765e+02 3.291e+02 3.723e+02 4.199e+02 5.918e+02, threshold=7.446e+02, percent-clipped=0.0 2026-09-24 08:14:53,395 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=238540.0, ans=0.125 2026-09-24 08:14:53,882 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=238540.0, ans=0.1 2026-09-24 08:15:06,055 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=238606.66666666666, ans=0.1 2026-09-24 08:15:08,966 INFO [train.py:1192] (1/2) Epoch 75, batch 700, loss[loss=0.2357, simple_loss=0.3553, pruned_loss=0.05803, over 24559.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3725, pruned_loss=0.06651, over 4688755.71 frames. ], batch size: 158, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:15:23,573 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=238706.66666666666, ans=0.125 2026-09-24 08:15:34,278 INFO [train.py:1192] (1/2) Epoch 75, batch 750, loss[loss=0.2473, simple_loss=0.3767, pruned_loss=0.05889, over 24619.00 frames. ], tot_loss[loss=0.2528, simple_loss=0.3721, pruned_loss=0.06671, over 4725420.05 frames. ], batch size: 175, lr: 2.81e-03, grad_scale: 32.0 2026-09-24 08:15:37,857 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=238806.66666666666, ans=0.0 2026-09-24 08:15:41,975 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.nonlin_attention.balancer.prob, batch_count=238840.0, ans=0.125 2026-09-24 08:15:43,332 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.908e+02 3.604e+02 4.002e+02 4.730e+02 6.100e+02, threshold=8.003e+02, percent-clipped=0.0 2026-09-24 08:15:50,822 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=4.57 vs. limit=15.0 2026-09-24 08:15:52,297 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=238906.66666666666, ans=0.125 2026-09-24 08:15:59,904 INFO [train.py:1192] (1/2) Epoch 75, batch 800, loss[loss=0.2245, simple_loss=0.3393, pruned_loss=0.05487, over 24528.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3723, pruned_loss=0.06681, over 4751513.93 frames. ], batch size: 137, lr: 2.80e-03, grad_scale: 32.0 2026-09-24 08:16:01,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.scale_min, batch_count=238973.33333333334, ans=0.2 2026-09-24 08:16:08,679 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=239006.66666666666, ans=0.125 2026-09-24 08:16:12,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=239040.0, ans=0.0 2026-09-24 08:16:14,451 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer2.prob, batch_count=239073.33333333334, ans=0.125 2026-09-24 08:16:15,411 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.whiten, num_groups=1, num_channels=192, metric=4.47 vs. limit=12.0 2026-09-24 08:16:16,855 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=239073.33333333334, ans=0.025 2026-09-24 08:16:17,379 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=239073.33333333334, ans=0.2 2026-09-24 08:16:17,416 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=239073.33333333334, ans=0.025 2026-09-24 08:16:19,741 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=239106.66666666666, ans=0.1 2026-09-24 08:16:24,276 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.11 vs. limit=15.0 2026-09-24 08:16:24,966 INFO [train.py:1192] (1/2) Epoch 75, batch 850, loss[loss=0.2647, simple_loss=0.3968, pruned_loss=0.0663, over 24590.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3718, pruned_loss=0.06645, over 4769877.65 frames. ], batch size: 204, lr: 2.80e-03, grad_scale: 32.0 2026-09-24 08:16:25,085 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:16:33,717 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.830e+02 3.516e+02 3.911e+02 4.413e+02 6.230e+02, threshold=7.822e+02, percent-clipped=0.0 2026-09-24 08:16:41,752 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.out_combiner.scale_min, batch_count=239240.0, ans=0.2 2026-09-24 08:16:43,054 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=239240.0, ans=0.025 2026-09-24 08:16:46,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.nonlin_attention.balancer.prob, batch_count=239273.33333333334, ans=0.125 2026-09-24 08:16:49,647 INFO [train.py:1192] (1/2) Epoch 75, batch 900, loss[loss=0.2205, simple_loss=0.3366, pruned_loss=0.05215, over 24554.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3716, pruned_loss=0.06624, over 4780649.37 frames. ], batch size: 137, lr: 2.80e-03, grad_scale: 32.0 2026-09-24 08:16:51,494 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.const_attention_rate, batch_count=239306.66666666666, ans=0.025 2026-09-24 08:17:08,640 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=239406.66666666666, ans=0.125 2026-09-24 08:17:09,909 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=239440.0, ans=0.125 2026-09-24 08:17:11,020 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.66 vs. limit=6.0 2026-09-24 08:17:11,951 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=239440.0, ans=0.2 2026-09-24 08:17:12,984 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=239440.0, ans=0.125 2026-09-24 08:17:14,895 INFO [train.py:1192] (1/2) Epoch 75, batch 950, loss[loss=0.3196, simple_loss=0.3975, pruned_loss=0.1209, over 10921.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3706, pruned_loss=0.06706, over 4711206.27 frames. ], batch size: 333, lr: 2.80e-03, grad_scale: 32.0 2026-09-24 08:17:26,061 INFO [train.py:1192] (1/2) Epoch 76, batch 0, loss[loss=0.2043, simple_loss=0.3301, pruned_loss=0.03929, over 24569.00 frames. ], tot_loss[loss=0.2043, simple_loss=0.3301, pruned_loss=0.03929, over 24569.00 frames. ], batch size: 137, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:17:26,061 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 08:17:37,693 INFO [train.py:1224] (1/2) Epoch 76, validation: loss=0.1691, simple_loss=0.2872, pruned_loss=0.0255, over 2564189.00 frames. 2026-09-24 08:17:37,693 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 08:17:42,601 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.621e+02 3.523e+02 3.917e+02 4.585e+02 8.203e+02, threshold=7.834e+02, percent-clipped=2.0 2026-09-24 08:17:43,632 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=239533.33333333334, ans=0.125 2026-09-24 08:17:48,627 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=239566.66666666666, ans=0.1 2026-09-24 08:17:50,934 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=239566.66666666666, ans=0.125 2026-09-24 08:17:54,441 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.21 vs. limit=22.5 2026-09-24 08:18:03,691 INFO [train.py:1192] (1/2) Epoch 76, batch 50, loss[loss=0.199, simple_loss=0.3114, pruned_loss=0.04334, over 24270.00 frames. ], tot_loss[loss=0.2567, simple_loss=0.3775, pruned_loss=0.06797, over 1081535.52 frames. ], batch size: 125, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:18:27,997 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=239800.0, ans=0.125 2026-09-24 08:18:28,936 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.bypass_mid.scale_min, batch_count=239800.0, ans=0.2 2026-09-24 08:18:29,758 INFO [train.py:1192] (1/2) Epoch 76, batch 100, loss[loss=0.2547, simple_loss=0.3717, pruned_loss=0.06888, over 24595.00 frames. ], tot_loss[loss=0.2606, simple_loss=0.3817, pruned_loss=0.06974, over 1915598.10 frames. ], batch size: 154, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:18:30,922 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=7.63 vs. limit=15.0 2026-09-24 08:18:32,140 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=239833.33333333334, ans=0.035 2026-09-24 08:18:34,251 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.830e+02 3.495e+02 3.949e+02 4.420e+02 6.131e+02, threshold=7.897e+02, percent-clipped=0.0 2026-09-24 08:18:36,507 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=239866.66666666666, ans=0.125 2026-09-24 08:18:47,364 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=8.96 vs. limit=22.5 2026-09-24 08:18:47,685 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=239933.33333333334, ans=0.025 2026-09-24 08:18:55,256 INFO [train.py:1192] (1/2) Epoch 76, batch 150, loss[loss=0.1909, simple_loss=0.31, pruned_loss=0.03585, over 24245.00 frames. ], tot_loss[loss=0.2559, simple_loss=0.3762, pruned_loss=0.06778, over 2560460.81 frames. ], batch size: 125, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:18:56,881 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=240000.0, ans=0.125 2026-09-24 08:19:01,806 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.hidden_balancer.prob, batch_count=240033.33333333334, ans=0.125 2026-09-24 08:19:05,004 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.00 vs. limit=6.0 2026-09-24 08:19:14,158 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff3_skip_rate, batch_count=240100.0, ans=0.0 2026-09-24 08:19:19,783 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=5.21 vs. limit=15.0 2026-09-24 08:19:21,031 INFO [train.py:1192] (1/2) Epoch 76, batch 200, loss[loss=0.2863, simple_loss=0.4154, pruned_loss=0.07859, over 24209.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3739, pruned_loss=0.06681, over 3059361.90 frames. ], batch size: 257, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:19:25,646 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.638e+02 3.457e+02 3.825e+02 4.494e+02 1.295e+03, threshold=7.650e+02, percent-clipped=1.0 2026-09-24 08:19:26,183 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.skip_rate, batch_count=240200.0, ans=0.04949747468305833 2026-09-24 08:19:28,536 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=240200.0, ans=0.0 2026-09-24 08:19:31,409 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=240233.33333333334, ans=0.125 2026-09-24 08:19:33,813 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=240233.33333333334, ans=0.125 2026-09-24 08:19:39,858 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.whiten1.whitening_limit, batch_count=240266.66666666666, ans=10.0 2026-09-24 08:19:46,106 INFO [train.py:1192] (1/2) Epoch 76, batch 250, loss[loss=0.2897, simple_loss=0.4132, pruned_loss=0.08312, over 24369.00 frames. ], tot_loss[loss=0.2536, simple_loss=0.3732, pruned_loss=0.06704, over 3443559.40 frames. ], batch size: 225, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:20:01,925 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.74 vs. limit=6.0 2026-09-24 08:20:02,333 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.07 vs. limit=15.0 2026-09-24 08:20:12,605 INFO [train.py:1192] (1/2) Epoch 76, batch 300, loss[loss=0.2754, simple_loss=0.4062, pruned_loss=0.07231, over 24534.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.3728, pruned_loss=0.06657, over 3756897.01 frames. ], batch size: 204, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:20:16,837 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.720e+02 3.464e+02 3.928e+02 4.450e+02 6.560e+02, threshold=7.856e+02, percent-clipped=0.0 2026-09-24 08:20:24,373 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=240566.66666666666, ans=0.0 2026-09-24 08:20:32,213 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=240633.33333333334, ans=0.125 2026-09-24 08:20:34,578 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=240633.33333333334, ans=0.0 2026-09-24 08:20:35,055 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=240633.33333333334, ans=0.1 2026-09-24 08:20:37,710 INFO [train.py:1192] (1/2) Epoch 76, batch 350, loss[loss=0.2177, simple_loss=0.331, pruned_loss=0.0522, over 24575.00 frames. ], tot_loss[loss=0.2529, simple_loss=0.373, pruned_loss=0.06635, over 3998221.78 frames. ], batch size: 137, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:20:37,801 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=240666.66666666666, ans=0.125 2026-09-24 08:20:44,355 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=240700.0, ans=0.125 2026-09-24 08:20:54,574 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=240766.66666666666, ans=0.125 2026-09-24 08:20:57,425 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=240800.0, ans=0.1 2026-09-24 08:20:58,588 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=240800.0, ans=0.0 2026-09-24 08:20:58,954 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass_mid.scale_min, batch_count=240800.0, ans=0.2 2026-09-24 08:21:02,085 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=240800.0, ans=0.1 2026-09-24 08:21:02,984 INFO [train.py:1192] (1/2) Epoch 76, batch 400, loss[loss=0.2521, simple_loss=0.3695, pruned_loss=0.06728, over 24554.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3719, pruned_loss=0.06563, over 4180021.02 frames. ], batch size: 170, lr: 2.78e-03, grad_scale: 32.0 2026-09-24 08:21:04,517 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.const_attention_rate, batch_count=240833.33333333334, ans=0.025 2026-09-24 08:21:07,147 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.763e+02 3.331e+02 3.627e+02 4.142e+02 8.048e+02, threshold=7.254e+02, percent-clipped=1.0 2026-09-24 08:21:23,048 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=240966.66666666666, ans=0.0 2026-09-24 08:21:23,470 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=240966.66666666666, ans=0.1 2026-09-24 08:21:26,499 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass_mid.scale_min, batch_count=240966.66666666666, ans=0.2 2026-09-24 08:21:27,872 INFO [train.py:1192] (1/2) Epoch 76, batch 450, loss[loss=0.2488, simple_loss=0.3727, pruned_loss=0.06249, over 24635.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3715, pruned_loss=0.06545, over 4319752.50 frames. ], batch size: 175, lr: 2.77e-03, grad_scale: 32.0 2026-09-24 08:21:29,321 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.14 vs. limit=10.0 2026-09-24 08:21:32,558 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=241033.33333333334, ans=0.125 2026-09-24 08:21:41,428 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=241066.66666666666, ans=0.0 2026-09-24 08:21:46,086 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:21:50,285 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=241133.33333333334, ans=0.125 2026-09-24 08:21:51,733 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=241133.33333333334, ans=0.125 2026-09-24 08:21:53,138 INFO [train.py:1192] (1/2) Epoch 76, batch 500, loss[loss=0.2928, simple_loss=0.4178, pruned_loss=0.08389, over 24525.00 frames. ], tot_loss[loss=0.2501, simple_loss=0.3703, pruned_loss=0.06497, over 4437572.12 frames. ], batch size: 218, lr: 2.77e-03, grad_scale: 16.0 2026-09-24 08:21:54,630 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:21:55,737 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.const_attention_rate, batch_count=241166.66666666666, ans=0.025 2026-09-24 08:21:55,743 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer_ff2.min_abs, batch_count=241166.66666666666, ans=0.1 2026-09-24 08:21:58,110 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.491e+02 3.246e+02 3.684e+02 4.227e+02 7.383e+02, threshold=7.368e+02, percent-clipped=1.0 2026-09-24 08:22:04,741 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=241233.33333333334, ans=0.1 2026-09-24 08:22:19,051 INFO [train.py:1192] (1/2) Epoch 76, batch 550, loss[loss=0.285, simple_loss=0.411, pruned_loss=0.0795, over 24287.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3714, pruned_loss=0.06536, over 4525112.87 frames. ], batch size: 257, lr: 2.77e-03, grad_scale: 16.0 2026-09-24 08:22:22,383 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.53 vs. limit=10.0 2026-09-24 08:22:35,169 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=241433.33333333334, ans=0.0 2026-09-24 08:22:39,495 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=192, metric=5.94 vs. limit=15.0 2026-09-24 08:22:41,341 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass_mid.scale_min, batch_count=241466.66666666666, ans=0.2 2026-09-24 08:22:45,484 INFO [train.py:1192] (1/2) Epoch 76, batch 600, loss[loss=0.286, simple_loss=0.4089, pruned_loss=0.08161, over 24352.00 frames. ], tot_loss[loss=0.2532, simple_loss=0.373, pruned_loss=0.06666, over 4590680.13 frames. ], batch size: 234, lr: 2.77e-03, grad_scale: 16.0 2026-09-24 08:22:50,573 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.795e+02 3.387e+02 3.819e+02 4.436e+02 6.290e+02, threshold=7.639e+02, percent-clipped=0.0 2026-09-24 08:22:51,732 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=241533.33333333334, ans=0.125 2026-09-24 08:22:53,684 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=241533.33333333334, ans=0.1 2026-09-24 08:22:59,111 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.balancer2.prob, batch_count=241566.66666666666, ans=0.125 2026-09-24 08:23:11,331 INFO [train.py:1192] (1/2) Epoch 76, batch 650, loss[loss=0.2705, simple_loss=0.3745, pruned_loss=0.08325, over 24604.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3718, pruned_loss=0.06588, over 4655021.37 frames. ], batch size: 154, lr: 2.77e-03, grad_scale: 16.0 2026-09-24 08:23:14,096 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer1.prob, batch_count=241666.66666666666, ans=0.125 2026-09-24 08:23:30,613 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=241766.66666666666, ans=0.0 2026-09-24 08:23:34,942 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=3.60 vs. limit=15.0 2026-09-24 08:23:37,769 INFO [train.py:1192] (1/2) Epoch 76, batch 700, loss[loss=0.2521, simple_loss=0.3693, pruned_loss=0.0675, over 24556.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3724, pruned_loss=0.06595, over 4691221.99 frames. ], batch size: 158, lr: 2.77e-03, grad_scale: 16.0 2026-09-24 08:23:42,330 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.balancer1.prob, batch_count=241866.66666666666, ans=0.125 2026-09-24 08:23:42,683 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.586e+02 3.452e+02 3.738e+02 4.196e+02 6.416e+02, threshold=7.476e+02, percent-clipped=0.0 2026-09-24 08:23:44,359 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=15.37 vs. limit=15.0 2026-09-24 08:23:50,105 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=4.74 vs. limit=15.0 2026-09-24 08:24:02,413 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=241966.66666666666, ans=0.0 2026-09-24 08:24:03,538 INFO [train.py:1192] (1/2) Epoch 76, batch 750, loss[loss=0.2572, simple_loss=0.375, pruned_loss=0.06969, over 24631.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3717, pruned_loss=0.06595, over 4724081.62 frames. ], batch size: 175, lr: 2.77e-03, grad_scale: 16.0 2026-09-24 08:24:04,496 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=242000.0, ans=0.1 2026-09-24 08:24:06,864 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=242000.0, ans=0.125 2026-09-24 08:24:10,257 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=242033.33333333334, ans=0.0 2026-09-24 08:24:17,619 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=242066.66666666666, ans=0.125 2026-09-24 08:24:20,462 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=242100.0, ans=0.0 2026-09-24 08:24:25,403 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=242133.33333333334, ans=0.0 2026-09-24 08:24:28,135 INFO [train.py:1192] (1/2) Epoch 76, batch 800, loss[loss=0.2023, simple_loss=0.3232, pruned_loss=0.04073, over 24538.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3715, pruned_loss=0.06586, over 4750171.89 frames. ], batch size: 137, lr: 2.77e-03, grad_scale: 32.0 2026-09-24 08:24:32,938 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.807e+02 3.450e+02 3.951e+02 4.539e+02 6.485e+02, threshold=7.903e+02, percent-clipped=0.0 2026-09-24 08:24:40,445 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer2.prob, batch_count=242233.33333333334, ans=0.125 2026-09-24 08:24:41,756 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.ff3_skip_rate, batch_count=242233.33333333334, ans=0.0 2026-09-24 08:24:43,043 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=242266.66666666666, ans=0.1 2026-09-24 08:24:45,971 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer1.min_positive, batch_count=242266.66666666666, ans=0.025 2026-09-24 08:24:46,434 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.ff3_skip_rate, batch_count=242266.66666666666, ans=0.0 2026-09-24 08:24:53,154 INFO [train.py:1192] (1/2) Epoch 76, batch 850, loss[loss=0.2622, simple_loss=0.3926, pruned_loss=0.06591, over 24560.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3708, pruned_loss=0.06557, over 4769768.71 frames. ], batch size: 204, lr: 2.77e-03, grad_scale: 32.0 2026-09-24 08:24:53,238 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.attention_skip_rate, batch_count=242333.33333333334, ans=0.0 2026-09-24 08:24:56,101 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=4.56 vs. limit=15.0 2026-09-24 08:25:04,415 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=242400.0, ans=0.015 2026-09-24 08:25:07,010 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=242400.0, ans=0.1 2026-09-24 08:25:07,984 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.ff2_skip_rate, batch_count=242433.33333333334, ans=0.0 2026-09-24 08:25:18,491 INFO [train.py:1192] (1/2) Epoch 76, batch 900, loss[loss=0.214, simple_loss=0.3315, pruned_loss=0.04831, over 24565.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3717, pruned_loss=0.06623, over 4779885.89 frames. ], batch size: 137, lr: 2.77e-03, grad_scale: 32.0 2026-09-24 08:25:18,627 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=242500.0, ans=0.125 2026-09-24 08:25:23,805 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.726e+02 3.589e+02 4.035e+02 4.771e+02 7.613e+02, threshold=8.071e+02, percent-clipped=0.0 2026-09-24 08:25:32,199 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.balancer2.prob, batch_count=242566.66666666666, ans=0.125 2026-09-24 08:25:36,232 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=242600.0, ans=0.125 2026-09-24 08:25:39,241 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass_mid.scale_min, batch_count=242633.33333333334, ans=0.2 2026-09-24 08:25:43,465 INFO [train.py:1192] (1/2) Epoch 76, batch 950, loss[loss=0.3201, simple_loss=0.4012, pruned_loss=0.1195, over 11767.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.37, pruned_loss=0.0668, over 4705921.48 frames. ], batch size: 334, lr: 2.76e-03, grad_scale: 16.0 2026-09-24 08:25:52,694 INFO [train.py:1192] (1/2) Epoch 77, batch 0, loss[loss=0.2151, simple_loss=0.3355, pruned_loss=0.04739, over 24580.00 frames. ], tot_loss[loss=0.2151, simple_loss=0.3355, pruned_loss=0.04739, over 24580.00 frames. ], batch size: 137, lr: 2.75e-03, grad_scale: 32.0 2026-09-24 08:25:52,702 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 08:26:02,687 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.6598, 2.5074, 2.1583, 3.0478], device='cuda:1') 2026-09-24 08:26:04,145 INFO [train.py:1224] (1/2) Epoch 77, validation: loss=0.1678, simple_loss=0.286, pruned_loss=0.0248, over 2564189.00 frames. 2026-09-24 08:26:04,145 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 08:26:09,241 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.88 vs. limit=6.0 2026-09-24 08:26:14,545 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.49 vs. limit=22.5 2026-09-24 08:26:16,057 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer2.prob, batch_count=242760.0, ans=0.125 2026-09-24 08:26:22,186 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=242793.33333333334, ans=0.0 2026-09-24 08:26:23,408 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=3.88 vs. limit=12.0 2026-09-24 08:26:25,650 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=242826.66666666666, ans=0.125 2026-09-24 08:26:28,228 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=242826.66666666666, ans=0.125 2026-09-24 08:26:29,670 INFO [train.py:1192] (1/2) Epoch 77, batch 50, loss[loss=0.1952, simple_loss=0.31, pruned_loss=0.04021, over 24296.00 frames. ], tot_loss[loss=0.257, simple_loss=0.3767, pruned_loss=0.06859, over 1081705.91 frames. ], batch size: 125, lr: 2.75e-03, grad_scale: 32.0 2026-09-24 08:26:30,220 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.4.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:26:31,055 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.642e+02 3.542e+02 4.135e+02 4.724e+02 7.671e+02, threshold=8.269e+02, percent-clipped=0.0 2026-09-24 08:26:31,170 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=242860.0, ans=0.125 2026-09-24 08:26:32,584 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=5.19 vs. limit=15.0 2026-09-24 08:26:35,614 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=242893.33333333334, ans=0.0 2026-09-24 08:26:35,624 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=242893.33333333334, ans=0.0 2026-09-24 08:26:41,572 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=242926.66666666666, ans=0.1 2026-09-24 08:26:48,588 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=242960.0, ans=0.125 2026-09-24 08:26:54,951 INFO [train.py:1192] (1/2) Epoch 77, batch 100, loss[loss=0.2496, simple_loss=0.3592, pruned_loss=0.06995, over 24606.00 frames. ], tot_loss[loss=0.2583, simple_loss=0.3796, pruned_loss=0.0685, over 1914213.64 frames. ], batch size: 154, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:27:00,629 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=243060.0, ans=0.1 2026-09-24 08:27:10,033 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=243126.66666666666, ans=0.125 2026-09-24 08:27:12,104 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=243126.66666666666, ans=0.015 2026-09-24 08:27:15,116 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module2.balancer1.min_positive, batch_count=243160.0, ans=0.025 2026-09-24 08:27:20,270 INFO [train.py:1192] (1/2) Epoch 77, batch 150, loss[loss=0.2081, simple_loss=0.3188, pruned_loss=0.04873, over 24321.00 frames. ], tot_loss[loss=0.2542, simple_loss=0.3747, pruned_loss=0.06688, over 2559821.82 frames. ], batch size: 125, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:27:21,585 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.750e+02 3.580e+02 3.897e+02 4.233e+02 6.143e+02, threshold=7.794e+02, percent-clipped=0.0 2026-09-24 08:27:27,151 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=243226.66666666666, ans=0.125 2026-09-24 08:27:37,349 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=9.79 vs. limit=15.0 2026-09-24 08:27:45,381 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.84 vs. limit=15.0 2026-09-24 08:27:45,599 INFO [train.py:1192] (1/2) Epoch 77, batch 200, loss[loss=0.2584, simple_loss=0.3927, pruned_loss=0.06203, over 24205.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3725, pruned_loss=0.06601, over 3059196.78 frames. ], batch size: 257, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:27:48,028 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=243360.0, ans=0.0 2026-09-24 08:27:58,096 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.1.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:28:00,280 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module1.balancer1.prob, batch_count=243426.66666666666, ans=0.125 2026-09-24 08:28:11,392 INFO [train.py:1192] (1/2) Epoch 77, batch 250, loss[loss=0.2861, simple_loss=0.4076, pruned_loss=0.0823, over 24370.00 frames. ], tot_loss[loss=0.2524, simple_loss=0.3721, pruned_loss=0.06638, over 3444026.56 frames. ], batch size: 225, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:28:11,938 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.balancer1.prob, batch_count=243526.66666666666, ans=0.125 2026-09-24 08:28:12,831 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.673e+02 3.585e+02 3.996e+02 4.594e+02 6.658e+02, threshold=7.992e+02, percent-clipped=0.0 2026-09-24 08:28:19,601 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=243560.0, ans=0.125 2026-09-24 08:28:24,573 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=243593.33333333334, ans=10.0 2026-09-24 08:28:30,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=243626.66666666666, ans=0.025 2026-09-24 08:28:30,615 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=243626.66666666666, ans=0.2 2026-09-24 08:28:31,853 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=243660.0, ans=0.125 2026-09-24 08:28:37,016 INFO [train.py:1192] (1/2) Epoch 77, batch 300, loss[loss=0.2867, simple_loss=0.4054, pruned_loss=0.08402, over 24559.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3712, pruned_loss=0.06585, over 3758232.72 frames. ], batch size: 204, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:29:02,461 INFO [train.py:1192] (1/2) Epoch 77, batch 350, loss[loss=0.2289, simple_loss=0.3396, pruned_loss=0.05909, over 24568.00 frames. ], tot_loss[loss=0.252, simple_loss=0.3719, pruned_loss=0.06605, over 3998491.23 frames. ], batch size: 137, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:29:03,943 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.724e+02 3.449e+02 3.803e+02 4.343e+02 6.609e+02, threshold=7.606e+02, percent-clipped=0.0 2026-09-24 08:29:04,055 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.bypass.scale_min, batch_count=243860.0, ans=0.2 2026-09-24 08:29:27,882 INFO [train.py:1192] (1/2) Epoch 77, batch 400, loss[loss=0.264, simple_loss=0.3832, pruned_loss=0.07243, over 24581.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3715, pruned_loss=0.06609, over 4180271.75 frames. ], batch size: 170, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:29:29,273 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=244026.66666666666, ans=0.05 2026-09-24 08:29:40,774 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.64 vs. limit=10.0 2026-09-24 08:29:41,150 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer2.prob, batch_count=244093.33333333334, ans=0.125 2026-09-24 08:29:43,985 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=244126.66666666666, ans=0.125 2026-09-24 08:29:53,393 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:29:53,821 INFO [train.py:1192] (1/2) Epoch 77, batch 450, loss[loss=0.2858, simple_loss=0.4047, pruned_loss=0.08347, over 24631.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3719, pruned_loss=0.066, over 4319660.93 frames. ], batch size: 175, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:29:55,137 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.791e+02 3.381e+02 3.816e+02 4.471e+02 6.865e+02, threshold=7.633e+02, percent-clipped=0.0 2026-09-24 08:30:13,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=244326.66666666666, ans=0.125 2026-09-24 08:30:18,923 INFO [train.py:1192] (1/2) Epoch 77, batch 500, loss[loss=0.2505, simple_loss=0.3844, pruned_loss=0.05833, over 24488.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3709, pruned_loss=0.06586, over 4437365.97 frames. ], batch size: 218, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:30:31,232 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=244426.66666666666, ans=0.0 2026-09-24 08:30:41,106 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.nonlin_attention.balancer.min_positive, batch_count=244493.33333333334, ans=0.05 2026-09-24 08:30:44,991 INFO [train.py:1192] (1/2) Epoch 77, batch 550, loss[loss=0.2919, simple_loss=0.4167, pruned_loss=0.08356, over 24287.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3717, pruned_loss=0.06621, over 4524042.71 frames. ], batch size: 257, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:30:46,547 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.668e+02 3.350e+02 3.888e+02 4.304e+02 6.561e+02, threshold=7.776e+02, percent-clipped=0.0 2026-09-24 08:31:10,906 INFO [train.py:1192] (1/2) Epoch 77, batch 600, loss[loss=0.2897, simple_loss=0.4086, pruned_loss=0.08539, over 24326.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3723, pruned_loss=0.06654, over 4591125.89 frames. ], batch size: 234, lr: 2.74e-03, grad_scale: 32.0 2026-09-24 08:31:18,462 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=244726.66666666666, ans=0.0 2026-09-24 08:31:21,452 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.ff2_skip_rate, batch_count=244760.0, ans=0.0 2026-09-24 08:31:37,034 INFO [train.py:1192] (1/2) Epoch 77, batch 650, loss[loss=0.25, simple_loss=0.361, pruned_loss=0.06951, over 24571.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3715, pruned_loss=0.06611, over 4655328.08 frames. ], batch size: 154, lr: 2.73e-03, grad_scale: 32.0 2026-09-24 08:31:38,308 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.482e+02 3.401e+02 3.705e+02 4.317e+02 6.289e+02, threshold=7.410e+02, percent-clipped=0.0 2026-09-24 08:31:42,603 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer2.prob, batch_count=244893.33333333334, ans=0.125 2026-09-24 08:31:50,030 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff2_skip_rate, batch_count=244926.66666666666, ans=0.0 2026-09-24 08:32:02,012 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=245026.66666666666, ans=0.125 2026-09-24 08:32:02,496 INFO [train.py:1192] (1/2) Epoch 77, batch 700, loss[loss=0.253, simple_loss=0.3701, pruned_loss=0.06795, over 24560.00 frames. ], tot_loss[loss=0.2521, simple_loss=0.3722, pruned_loss=0.06604, over 4690622.24 frames. ], batch size: 158, lr: 2.73e-03, grad_scale: 32.0 2026-09-24 08:32:04,366 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.layerdrop_rate, batch_count=245026.66666666666, ans=0.015 2026-09-24 08:32:06,032 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=6.21 vs. limit=15.0 2026-09-24 08:32:14,634 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.11 vs. limit=15.0 2026-09-24 08:32:19,356 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=11.71 vs. limit=15.0 2026-09-24 08:32:24,723 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:32:26,720 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=245160.0, ans=0.1 2026-09-24 08:32:27,970 INFO [train.py:1192] (1/2) Epoch 77, batch 750, loss[loss=0.27, simple_loss=0.387, pruned_loss=0.07652, over 24623.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3715, pruned_loss=0.0659, over 4726565.38 frames. ], batch size: 175, lr: 2.73e-03, grad_scale: 32.0 2026-09-24 08:32:28,060 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=245193.33333333334, ans=0.125 2026-09-24 08:32:29,822 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 3.022e+02 3.446e+02 3.846e+02 4.397e+02 7.250e+02, threshold=7.693e+02, percent-clipped=0.0 2026-09-24 08:32:30,496 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.47 vs. limit=15.0 2026-09-24 08:32:31,796 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.prob, batch_count=245193.33333333334, ans=0.125 2026-09-24 08:32:36,105 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.const_attention_rate, batch_count=245226.66666666666, ans=0.025 2026-09-24 08:32:37,596 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=245260.0, ans=0.1 2026-09-24 08:32:38,009 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.balancer2.prob, batch_count=245260.0, ans=0.125 2026-09-24 08:32:43,550 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.73 vs. limit=15.0 2026-09-24 08:32:43,929 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.attention_skip_rate, batch_count=245293.33333333334, ans=0.0 2026-09-24 08:32:53,145 INFO [train.py:1192] (1/2) Epoch 77, batch 800, loss[loss=0.2053, simple_loss=0.3276, pruned_loss=0.04148, over 24530.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3714, pruned_loss=0.06599, over 4753736.10 frames. ], batch size: 137, lr: 2.73e-03, grad_scale: 32.0 2026-09-24 08:33:03,234 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=245426.66666666666, ans=0.1 2026-09-24 08:33:05,073 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=245426.66666666666, ans=0.125 2026-09-24 08:33:08,225 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=8, num_channels=256, metric=4.88 vs. limit=6.0 2026-09-24 08:33:13,416 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=9.96 vs. limit=22.5 2026-09-24 08:33:18,387 INFO [train.py:1192] (1/2) Epoch 77, batch 850, loss[loss=0.2612, simple_loss=0.3964, pruned_loss=0.06296, over 24544.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3705, pruned_loss=0.06523, over 4772118.81 frames. ], batch size: 204, lr: 2.73e-03, grad_scale: 32.0 2026-09-24 08:33:19,866 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.524e+02 3.493e+02 4.068e+02 4.856e+02 6.731e+02, threshold=8.137e+02, percent-clipped=0.0 2026-09-24 08:33:21,229 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.out_combiner.scale_min, batch_count=245526.66666666666, ans=0.2 2026-09-24 08:33:32,291 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.01 vs. limit=15.0 2026-09-24 08:33:37,331 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=245626.66666666666, ans=0.125 2026-09-24 08:33:40,779 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.62 vs. limit=22.5 2026-09-24 08:33:43,723 INFO [train.py:1192] (1/2) Epoch 77, batch 900, loss[loss=0.2137, simple_loss=0.334, pruned_loss=0.0467, over 24550.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3708, pruned_loss=0.06531, over 4782291.22 frames. ], batch size: 137, lr: 2.73e-03, grad_scale: 32.0 2026-09-24 08:33:48,519 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=245726.66666666666, ans=0.125 2026-09-24 08:33:52,531 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn2.whiten, num_groups=1, num_channels=192, metric=11.92 vs. limit=22.5 2026-09-24 08:34:02,808 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_skip_rate, batch_count=245793.33333333334, ans=0.0 2026-09-24 08:34:09,055 INFO [train.py:1192] (1/2) Epoch 77, batch 950, loss[loss=0.3677, simple_loss=0.4309, pruned_loss=0.1523, over 11072.00 frames. ], tot_loss[loss=0.2518, simple_loss=0.3699, pruned_loss=0.0668, over 4706105.69 frames. ], batch size: 333, lr: 2.73e-03, grad_scale: 32.0 2026-09-24 08:34:10,525 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.720e+02 3.408e+02 3.913e+02 4.584e+02 6.615e+02, threshold=7.826e+02, percent-clipped=0.0 2026-09-24 08:34:20,058 INFO [train.py:1192] (1/2) Epoch 78, batch 0, loss[loss=0.2131, simple_loss=0.3324, pruned_loss=0.04687, over 24570.00 frames. ], tot_loss[loss=0.2131, simple_loss=0.3324, pruned_loss=0.04687, over 24570.00 frames. ], batch size: 137, lr: 2.71e-03, grad_scale: 32.0 2026-09-24 08:34:20,058 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 08:34:21,708 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([1.7227, 1.3432, 2.5007, 1.5130], device='cuda:1') 2026-09-24 08:34:24,040 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.2.encoder.layers.1.self_attn_weights, attn_weights_entropy = tensor([2.2904, 3.9872, 3.7743, 3.2295], device='cuda:1') 2026-09-24 08:34:26,497 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.8257, 2.2083, 3.0292, 1.4983], device='cuda:1') 2026-09-24 08:34:31,824 INFO [train.py:1224] (1/2) Epoch 78, validation: loss=0.169, simple_loss=0.2871, pruned_loss=0.02545, over 2564189.00 frames. 2026-09-24 08:34:31,824 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 08:34:35,291 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.bypass.scale_min, batch_count=245886.66666666666, ans=0.2 2026-09-24 08:34:41,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass_mid.scale_min, batch_count=245953.33333333334, ans=0.2 2026-09-24 08:34:46,251 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=245986.66666666666, ans=0.0 2026-09-24 08:34:47,437 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=12.37 vs. limit=22.5 2026-09-24 08:34:48,384 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=192, metric=11.00 vs. limit=15.0 2026-09-24 08:34:56,846 INFO [train.py:1192] (1/2) Epoch 78, batch 50, loss[loss=0.2092, simple_loss=0.3224, pruned_loss=0.04794, over 24316.00 frames. ], tot_loss[loss=0.2558, simple_loss=0.3759, pruned_loss=0.0678, over 1081348.01 frames. ], batch size: 125, lr: 2.71e-03, grad_scale: 32.0 2026-09-24 08:35:13,319 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.const_attention_rate, batch_count=246153.33333333334, ans=0.025 2026-09-24 08:35:18,203 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.45 vs. limit=22.5 2026-09-24 08:35:19,544 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=246186.66666666666, ans=0.0 2026-09-24 08:35:20,367 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.706e+02 3.564e+02 3.879e+02 4.906e+02 6.303e+02, threshold=7.759e+02, percent-clipped=0.0 2026-09-24 08:35:22,859 INFO [train.py:1192] (1/2) Epoch 78, batch 100, loss[loss=0.2396, simple_loss=0.355, pruned_loss=0.06206, over 24591.00 frames. ], tot_loss[loss=0.2593, simple_loss=0.3801, pruned_loss=0.06923, over 1914694.19 frames. ], batch size: 154, lr: 2.71e-03, grad_scale: 32.0 2026-09-24 08:35:24,322 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=246220.0, ans=0.125 2026-09-24 08:35:25,696 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=246220.0, ans=0.1 2026-09-24 08:35:26,187 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=246220.0, ans=0.125 2026-09-24 08:35:26,655 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=246220.0, ans=0.125 2026-09-24 08:35:30,383 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:35:35,201 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.feed_forward1.out_proj.dropout_p, batch_count=246286.66666666666, ans=0.1 2026-09-24 08:35:48,804 INFO [train.py:1192] (1/2) Epoch 78, batch 150, loss[loss=0.2232, simple_loss=0.332, pruned_loss=0.05716, over 24293.00 frames. ], tot_loss[loss=0.256, simple_loss=0.3757, pruned_loss=0.06815, over 2560274.08 frames. ], batch size: 125, lr: 2.71e-03, grad_scale: 16.0 2026-09-24 08:35:54,857 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=246420.0, ans=0.0 2026-09-24 08:35:55,714 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=246420.0, ans=0.2 2026-09-24 08:36:13,175 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.731e+02 3.462e+02 3.795e+02 4.434e+02 6.400e+02, threshold=7.589e+02, percent-clipped=0.0 2026-09-24 08:36:15,516 INFO [train.py:1192] (1/2) Epoch 78, batch 200, loss[loss=0.2773, simple_loss=0.403, pruned_loss=0.07577, over 24222.00 frames. ], tot_loss[loss=0.2541, simple_loss=0.3738, pruned_loss=0.06714, over 3058380.96 frames. ], batch size: 257, lr: 2.71e-03, grad_scale: 16.0 2026-09-24 08:36:20,963 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer1.prob, batch_count=246586.66666666666, ans=0.125 2026-09-24 08:36:23,688 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=246586.66666666666, ans=0.0 2026-09-24 08:36:24,951 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.whiten, num_groups=1, num_channels=192, metric=4.11 vs. limit=12.0 2026-09-24 08:36:41,359 INFO [train.py:1192] (1/2) Epoch 78, batch 250, loss[loss=0.2772, simple_loss=0.3988, pruned_loss=0.07774, over 24353.00 frames. ], tot_loss[loss=0.2538, simple_loss=0.3733, pruned_loss=0.06719, over 3442662.77 frames. ], batch size: 225, lr: 2.71e-03, grad_scale: 16.0 2026-09-24 08:36:46,487 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.1.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:36:50,520 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=6.29 vs. limit=12.0 2026-09-24 08:36:50,607 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=5.15 vs. limit=15.0 2026-09-24 08:36:50,983 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.ff3_skip_rate, batch_count=246753.33333333334, ans=0.0 2026-09-24 08:36:59,857 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=246820.0, ans=0.125 2026-09-24 08:37:04,159 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer1.prob, batch_count=246853.33333333334, ans=0.125 2026-09-24 08:37:04,963 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.840e+02 3.577e+02 4.129e+02 4.659e+02 7.262e+02, threshold=8.259e+02, percent-clipped=0.0 2026-09-24 08:37:06,940 INFO [train.py:1192] (1/2) Epoch 78, batch 300, loss[loss=0.2711, simple_loss=0.3976, pruned_loss=0.07229, over 24536.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.372, pruned_loss=0.06632, over 3756001.46 frames. ], batch size: 204, lr: 2.71e-03, grad_scale: 16.0 2026-09-24 08:37:10,288 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer1.prob, batch_count=246886.66666666666, ans=0.125 2026-09-24 08:37:14,422 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=9.58 vs. limit=22.5 2026-09-24 08:37:15,442 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_skip_rate, batch_count=246920.0, ans=0.0 2026-09-24 08:37:31,133 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:37:32,108 INFO [train.py:1192] (1/2) Epoch 78, batch 350, loss[loss=0.206, simple_loss=0.3217, pruned_loss=0.04512, over 24563.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3725, pruned_loss=0.06648, over 3996859.94 frames. ], batch size: 137, lr: 2.70e-03, grad_scale: 16.0 2026-09-24 08:37:38,063 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=247086.66666666666, ans=0.0 2026-09-24 08:37:47,919 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=192, metric=4.39 vs. limit=15.0 2026-09-24 08:37:48,721 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module1.balancer2.prob, batch_count=247153.33333333334, ans=0.125 2026-09-24 08:37:50,994 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=247153.33333333334, ans=0.125 2026-09-24 08:37:56,125 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.713e+02 3.444e+02 3.822e+02 4.330e+02 1.621e+03, threshold=7.644e+02, percent-clipped=1.0 2026-09-24 08:37:58,173 INFO [train.py:1192] (1/2) Epoch 78, batch 400, loss[loss=0.2385, simple_loss=0.3641, pruned_loss=0.05643, over 24563.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3714, pruned_loss=0.06579, over 4178618.33 frames. ], batch size: 170, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:38:09,601 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.73 vs. limit=15.0 2026-09-24 08:38:17,310 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=247320.0, ans=0.0 2026-09-24 08:38:24,362 INFO [train.py:1192] (1/2) Epoch 78, batch 450, loss[loss=0.2583, simple_loss=0.3881, pruned_loss=0.06426, over 24634.00 frames. ], tot_loss[loss=0.2527, simple_loss=0.3726, pruned_loss=0.06644, over 4319554.59 frames. ], batch size: 175, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:38:32,923 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.bypass.skip_rate, batch_count=247420.0, ans=0.07 2026-09-24 08:38:39,604 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=247453.33333333334, ans=0.125 2026-09-24 08:38:48,573 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.706e+02 3.246e+02 3.816e+02 4.281e+02 5.996e+02, threshold=7.631e+02, percent-clipped=0.0 2026-09-24 08:38:50,692 INFO [train.py:1192] (1/2) Epoch 78, batch 500, loss[loss=0.2429, simple_loss=0.3769, pruned_loss=0.0544, over 24485.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3715, pruned_loss=0.06611, over 4437893.93 frames. ], batch size: 218, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:38:58,659 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer1.prob, batch_count=247586.66666666666, ans=0.125 2026-09-24 08:39:00,051 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=247586.66666666666, ans=0.125 2026-09-24 08:39:09,889 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=247653.33333333334, ans=0.125 2026-09-24 08:39:14,929 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.whiten, num_groups=1, num_channels=256, metric=4.01 vs. limit=12.0 2026-09-24 08:39:16,618 INFO [train.py:1192] (1/2) Epoch 78, batch 550, loss[loss=0.2624, simple_loss=0.3936, pruned_loss=0.06557, over 24264.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3715, pruned_loss=0.06595, over 4523093.16 frames. ], batch size: 257, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:39:26,756 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=247786.66666666666, ans=0.125 2026-09-24 08:39:28,273 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff3_skip_rate, batch_count=247786.66666666666, ans=0.0 2026-09-24 08:39:34,157 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module1.balancer1.prob, batch_count=247820.0, ans=0.125 2026-09-24 08:39:35,645 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=7.07 vs. limit=15.0 2026-09-24 08:39:36,515 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_skip_rate, batch_count=247853.33333333334, ans=0.0 2026-09-24 08:39:40,766 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.871e+02 3.377e+02 3.784e+02 4.275e+02 5.619e+02, threshold=7.568e+02, percent-clipped=0.0 2026-09-24 08:39:42,788 INFO [train.py:1192] (1/2) Epoch 78, batch 600, loss[loss=0.2536, simple_loss=0.391, pruned_loss=0.05809, over 24300.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3717, pruned_loss=0.06571, over 4590363.48 frames. ], batch size: 234, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:39:54,742 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.conv_module1.balancer2.prob, batch_count=247953.33333333334, ans=0.125 2026-09-24 08:39:59,683 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=247986.66666666666, ans=0.04949747468305833 2026-09-24 08:40:08,342 INFO [train.py:1192] (1/2) Epoch 78, batch 650, loss[loss=0.2499, simple_loss=0.3604, pruned_loss=0.06969, over 24589.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3715, pruned_loss=0.06586, over 4655102.88 frames. ], batch size: 154, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:40:08,447 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=248053.33333333334, ans=0.0 2026-09-24 08:40:18,470 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=248120.0, ans=0.0 2026-09-24 08:40:23,177 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=144, metric=6.23 vs. limit=10.0 2026-09-24 08:40:31,926 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.810e+02 3.472e+02 3.856e+02 4.289e+02 7.171e+02, threshold=7.712e+02, percent-clipped=0.0 2026-09-24 08:40:34,067 INFO [train.py:1192] (1/2) Epoch 78, batch 700, loss[loss=0.264, simple_loss=0.3746, pruned_loss=0.07667, over 24556.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.3727, pruned_loss=0.06627, over 4689489.10 frames. ], batch size: 158, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:40:40,412 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=6.13 vs. limit=15.0 2026-09-24 08:40:48,363 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.balancer1.prob, batch_count=248286.66666666666, ans=0.125 2026-09-24 08:40:55,282 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=248353.33333333334, ans=0.0 2026-09-24 08:40:58,206 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_skip_rate, batch_count=248353.33333333334, ans=0.0 2026-09-24 08:40:59,691 INFO [train.py:1192] (1/2) Epoch 78, batch 750, loss[loss=0.2595, simple_loss=0.3806, pruned_loss=0.06925, over 24619.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3723, pruned_loss=0.06637, over 4725717.02 frames. ], batch size: 175, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:41:17,104 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=5.43 vs. limit=10.0 2026-09-24 08:41:23,175 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.910e+02 3.459e+02 3.880e+02 4.813e+02 7.336e+02, threshold=7.760e+02, percent-clipped=0.0 2026-09-24 08:41:25,113 INFO [train.py:1192] (1/2) Epoch 78, batch 800, loss[loss=0.2071, simple_loss=0.3248, pruned_loss=0.04472, over 24533.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3712, pruned_loss=0.06581, over 4752047.42 frames. ], batch size: 137, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:41:42,921 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.min_abs, batch_count=248653.33333333334, ans=0.5 2026-09-24 08:41:50,416 INFO [train.py:1192] (1/2) Epoch 78, batch 850, loss[loss=0.2611, simple_loss=0.3865, pruned_loss=0.06788, over 24569.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.3707, pruned_loss=0.06551, over 4771003.96 frames. ], batch size: 204, lr: 2.70e-03, grad_scale: 32.0 2026-09-24 08:41:50,522 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module2.balancer1.prob, batch_count=248720.0, ans=0.125 2026-09-24 08:41:54,781 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=248720.0, ans=0.125 2026-09-24 08:42:01,772 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=248786.66666666666, ans=0.125 2026-09-24 08:42:12,114 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward2.hidden_balancer.prob, batch_count=248853.33333333334, ans=0.125 2026-09-24 08:42:12,569 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=7.04 vs. limit=15.0 2026-09-24 08:42:13,653 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.whiten2.whitening_limit, batch_count=248853.33333333334, ans=15.0 2026-09-24 08:42:13,894 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.596e+02 3.659e+02 4.152e+02 4.826e+02 6.895e+02, threshold=8.305e+02, percent-clipped=0.0 2026-09-24 08:42:16,182 INFO [train.py:1192] (1/2) Epoch 78, batch 900, loss[loss=0.2468, simple_loss=0.3536, pruned_loss=0.07007, over 24541.00 frames. ], tot_loss[loss=0.251, simple_loss=0.3707, pruned_loss=0.0656, over 4781011.18 frames. ], batch size: 137, lr: 2.69e-03, grad_scale: 32.0 2026-09-24 08:42:18,961 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=248886.66666666666, ans=0.0 2026-09-24 08:42:22,988 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer_ff2.min_abs, batch_count=248920.0, ans=0.1 2026-09-24 08:42:31,824 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=248986.66666666666, ans=0.0 2026-09-24 08:42:32,843 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=248986.66666666666, ans=0.035 2026-09-24 08:42:39,406 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward2.hidden_balancer.prob, batch_count=249020.0, ans=0.125 2026-09-24 08:42:41,709 INFO [train.py:1192] (1/2) Epoch 78, batch 950, loss[loss=0.3346, simple_loss=0.4077, pruned_loss=0.1308, over 11726.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3706, pruned_loss=0.067, over 4717465.45 frames. ], batch size: 333, lr: 2.69e-03, grad_scale: 32.0 2026-09-24 08:42:51,047 INFO [train.py:1192] (1/2) Epoch 79, batch 0, loss[loss=0.2076, simple_loss=0.3262, pruned_loss=0.04446, over 24541.00 frames. ], tot_loss[loss=0.2076, simple_loss=0.3262, pruned_loss=0.04446, over 24541.00 frames. ], batch size: 137, lr: 2.68e-03, grad_scale: 32.0 2026-09-24 08:42:51,047 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 08:43:02,754 INFO [train.py:1224] (1/2) Epoch 79, validation: loss=0.1687, simple_loss=0.2865, pruned_loss=0.02543, over 2564189.00 frames. 2026-09-24 08:43:02,754 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 08:43:12,534 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.out_combiner.scale_min, batch_count=249146.66666666666, ans=0.2 2026-09-24 08:43:15,858 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=249146.66666666666, ans=0.125 2026-09-24 08:43:16,951 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.whiten2, num_groups=1, num_channels=256, metric=6.37 vs. limit=15.0 2026-09-24 08:43:21,940 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.542e+02 3.541e+02 3.974e+02 4.654e+02 7.476e+02, threshold=7.947e+02, percent-clipped=0.0 2026-09-24 08:43:28,497 INFO [train.py:1192] (1/2) Epoch 79, batch 50, loss[loss=0.2129, simple_loss=0.3228, pruned_loss=0.05146, over 24215.00 frames. ], tot_loss[loss=0.2546, simple_loss=0.3742, pruned_loss=0.06748, over 1083136.33 frames. ], batch size: 125, lr: 2.68e-03, grad_scale: 32.0 2026-09-24 08:43:31,363 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.scale_min, batch_count=249246.66666666666, ans=0.2 2026-09-24 08:43:33,493 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=11.65 vs. limit=15.0 2026-09-24 08:43:48,478 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.3.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:43:50,340 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer2.prob, batch_count=249380.0, ans=0.125 2026-09-24 08:43:50,345 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_skip_rate, batch_count=249380.0, ans=0.0 2026-09-24 08:43:52,672 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.0.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=7.68 vs. limit=15.0 2026-09-24 08:43:53,999 INFO [train.py:1192] (1/2) Epoch 79, batch 100, loss[loss=0.2522, simple_loss=0.3678, pruned_loss=0.0683, over 24586.00 frames. ], tot_loss[loss=0.2566, simple_loss=0.3781, pruned_loss=0.06753, over 1916651.63 frames. ], batch size: 154, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:44:01,820 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.hidden_balancer.prob, batch_count=249446.66666666666, ans=0.125 2026-09-24 08:44:02,669 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn1.whiten, num_groups=1, num_channels=192, metric=12.35 vs. limit=22.5 2026-09-24 08:44:02,866 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module1.balancer1.max_abs, batch_count=249446.66666666666, ans=10.0 2026-09-24 08:44:10,111 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=249513.33333333334, ans=0.125 2026-09-24 08:44:12,888 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.912e+02 3.453e+02 3.822e+02 4.249e+02 5.886e+02, threshold=7.645e+02, percent-clipped=0.0 2026-09-24 08:44:19,340 INFO [train.py:1192] (1/2) Epoch 79, batch 150, loss[loss=0.2182, simple_loss=0.331, pruned_loss=0.05268, over 24214.00 frames. ], tot_loss[loss=0.2523, simple_loss=0.3734, pruned_loss=0.06559, over 2562027.08 frames. ], batch size: 125, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:44:25,193 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=8.41 vs. limit=15.0 2026-09-24 08:44:34,793 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.convnext.hidden_balancer.prob, batch_count=249680.0, ans=0.125 2026-09-24 08:44:37,430 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=249680.0, ans=0.025 2026-09-24 08:44:41,075 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=249713.33333333334, ans=0.125 2026-09-24 08:44:43,370 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_module2.balancer1.prob, batch_count=249713.33333333334, ans=0.125 2026-09-24 08:44:44,965 INFO [train.py:1192] (1/2) Epoch 79, batch 200, loss[loss=0.2935, simple_loss=0.4178, pruned_loss=0.08462, over 24204.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3716, pruned_loss=0.06451, over 3061075.84 frames. ], batch size: 257, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:44:48,936 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward2.out_whiten, num_groups=1, num_channels=256, metric=11.21 vs. limit=15.0 2026-09-24 08:44:54,282 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_skip_rate, batch_count=249780.0, ans=0.0 2026-09-24 08:44:57,229 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=249813.33333333334, ans=0.125 2026-09-24 08:45:03,703 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.95 vs. limit=22.5 2026-09-24 08:45:04,799 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.847e+02 3.520e+02 4.008e+02 4.520e+02 7.241e+02, threshold=8.015e+02, percent-clipped=0.0 2026-09-24 08:45:05,785 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=249880.0, ans=0.125 2026-09-24 08:45:08,258 INFO [scaling.py:1024] (1/2) Whitening: name=encoder_embed.out_whiten, num_groups=1, num_channels=192, metric=5.42 vs. limit=8.0 2026-09-24 08:45:09,838 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_module2.balancer1.prob, batch_count=249880.0, ans=0.125 2026-09-24 08:45:11,303 INFO [train.py:1192] (1/2) Epoch 79, batch 250, loss[loss=0.283, simple_loss=0.4087, pruned_loss=0.07869, over 24353.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3717, pruned_loss=0.06537, over 3444501.46 frames. ], batch size: 225, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:45:11,836 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.attention_skip_rate, batch_count=249913.33333333334, ans=0.0 2026-09-24 08:45:13,388 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=249913.33333333334, ans=0.95 2026-09-24 08:45:15,934 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.26 vs. limit=22.5 2026-09-24 08:45:19,339 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=249946.66666666666, ans=0.125 2026-09-24 08:45:26,384 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=250013.33333333334, ans=0.125 2026-09-24 08:45:26,398 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=250013.33333333334, ans=0.0 2026-09-24 08:45:34,987 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.ff2_skip_rate, batch_count=250046.66666666666, ans=0.0 2026-09-24 08:45:36,641 INFO [train.py:1192] (1/2) Epoch 79, batch 300, loss[loss=0.2675, simple_loss=0.4026, pruned_loss=0.06624, over 24569.00 frames. ], tot_loss[loss=0.2511, simple_loss=0.3715, pruned_loss=0.06532, over 3758700.75 frames. ], batch size: 204, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:45:42,897 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module2.balancer2.prob, batch_count=250113.33333333334, ans=0.125 2026-09-24 08:45:49,002 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module2.balancer1.prob, batch_count=250146.66666666666, ans=0.125 2026-09-24 08:45:49,884 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=250146.66666666666, ans=0.0 2026-09-24 08:45:54,859 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.81 vs. limit=15.0 2026-09-24 08:45:55,603 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.824e+02 3.495e+02 3.883e+02 4.427e+02 6.886e+02, threshold=7.766e+02, percent-clipped=0.0 2026-09-24 08:45:59,800 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.ff2_skip_rate, batch_count=250213.33333333334, ans=0.0 2026-09-24 08:46:00,808 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.bypass.skip_rate, batch_count=250213.33333333334, ans=0.04949747468305833 2026-09-24 08:46:02,106 INFO [train.py:1192] (1/2) Epoch 79, batch 350, loss[loss=0.1951, simple_loss=0.3169, pruned_loss=0.03665, over 24543.00 frames. ], tot_loss[loss=0.2525, simple_loss=0.3728, pruned_loss=0.06611, over 3995400.29 frames. ], batch size: 137, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:46:06,542 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=250280.0, ans=0.125 2026-09-24 08:46:12,047 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.bypass.skip_rate, batch_count=250313.33333333334, ans=0.09899494936611666 2026-09-24 08:46:16,683 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=250313.33333333334, ans=0.2 2026-09-24 08:46:19,119 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer2.prob, batch_count=250346.66666666666, ans=0.125 2026-09-24 08:46:23,528 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.0.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.29 vs. limit=10.0 2026-09-24 08:46:23,880 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.bypass.scale_min, batch_count=250380.0, ans=0.2 2026-09-24 08:46:27,840 INFO [train.py:1192] (1/2) Epoch 79, batch 400, loss[loss=0.2517, simple_loss=0.3741, pruned_loss=0.06463, over 24577.00 frames. ], tot_loss[loss=0.2513, simple_loss=0.3716, pruned_loss=0.06557, over 4182428.39 frames. ], batch size: 170, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:46:34,465 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.42 vs. limit=10.0 2026-09-24 08:46:39,904 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=250480.0, ans=0.1 2026-09-24 08:46:46,704 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.633e+02 3.490e+02 3.885e+02 4.618e+02 6.233e+02, threshold=7.769e+02, percent-clipped=0.0 2026-09-24 08:46:47,312 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward3.hidden_balancer.prob, batch_count=250546.66666666666, ans=0.125 2026-09-24 08:46:53,076 INFO [train.py:1192] (1/2) Epoch 79, batch 450, loss[loss=0.2569, simple_loss=0.3865, pruned_loss=0.06367, over 24634.00 frames. ], tot_loss[loss=0.2517, simple_loss=0.3718, pruned_loss=0.06576, over 4318988.35 frames. ], batch size: 175, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:46:53,177 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=250580.0, ans=0.025 2026-09-24 08:47:18,494 INFO [train.py:1192] (1/2) Epoch 79, batch 500, loss[loss=0.306, simple_loss=0.4238, pruned_loss=0.09411, over 24494.00 frames. ], tot_loss[loss=0.2505, simple_loss=0.3704, pruned_loss=0.06528, over 4436613.37 frames. ], batch size: 218, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:47:23,930 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer2.min_positive, batch_count=250780.0, ans=0.05 2026-09-24 08:47:29,975 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff2_skip_rate, batch_count=250813.33333333334, ans=0.0 2026-09-24 08:47:37,919 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.693e+02 3.381e+02 3.663e+02 4.213e+02 5.999e+02, threshold=7.326e+02, percent-clipped=0.0 2026-09-24 08:47:44,126 INFO [train.py:1192] (1/2) Epoch 79, batch 550, loss[loss=0.2647, simple_loss=0.3939, pruned_loss=0.06773, over 24252.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3711, pruned_loss=0.06564, over 4522107.25 frames. ], batch size: 257, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:47:48,443 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.balancer1.prob, batch_count=250913.33333333334, ans=0.125 2026-09-24 08:47:50,889 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=250946.66666666666, ans=0.125 2026-09-24 08:47:53,854 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=250980.0, ans=0.05 2026-09-24 08:47:59,786 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=251013.33333333334, ans=0.125 2026-09-24 08:48:06,366 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.const_attention_rate, batch_count=251046.66666666666, ans=0.025 2026-09-24 08:48:09,490 INFO [train.py:1192] (1/2) Epoch 79, batch 600, loss[loss=0.2638, simple_loss=0.3914, pruned_loss=0.06815, over 24358.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3718, pruned_loss=0.066, over 4588455.77 frames. ], batch size: 234, lr: 2.67e-03, grad_scale: 32.0 2026-09-24 08:48:17,388 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.46 vs. limit=15.0 2026-09-24 08:48:28,256 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.682e+02 3.335e+02 3.736e+02 4.552e+02 5.747e+02, threshold=7.472e+02, percent-clipped=0.0 2026-09-24 08:48:34,679 INFO [train.py:1192] (1/2) Epoch 79, batch 650, loss[loss=0.2177, simple_loss=0.3398, pruned_loss=0.04779, over 24602.00 frames. ], tot_loss[loss=0.2508, simple_loss=0.3708, pruned_loss=0.06534, over 4653246.14 frames. ], batch size: 154, lr: 2.66e-03, grad_scale: 32.0 2026-09-24 08:48:39,513 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.conv_module2.balancer2.prob, batch_count=251280.0, ans=0.125 2026-09-24 08:48:54,079 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=251380.0, ans=0.0 2026-09-24 08:49:00,176 INFO [train.py:1192] (1/2) Epoch 79, batch 700, loss[loss=0.249, simple_loss=0.3662, pruned_loss=0.06597, over 24555.00 frames. ], tot_loss[loss=0.2516, simple_loss=0.3719, pruned_loss=0.06568, over 4689149.53 frames. ], batch size: 158, lr: 2.66e-03, grad_scale: 32.0 2026-09-24 08:49:03,993 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=251413.33333333334, ans=0.1 2026-09-24 08:49:10,632 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=251480.0, ans=0.1 2026-09-24 08:49:19,127 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.914e+02 3.511e+02 3.902e+02 4.459e+02 6.947e+02, threshold=7.805e+02, percent-clipped=0.0 2026-09-24 08:49:19,574 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.conv_skip_rate, batch_count=251513.33333333334, ans=0.0 2026-09-24 08:49:21,064 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=251546.66666666666, ans=0.1 2026-09-24 08:49:21,981 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.feed_forward3.hidden_balancer.prob, batch_count=251546.66666666666, ans=0.125 2026-09-24 08:49:25,065 INFO [train.py:1192] (1/2) Epoch 79, batch 750, loss[loss=0.2554, simple_loss=0.3781, pruned_loss=0.06629, over 24651.00 frames. ], tot_loss[loss=0.2509, simple_loss=0.371, pruned_loss=0.06545, over 4725620.55 frames. ], batch size: 175, lr: 2.66e-03, grad_scale: 32.0 2026-09-24 08:49:25,168 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.conv_module1.balancer1.prob, batch_count=251580.0, ans=0.125 2026-09-24 08:49:50,418 INFO [train.py:1192] (1/2) Epoch 79, batch 800, loss[loss=0.2064, simple_loss=0.3267, pruned_loss=0.04301, over 24539.00 frames. ], tot_loss[loss=0.2504, simple_loss=0.3706, pruned_loss=0.06507, over 4751475.24 frames. ], batch size: 137, lr: 2.66e-03, grad_scale: 32.0 2026-09-24 08:49:53,246 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.nonlin_attention.whiten1, num_groups=1, num_channels=192, metric=4.18 vs. limit=10.0 2026-09-24 08:49:53,931 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.min_positive, batch_count=251746.66666666666, ans=0.05 2026-09-24 08:50:10,036 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.832e+02 3.485e+02 3.865e+02 4.570e+02 6.004e+02, threshold=7.731e+02, percent-clipped=0.0 2026-09-24 08:50:15,841 INFO [train.py:1192] (1/2) Epoch 79, batch 850, loss[loss=0.2815, simple_loss=0.403, pruned_loss=0.08001, over 24552.00 frames. ], tot_loss[loss=0.25, simple_loss=0.3702, pruned_loss=0.06492, over 4770159.13 frames. ], batch size: 204, lr: 2.66e-03, grad_scale: 32.0 2026-09-24 08:50:20,619 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=11.06 vs. limit=22.5 2026-09-24 08:50:23,546 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=251946.66666666666, ans=0.0 2026-09-24 08:50:24,117 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_skip_rate, batch_count=251946.66666666666, ans=0.0 2026-09-24 08:50:39,329 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.const_attention_rate, batch_count=252046.66666666666, ans=0.025 2026-09-24 08:50:41,599 INFO [train.py:1192] (1/2) Epoch 79, batch 900, loss[loss=0.2157, simple_loss=0.3341, pruned_loss=0.0486, over 24569.00 frames. ], tot_loss[loss=0.2503, simple_loss=0.3703, pruned_loss=0.06509, over 4780823.53 frames. ], batch size: 137, lr: 2.66e-03, grad_scale: 32.0 2026-09-24 08:50:57,875 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=252180.0, ans=0.0 2026-09-24 08:50:58,320 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=252180.0, ans=0.125 2026-09-24 08:51:01,709 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.431e+02 3.649e+02 3.988e+02 4.637e+02 8.380e+02, threshold=7.977e+02, percent-clipped=1.0 2026-09-24 08:51:04,194 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module1.balancer2.prob, batch_count=252213.33333333334, ans=0.125 2026-09-24 08:51:06,979 INFO [train.py:1192] (1/2) Epoch 79, batch 950, loss[loss=0.3687, simple_loss=0.4399, pruned_loss=0.1487, over 11240.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.37, pruned_loss=0.06644, over 4711820.95 frames. ], batch size: 334, lr: 2.66e-03, grad_scale: 16.0 2026-09-24 08:51:07,503 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=252246.66666666666, ans=0.035 2026-09-24 08:51:08,906 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn2.whiten, num_groups=1, num_channels=192, metric=20.58 vs. limit=22.5 2026-09-24 08:51:16,296 INFO [train.py:1192] (1/2) Epoch 80, batch 0, loss[loss=0.2038, simple_loss=0.3267, pruned_loss=0.04043, over 24597.00 frames. ], tot_loss[loss=0.2038, simple_loss=0.3267, pruned_loss=0.04043, over 24597.00 frames. ], batch size: 137, lr: 2.64e-03, grad_scale: 32.0 2026-09-24 08:51:16,296 INFO [train.py:1215] (1/2) Computing validation loss 2026-09-24 08:51:18,151 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.5.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.6814, 2.1990, 2.9140, 1.5698], device='cuda:1') 2026-09-24 08:51:24,517 INFO [zipformer.py:1764] (1/2) name=encoder.encoders.4.encoder.layers.0.self_attn_weights, attn_weights_entropy = tensor([2.0559, 2.9867, 2.5848, 2.2621], device='cuda:1') 2026-09-24 08:51:27,873 INFO [train.py:1224] (1/2) Epoch 80, validation: loss=0.1689, simple_loss=0.2867, pruned_loss=0.02552, over 2564189.00 frames. 2026-09-24 08:51:27,873 INFO [train.py:1225] (1/2) Maximum memory allocated so far is 13362MB 2026-09-24 08:51:32,298 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_module1.balancer1.prob, batch_count=252273.33333333334, ans=0.125 2026-09-24 08:51:37,284 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.self_attn_weights.pos_emb_skip_rate, batch_count=252306.66666666666, ans=0.0 2026-09-24 08:51:41,725 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.3.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=6.73 vs. limit=15.0 2026-09-24 08:51:52,190 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.conv_skip_rate, batch_count=252406.66666666666, ans=0.0 2026-09-24 08:51:54,048 INFO [train.py:1192] (1/2) Epoch 80, batch 50, loss[loss=0.1915, simple_loss=0.3075, pruned_loss=0.0378, over 24303.00 frames. ], tot_loss[loss=0.2561, simple_loss=0.3759, pruned_loss=0.06812, over 1081313.37 frames. ], batch size: 125, lr: 2.64e-03, grad_scale: 32.0 2026-09-24 08:52:09,732 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.794e+02 3.586e+02 3.951e+02 4.531e+02 1.002e+03, threshold=7.902e+02, percent-clipped=1.0 2026-09-24 08:52:19,588 INFO [train.py:1192] (1/2) Epoch 80, batch 100, loss[loss=0.2407, simple_loss=0.3595, pruned_loss=0.06094, over 24610.00 frames. ], tot_loss[loss=0.2591, simple_loss=0.3799, pruned_loss=0.06911, over 1914095.86 frames. ], batch size: 154, lr: 2.64e-03, grad_scale: 32.0 2026-09-24 08:52:26,991 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=252640.0, ans=0.1 2026-09-24 08:52:30,780 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.attention_skip_rate, batch_count=252673.33333333334, ans=0.0 2026-09-24 08:52:33,212 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.conv_skip_rate, batch_count=252673.33333333334, ans=0.0 2026-09-24 08:52:44,899 INFO [train.py:1192] (1/2) Epoch 80, batch 150, loss[loss=0.1856, simple_loss=0.3031, pruned_loss=0.03409, over 24241.00 frames. ], tot_loss[loss=0.2544, simple_loss=0.3745, pruned_loss=0.06713, over 2559776.12 frames. ], batch size: 125, lr: 2.64e-03, grad_scale: 32.0 2026-09-24 08:52:51,215 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.ff3_skip_rate, batch_count=252806.66666666666, ans=0.0 2026-09-24 08:53:00,765 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.639e+02 3.480e+02 3.903e+02 4.526e+02 1.036e+03, threshold=7.805e+02, percent-clipped=1.0 2026-09-24 08:53:02,543 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=252873.33333333334, ans=0.1 2026-09-24 08:53:06,410 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=15.94 vs. limit=22.5 2026-09-24 08:53:10,449 INFO [train.py:1192] (1/2) Epoch 80, batch 200, loss[loss=0.2741, simple_loss=0.4101, pruned_loss=0.06907, over 24238.00 frames. ], tot_loss[loss=0.2526, simple_loss=0.3726, pruned_loss=0.06629, over 3058247.05 frames. ], batch size: 257, lr: 2.64e-03, grad_scale: 32.0 2026-09-24 08:53:16,832 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.feed_forward1.out_proj.dropout_p, batch_count=252973.33333333334, ans=0.1 2026-09-24 08:53:21,291 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.conv_module2.balancer1.min_positive, batch_count=253006.66666666666, ans=0.025 2026-09-24 08:53:21,371 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.conv_module1.whiten, num_groups=1, num_channels=256, metric=3.38 vs. limit=15.0 2026-09-24 08:53:29,095 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=5.10 vs. limit=12.0 2026-09-24 08:53:34,830 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.58 vs. limit=6.0 2026-09-24 08:53:35,457 INFO [train.py:1192] (1/2) Epoch 80, batch 250, loss[loss=0.2612, simple_loss=0.3898, pruned_loss=0.0663, over 24375.00 frames. ], tot_loss[loss=0.2519, simple_loss=0.3718, pruned_loss=0.06603, over 3442704.86 frames. ], batch size: 225, lr: 2.64e-03, grad_scale: 16.0 2026-09-24 08:53:46,656 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module1.balancer1.prob, batch_count=253173.33333333334, ans=0.125 2026-09-24 08:53:51,625 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.914e+02 3.683e+02 4.085e+02 4.754e+02 7.887e+02, threshold=8.170e+02, percent-clipped=1.0 2026-09-24 08:54:00,955 INFO [train.py:1192] (1/2) Epoch 80, batch 300, loss[loss=0.2604, simple_loss=0.3856, pruned_loss=0.06763, over 24559.00 frames. ], tot_loss[loss=0.25, simple_loss=0.3702, pruned_loss=0.06491, over 3756106.91 frames. ], batch size: 204, lr: 2.64e-03, grad_scale: 16.0 2026-09-24 08:54:02,945 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.balancer1.prob, batch_count=253273.33333333334, ans=0.125 2026-09-24 08:54:04,467 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.attention_skip_rate, batch_count=253273.33333333334, ans=0.0 2026-09-24 08:54:12,553 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.nonlin_attention.balancer.max_positive, batch_count=253340.0, ans=0.95 2026-09-24 08:54:20,137 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.attention_skip_rate, batch_count=253373.33333333334, ans=0.0 2026-09-24 08:54:20,913 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.2.encoder.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:54:26,228 INFO [train.py:1192] (1/2) Epoch 80, batch 350, loss[loss=0.2063, simple_loss=0.3227, pruned_loss=0.04491, over 24565.00 frames. ], tot_loss[loss=0.2512, simple_loss=0.3714, pruned_loss=0.0655, over 3997118.62 frames. ], batch size: 137, lr: 2.64e-03, grad_scale: 16.0 2026-09-24 08:54:31,632 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.4.encoder.layers.1.feed_forward3.out_whiten, num_groups=1, num_channels=256, metric=9.43 vs. limit=15.0 2026-09-24 08:54:34,018 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=253473.33333333334, ans=0.0 2026-09-24 08:54:42,597 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.556e+02 3.480e+02 3.784e+02 4.358e+02 6.303e+02, threshold=7.569e+02, percent-clipped=0.0 2026-09-24 08:54:43,656 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.feed_forward2.hidden_balancer.prob, batch_count=253540.0, ans=0.125 2026-09-24 08:54:50,137 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=14.76 vs. limit=22.5 2026-09-24 08:54:51,738 INFO [train.py:1192] (1/2) Epoch 80, batch 400, loss[loss=0.2505, simple_loss=0.3737, pruned_loss=0.06372, over 24565.00 frames. ], tot_loss[loss=0.2506, simple_loss=0.3709, pruned_loss=0.06521, over 4178400.09 frames. ], batch size: 170, lr: 2.64e-03, grad_scale: 32.0 2026-09-24 08:54:55,703 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.ff3_skip_rate, batch_count=253606.66666666666, ans=0.0 2026-09-24 08:55:04,351 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.1.whiten, num_groups=1, num_channels=256, metric=10.79 vs. limit=12.0 2026-09-24 08:55:17,534 INFO [train.py:1192] (1/2) Epoch 80, batch 450, loss[loss=0.2516, simple_loss=0.3776, pruned_loss=0.06281, over 24621.00 frames. ], tot_loss[loss=0.2514, simple_loss=0.3716, pruned_loss=0.06561, over 4319664.77 frames. ], batch size: 175, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:55:22,019 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.5.prob, batch_count=253806.66666666666, ans=0.125 2026-09-24 08:55:27,610 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=253840.0, ans=0.035 2026-09-24 08:55:33,259 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_module2.balancer2.prob, batch_count=253873.33333333334, ans=0.125 2026-09-24 08:55:33,644 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.615e+02 3.296e+02 3.721e+02 4.183e+02 6.082e+02, threshold=7.443e+02, percent-clipped=0.0 2026-09-24 08:55:43,183 INFO [train.py:1192] (1/2) Epoch 80, batch 500, loss[loss=0.2986, simple_loss=0.417, pruned_loss=0.09013, over 24505.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3704, pruned_loss=0.06553, over 4437324.21 frames. ], batch size: 218, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:56:04,134 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module2.balancer1.prob, batch_count=254073.33333333334, ans=0.125 2026-09-24 08:56:06,364 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.1.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=2.85 vs. limit=6.0 2026-09-24 08:56:07,410 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.0.conv_module1.balancer1.prob, batch_count=254073.33333333334, ans=0.125 2026-09-24 08:56:09,265 INFO [train.py:1192] (1/2) Epoch 80, batch 550, loss[loss=0.2553, simple_loss=0.3911, pruned_loss=0.05981, over 24260.00 frames. ], tot_loss[loss=0.2507, simple_loss=0.3705, pruned_loss=0.06542, over 4521898.51 frames. ], batch size: 257, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:56:22,043 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=254173.33333333334, ans=0.0 2026-09-24 08:56:22,844 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.1.feed_forward1.hidden_balancer.prob, batch_count=254173.33333333334, ans=0.125 2026-09-24 08:56:25,532 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.819e+02 3.448e+02 3.764e+02 4.262e+02 8.012e+02, threshold=7.528e+02, percent-clipped=1.0 2026-09-24 08:56:30,364 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=254240.0, ans=0.2 2026-09-24 08:56:34,695 INFO [train.py:1192] (1/2) Epoch 80, batch 600, loss[loss=0.2458, simple_loss=0.3808, pruned_loss=0.05536, over 24334.00 frames. ], tot_loss[loss=0.2515, simple_loss=0.3714, pruned_loss=0.06582, over 4588746.56 frames. ], batch size: 234, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:56:36,675 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.conv_skip_rate, batch_count=254273.33333333334, ans=0.0 2026-09-24 08:56:43,932 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.self_attn_weights.whiten_keys, num_groups=4, num_channels=128, metric=3.27 vs. limit=6.0 2026-09-24 08:56:53,314 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.bypass.scale_min, batch_count=254373.33333333334, ans=0.2 2026-09-24 08:56:55,628 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.conv_module1.balancer2.prob, batch_count=254406.66666666666, ans=0.125 2026-09-24 08:56:59,897 INFO [train.py:1192] (1/2) Epoch 80, batch 650, loss[loss=0.2473, simple_loss=0.3664, pruned_loss=0.06406, over 24598.00 frames. ], tot_loss[loss=0.2498, simple_loss=0.3701, pruned_loss=0.06476, over 4653749.97 frames. ], batch size: 154, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:57:13,485 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.attention_skip_rate, batch_count=254506.66666666666, ans=0.0 2026-09-24 08:57:15,960 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.878e+02 3.361e+02 3.729e+02 4.333e+02 5.678e+02, threshold=7.458e+02, percent-clipped=0.0 2026-09-24 08:57:17,007 INFO [scaling.py:1120] (1/2) WithLoss: name=encoder.encoders.0.layers.0.self_attn_weights, loss-sum=0.000e+00 2026-09-24 08:57:20,516 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=254573.33333333334, ans=0.125 2026-09-24 08:57:25,385 INFO [train.py:1192] (1/2) Epoch 80, batch 700, loss[loss=0.2352, simple_loss=0.3546, pruned_loss=0.0579, over 24567.00 frames. ], tot_loss[loss=0.2502, simple_loss=0.3708, pruned_loss=0.06483, over 4688928.68 frames. ], batch size: 158, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:57:26,584 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.0.self_attn2.whiten, num_groups=1, num_channels=256, metric=10.44 vs. limit=22.5 2026-09-24 08:57:30,931 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.0.layers.0.feed_forward1.out_whiten, num_groups=1, num_channels=192, metric=8.02 vs. limit=15.0 2026-09-24 08:57:41,736 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder_embed.conv.8.prob, batch_count=254706.66666666666, ans=0.125 2026-09-24 08:57:43,893 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.4.encoder.layers.0.ff3_skip_rate, batch_count=254706.66666666666, ans=0.0 2026-09-24 08:57:50,349 INFO [train.py:1192] (1/2) Epoch 80, batch 750, loss[loss=0.2662, simple_loss=0.3882, pruned_loss=0.07212, over 24607.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3702, pruned_loss=0.06481, over 4725086.63 frames. ], batch size: 175, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:57:50,894 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.balancer1.prob, batch_count=254773.33333333334, ans=0.125 2026-09-24 08:58:05,983 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.1.conv_module2.balancer1.prob, batch_count=254873.33333333334, ans=0.125 2026-09-24 08:58:06,371 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.846e+02 3.517e+02 4.029e+02 4.805e+02 7.034e+02, threshold=8.058e+02, percent-clipped=0.0 2026-09-24 08:58:06,458 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.balancer2.prob, batch_count=254873.33333333334, ans=0.125 2026-09-24 08:58:15,523 INFO [train.py:1192] (1/2) Epoch 80, batch 800, loss[loss=0.2215, simple_loss=0.337, pruned_loss=0.05305, over 24577.00 frames. ], tot_loss[loss=0.2497, simple_loss=0.3699, pruned_loss=0.06474, over 4751773.82 frames. ], batch size: 137, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:58:18,739 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.1.encoder.layers.1.feed_forward1.out_whiten, num_groups=1, num_channels=256, metric=8.63 vs. limit=15.0 2026-09-24 08:58:24,256 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.bypass.scale_min, batch_count=254973.33333333334, ans=0.2 2026-09-24 08:58:35,509 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.feed_forward1.out_proj.dropout_p, batch_count=255073.33333333334, ans=0.1 2026-09-24 08:58:38,389 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.1.bypass.scale_min, batch_count=255073.33333333334, ans=0.2 2026-09-24 08:58:40,280 INFO [train.py:1192] (1/2) Epoch 80, batch 850, loss[loss=0.2743, simple_loss=0.4011, pruned_loss=0.07372, over 24526.00 frames. ], tot_loss[loss=0.2492, simple_loss=0.3694, pruned_loss=0.06447, over 4770608.84 frames. ], batch size: 204, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:58:55,772 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.5.encoder.layers.0.conv_module2.whiten, num_groups=1, num_channels=256, metric=3.23 vs. limit=15.0 2026-09-24 08:58:56,878 WARNING [optim.py:487] (1/2) Clipping_scale=2.0, grad-norm quartiles 2.839e+02 3.514e+02 3.983e+02 4.549e+02 6.342e+02, threshold=7.965e+02, percent-clipped=0.0 2026-09-24 08:58:58,204 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.0.ff3_skip_rate, batch_count=255206.66666666666, ans=0.0 2026-09-24 08:59:03,295 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.1.nonlin_attention.balancer.prob, batch_count=255240.0, ans=0.125 2026-09-24 08:59:06,377 INFO [train.py:1192] (1/2) Epoch 80, batch 900, loss[loss=0.2296, simple_loss=0.3434, pruned_loss=0.05791, over 24556.00 frames. ], tot_loss[loss=0.2499, simple_loss=0.3701, pruned_loss=0.06482, over 4781342.24 frames. ], batch size: 137, lr: 2.63e-03, grad_scale: 32.0 2026-09-24 08:59:13,860 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.2.encoder.layers.1.self_attn_weights.pos_emb_skip_rate, batch_count=255306.66666666666, ans=0.0 2026-09-24 08:59:15,220 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.0.layers.1.bypass.skip_rate, batch_count=255306.66666666666, ans=0.035 2026-09-24 08:59:23,069 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.5.encoder.layers.0.conv_module2.balancer1.prob, batch_count=255373.33333333334, ans=0.125 2026-09-24 08:59:30,775 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.1.encoder.layers.0.feed_forward1.out_proj.dropout_p, batch_count=255406.66666666666, ans=0.1 2026-09-24 08:59:30,787 INFO [scaling.py:214] (1/2) ScheduledFloat: name=encoder.encoders.3.encoder.layers.0.nonlin_attention.balancer.prob, batch_count=255406.66666666666, ans=0.125 2026-09-24 08:59:31,832 INFO [train.py:1192] (1/2) Epoch 80, batch 950, loss[loss=0.4215, simple_loss=0.4629, pruned_loss=0.19, over 10330.00 frames. ], tot_loss[loss=0.25, simple_loss=0.3688, pruned_loss=0.06559, over 4708498.29 frames. ], batch size: 333, lr: 2.63e-03, grad_scale: 16.0 2026-09-24 08:59:33,031 INFO [scaling.py:1024] (1/2) Whitening: name=encoder.encoders.2.encoder.layers.1.self_attn1.whiten, num_groups=1, num_channels=256, metric=22.69 vs. limit=22.5 2026-09-24 08:59:36,234 INFO [train.py:1469] (1/2) Done!